Everyone hates Ben but look past that. Ben was doing something good, he stopped the Raider attacks and was in some way contributing to the group. If you look and see, those Raiders just walked straight through the camp and took everyone captive. Why? Because of Lee and Lilly.
Now think if that never happened? No raiders indefinitely. But look at it this way, no one is really to blame. Ben should've told them and Lee and Lilly should have never looked into it.
To me Ben was a good member but you guys are in a uproar because of what YOU did.