This discussion may contain spoilers!
Let's add this to the conspiracy theory pot. The Governor talks to Clem. He's the only "canon" bad guy in the walking dead series. TTG did say we'd meet some characters from the comics, TV series, and original characters they made.
That's the only other bad guy who would be any sorta trouble ... Perhaps it is even the bandits still. And yeah, I know it has nothing to do with Rick Grimes and his group, but why not debate? LOL
Also.. doesn't the governor die in the comics?
Perhaps it isn't him.