I was also very confused with this one. I payed attention to the shirts, since only 2 were red, so the one had to be green in inverted.
However, all of them I choose were wrong, making me assume the colors weren't inverted at all and then all hell broke loose.
After the hat tip it became easy though... that was pretty weird. Why could I solve it based on hats, but not shirts?
