I always keep subtitles on, as I tend to parse text better than audio. I agree that it can kill a joke, but it's worth it to know I'm not missing anything. Also, I never skip dialog just because I read the subtitle too quickly... it seems like a slight to the VAs, and it ruins the flow.
"This grass feels funny. It feels like... pants." - Kirby
Also a song.