Lets first pretend adventure games dont include games like Mario 64 and the Legend of Zelda because thats clearly not what you were reffering too.
I agree for the most part with Udvarnoky. If the action is done well, does it matter? Perhaps your problem is it hasnt been executed well in the examples you have played.
However, I think your whole line of thought is a little deceptive. Adventure games can really be a lot more than that, the genre
should keep evolving, look at the game Fahrenheit (which you have brought up) for modern day examples. Im not saying it's better or worse than traditional adventure games or even text adventure games, i'm arguing that mutliple types of adventure games can be developed concurrently and their is merit in exploring all possibly game play dynamics.
Hell, I would love nothing more than an adventure game for the Nintendo Revolution operated via that remote, just pointing at different locations on the screen. (Hey, its not
as far fetched as it seems, Nintendo have been releasing adventure games for the Nintendo

S operated via the stylus, with any luck this could stimulate interest in adventure games by Japanese developers).