The Drupal community is notorious for producing excellent modules to meet specific needs without much more than a cursory acknowledgement of the interrelationship of other / complementary modules. This is no more apparent that with performance modules.

Different sites definitely call for different configurations and combinations or performance modules but there is a standard set that should be applied to virtually every production Drupal 7 site.

1. At a bare minimum, core + advagg+ httprl with proper configuration and disable performance degrading modules (see gist)
2. Consider enabling css_emimage+ speedy for most circumstances.
3. Drupal caching is not a one-size-fits all solution, but for any non-Commerce site using Views, views_cache_bully is a highly recommended easy win.
4. Utilize entitycache.
5. If you have more than 20k taxonomy terms, start leveraging taxonomy_edge.

See the complete presentation I recently gave at Drupal Camp SC attached below.