Dunno about a checklist for something like that, but off the top of my head, I would say you have one or both of two overarching issues that affect many sites:

1) The quality of your backlink profile may not be as good as some of your competitors (I know you feel the quality is good, but can't tell you how many times I've heard that, but disagreed when looking into it; how have you assessed quality?).

2) You may have a less than well optimized site, which often means improvement is called for in:
-- site nav (including clarity, hierarchy, flat versus pyramid, redundancy),
-- nav anchor text,
-- page titles,
-- META descriptions (don't let ANYONE tell you these are not important),
-- Hx,
-- number and quality of outbound links.

Also have a look at potential dup issues within the site (too many too-similar pages).

<added> FWIW, I don't start with a checklist. I start by looking at the SERP's to see what does and does not make sense to me given a general knowledge of the site in question. That usually leads to identifying dislocations that reveal issues. Well, that's one of the things I start with, anyway. ;-) </added>

Not a checklist either per se but here's my general approach

- technical issues with hosting and code
- historical trends - log files and lots of them - review for how the site is being found and how it's being used
- SEO issues (see caveman's list and then add number and quality of INbound links and the link neighborhoods they come from)
- general sense of the market space - target keywords, the competition for them, and how well the site in question competes

- One of the things I check is the semantic structure of pages, which is likely a similar line of thought to Caveman's "Hx" comment.
- Also check for code validation, and see if there's anything that can be done to improve the ratio of unique content to code junk and/or template overhead.