Ideally, you remove the pages with a low conversion rate - but this happens at the advertiser's site, so its impossible for you to know what it is. :(
I would be more guided by ECPM than CTR, but as we are working in the dark, its a little more intuitive than that. Look at the pages. Which are more likely to convert better? What sort of traffic do each of them get? Remove some, observe results, draw conclusions.
Google doesn't give us enough information to determine how to beat smartpricing, so its all about trial and error :(

I am experimenting with this right now as you are reading this thread. I use to have 2 or 3 ad units per page, now I am trying only one per page to see if my eCPM improves any. I have a pretty good CTR, so I just need to work on that darn eCPM! I will post the results in a few days.

Removing pages that have an eCPM that is very low compared to the majority of your pages is often a good idea.

But I think it's more important to look at the ads on your pages, and think about what sort of person visits that page and why. If the ads on the page are offering products that are very unlikely to be purchased by the visitors to your page, then that ad is not likely to convert.

Removing AdSense from pages that just don't show relevant ads is probably the best thing you can do. Some topics just don't come with a decent bank of adverts, and there's no point running AdSense on such pages.