If the directory is a large directory that specializes in your niche, and it takes its name from the same keywords that are your main target phrase, then it might be formidable competition. That's particulary so if it's a good directory, one which has support throughout your industry.

Eg, if your keywords are widget supplies and the directory is the 'Widget Supplies Directory', you could well have a tough time. A good niche directory might outscore you in both inbound links and traffic from the niche.

But if you're going after geo-targeted terms, like "placename widgets" and you're located in "Placename", you may have a very good shot at ranking better than the directory. That's because Google tends to favor local sites over geo-based directories in many verticals.

I have a B2B client that's in the first of the situations I describe, and while we dominate most of the queries in the niche, we've never been able to beat the Widget Directory for a query that's included "widget".