Removing 404 erors

Forum Moderators: phranque

Message Too Old, No Replies

Removing 404 erors

will google spider urls after block by robots txt

cheaperholidays

7:29 am on Aug 22, 2006 (gmt 0)

Hi all
A while ago our news blog went of on a tangent and threw up over 400 404 erros.

I have given these error pages a 410 status and blocked them in the robots.txt

However does anyone know if google will check the urls any way and get the 410 gone or will it just ignore them.

If its just ignore them, should i remove the robots txt exclusion and let google spider these urls

Kind Regards

Pete

physics

5:28 am on Aug 23, 2006 (gmt 0)

I think you're worrying a bit too much, but blocking them in the robots.txt and throwing up 410's sounds like the right thing to do in this case, unless those pages are normally good and just temporarily were causing errors.