Googlebot "200" vs. "304" Status Codes

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Googlebot "200" vs. "304" Status Codes

Ramifications of these different status

doughayman

1:26 am on Jun 12, 2007 (gmt 0)

I have a situation where Googlebot fetches a major page of my site, and in my web log it returns a Status 200, with the byte size. This page has not been changed in a while. Sometimes when it is spidered it returns the code I would expect - "304", which means that it is the same as the cached version, and it hasn't detected change.

However, whenever Googlebot returns a "200", my page is flushed from the index, to only return days later. When the "304" is encountered (as it should), the page remains in the index, and my SERPS are unaffected. My SERPs are dramatically effected for days, whenever the "200" is recorded.

What would precipitate a "200" vs. a "304"? Why would I get a "200" sometimes, and a "304" other times, if the page has not changed?

Any thoughts would be most appreciated.

tedster

6:12 am on Jun 12, 2007 (gmt 0)

Those status codes are sent by your server, not googlebot. So in terms of your server's record of the file, a 200 status should mean that something about the file has changed since the time/date that googlebot sent in its If-Modified-Since header.

It's always possible that googlebot is sending an older date than what you see as the last googlebot crawl - they do have a lot of data to coordinate.

The only ramification I know of is that a 200 is using more of your bandwidth (and possibly your googlbot crawl quota) -- because with a 304 status, the server doesn't bother sending the actual file another time.

Also, I'm not sure that every googlebot always sends the If-Modified-Since header every time. I haven't studied my logs that closely.