Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Indexing Failed: DNS server unresponsive (304 Redirect)

         

SEO Learner 04

5:40 am on Apr 6, 2024 (gmt 0)

Top Contributors Of The Month



Guys, my site is 7 days old and I have submitted it in gsc 4 days ago. But it isn't still indexed due to a redirect i.e. 304 Redirect. As this redirect happens when we open a site that hasn't been modified since the last visit, as the browser stores the recent version of the site and serves on requesting again with 304 redirect. But in gsc on doing a live test to the homepage I am facing this error "DNS server unresponsive" on page fetch. ... Need your assistance on resolving this error.

not2easy

11:36 am on Apr 6, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



You can clear your cache, but that should not have any effect on Google's verify efforts. You can use a different browser set to no cache. Or you can verify your domain with a .txt file, see https://support.google.com/a/answer/183895

But if you are having DNS issues, maybe you should look at that first. There are places to help you find and fix DNS situations and it's free: https://www.dnsstuff.com/

lucy24

2:53 pm on Apr 6, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



304? Are you sure? In spite of the 300-class, that’s not actually a redirect; it means “no change since last time”. Ordinarily it only applies to static files such as images, and to pages that are pure hard-coded html with no php-or-similar inclusions.

not2easy

4:31 pm on Apr 6, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I was wondering where that "(304 Redirect)" came from.

phranque

6:34 pm on Apr 6, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



a 304 is a "Not Modified" response to a conditional request with a If-Modified-Since or If-None-Match header that specifies a false condition.
it is an "implied redirect" to the cached version of the requested resource.
https://www.rfc-editor.org/rfc/rfc9110.html#status.304

lucy24

9:04 pm on Apr 6, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Re-reading OP, I do wonder if these are two separate and unrelated issues. If the server is unresponsive, then you wouldn’t expect any numerical response, because the request never got that far.

this redirect happens when we open a site that hasn't been modified since the last visit, as the browser stores the recent version of the site
If you want to split hairs, it’s when you request some specific file that hasn’t been modified. Search engine crawlers don’t request sites or even complete pages (unless you’re talking about a supplemental function such as Preview); they request individual files, one by one.

Any time you’re testing something that involves Googlebot (or some other robot) visiting your site, be sure to check your access logs immediately afterward, to see that the request was received and the expected response was sent.

SEO Learner 04

9:59 am on Apr 8, 2024 (gmt 0)

Top Contributors Of The Month



I have testified the DNS server and it's okay... But I am surprised at why this is so for Google's user-agents. This should be only for the browser's user agent.

not2easy

10:32 am on Apr 8, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I would not expect that a new site might be indexed within days of submitting it. 304 is telling Google that it is the same as it was last time they looked at it. Multiple submissions of the same content will not make them work faster.

Others are noticing the same situation, here is another current example: [webmasterworld.com...]

lucy24

4:22 pm on Apr 8, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



But I am surprised at why this is so for Google's user-agents. This should be only for the browser's user agent.
It’s not about the user-agent, it’s about whether the request includes the If-Modified-Since header. A 304 response is smaller than a complete file (of whatever kind); multiply it by G###’s millions of daily requests and it adds up.

:: quick detour to logs ::

Looks like a bit under half of Googlebot-Image requests include that header, leading to a 304 response. Interestingly, they don’t seem to use it with css requests; the only 304s I find with regular Googlebot are for pdfs.

SEO Learner 04

6:57 am on Apr 15, 2024 (gmt 0)

Top Contributors Of The Month



Thanks everyone for your kind suggestions, the site has been indexed. What I did is that I resubmitted the site with non-www URL in gsc, removed old sitemap and resubmitted it and sent for validation to the homepage. Later on the site got crawled and indexed.

not2easy

11:21 am on Apr 15, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



If you have changed the URL in GSC, have you also 301 redirected all requests to the non-www version of the site? I ask because Google sees the two URLs as different sites and can penalize for duplicate content if both versions can be visited.

SEO Learner 04

11:39 am on Apr 15, 2024 (gmt 0)

Top Contributors Of The Month



Yup... I have this version...so no worries...

not2easy

12:16 pm on Apr 15, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



The problem is not whether the non-www version exists, it is whether when you visit the www version it displays the non-www version and your logs show a 301 redirect with a 200 result.

SEO Learner 04

12:50 pm on Apr 15, 2024 (gmt 0)

Top Contributors Of The Month



you are right..it is doing the same...but I don't think this could cause an issue... because the sites we mostly visit and on returning the GET request we always get a non-www version.

lucy24

4:48 pm on Apr 15, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It isn't clear from your answer, so let’s phrase it differently:

When there is a request for https://www.example.com, is it redirected to https://example.com ?

SEO Learner 04

7:39 am on Apr 16, 2024 (gmt 0)

Top Contributors Of The Month



I mean this term: In recent years, the use of the "www" subdomain has declined, as modern web browsers automatically redirect users to the website's main URL, whether it has the "www" subdomain or not. As a result, many website owners choose not to use the "www" subdomain in their URLs to keep things simple and consistent.

phranque

9:20 am on Apr 16, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



modern web browsers will automatically redirect users to the website's main URL only if the web server responds to a non-canonical get request with a 301 status code and with a Location header referring to the site's main URL.
the browser has no way to know the preference otherwise.

in other words if you prefer non-www, then you must redirect the www requests with a 301 response.

SEO Learner 04

2:08 pm on Apr 16, 2024 (gmt 0)

Top Contributors Of The Month



You are right.. but a redirect will be made only when the resource is requested in www version..otherwise it will be served as nonwww. Now as we know, nobody requests a site using "www" in the URL.. But I have still my site serving in both versions.

not2easy

2:22 pm on Apr 16, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



But I have still my site serving in both versions.
This will cause you problems of duplicate content if both versions of your site can be accessed. This is the reason people have been asking you to look at your server's access logs, to ensure that there are no 302 (temporary) redirects happening.

The Apache [webmasterworld.com] Forum is where you can go to read (or ask) about it.

lucy24

6:08 pm on Apr 16, 2024 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The only role a browser plays in redirects is in responding to instructions from the server. If it redirects on its own volition, it is because it has previously received a 301 response on the same URL.

Your site should respond to all forms--typically http vs https, and with/without www--but should only serve content to the site's preferred canonical form. Search engines can and will request all forms of an URL, and will note the sites that don't canonicalize.

Now as we know, nobody requests a site using "www" in the URL.
It is possible you are getting your information from less-that-reliable sources, because this is a wild generalization, by no means universally true. For the simplest of illustrations, look up at your browser's address bar right now. The present site prefers with-www, and will redirect if you request without-www.

SEO Learner 04

7:08 am on Apr 17, 2024 (gmt 0)

Top Contributors Of The Month



Okay.. So now I should reindex my site with www version...with proper redirects and canonical tags..

not2easy

11:26 am on Apr 17, 2024 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



It is up to you which you prefer, but Google sees https://www.example.com and https://example.com as two different websites that are the same. You can't have two live accounts that are the same site. Just decide which one you will use and then implement a 301 (permanent) redirect to that preferred version.

Given that it has taken two weeks to explain the problem and we are still in the Google SEO forum I'd suggest you try looking in the Apache forum to find out how to do that. But no one can offer much help without an understanding of the site's platform/environment. Where and how do you manage redirection?