404 Page Indexing Issues in Search Console - Non existing page listed - Webmaster General forum at WebmasterWorld

Forum Moderators: phranque

Message Too Old, No Replies

404 Page Indexing Issues in Search Console - Non existing page listed

Not able to find how the referring page is generating these 404 URLs

NawazMohammed

7:36 am on Jan 26, 2023 (gmt 0)

I am having an issue with page indexing in Google Search Console. Under the "Not found (404)" section, I am seeing pages with URLs like "https://example.com/1000-dollar-to-euro", however, these links do not exist on my website. When I inspect the URLs, under details it shows the Referring page as https://example.com/convert/1000-dollar-to-euro. This referring URL is the correct page URL and when I check this referring page I cannot find it generating or pointing to the incorrect URL https://example.com/1000-dollar-to-euro.

I have checked my website for broken links, redirects, and hardcoded links, but I cannot find any issues. I have also checked for any third-party scripts or plugins that may be causing the problem, but I have not found any. I am looking for help in identifying and resolving this issue.

[edited by: phranque at 9:03 am (utc) on Jan 26, 2023]
[edit reason] exemplified urls [/edit]

phranque

9:07 am on Jan 26, 2023 (gmt 0)

welcome to WebmasterWorld [webmasterworld.com], NawazMohammed!

have you checked your web server access log file for clues regarding requests for these urls?
you might also try crawling your site as googlebot to see if those urls are shown to that user agent for some reason.

NawazMohammed

10:47 am on Jan 26, 2023 (gmt 0)

Hi Phranque, The web server access log file shows that only Googlebots are accessing these non existing URLs.
What tool/s do you suggest for us to try crawling the site as googlebot to see if those urls are shown to given user agent?

Thank you.

Best Regards.

not2easy

12:45 pm on Jan 26, 2023 (gmt 0)

You can find UserAgent tools for most browsers available as built in or add-on/plugin tools from the browser you use.

If you are seeing these non-existing URLs being indexed that might be troubling, but it should not cause you any grief to see them listed as 404 results. Isn't that what you would expect to see? If Googlebot was able to access those URLs, they would not be shown as "Not found (404)" results.

Am I missing something?

NawazMohammed

1:06 pm on Jan 26, 2023 (gmt 0)

Hi not2easy+,

Yes them having a 404 is okay. But I was still curious to find from Googlebot was getting these non-existing URLs. I will try the way to suggested using a plugin tool.

Thank you.

Best Regards.

lucy24

5:12 pm on Jan 26, 2023 (gmt 0)

Funny, I see this more often with bingbot. Something in their computer gets the hiccups and they start appending some other site's paths onto my domain name. For example, for quite a while I was seeing assorted requests for what was clearly a site in Norway.

9 times out of 10, when GSC tells you something, it's just a heads-up to take a look and make sure things are happening the way they're supposed to. If they're reporting a 404 on URLs that do not, in fact, exist on your site, that's very much correct and desirable. Nothing is wrong and you need take no action. It's only a problem if they report 404s on something that does exist--or, worse, if they're crawling and indexing pages that you didn't know you had.

nickZ

7:43 pm on Jan 26, 2023 (gmt 0)

At a time people where complaining about sites not indexing fast enough, now we have the opposite, urls from whatever source are indexed and some como from test of someone trying to fetch that url.

phranque

9:06 pm on Jan 26, 2023 (gmt 0)

What tool/s do you suggest for us to try crawling the site as googlebot to see if those urls are shown to given user agent?

the Xenu LinkSleuth and Screaming Frog SEO crawlers both have Settings to specify the User Agent string for HTTP Requests of the provided url(s).

lucy24

10:05 pm on Jan 26, 2023 (gmt 0)

... assuming for the sake of discussion that the site allows visitors from entities calling themselves Googlebot in spite of coming from a non-Googlebot IP. Granted, the behavior is not nearly as common as it was in years past, but it�s not time to fling open the doors just yet.

phranque

11:29 pm on Jan 26, 2023 (gmt 0)

since you only need to crawl a single url in this case, i would use Fetch as Googlebot in your Google Search Console to see the HTML that is served to googlebot when it requests https://example.com/convert/1000-dollar-to-euro