Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Googlebot reading JavaScript?

Bot identified as googlebot hitting javascript links

         

Duskrider

3:18 am on Oct 10, 2007 (gmt 0)

10+ Year Member



One of the sites I run has user submitted content, and on those pages I have a link for reporting abuse. Pretty standard. At this point all it does is send me an email so I can go look at the page and verify if a TOS violation is on there somewhere. The link is inside a document.write() and it also pops up a confirm box to ensure the user actually wants to report abuse.

For the last two days Googlebot (verified by IP 66.249.65.112) has been hitting that link and evidently clicking ok... because that's the only way the abuse reporting script will load. I've gotten hundreds of e-mails the last couple of days reporting abuse and figured it was a scraper who had a javascript enabled spider or something. I just modified my script to also include the IP, UA, and Referer of the computer doing the reporting as to catch the scraper and ban the IP... and lo and behold Google shows up the next time I get the abuse e-mail.

I had to edit the script to redirect the Googlebot UA to my index just to keep from getting the spam in my inbox.

Am I missing something, or did Googlebot just start parsing JavaScript?

tedster

6:07 am on Oct 10, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Google has been "reading" JavaScript for quite a while - but that doesn't mean that they are "parsing" it. They mostly just look at the script as plain text rather than trying to execute it. The main purpose has apparently been to look for character strings that look like urls so they can find pages that they may have missed.

I've also heard that Google experiments with some of the scripts they download in a safe "sandbox" environment, to see what sites are doing. This doesn't mean that Google is now automatically and universally understanding the effects of JavaScript and using that information in their algo.

Your report of getting abuse reports from the Googlebot IP is uncommon - and certainly not the kind of thing they would want to be doing. Sounds like an "alpha" experiment gone bad to me.