In particular, Google can now index Facebook comments as well as those on some other social sites such as Disqus or Intense Debate. See http://www.techmeme.com/111031/p54#a111031p54 for more detail.

(Note: the fragment identifier gets broken if we post a live link - so please copy/paste.)

This is really interesting.

Are they going to base serps on what is trending in comments and of course all of the other variables (location, etc)?

In particular, Google can now index Facebook comments

OK, so you're going to talk about this next week at pubcon then? Or someone's got to!

wouldn't they need to be a friend to read your fb comments? :)

[edited by: santapaws at 9:06 pm (utc) on Nov 1, 2011]

As I understand it, this is about the Facebook Comments Plug-in [news.softpedia.com] that's in use on many websites.

The ex-Googler @pedrodias tweeted about it a couple days ago.
[twitter.com ]

It's realy a good news.

Big move - Google will crawl more contents and links. But we need to keep moderate the links on comment (this will create spam opportunity to spammers). Can we set up no-follow links on Facebook comment box?

I think this is really interesting too. I wonder if Facebook will put a stop to it asap. Loophole or intentional? Facebook does not want Google on their side at all, just look at their robots.txt

Facebook does not want Google on their side at all, just look at their robots.txt

Those are pretty sensible exclusions, preventing the indexing of search pages, duplicate content, and so on. Google reports over 5.6 billion results from FB in its index.

This is good news for FB if if encourages wider adoption of its comment system.

Be careful to keep this in perspective.

Matt said Google can

"...index some dynamic comments..."

Matt did not say Google can index ALL dynamic comments. Webmasters should still be careful when adding AJAX to their site.

Likely a case of 'it can parse well known frameworks' to varying extent rather than "ajax/js".

I'm glad to see it. Fits in quite well with my evil empire.

Skynet could also execute AJAX, hmmmm....

OMG! It's indexing Facebook to find Sarah Connor!

[edited by: incrediBILL at 2:23 pm (utc) on Nov 2, 2011]

Big move - Google will crawl more contents and links. But we need to keep moderate the links on comment (this will create spam opportunity to spammers). Can we set up no-follow links on Facebook comment box?

I think that all links on facebook comments are no-followed and I think you can't change this!

In particular, Google can now index Facebook comments as well as those on some other social sites such as Disqus or Intense Debate. See [techmeme.com...] for more detail.

I use Wikipedia text in some of my image heavy pages, but I load the text through AJAX so bots can't see it. Could this now possibly affect my rankings, if G is able to load that text?.

The big danger here is that Google is now going to index some of the data we thought we had blocked it from crawling.

From the GorgBlog [googlewebmastercentral.blogspot.com]

Google crawls the URL, yummy-sundae.html.
Google begins indexing yummy-sundae.html and...decides to attempt to render the page.
During the render, yummy-sundae.html automatically sends an XMLHttpRequest for a resource, hot-fudge-info.html, using the POST method.

I use Wikipedia text in some of my image heavy pages, but I load the text through AJAX so bots can't see it. Could this now possibly affect my rankings, if G is able to load that text?.

If I understood this video from Matt Cutts, something like that would probably be considered cloaking, which could possibly get you into trouble.

[youtube.com...]

If I understood this video from Matt Cutts, something like that would probably be considered cloaking, which could possibly get you into trouble.

Well that sounds scary. The way I understood the video is that if you target Googlebot and present it with different data, then that would be considered cloaking. I'm basically showing my pages this way to all users with javascript enabled.

Edit
Just saw the last part of the video. Guess I'll be changing my pages.

Probably the safest way to go.

This is old news, Google's been indexing words that do not appear on page if you view source for over a year now. I suppose they had to make it official though.

I noticed Google indexing my Ajax content more than a year ago.

So, if anyone was thinking ajax was a way of hiding content from google then it has been a no-go for ages.

I totally hide anything I don't want indexed by Google - that is stuff like ads, dupe content etc. - and yes it is cloaking, but I simply don't care if there is an issue - I need to pay the bills for my family.

In fact, I have started doing anything that works - forget Google guidelines etc. or if it is "ethical" and you know what, it pays better than a strategy that looks at ranking well in Google.

Over the past year I have shifted to simply thinking about users and then manipulating Google as I can to get traffic to my content.

I only added the ajax stuff yesterday, that's why it's easy for me to undo. But, yeah one of my competitors has been using that since ever and is ranking well.

Well, I've decided to put the Wikipedia snipped back. I'm not loading the text through JavaScript this time.

They've been executing JS for quite some time now (3-4 years I think) to fish for hidden links outside HTML code. I understand that could be done during indexing, i.e. not in real time.

But are they now executing it during crawl or still during indexing/processing stage? I mean, those Facebook comments and the like are all time-sensitive info. It would make no sense to execute THAT JS later - so are they going to run all JS on all pages they download right that instant?

I did not think there's enough computing power in the world that would let anyone do that, not at their scale anyhow. How about JS with errors? Endless loops and otherwise bad code? Wow, this looks like a can of worm I would prefer to keep sealed ...

This is, as others have said, hardly new news.

So long as the link contains hard URL data, Google will be able to find it, and has been for some time. I've seen sites, for example, who's JS links contain just a string to add to the end of the URL, for example "parameter=filter1", and Google has tried sticking it on the end of that pages URL and spidering the result.

Rather handily, when its a navigation element, Googlebot seemed to get stuck in a loop for a bit, resulting in it spidering pages that had the query "Parameter1 + Paremeter1 + Parameter1 + Parameter1".

@dududoseo

I think that all links on facebook comments are no-followed

So you think that Googlebot doesn't follow no-follow links ;)

In practical terms... I noticed that the major search engines have started picking up javascript pages. Fine, if it's intrinsic to the page content. But how the ### do you keep them from picking up things like piwik code that they have no business snooping around in?

:: not happy about the prospect of manually changing a bunch of links to go along with relocating code snippet to roboted-out directory where it doesn't really belong ::