"promises to behave better" are an acceptable way of settling the issue - provided the promises are kept.

I didn't realize that Webmaster World has so many anti-trust law experts, and that they are better judges of what constitutes a violation than the investigators at the FTC

Apparently we are, to judge by the Commissioners' decision.

toidi wrote:
And they still call it an algorithm even though it is operated manually?

That's a great point. Google secretly made manual adjustments to their search results to benefit its own services. This is how the WSJ article reported it:

On the most important issue, that of Google’s prized search engine, the FTC report said Google altered it to benefit its own services at the expense of rivals. The report said Google “adopted a strategy of demoting, or refusing to display, links to certain vertical websites in highly commercial categories.” --- [wsj.com...]

How could anyone, except for the most naive, continue to trust Google after these revelations?

except for the most naive

Sadly, most searchers fall into this category...

> It isn't "operated manually." Focus groups, user testing, etc.
> provide insights for use in creating and refining the algorithm.

There is no public evidence to support that claim.

There is no public evidence to support that claim

While it was not meant to be public, one particular excerpt from the FTC report published by the Wall Street Journal (page 132, footnote 154) does seem to support it, describing the process in detail.

[graphics.wsj.com...]

...

That actually proves my point that they use Raters to rate search results. aka: it *is* operated manually in many (how high?) cases. There is a growing body of consensus that a major portion of Googles current "algo" consists of thousands of raters that score results for ranking purposes. The "algorithm" by machine, on the majority of results seen by a high percentage of people, is almost non-existent.

Remember a few years ago when Google try to peddle, "Google news is machine generated". That was a complete fabrication. Humans wrote the code, told the code what sites to visit, told the code how to score results, and everything else there is to know about "Google news". eg: it's humans stupid - it is simply code carrying out criteria set by Google.

it's humans stupid - it is simply code carrying out criteria set by Google

I would be stupid if I thought it was some kind of magic.

The cited document is public evidence to support the stated claim.

...

There is a growing body of consensus that a major portion of Googles current "algo" consists of thousands of raters that score results for ranking purposes

.... and the criteria is, has, and was, strongly favouring those that spend the money in areas of monetized search. Not rocket science really, but at least everyone's getting to a point of understanding. If you pay, you stay [ or more likely to ].

In highly competitive verticals the algorithm is simply a more efficient content detection tool sitting over favored sites. So in the light of this why wouldn't Google be driven to promote it's own content / assets ahead of "fill" provided by rivals. I don't believe their practice will stop either - they just need to manage it well.

The average search punter has no clue what the difference is now between paid and non paid listings on Google assets.

The whole thing is a legal / rights / commercial minefield, far more complex than the Microsoft anti trust actions. Unless you define "search" as public property, and then who's going to run it and on what basis. Ask Baidu and Yandex what favors are involved in ranking with them, and who the ultimate beneficiaries are. It's a fun / reality World. By the time the regulators / business / public work it out, search will be long gone, and technically defunct

Doesn't look good for Google and its fans. In fact it pretty well supports a lot of the anti-Google commentary of recent years.

Regards...jmcc

As far as I can see the main complaint about search results seems to be that Google demoted comparison shopping sites in the main SERPS. Surely everyone knew this already, so I see no new support for anti-Google commentary.

The key new information seems to be:
1) Google's algorithm rating process was twisted to favour algos that demoted comparison shopping sites. Google were determined to demote them in the organic SEPRS.
2) Google used exclusivity agreements toe exclude competition. This is the most serious allegation, and I cannot find more detail on it.

In my previous post i mistakenly grouped two seperate thoughts together and they are being looked at as one.

the part about the manually run algo come from the quotes below and not from the part about focus groups.

"adopted a strategy of demoting or refusing to display, links to certain vertical websites in highly commercial categories."

Although Google originally sought to demote all comparison shopping websites,

“This document appears to show that the FTC had direct evidence from Google of intentional search bias,” 

There is a growing body of consensus that a major portion of Googles current "algo" consists of thousands of raters that score results for ranking purposes. The "algorithm" by machine, on the majority of results seen by a high percentage of people, is almost non-existent.

Only thousands of raters to score the results manually? They must be really hustling to keep up with the 3.5 billion searches that Google processes every day.

oh the spin... It's not such a daunting task when you realize that 3.4 billion of those searches are Kardashian related.

Google originally sought to demote all comparison shopping websites

You say that like it was a bad thing.

Unless the search term contained a variation on the word "compare" the dominance of such sites was intensely annoying to many searchers. The results were not what they were looking for.

The FTC document that the quote comes from goes on to describe how various adjustments to the algorithm were made, tested and revised. Nowhere does it suggest that any individual site was marked down "manually".

The fact is that searchers do not want the SERPs overrun with content farms and other useless results, and people here on WebmasterWorld are the first to complain when it happens.

Adjustments to the algorithm are a normal and perfectly reasonable response.

There are many reasons to criticise Google, but this is not one of them.

...

 it is simply code carrying out criteria set by Google. 

Exactly!

The FTC document that the quote comes from goes on to describe how various adjustments to the algorithm were made, tested and revised. Nowhere does it suggest that any individual site was marked down "manually".

I recall a similar situation in which tranches of loans were resubmitted to regulators until the desired result was achieved.

If we turn our attention to Europe Google has possibly a bigger antitrust case likely to move forward where Google dominates the search with a 90% market share by some measures. There is so much about Google that I like, so don't get me wrong, but I think at some point Google is going to be faced with being a victim of their own success, and needing to realize that competition in the market is important. When a company comes to overly dominate, governments play a role in balancing the playing field.

it is simply code carrying out criteria set by Google.

Exactly!

That will doubtless come as a shock to anyone who thought that Google's algorithm was magically burned into stone tablets in the Sinai peninsula by an unseen deity several millennia ago, but to everyone else it is merely a statement of the bleedin' obvious.

Now imagine the SERPs if Google set no criteria.

Enjoy your feast of spam.

...

The question is whether the criteria were deliberately designed in a way that is an abuse of monopoly. That really depends on whether you regard Google's vertical search offerings as content that should compete with other content in the SERPS, or as simply a way of searching for vendors so excluding it is as reasonable as Google's long standing policy of not showing results from other search engines in the SERPS.

I agreed Samizdata it is pretty easy to pick up a torch and pitchfork at this point, but what is being implied by the FTC is that Googles criteria was:

GoogleBot +10 all Yelp content (strip mine all Yelp reviews to build their database).

GoogleSerps -10 all yelp content (downgrade them in the rankings and claim they aren't showing serps in serps).

That is anticompetitive criteria that was manually set.

But its not Yelp's content. Yelp (UK) T & C say "you own Your Content". Of course Yelp claims it "owns" the compiled content (how valid is that legally in each country they operate in, I wonder), but if Google crawled user content and compiled it separately.... Anyway Yelp is not a content creator and the argument gets complicated.

What would be much less arguable would be a smoking gun that showed Google intimidated people to play along with this by threatening to drop them from the SERPS altogether. This is briefly mentioned in the story, but the relevant bits of the documents are not available. Exlusivity arrangements would be even worse (also briefly mentioned without details).

They intentionally (and secretly) rigged their results to put their own comparison sites at the top. In order to do so, they sometimes had to take special measures (manual interventions) to push other comparison sites down to get them out of the way. They avoided being sued by promising to stop their dishonest manipulations.

And what if Google used Yelp as an initial data source to understand the review process, and understand which sites and methods would be most useful once they demoted Yelp?

I believe a similar practice took place early in Google's history with Yahoo, Dmoz and many other directories. In a way that is smart, so kudos to Google. But then we get to this place where Google is dominating 90% of the searches having outsmarted, out financed and out maneuvered the competition. Monopoly has been achieved.

what is being implied by the FTC

The report and recommendation was the work of lower-echelon staff at the FTC, which was countered by another department and overruled by those in the organisation who actually make the decisions.

I can find no mention of Yelp in the published excerpts.

Key staff of the Federal Trade Commission concluded

The term "key staff" customarily applies to people with decision making powers, so why it was used by the WSJ in this instance is open to question.

I wouldn't trust Google as far as I could throw a self-driving car.

And I wouldn't trust a NewsCorp title to publish anything that did not serve the interests of its proprietor.

Rupert Murdoch is 85.

...

lower-echelon staff at the FTC are the ones that make the recommendations because it's their job and they are highly qualified to make such recommendations. The five "friends of the current administration", or commissioners, that make the final decision don't need to have any specific requirements, they only need to get appointed. This is why sitting commissioners generally act on decisions as staff recommends.

Yelp led anti-google consortium:
[focusontheuser.eu...]

The company was also accused of other misbehavior, including scraping material, such as product ratings, from rivals, including Yelp, TripAdvisor, and Amazon, and using it to improve its own search results. Then, when the companies complained, Google reportedly threatened to take them out of its search results entirely.

[newyorker.com...]

On this site we are computer geeks (webmaster, et al)... but there is that other side out there. Here's how the political news geeks are reporting this:

In what is sure to lead to a customer scandal and heighten a U.S. Antitrust Probe, Federal Trade Commission (FTC) staffers determined in an undisclosed report that Google, Inc. allegedly used an algorithm to manipulate search results to favor their own less relevant search over competitors. The alleged Google fraudulent practice only became public when FTC staffers inadvertently shared the document with the Wall Street Journal.

In a stunning revelation, FTC staff found that to improve Google’s shopping results, the company allegedly “scraped ratings and user reviews from Amazon.com site.” Google also allegedly used Amazon’s product rankings to determine the order to rank products in Google Product Search. Although Google provided search services on Amazon’s website, which generated almost $170 million in revenue, Amazon shifted some search traffic to less-profitable Microsoft’s Bing in an effort “to try to foster a more competitive marketplace.”

[breitbart.com...]

We've know this for years... we see the results daily and over time know the trend of such manipulation... as well as the other penalties, usually announced as preventing gaming the system, but each change has had one effect: to solidify Google's Products over the last few years.

There is no doubt that G has the tech and ability to truly search the web, but when humans with corporate dollars get involved, the search becomes tainted... and I suppose that can't be helped... after all, we are all greedy little fellows! :)

REALLY! are we going to have a *gasp* moment when we have known for years this has been going on? I know you all want to rank 1 in goog but do you really turn a blind eye?


don't do evil * giggle*

staff at the FTC are the ones that make the recommendations because it's their job

Commissioners at the FTC are the ones who make the decisions because it's their job. They are the "key staff".

They found that Google used some anti-competitive practices, and demanded changes - mounting a prosecution would have meant spending several years (and several million tax dollars) on a court case with an unpredictable result.

Eric Schmidt in an orange jumpsuit was never a realistic outcome.

Yelp led anti-google consortium

My point was that there is no mention of Yelp in the published excerpts from the report, and no suggestion that any individual site was "manually" demoted.

You will know better than me, but my recollection is that on WebmasterWorld a "manual" penalty has always referred to action against a specific site rather than an algorithm change.

Terminology is important, on which point:

search engine illegally took information from rival websites

That apparently refers to Googlebot crawling the sites, which is not illegal even if robots.txt is ignored.

"Never let the facts get in the way of a good story."

...

The commissioners are appointed figure heads and are never referred to as staff, key or otherwise.

The commissioner "demanded" no changes, google offered to allow websites to opt out of having their content scraped AND not be listed in the search results (package deal) instead of threatening to remove websites from the search results if they complained about their content being scraped. Seems like a pretty good deal for only $24+ million...