Snap reaction: Please please please... I don't know what other people use the search information for; I use it to make the site better.*

There are two different tiers of withholding information. One is when the referer is a bare "https google" (paraphrasing to sidestep auto-linking). The other is the form that says "cd=some-number-here" so you know only that you came up on such-and-such page for, uh, something.


* Well, sometimes. If the search engine thinks that volume three of the Paston letters will answer a user's question about whether her husband is gay-- long, complicated query typed in on, so help me, a cell phone-- there's not much I can do about it.

Well it's good that they're considering it. Of the options mentioned in the article...

>> Google makes all organic click data available through Google Webmaster Tools

Seems the most likely to me.

>> Clicks may pass only to sites that run secure servers

Would likely see a large rise in sites converting to HTTPS only. More latency and cycles but more private for the end-user, which is apparently Google's reasoning for hiding the referral data in the first place.

They should allow the webmaster to see them via webmaster tools. Create a hash lookup that can be anonymously cross correlated after the fact. This would keep privacy, and allow webmasters to gain valuable insight for improvement. Has the additional benefit of encouraging webmasters going to google in a verified way.

Interesting; I expected them to roll out some sort of paid service. I wonder what prompted this? Worries of another class action lawsuit from a consortium of large businesses, similar to the antitrust investigations in the EC? You could argue that, if this is about privacy, then Google should not gather the data either, they should hold it only in user session and then destroy it, and be required to show that this is what they do.

The fact that THEY know exactly what was searched for in connection with a referral, but the site owner isn't allowed to see, does to my mind weaken any 'it's for privacy, honest' argument - especially given their penchant for ads that follow you around from Facebook to Google and back again, or match what you type into GMail.

The fact that THEY know exactly what was searched for in connection with a referral, but the site owner isn't allowed to see, does to my mind weaken any 'it's for privacy, honest' argument

According to Internet Protocol, the information should not be sent to a non-secure site.

Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.

https://tools.ietf.org/html/rfc2616#section-15.1.3

So, anyone with a non-secure site really has nothing to complain about at all.

Worth clarifying that goes beyond the RFC though. You *should* get referrer data when it's a HTTPS to HTTPS page flow, but Google goes out its way to redirect in such a way that you don't get the referrer data you're after (keywords). At the moment it doesn't matter if a site is HTTPS or not, you won't get the keywords in the referral string.

True, and good point, but even if they made the data available, most people here still wouldn't get it and really have no room to complain about not getting it, because a compliant browser will not send it to a non-secure site and most sites people here operate are non-secure.

My main point was: If someone isn't running an https site, then getting upset at Google is pointless, because unless a site is https it doesn't matter if Google makes the info available or not, they're not getting it either way.

ADDED: Google *could* retain the info internally and make it available via WMT or Analytics internally without breaking any type of protocol, since they are both secure connections, but that would be "cool" since they're definitely not required to in any way.

Also, an HTML5 compliant browser will not send referrer data if the link clicked contains rel="noreferrer", so even without a redirect or anything else, anyone can stop referrer data from being passed via a link, meaning it's still likely *not* "lawsuit material" or anything like that for Google to withhold the data -- I don't like it very much [that's polite for "I think it sucks #$*@"] but it's their option.

Requires that the user agent not send an HTTP Referer (sic) header if the user follows the hyperlink.

http://www.w3.org/html/wg/drafts/html/master/links.html#linkTypes

Most people wouldn't shell out for SSL? Really? The costs aren't prohibitive, and they'd come down further if Google announced that they send info to secure sites - but this has been tested and they don't.

The IETF link is interesting but I'm sure you don't believe that Google took this course of action because they strongly believe in the Internet Engineering Task Force guidelines and realised that they hadn't followed them for the first 14 years of their existence.

Remember that the https has been slowly introduced as default - first for logged in users, then for all users. Google don't give anyone the option of browsing non-https (although I think there are workarounds) so it's not really a conscious choice made by the searcher.

I'm not an expert on the IETF (just had a quick look on their site now) but what I saw indicated to me that Google continue to break a lot more of their guidelines than they adhere to.

I believe I may have seen a tweet from someone at SMX to the effect that Matt Cutts' interpretation of what Amit said was not to expect keyword data anytime soon.

(talk about second hand third hand fourth hand information)

Requires that the user agent not send an HTTP Referer (sic) header if the user follows the hyperlink.

You mean not at all? That's ridiculous. It's like saying "I'm from the CIA but you'll have to take my unsupported word for it because I'm not allowed to tell you anything more."

I'll throw a "dumb question" out there: why can't a browser not send the IP address, but send the referral URL? Then the referral URL is 100% anonymous, yet still useful for the site receiving such visitor data? It seems the privacy issues concerns IP address + referral data = privacy issue. Remove IP address = no privacy issue.

Uh... what IP address? The one making the request? Would any website in its right mind admit anonymous visitors? Besides, the IP isn't sent by the browser. That's why referers and UAs can be easily faked but IPs can't.

I really hope I have completely misunderstood the question.

I believe I may have seen a tweet from someone at SMX to the effect that Matt Cutts' interpretation of what Amit said was not to expect keyword data anytime soon.

I saw the same thing and think you're correct.

You mean not at all?

Yup, that's what I mean. Whether we like it or not, the doc I linked is very explicit in what it says -- rel="noreferrer" means "send no referrer header."

Whew. I read too fast and thought the rule applied to all links on https pages. "rel = 'noreferrer'" at least is an individual choice-- and sites can individually choose what to do about it. (Is it really spelled with two r's in the middle? Four in all? How do they expect us to remember when to misspell it if they themselves can't be consistent?)

Is it really spelled with two r's in the middle? Four in all? How do they expect us to remember when to misspell it if they themselves can't be consistent?

Yes, it's spelled correctly with two r's in the middle when used in a rel="noreferrer", but don't go asking more questions that confuse people with facts -- It's just not right! lol

Whew. I read too fast and thought the rule applied to all links on https pages.

It applies to https pages when the link clicked is to an http [no s] page -- In the case of someone clicking a link on a secure page/site to a non-secure page/site, referrer [whatever anyone's choice of spelling referrer is] should *not* be sent by a compliant browser.

I don't see where it says all links on all https pages. I only see the "rel='noreferrrrrrer'" part :( Even searched the whole page for any occurrences of "https" or "secure". Is that part on a different page somewhere? I don't like reading html5 docs. They make me tired. And it's unnerving to hit an article with today's date when today is only three hours old. Sunday, no less.

It doesn't there [HTML5 docs]. It does here [RFC 2616]:

Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.

[tools.ietf.org...]

If Google decides to show "not provided" data and I have a website, http://www.example.tld, and someone arrives at the site by doing a keyword search on Google and then clicking on the site in the SERP, would I be able to see in my server logs and in Google Analytics that someone arrived to the site by searching for that keyword phrase in Google or would my site have to be https://www.example.tld for me to see this?

It does here

But now wait. That's dated 1999. Is anyone at this late date going to say "Oh, sorry, I've been doing it wrong for the last 15 years"? It seems more like a rule whose window of opportunity has passed, especially when you consider how much rarer https was in 1999.

As long as HTTP 1.1 is with us and not edited, it's the "rule" for secure to non-secure connections -- I also don't know of any browser that "does it wrong".

The recommendation isn't for sites to send or not send anything, it's for browsers and says to not send the referrer header from https to an http connection, which is why I was saying earlier, "Unless someone has a secure site, they really have nothing to complain about," because until HTML5 and the introduction of noreferrer, browsers at their sole discretion made the decisions on whether to send a referrer header or not, and they get it right according to the RFC AFAIK.

Now a site can tell a browser to not send a referred header, but it wasn't that way until HTML5 was released. Previously, it was totally up to the browser's coders and, again, I don't know of any browser "getting it wrong" when a visitor clicks a link from an https page to an http page -- According to the RFC, the referrer header should *not* be sent by the browser.

I'm missing something here. Logs are filled with entries that give [google...] as referer. According to the rules, they're not supposed to do this? It doesn't say "leave off the query string". It says no referer at all.

The SERPs are served on an HTTPS page.

The redirect on Google is HTTP, so you get a referrer but without the keywords in it.

All academic really. Google chooses not to provide keyword data... at this moment in time.

I don't remember exactly where it is Lucy24, but there's something about sending the site but not the exact page or query string or something like that somewhere -- Basically, the theory is https protects someone's privacy, so the site knows you were there and what page you visited but broadcasting exactly where you were over a non-secure connection so anyone can find out shouldn't be done.

I really don't disagree with it either, because if I'm at PayPal or a bank's website or where ever else a person thinks is private, no one needs to [or should imo] know what URL was being visited and it certainly doesn't need to get broadcast in a non-secure [http] manner.