I MAY have a clue about this...

I was using ChatGPT for help with some research into a local history subject. It quoted one of my websites as a source. I stumbled across a very similar entry in my logs that I believe was due to my interaction. It could just be a coincidence, but the timing makes me think it was the page being requested during my "chat" with ChatGPT.

Side note, it felt very strange asking ChatGPT for information and being presented with information I wrote!

Mack.

In my case I have a lot of PDF files, I don't know how ChatGPT knows it's looking for a specific file, it doesn't seem to keep them in any local data base because its constantly downloading the same files.

But then again googlebot has been doing this for years. I don't know why it constantly has to request my entire PDF inventory on an almost daily basis. Some requests are just HEAD's, but mostly it wants the entire file. Bing is like that also. Do they think that content is changing in a PDF file?

I guess I'm wondering if during a user's interaction with ChatGPT, is the bot grabbing these files and showing them to the user. I really don't use any chatbots so I don't know what their user-interface looks like. In other words, can people use the bot interface to directly view web content (ie pdf files in this case) and so I see this as a hit from the bot IP and not the user's IP.

Heh. You don’t make it easy, do you. A PDF file is a free-standing entity, so requests could be anything. The question is, what do they request when it’s an ordinary web page?

:: quick run to logs ::

Total requests within this calendar year: around 60,000 (from a variety of IPs, so that’s assuming all are legit--probably yes, since there are no header deficits), of which about 4000 blocked on IP grounds *
Requests that are not for pages, pdf or robots.txt: 23 (twenty-three, i.e. less than one in a thousand), all of which are images)
Requests for css: 0 (zero)

That doesn’t really look like human viewing to me.

* This suggests that if I wanted to be kind, I would change a few “Require ip” to bad_range, for which this UA gets a hole poked. But I don’t know that I’ll bother.