This is the kind of traffic you don't want to opt out of, but instead being immune for it. Smart firewall rules which detect and block port scanners should do the trick. Any IP trying to access port 3306 or similar ports on my servers will have a hard time to get any valuable information later on.

like a robot.txt for exclusion

A robots.txt Disallow only works if the robot #1 requests and #2 honors robots.txt. (In that order. Asking for robots.txt after you have already been 403d doesn't count.) Do they?

@lammert I don't agree. They don't have a right to scan and index me and put ANY information in their 'service.' I do agree that it is *MY* job to prevent that stuff. I'm just saying they should have some standard of behavior and option to opt out that doesn't involve me blocking large ranges of IPs.

blocking large ranges of IPs

If they crawl from a wide range of IPs (I think of these as distributed robots, though I'm told this is not technically correct), you may be better off blocking the User-Agent by name. That's assuming it is included in the UA string; it isn't entirely clear from the initial post if “Leakix” is part of the UA string, or just the owner of the IP. If possible, you do want to avoid methods that requires the server to do extra work--such as a lookup--on every request.

LeakIX does port scans over the whole range of vulnerable ports, 22, 80, 443, 3306, you name it. Using robots.txt or blocking by User-Agent makes no sense. They are just black-hat hackers with a with a white-hat 'security research sauce'.

@lammert Agreed. When I read the info on the LeakIX site it looks more like a nation state actor looking for a community to do the work of finding vulnerable targets FOR them. I see them of 0 use to the Internet and they waste my resources and cost me $$$ by connecting to my infrastructure. They seem as useless as the turds at ltx71.

They ought to have a code of ethics and an option to opt out or be considered an illegitimate 'project.' Perhaps I ask too much from a bunch of 'researchers.'

They seem as useless as the turds at ltx71.

Well, now, that's an interesting digression, since I know ltx71 only as “robots I block because they don't honor robots.txt and I have no idea what they’re ostensibly for”.

@lucy24 Seems like Ltx71 is most likely a front for something else.

This BS description made me laugh:
'We continuously scan the internet for security research purposes. Our crawling is not malicious and only notes summary information for a page.'

They don't obey robots.txt
They don't offer a way to opt out. All of them need to do this or we need a new standard.
They don't even know how to setup TLS.

How am I supposed to take them seriously?