Forum Moderators: mack
It looks like something you will probably wish to block.
[msdn.microsoft.com...]
Although this isn't documented in MSDN, the control always set the user-agent to "Microsoft URL Control" plus a version number. It was possible to change that, but it was quite a pain, so few did.
This actually caused some problems when people upgraded to the .NET Framework because the replacement API actually required a user-agent string, and some customers coudln't figure out what to put there. (Apparently some opted for the space character. Has anyone seen that in the real world?)
Of course there are plenty of old machines (and old software) still around. It is perfectly possible that a legitimate program is somehow doing this, but most of the traffic I see on the web suggests that the typical program that does this is an e-mail-address-stealing crawler. I'd guess that someone wrote one of these once upon a time and that lots of other people have simply copied and tweaked it since then.
You cannot block this with robots.txt; the code is older than the robots.txt specification, so the programmer would actually have to write extra code to read, parse, and respect robots.txt. That seems very, very unlikely. :-)
On the other hand, if you do go to extra trouble to block the user-agent, it won't be completely trivial for them to update their programs to work around it. A real programmer can do it, of course, but a script kiddie might be stymied. Like I said, we didn't make it easy to do, and there is no support for this now.
Hope that helps. Sorry to hear that someone is doing this to you.
