New Russian Spider

Forum Moderators: open

Message Too Old, No Replies

New Russian Spider

webalta.net

GaryK

3:20 pm on Aug 13, 2006 (gmt 0)

This was sent to me by someone using my contact form:

New Russian Spider
[webalta.net...]

They didn't send a user agent but a quick search turned up this user agent:

WebAlta Crawler/1.3.18 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1;

Apparently there is no closing parenthesis after Windows NT 5.1;

Has anyone seen this? If so can you tell me if it's well-behaved. Thanks.

incrediBILL

12:33 am on Aug 14, 2006 (gmt 0)

Sorry, it's been around a few months ;)

81.222.146.132 "WebAlta Crawler/1.3.11 (http://www.webalta.ru/bot.html) (Windows; U; Windows NT 5.1; ru-RU)"

81.222.146.134 "WebAlta Crawler/1.3.11 (http://www.webalta.ru/bot.html) (Windows; U; Windows NT 5.1; ru-RU)"

Always seems to come from 81.222.146.*

GaryK

2:10 am on Aug 14, 2006 (gmt 0)

Thank you Bill.

EDIT: I forgot to ask. Does it read and respect robots.txt? Is there anything else we should know about it?

[edited by: GaryK at 2:13 am (utc) on Aug. 14, 2006]

incrediBILL

4:19 am on Aug 14, 2006 (gmt 0)

I don't know for sure, my old archives don't track all the specifics, but it never tried crawling more than my home page but that could be because it got slapped with an error page ;)

GaryK

3:26 pm on Aug 14, 2006 (gmt 0)

I'll see what I can do to get this bot to crawl at least one of my sites. Hopefully it'll be well behaved. Right now Yandex is the only Russian SE that can legitimately crawl my sites.

Lord Majestic

6:38 pm on Aug 15, 2006 (gmt 0)

Allegedly WebAlta is run by the same people who founded UmaxForum and UmaxSearch. This is meant to be Russian centered search engine.

GaryK

7:59 pm on Aug 20, 2006 (gmt 0)

They were in my logs for last week. It did not read robots.txt. It came from 87.224.173.*

What I noticed was each time a Firefox ua took a page and got away with it the bot would immediately take the same page. Is it trying to sniff its way through my site without reading robots.txt? If so it's not working because not reading robots.txt will get my attention every time.

This was the Fx ua:
Mozilla/5.0 (Windows; U; Windows NT 5.0; ru-RU; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4

[edited by: GaryK at 8:00 pm (utc) on Aug. 20, 2006]

incrediBILL

8:22 pm on Aug 20, 2006 (gmt 0)

UmaxForum and UmaxSearch

Any relation to the notorious Umax scrapers and malware injection sites?

Those all seem to be Ukrainian/Russian owned.

GaryK

10:10 pm on Aug 20, 2006 (gmt 0)

I don't know.

RU-TELESET-20050614
Teleset-Servis Ltd.
Ekaterinburg, Russia