|
|
|
Frequently spidering bots
The webpages which we use to put submitted websites on to ensure rapide inclusion in search engines are regularly visited by the following bots:
Yahoo! Slurp - www.yahoo.com
Gigabot/2.0 - www.gigablast.com
Becomebot/3.0 - www.become.com
Snapbot/1.0
Googlebot/2.1 - www.google.com
psbot/0.1 - www.picsearch.com
NaverBot-1.0 - www.naver.com
Msnbot/1.0 - search.msn.com
Msnbot/0.9 - search.msn.com
Exabot/3.0 - www.exabot.com
MJ12bot/v1.0.8 - www.majestic12.co.uk
lanshanbot/1.0
appie 1.1
nicebot
NutchCVS/0.7.2 - lucene.apache.org
Jyxobot/1 - jyxo.cz
myahintbot/1.0
robot@sportspy.de
feedback@surveybot.com
NutchCVS/0.7.1 - lucene.apache.org
Ingrid/3.0 - www.ilse.nl
Vagabondo/3.0 - www.wise-guys.nl
ZyBorg/1.0 - www.wisenut.com www.looksmart.com
Ask Jeeves/Teoma
HappyFunBot/1.1 - www.happyfunsearch.com
Girafabot - girafa.com
Szukacz/1.5 - www.szukacz.pl
Lobot/0.03
Seekbot/1.0 - www.seekbot.net
BilgiBetaBot/0.8-dev/0.8-dev - bilgi.com
HenryTheMiragoRobot - www.miragorobot.com
ScollSpider - www.webwobot.com
LinksManager.com_bot - linksmanager.com
Gaisbot/3.0+ - gais.cs.ccu.edu.tw
Healthbot/Health_and_Longevity_Project_ - HealthHaven.com
i1searchbot/2.0 - www.i1search.com
archive.org_bot/heritrix-1.9.0-200608171144 - pandora.nla.gov.au
BilgiBot/1.0 - bilgi.com
OutfoxBot/0.5
yacybot
sproose/0.1 - www.sproose.com
factbot - www.factbites.com
sdcresearchlabs-testbot/0.8-dev - www.shopping.com
schibstedsokbot - www.schibstedsok.no
Googlebot-Mobile/2.1 - www.google.com
TurnitinBot/2.1 - www.turnitin.com
PlantyNet_WebRobot_V1.9 - www.plantynet.com
QweeryBot/3.01 - www.qweery.nl
Ixquick
ia_archiver-web.archive.org
AgentName/0.1
wwwster/1.4 - cis.uni-muenchen.de
lwp-trivial/1.41
larbin_2.6.3
Baiduspider - www.baidu.com
SBIder/0.8-dev - www.sitesell.com
findlinks/1.1.3-beta8 - wortschatz.uni-leipzig.de/findlinks
ichiro/2.0 - www.goo.ne.jp
InsurancoBot - www.insuranco.de
Mister Pix II 2.15
Snoopy v1.01
Konqueror/3.1-rc1
DART TCP Control/2.0.23
Monstro/1.0
http://www.almaden.ibm.com/cs/crawler
Xenu Link Sleuth 1.2g
Poirot
McBot/5.001
WebTrends/3.0
Misterbot
MyEngines-Bot - www.bodysytlenet.de
EnfinBot V.4
Shim-Crawler
WebSauger 1.20b
Sleipnir
Wells Search II
Random
EasyDL/3.04 - www.keywen.com
RufusBot
KummHttp/1.1
Sensis Web Crawler - www.sensis.com.au
Pompos/1.3 - dir.com
ichiro/2.0 - www.nttr.co.jp)
DataSpearSpiderBot/0.2 - dssb.dataspear.com
OCP HRS 1.0
lwp-trivial/1.38
LWP::Simple/5.803
CFNetwork/10.4.3
ccubee/3.2
Jakarta Commons-HttpClient/3.0-rc1
Zeus 19132 Webster Pro V2.9 Win32
CheckLinkbacks.exe/0.99 - www.waterionizer.org
SportSpy-URLChecker/0.1
BenditBot1.1 - www.suche-bendit.de
Infoseek SideWinder/2.0B
NG/2.0
Norbert the Spider - www.burf.com
IlTrovatore-Setaccio/1.2 - www.iltrovatore.it
PrivacyFinder/1.1
gonzo1
sohu agent
Oracle Ultra Search
boitho.com-dc/0.83 - www.boitho.com
lilreno
curl/7.10.7
JoeDog/1.00
ELinks/0.11.CVS
Download Ninja 2.0
OutfoxBot/0.3
WebCopier v4.3
RMA/1.0
SlimBrowser
Poodle predictor 1.0
NavissoBot
NG-Search/0.86 - www.ng-search.com
SearchIt-Bot/2.3.1 - www.searchit.com
LexiteriaCorpus/1.2
MySuMa Crawler/Bot
sohu-search
DiamondBot
Istwo-Bot - www.istwo.de
BLA
Nokia6021/2.0
http://www.Link-Directory-Submitter.com will automate link popularity increase for your site.
The biggest b2b search engine in the word! - http://365sources.com
MyEngines-Bot - www.bodysytlenet.de
NuSearch Spider
MnoGoSearch/3.2.34
EARTHCOM.info/1.98 - www.earthcom.info
AnswerBus - www.answerbus.com
SCrawlTest/CR1 - sproose.com
SonyEricssonT630/R601
Extreme Picture Finder
cfetch/1.0
| |
