View Single Post
Old 09-05-2007, 05:23 AM   #1
dplax
Jack Burton
 

Join Date: July 19, 2003
Location: an expat living in France
Age: 38
Posts: 5,577
Default Yahoo webcrawling bot

Currently the active users on the forum are:

Quote:
Currently Active Users: 91 (7 members & 1 guests & 83 bots)
View Who's Online
Most users ever online was 536, 08-29-2007 at 12:56 AM.
dplax, Armen, Dundee Slaytern, Hivetyrant, wellard, Wolvenrick, Yahoo! Slurp (82), Google (1)
More than 80% of all is made up by Yahoo. Now I know that Yahoo recently started using several different IP addresses and thus the difference between Yahoo and Google web crawling isn't as big as it seems, but still...

A forum where I'm admin has the following hits from bots, according to number of requests made in September so far:

Yahoo! Slurp: 509
MSN: 109
Google: 51
Ask Jeeves: 3
Archive.org: 1

Yahoo easily outnumbers the rest combined.

Interesting discussion about it at http://www.ipsbeyond.com/forums/inde...howtopic=24698 (also contains link to a thread at Yahoo about it)

In case that first link requires logging in here's a link to the Yahoo page about it: http://suggestions.yahoo.com/detail/...orer&fid=31431 and also a link to an article on digg http://www.digg.com/software/Yahoo_S...guest_tracking

So, is Yahoo just indexing webpages or are they adding unnecessary bandwidth to servers?
__________________

dplax is offline   Reply With Quote