Page 1 of 1
Bot or Crawler??
Posted:
Wednesday Feb 20, 2008 3:22 pmby DJ
At the bottom of the front screen in the forum it shows who is online.
For example
DJ, drsmurto, Google [Bot], gregb, Heritrix [Crawler], rwh, scooter75, Yahoo [Bot]
So who or what is a BOT or CRAWLER?
Re: Bot or Crawler??
Posted:
Wednesday Feb 20, 2008 3:46 pmby Kevnlis
Crawlers and Bots are much in the same except that those bots are sent out to collect info for search engines like Google, Yahoo, etc. The crawler is probably trying to harvest e-mail addresses and other personal data to sell to spam companies.
http://crawler.archive.org/
Re: Bot or Crawler??
Posted:
Wednesday Apr 16, 2008 8:31 pmby Chris
Is there any way to kill that google bot? It worries me that it is there for some reason. Is this a justified concern?
Re: Bot or Crawler??
Posted:
Wednesday Apr 16, 2008 9:27 pmby Kevnlis
That google bot is what makes people find us. But for it, we would not have many members at all (me included)!
Everything you find when you type a search into a google search box was indexed by a google bot.
Re: Bot or Crawler??
Posted:
Thursday Apr 17, 2008 1:03 amby KEG
they are harmless.
Re: Bot or Crawler??
Posted:
Thursday Apr 17, 2008 1:32 pmby Chris
Fair enough. As long as they are safe...
Re: Bot or Crawler??
Posted:
Thursday Dec 02, 2010 3:27 pmby Oliver
Indeed, bots are harmless.
But I've now worked out how to hide them from the user list.
They have to be done one by one, so I've only hidden the 20 or so that have ever visited homebrewandbeer.com. Another 30 have never visited so I didn't worry hiding them.
If anyone notices any appearing (they'll probably be for new search engines) let me know and I'll hide them too.
Cheers,
Oliver