Bot or Crawler??

Having trouble with an aspect of the forum? Have a suggestion for an improvement?
Post Reply
DJ
Posts: 271
Joined: Wednesday May 24, 2006 3:03 pm
Location: Mt Annan, NSW

Bot or Crawler??

Post by DJ »

At the bottom of the front screen in the forum it shows who is online.

For example

DJ, drsmurto, Google [Bot], gregb, Heritrix [Crawler], rwh, scooter75, Yahoo [Bot]


So who or what is a BOT or CRAWLER?
DJ


"No Excuses" - Kostya Tszyu
Kevnlis
Posts: 3380
Joined: Tuesday Jul 10, 2007 5:15 pm
Location: B-Rat
Contact:

Re: Bot or Crawler??

Post by Kevnlis »

Crawlers and Bots are much in the same except that those bots are sent out to collect info for search engines like Google, Yahoo, etc. The crawler is probably trying to harvest e-mail addresses and other personal data to sell to spam companies.

http://crawler.archive.org/
Prost and happy brewing!

Image
O'Brien Gluten Free Beer
Chris
Posts: 3716
Joined: Tuesday Oct 04, 2005 1:35 pm
Location: Northern Canberra

Re: Bot or Crawler??

Post by Chris »

Is there any way to kill that google bot? It worries me that it is there for some reason. Is this a justified concern?
A beer in the hand is worth two in George Bush...

"They say beer will make me dumb. It are go good with pizza"
Psychostick
Kevnlis
Posts: 3380
Joined: Tuesday Jul 10, 2007 5:15 pm
Location: B-Rat
Contact:

Re: Bot or Crawler??

Post by Kevnlis »

That google bot is what makes people find us. But for it, we would not have many members at all (me included)!

Everything you find when you type a search into a google search box was indexed by a google bot.
Prost and happy brewing!

Image
O'Brien Gluten Free Beer
User avatar
KEG
Posts: 1682
Joined: Thursday Dec 21, 2006 9:02 am

Re: Bot or Crawler??

Post by KEG »

they are harmless.
Image
Chris
Posts: 3716
Joined: Tuesday Oct 04, 2005 1:35 pm
Location: Northern Canberra

Re: Bot or Crawler??

Post by Chris »

Fair enough. As long as they are safe...
A beer in the hand is worth two in George Bush...

"They say beer will make me dumb. It are go good with pizza"
Psychostick
Oliver
Administrator
Posts: 3424
Joined: Thursday Jul 22, 2004 1:22 am
Location: West Melbourne, Victoria, Australia

Re: Bot or Crawler??

Post by Oliver »

Indeed, bots are harmless.

But I've now worked out how to hide them from the user list.

They have to be done one by one, so I've only hidden the 20 or so that have ever visited homebrewandbeer.com. Another 30 have never visited so I didn't worry hiding them.

If anyone notices any appearing (they'll probably be for new search engines) let me know and I'll hide them too.

Cheers,

Oliver
Post Reply