How to Build Bot Traps in PHP

by Scott Allen

“Code Red! Unidentified Robots in Sector 17…”
The other day at work, we were going over the server site stats for one of our clients’ web site. The stats showed that there were Robots hitting the site that were ignoring our Robots.txt file. I pointed this out to my colleague and said “We’ve got unidentified robots…we need to build a bot trap.” One of the other web designers, who wasn’t part of the conversation, had just taken off her headphones and overheard just that part of the conversation, and started cracking up. She said, “if the print designers had randomly overheard that statement they’d think you guys were living in a Sci-Fi movie.” We had a good laugh. True, we’re geeks.

There may not be any alien robots out there that we have to fight, but rogue web-bots can be much more than a minor irritation. Bots can be web worms, gathering information or hacking your site; they can be email harvesters or web site downloaders, sucking files or email addresses off your servers in bulk; they can send spam through your mailserver; or they can be part of a denial-of-service attack, etc, etc. (For more info see my later post.) You don’t have to worry about bots from major search engines as they will almost always respect your robots.txt file. However, when bots don’t obey, it’s a sign they are up to no good. You can fix this quite easily with a good Bot Trap. I’ll go into more detail in a later post, but for now check out the following to get you started:

Good Articles on Spider Traps / Bot Traps:
http://www.kloth.net/internet/bottrap.php
http://www.fleiner.com/bots/
http://www.neilgunton.com/spambot_trap/
http://manly.delconet.com/klahn/privacy/spidertrap.html

Technorati Tags:
| | |

Bookmark or Share with Friends: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • StumbleUpon
  • del.icio.us
  • Sphinn
  • Digg
  • Reddit


If you enjoyed this post, make sure you subscribe to the RSS feed!


Email This to a Friend Email This to a Friend

Print This Post Print This Post


Related Posts:

  • Look Up IP Address Info
  • Search Engine Optimization (SEO) Tools #2 - Robots.txt Generator
  • Web Site Security - Bot Traps
  • Cyber-Surveillance and Internet Data-Mining
  • Link Building Resources

    • About This Entry

    You’re currently reading “How to Build Bot Traps in PHP,” an entry on WebGeek

    Published:
    01.15.06 / 9pm
    Category:
    Bad Bots, PHP, Website Security
    Related Posts:
  • Look Up IP Address Info
  • Search Engine Optimization (SEO) Tools #2 - Robots.txt Generator
  • Web Site Security - Bot Traps
  • Cyber-Surveillance and Internet Data-Mining
  • Link Building Resources
    • RSS Feeds:
  • Subscribe to Blog
  • Subscribe to Comments
    • Share:
  • Email This to a Friend
  • WebGeek Badges
    • WordPress Plugins:
  • WP-SpamFree: Blog Anti-Spam
    • About Us:
  • Hybrid6 Studios is a
    web design and SEO firm
    based in Los Angeles, CA.