[PLUG] Do no robots texts work?

["kaze" <kaze@voicenet.com>]

I know I'm wasting time thinking about how to out think the spammers, but:
say I want to post an email link on a web page. An email address harvester
would probably ignore a robots.txt, but what about a harvester which used a
search engine like google, would this then thwart it?

Inotherwords would it be correct to say that a malevolent harvester spider
would ignore the robots.txt, but an engine built on or seeded by Google
would 'honor' the robots text. (More convoluted still, might a robots.txt
expose you more as some would search just for them figuring there is
something hidden?)

- Zake

___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug