Where


When

Who

< QOTD | Main | Tracking Back >

March 18, 2003

Email Crawlers are Evil

Phil Ringnalda writes about Dave upgrading the weblogs.com machine and finding someone crawling it.

I'm trying really hard not to think about how Dave was seeing a heavy load during the changeover because someone was crawling all over the Radio discussion group. Whether he meant radio.userland.com/discuss/ or radiocomments.userland.com/discuss/, in either case if I were crawling it, it would be because of all those juicy email addresses, sitting out at the end of /profiles/$ URLs all over the place. I've always thought those were fat enough targets to be well worth writing a special purpose crawler. (Note for the irony impaired: I'm not actually a spammer, or a writer of email harvesters.)

I hate to break it to you, but they've been crawled a bunch (and I know I've griped about it a few times here). They grabbed one of my addresses and I've gotten span from it. I personally think Userland needs to start encoding email addresses anywhere it prints them out on a web page (or find a way not to print them out at all, I personally don't trust the endcoding thing).

Posted by snooze at March 18, 2003 5:48 AM