.htaccess ban list

[KVN]

Siguie

Click on the "fight spam here" link on the page
It brings you to a list of known spammer emails - with the links on this page generating another list of known spammer emails.. etc. etc.

areidmtm

There's no automatic way to get rid of spambots as they will always mimic valid users - fact.
You can manually block them as you have done (excellent list by the way - nice one mate) or you can try to deter them (i.e. spampoison).

Either way peeps

No need to dis the suggestion - absorb, accept or discard as you please.
It's all good either way.

Happy spam hunting
KVN

[dvessel]

Uh, with all those rules couldn't it stress the server? And all those spam bots out there.. Lotta work going through your logs and adding & adding.

[areidmtm]

Uh, with all those rules couldn't it stress the server? And all those spam bots out there.. Lotta work going through your logs and adding & adding.

It actually doesn't stress the server at all, with a properly writen file. I've seen some HUGE .htaccess files and they've worked fine. However the bigger the file, the longer it will take your site to load...but the difference is less than .01 of a second. It is faster if you put the rules all one line rather then each rule on its own line. But again, .01 of a second.

My list blocks most of the ones that go to my site. Rarely do I see a new one and if I do, I add it, no big deal. And that's why if anyone else see anything diffrent, to add here. If there is any other better way of blocking all the bots, I’d like to hear it. I’ve tried other methods and have done a little bit of research, and this seam to be the best for right now.

[dvessel]

Ah, that's good to know. I've read of other rewrite rules that can do more harm than good.

I tried using bad behavior and it block more visitors than spam bots.
http://www.homelandstupidity.us/software/bad-behavior/

Now I'm using a captcha. Any poster will only be asked once for the image code then it gets stored in a cookie which I think is reasonable.

btw, the 2 solutions are modules for drupal as I wouldn't know how to implement them myself.

[areidmtm]

Yeah, .htaccess can be tricky sometimes. If you're not careful, you can run into some problems. As far as I can tell with my list so far, it isn't creating any problems.

[T_S_Kimball]

dvessel - I've not had any big issue with BB at this point (though I'm using it with WordPress). There's a second module someone else wrote so I can see the BB logs from my admin pages (handy at work, only GET/POST allowed by proxy apparently); So far nothing serious. Out of curiosity, how did you determine that users were being prevented (so I can keep an eye out)?

areidmtm - Quite an extensive list. I'll look into it if I have time this weekend. Unfortunately, the only issue I can see getting around this is bots and spiders starting to just spoof the IE and FireFox/Moz IDs.

--TSK

[siguie]

OH that's a neat site! Though I was just looking to see how spampoison was supposed to work and I couldn't find the email generating code.

[areidmtm]

Unfortunately, the only issue I can see getting around this is bots and spiders starting to just spoof the IE and FireFox/Moz IDs.

Yeah this is the draw back, and unfortunately there is not much you can do to combat these types of attacks.

[dvessel]

dvessel - I've not had any big issue with BB at this point (though I'm using it with WordPress). There's a second module someone else wrote so I can see the BB logs from my admin pages (handy at work, only GET/POST allowed by proxy apparently); So far nothing serious. Out of curiosity, how did you determine that users were being prevented (so I can keep an eye out)?

The way it was implemented as a drupal module I could check the logs on why they were rejected + a handy link to whois with the ip of the visitor. Whois has a spam blacklist listed and always they were clear.

Could be related to proxies I guess.. not sure but it was always because the visitor couldn't accept headers. It just didn't add up so I removed it.

[chadderuski]

I take it that this method is used for bots that do NOT respect the robots.txt file? Is this really that common a problem?

I moved to BH from an SBC server because they were having lots of problems getting blacklisted by our pals in england and china and such. Interestingly enough, our website gained a point in rank within a week of the move.

It would be great if BH could take some time to prepare more technical documents along these lines. It is fine to dig around in the forums looking for answers, but would be far better if these could be ported over the the "support" site. Also, a link to this forum would also be nice... I actually found this forum through google!

I have talked to several techs at BH since our move and have been very satisfied with their level of knowledge. True hard core geeks ... the way it should be! Keep up the good work.