Forums

These errors seem to be getting more frequent and I notice the site slow down in general.

I think it's definitely worth giving something like CloudFlare a shot.

I second the slowdown experience and 503 errors.

I’m a little hesitant to go to Cloudflare because they are not necessarily one of the good guys on the internet, and make access from some places in the world very hard (recaptcha x10 per page, for example). However I don’t have a lot of tools to fight the abusive spiders that are occasionally hammering us - I’ve added rules to give them 403 errors but I don’t have easy access to check what’s going on much of the time. Right now on my phone I can’t easily see the activity or edit the bot patterns.

Well there are only really two options as I see it, move to a stronger host or use a CDN like Cloudflare.

As far as ethics I've not heard anything bad, infact they go out of their way to protect free speech etc, they should (usually) only block traffic identified as part of an attack.

https://www.wired.com/story/free-speech-issue-cloudflare/

Well, we have been using Cloudflare for a few days now and so far all is well. In particular we added rules to block the specific crawlers that were harassing us (~1.2 million hits per day compared to ~200,000 for everything else), and it's achieving a high cache rate (for images and the like). It is doing that thing that Cloudflare does where it gives you captchas if it thinks your browser or IP is weird, so please let us know if that is causing trouble.

Author	Message
djbass Joined: 25 Jul 2007 Posts: 562 Location: Melbourne, Australia	503 backend errors and slowdowns Posted: Sun Jan 12, 2020 12:24 am
	These errors seem to be getting more frequent and I notice the site slow down in general. I think it's definitely worth giving something like CloudFlare a shot.

Niloct Joined: 25 Dec 2005 Posts: 569 Location: São Paulo - Brazil	Posted: Sun Jan 12, 2020 12:58 am
	I second the slowdown experience and 503 errors.

Maxim Site Admin Joined: 19 Oct 1999 Posts: 13038 Location: London	Posted: Sun Jan 12, 2020 10:32 am
	I’m a little hesitant to go to Cloudflare because they are not necessarily one of the good guys on the internet, and make access from some places in the world very hard (recaptcha x10 per page, for example). However I don’t have a lot of tools to fight the abusive spiders that are occasionally hammering us - I’ve added rules to give them 403 errors but I don’t have easy access to check what’s going on much of the time. Right now on my phone I can’t easily see the activity or edit the bot patterns.

djbass Joined: 25 Jul 2007 Posts: 562 Location: Melbourne, Australia	Posted: Sun Jan 12, 2020 11:46 am
	Well there are only really two options as I see it, move to a stronger host or use a CDN like Cloudflare. As far as ethics I've not heard anything bad, infact they go out of their way to protect free speech etc, they should (usually) only block traffic identified as part of an attack. https://www.wired.com/story/free-speech-issue-cloudflare/

Maxim Site Admin Joined: 19 Oct 1999 Posts: 13038 Location: London	Posted: Wed Jan 22, 2020 9:48 am
	Well, we have been using Cloudflare for a few days now and so far all is well. In particular we added rules to block the specific crawlers that were harassing us (~1.2 million hits per day compared to ~200,000 for everything else), and it's achieving a high cache rate (for images and the like). It is doing that thing that Cloudflare does where it gives you captchas if it thinks your browser or IP is weird, so please let us know if that is causing trouble.

Forums

View topic - 503 backend errors and slowdowns