{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreidmyhpbukodw3zh3bji4ufoskyd6if45qttxe5mnt23oynjrgsz7e",
    "uri": "at://did:plc:djqqegmazb2f32esguwok4xg/app.bsky.feed.post/3miz4eexk2ju2"
  },
  "description": "Sean Conner at The Boston Diaries wrote about trying to block annoying and/or malicious bots from crawling his website. It is a good read.\n\nBeginning in late 2024, I started noticing The New Leaf Journal going down periodically. The server logs suggested that it was being overrun by bots and crawlers. I tried various methods to ensure that NLJ would be up all the time, including fiddling with Apache configs and my .htaccess file. I stumbled upon the \"solution\" (for now, at least) in late […]",
  "path": "/naferrell/blocking-bad-bots-with-abuseipd-blacklist-04-08-26/",
  "publishedAt": "2026-04-08T20:16:03.000Z",
  "site": "https://social.emucafe.org",
  "tags": [
    "bots",
    "cloudron",
    "firewalls",
    "troubleshooting",
    "wrote about trying to block annoying and/or malicious bots from crawling his website",
    "Using Cloudron’s IP Firewall",
    "readily available",
    "LittleJake’s GitHub repository",
    "Webmention"
  ],
  "textContent": "Sean Conner at The Boston Diaries wrote about trying to block annoying and/or malicious bots from crawling his website. It is a good read.\n\nBeginning in late 2024, I started noticing The New Leaf Journal going down periodically. The server logs suggested that it was being overrun by bots and crawlers. I tried various methods to ensure that NLJ would be up all the time, including fiddling with Apache configs and my .htaccess file. I stumbled upon the “solution” (for now, at least) in late January. I use Cloudron to manage this server (which is home to both this site and NLJ). Cloudron has a built-in firewall. Back on January 30, I published Using Cloudron’s IP Firewall. I populated its firewall with IPs from the AbuseIPD blacklist, which is readily available in LittleJake’s GitHub repository. NLJ’s resource usage went  _way down_(see the chart in my linked post) and my crashing issues have been gone for more than two months. The AbuseIPD blacklist is updated approximately every six hours. I have been adding the IPs manually but I have been meaning to look into a way to automate the process.\n\nYou can reply to this article from your own site by sending a Webmention.",
  "title": "Blocking Bad Bots With AbuseIPD Blacklist",
  "updatedAt": "2026-04-10T16:08:32.000Z"
}