ClaudeBot is hammering my server with almost a million requests in one day
Just checked my crawler logs for the last 24 hours and ClaudeBot (Anthropic) hit my site **\~881,000 times**. That’s basically my entire traffic for the day.
I don’t mind legit crawlers like Googlebot/Bingbot since they at least help with indexing, but this thing is just **sucking bandwidth for free training** and giving nothing back.
Couple of questions for others here:
* Are you seeing the same ridiculous traffic from ClaudeBot?
* Does it respect `robots.txt`, or do I need to block it at the firewall?
* Any downsides to just outright banning it (and other AI crawlers)?
Feels like we’re all getting turned into free API fodder without consent.