Block crawlers with Caddy
A quick tip on how to block crawlers from accessing your website by rejecting requests containing their user agent tokens in Caddy.
A quick tip on how to block crawlers from accessing your website by rejecting requests containing their user agent tokens in Caddy.
A quick tip on how to use UFW to block all IP adresses used by Google and by AI-related crawlers.