Ajouter robots.txt #264

New issue

Closed

opened 2024-10-09 13:10:21 +02:00 by kazhnuz · 0 comments

kazhnuz commented

2024-10-09 13:10:21 +02:00

Owner

# robots.txt
# @url: https://kazhnuz.space

User-agent: *
Allow: /

User-agent: anthropic-ai
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: CCbot
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: PiplBot
Disallow: /

``` # robots.txt # @url: https://kazhnuz.space User-agent: * Allow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: CCbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: PiplBot Disallow: / ```