takahe/static/robots.txt

9 lines
153 B
Text
Raw Normal View History

User-Agent: *
# Don't allow any bot to crawl tags.
Disallow: /tags/
Disallow: /tags/*
2023-01-14 18:15:18 +00:00
# Don't allow bots to crawl through the proxy
Disallow: /proxy/*