takahe/static/robots.txt

5 lines
87 B
Text

User-Agent: *
# Don't allow any bot to crawl tags.
Disallow: /tags/
Disallow: /tags/*