mirror of
https://github.com/zedeus/nitter.git
synced 2024-06-02 11:20:41 +00:00
Block search engines via robots.txt
Prevents instances from being rate limited due to being senselessly crawled by search engines. Since there is no reason to index Nitter instances, simply block all robots. Notably, this does *not* affect link previews (e.g. in various chat software).
This commit is contained in:
parent
778c6c64cb
commit
afbb700d2b
2
public/robots.txt
Normal file
2
public/robots.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
User-agent: *
|
||||
Disallow: /
|
Loading…
Reference in a new issue