mirrors/wallabag
1
0
Fork 0
mirror of https://github.com/wallabag/wallabag.git synced 2024-11-15 21:41:06 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #3510 from wallabag/update-robotstxt

Browse source 
Updated robots.txt to prevent crawling
This commit is contained in:
Nicolas Lœuillet 2017-12-16 09:02:40 +01:00 committed by GitHub
commit 9b606725fe
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
web/robots.txt
View file

@ -1,4 +1,2 @@
# www.robotstxt.org/
# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
User-agent: * User-agent: *
Disallow: /