mirror of
https://github.com/searxng/searxng.git
synced 2024-12-01 15:11:03 +00:00
[mod] limiter: block requests from PetalBot
Block requests from PetalBlock. Normally robots.txt is enough to stop PetalBlock from making requests [1]. However, if SearXNG is offered below a path (example.org/search), then the robots.txt is not available in the root paths of the domain / subdomain. [1] https://webmaster.petalsearch.com/site/petalbot Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
parent
9967fbb7d1
commit
823c490c84
1 changed files with 1 additions and 0 deletions
|
@ -37,6 +37,7 @@ block_user_agent = re.compile(
|
||||||
# unmaintained Farside instances
|
# unmaintained Farside instances
|
||||||
+ r'|'
|
+ r'|'
|
||||||
+ re.escape(r'Mozilla/5.0 (compatible; Farside/0.1.0; +https://farside.link)')
|
+ re.escape(r'Mozilla/5.0 (compatible; Farside/0.1.0; +https://farside.link)')
|
||||||
|
+ '|.*PetalBot.*'
|
||||||
+ r')'
|
+ r')'
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue