searxng/searx/plugins/limiter.py
Markus Heiser 1ec325adcc [mod] limiter -> botdetection: modularization and documentation
In order to be able to meet the outstanding requirements, the implementation is
modularized and supplemented with documentation.

This patch does not contain functional change, except it fixes issue #2455

----

Aktivate limiter in the settings.yml and simulate a bot request by::

    curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \
         -H 'Accept: text/html'
         -H 'User-Agent: xyz' \
         -H 'Accept-Encoding: gzip' \
         'http://127.0.0.1:8888/search?q=foo'

In the LOG:

    DEBUG   searx.botdetection.link_token : missing ping for this request: .....

Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time
before you get a "Too Many Requests" response.

Closes: https://github.com/searxng/searxng/issues/2455
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-29 14:54:56 +02:00

43 lines
1.1 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-or-later
# lint: pylint
# pyright: basic
"""see :ref:`limiter src`"""
import flask
from searx import redisdb
from searx.plugins import logger
from searx.botdetection import limiter
from searx.botdetection import dump_request
name = "Request limiter"
description = "Limit the number of request"
default_on = False
preference_section = 'service'
logger = logger.getChild('limiter')
def pre_request():
"""See :ref:`flask.Flask.before_request`"""
val = limiter.filter_request(flask.request)
if val is not None:
http_status, msg = val
client_ip = flask.request.headers.get('X-Forwarded-For', '<unknown>')
logger.error("BLOCK (IP %s): %s" % (client_ip, msg))
return 'Too Many Requests', http_status
logger.debug("OK: %s" % dump_request(flask.request))
return None
def init(app: flask.Flask, settings) -> bool:
if not settings['server']['limiter']:
return False
if not redisdb.client():
logger.error("The limiter requires Redis")
return False
app.before_request(pre_request)
return True