searxng/docs/dev/makefile.rst

Makefile

build environment

Before looking deeper at the targets, first read about make install.

To install system requirements follow buildhosts.

All relevant build tasks are implemented in manage.sh and for CI or IDE integration a small Makefile wrapper is available. If you are not familiar with Makefiles, we recommend to read gnu-make introduction.

The usage is simple, just type make {target-name} to build a target. Calling the help target gives a first overview (make help):

bash -c "cd ..; make --no-print-directory help"

Contents

Python Environment (make install)

activate environment

source ./local/py3/bin/activate

We do no longer need to build up the virtualenv manually. Jump into your git working tree and release a make install to get a virtualenv with a developer install of searx (setup.py). :

$ cd ~/searx-clone
$ make install
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

If you release make install multiple times the installation will only rebuild if the sha256 sum of the requirement files fails. With other words: the check fails if you edit the requirements listed in requirements-dev.txt and requirements.txt). :

$ make install
PYENV     OK
PYENV     [virtualenv] requirements.sha256 failed
          [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e...  ./requirements-dev.txt
          [virtualenv] - 471efef6c73558e391c3adb35f4...  ./requirements.txt
...
PYENV     [virtualenv] installing ./requirements*.txt into local/py3
...
PYENV     OK
PYENV     [install] pip install -e 'searx[test]'
...
Successfully installed argparse-1.4.0 searx
BUILDENV  INFO:searx:load the default settings from ./searx/settings.yml
BUILDENV  INFO:searx:Initialisation done
BUILDENV  build utils/brand.env

drop environment

To get rid of the existing environment before re-build use clean target <make clean> first.

If you think, something goes wrong with your ./local environment or you change the setup.py file, you have to call make clean.

make buildenv

Rebuild instance's environment with the modified settings from the settings global brand and settings global server section of your settings.yml <settings location>.

We have all SearXNG setups are centralized in the settings.yml file. This setup is available as long we are in a installed instance. E.g. the installed instance on the server or the installed developer instance at ./local (the later one is created by a make install <make install> or make run <make run>).

Tasks running outside of an installed instance, especially those tasks and scripts running at (pre-) installation time do not have access to the SearXNG setup (from a installed instance). Those tasks need a build environment.

The make buildenv target will update the build environment in:

• utils/brand.env

Tasks running outside of an installed instance, need the following settings from the YAML configuration:

• GIT_URL from git configuration
• GIT_BRANCH from git configuration
• SEARX_URL from server.base_url <settings global server> (aka PUBLIC_URL)
• SEARX_BIND_ADDRESS from server.bind_address <settings global server>
• SEARX_PORT from server.port <settings global server>

make run

To get up a running a developer instance simply call make run. This enables debug option in searx/settings.yml, starts a ./searx/webapp.py instance, disables debug option again and opens the URL in your favorite WEB browser (xdg-open):

$ make run
PYENV     OK
SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
...
INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)

make clean

Drop all intermediate files, all builds, but keep sources untouched. Before calling make clean stop all processes using make install. :

$ make clean
CLEAN     pyenv
PYENV     [virtualenv] drop ./local/py3
CLEAN     docs -- ./build/docs ./dist/docs
CLEAN     locally installed npm dependencies
CLEAN     test stuff
CLEAN     common files

make docs docs.autobuild docs.clean

We describe the usage of the doc.* targets in the How to contribute / Documentation <contrib docs> section. If you want to edit the documentation read our make docs.live section. If you are working in your own brand, adjust your settings global.

make docs.gh-pages

To deploy on github.io first adjust your settings global. For any further read deploy on github.io.

make test

Runs a series of tests: make test.pylint, test.pep8, test.unit and test.robot. You can run tests selective, e.g.:

$ make test.pep8 test.unit test.sh
TEST      test.pep8 OK
...
TEST      test.unit OK
...
TEST      test.sh OK

make test.sh

sh lint / if you have changed some bash scripting run this test before commit.

make test.pylint

Pylint is known as one of the best source-code, bug and quality checker for the Python programming language. The pylint profile we use at searx project is found in project's root folder .pylintrc.

search.checker.{engine name}

To check all engines:

make search.checker

To check a engine with whitespace in the name like google news replace space by underline:

make search.checker.google_news

To see HTTP requests and more use SEARX_DEBUG:

make SEARX_DEBUG=1 search.checker.google_news

To filter out HTTP redirects (3xx):

make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
...
Engine google news                   Checking
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
--

make pybuild

Build Python packages in ./dist/py:

$ make pybuild
...
BUILD     pybuild
running sdist
running egg_info
...
running bdist_wheel

$ ls  ./dist
searx-0.18.0-py3-none-any.whl  searx-0.18.0.tar.gz

To upload packages to PyPi, there is also a pypi.upload target (to test use pypi.upload.test). Since you are not the owner of searx you will never need to upload.