Commit graph

60 commits

Author SHA1 Message Date
Markus Heiser
40956e8e6b [data.all] add searx_extra/update/update_osm_keys_tags.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Markus Heiser
393c06f537 [enh] add test.yamllint - lint yaml files
Usage::

    make test.yamllint

    ./manage test.yamllint

test.yamllint is also added to the `test` makefile target.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Alexandre Flament
20580bcbd4 [docker] multiarch support: linux/amd64,linux/arm64,linux/arm/v7
make docker.buildx : build and push multiarch build.
(it can't be only build)

use buildx with the --cache-from and --cache-to options to cache the layers
(only the last built is cached)
2021-04-28 14:40:00 +02:00
Markus Heiser
3bd2f3dc81 [fix] manage - fix miss usage of 'set -e'
The philosophy of set -e is typically that it only exits upon uncaught
errors. Here, the presence of || outside the subshell seems to tell the shell
that the error inside the subshell is 'caught' and therefore set -e does not
cause an exit after false [1].

The shell does not exit if the command that fails is ... part of any command
executed in a && or || list except the command following the final && or ||, any
command in a pipeline but the last, or if the command’s return status is being
inverted with ! [2]

[1] https://unix.stackexchange.com/questions/296526/set-e-in-a-subshell
[2] https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html#The-Set-Builtin

BTW: fix error reported by 'make test.shell'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-27 13:02:55 +02:00
Markus Heiser
a92a4cb443 [pylint] get PYLINT_FILES from tag '# lint: pylint'
These py files are linted by test.pylint(), all other files are linted by
test.pep8()

close: https://github.com/searxng/searxng/issues/21
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-26 20:52:52 +02:00
Markus Heiser
8efabd3ab7 [mod] core.ac.uk engine
- add to list of pylint scripts
- add debug log messages
- move API key int `settings.yml`
- improved readability
- add some metadata to results

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-24 09:00:53 +02:00
Alexandre Flament
4863c1933e Rerevert c54bf42 2021-04-24 07:59:16 +02:00
Markus Heiser
c69fc20be2 [fix] ./manage pyenv.install - die when pyenv build fails
ERROR: When pyenv build fails, the pyenv command runs into a recursion [1]:

    ....
    BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  ERROR: Cannot install idna==3.1 and requests[socks]==2.25.1 because these package versions have conflicting dependencies.
    BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV
    BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  BUILDENV  The conflict is caused by:

[1] https://github.com/return42/searx-next/pull/4#issuecomment-821999497

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-23 11:26:22 +02:00
Markus Heiser
d0e371f474 [mod] replace makefile boilerplate by 'manage' script
Replaces the make targets with the bash scripts

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:26:22 +02:00
Markus Heiser
720d0d48ad [enh] implement all build task in a bash script (manage)
note: in further patches script 'manage' will replace 'manage.sh'

pyenv.* :
  assert    : build virtualenv if not exists
  intsall   : developer install of searx into virtualenv
  unintsall : uninstall developer installation
  cmd ...   : run command ... in virtualenv
  OK        : test if virtualenv is OK
pypi.upload:
  Upload python packages to PyPi (to test use pypi.upload.test)
pybuild :
  Build python packages at ./${PYDIST}
pyclean :
  delete virtualenv and intermediate py files
test.* :
  pylint    : lint PYLINT_FILES, searx/engines, searx & tests
  pep8      : pycodestyle (pep8) for all files except PYLINT_FILES
  unit      : run unit tests
  coverage  : run unit tests with coverage
  robot     : run robot test
  clean     : clean intermediate test stuff
node.* :
  env       : download & install npm dependencies locally
  clean     : drop npm installations
buildenv :
  rebuild ./utils/brand.env
data.* :
  all       : update searx/languages.py and ./data/*
  languages : update searx/data/engines_languages.json & searx/languages.py
  useragents: update searx/data/useragents.json with the most recent versions of Firefox.
themes.* :
  all       : build all themes
  oscar     : build oscar theme
  simple    : build simple theme
  bootstrap : less compile bootstrap.min.css CSS
babel.compile :
  pybabel compile ./searx/translations
docs.* :
  html      : build HTML documentation
  gh-pages  : deploy on gh-pages branch
  autobuild : autobuild HTML documentation while editing
  prebuild  : build reST include files (./${DOCS_BUILD}/includes)
  clean     : clean documentation build
docker.build [push] :
  build (and push) docker image
gecko.driver :
  download & install geckodriver if not already installed (required for
  robot_tests)

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:14:29 +02:00