Commit graph

945 commits

Author SHA1 Message Date
Noémi Ványi
f0ca1c3483
[enh] Add command line engines: git grep, find, etc. (#2128)
A new "base" engine called command is introduced. It is the foundation for all command line engines for now.
You can use this engine to create your own command line engine.

Add some engines (commented out to make sure no one enables anything accidentally):
* git grep: This engine lets you grep in the searx repo.
* locate: If locate is installed and initialized, you can search on the FS.
* find: You can find files with a specific name from where you started searx.
* pattern search in files: This engine utilizes the command fgrep.
* regex search in files: This engine runs `grep` to find a file based on its contents.
2020-09-08 09:51:53 +02:00
Alexandre Flament
3397382754
[enh] stop searx when an engine raise an SyntaxError exception (#2177)
and some other exceptions:
* KeyboardInterrupt
* SystemExit
* RuntimeError
* SystemError
* ImportError: an engine with an unmet dependency will stop everything.
2020-09-07 15:39:26 +02:00
Alexandre Flament
b329058c1a Revert "[enh] test: load each engine to check for syntax errors"
This reverts commit 4fb3ed2c63.
2020-08-31 19:00:06 +02:00
Adam Tauber
6f9aa0e258
Merge pull request #2160 from dalf/test_load_engine
[enh] test: load each engine to check for syntax errors
2020-08-31 14:29:52 +02:00
Adam Tauber
6ded6e7a9a [fix] skip uncomplete image results - closes #1496 2020-08-31 14:07:45 +02:00
Dalf
4fb3ed2c63 [enh] test: load each engine to check for syntax errors 2020-08-28 12:12:32 +02:00
Marc Abonce Seguin
0d8970c8f2
only return one url per "type" in Wikidata (#2151)
i.e. only one official website, one Twitter, etc.
2020-08-27 21:44:48 +02:00
Émilien Devos
27d74826f1
[enh] add yggtorrent engine (#2135) 2020-08-18 18:02:41 +02:00
Emilien Devos
c15a91a534 [fix] piratebay engine date and pep8 indentation 2020-08-10 23:44:53 +02:00
Emilien Devos
52d78d8418 [fix] piratebay engine 2020-08-10 20:26:59 +02:00
Adam Tauber
77103c7874
Merge pull request #2116 from mikeri/invidiousres
Include author and video length in Invidious results
2020-08-10 12:49:17 +02:00
Vlad
f678388dbc
Fix google images 'get image' button bug from issue #2103 (#2115)
Closes #2103
2020-08-08 19:35:22 +02:00
Michael Ilsaas
a1ce141c99
add peertube engine (#2109) 2020-08-08 19:22:53 +02:00
Michael Ilsaas
2ed8ad7691 include length in invidious results 2020-08-02 13:31:04 +02:00
Michael Ilsaas
0305fe0dd5 include author in invidious results 2020-08-02 13:30:38 +02:00
Marc Abonce Seguin
77b9faa8df fix Wikipedia's paragraph extraction 2020-07-26 23:53:40 -07:00
Michael Ilsaas
98cb6b6701 Update torrentz2 URL from .eu to .is 2020-07-26 15:56:54 +02:00
xywei
1d4657b714
Fix relative urls that do not start with '/' 2020-07-23 11:12:19 -05:00
Gaspard d'Hautefeuille
4e346e741a
fix python 3 support 2020-07-12 23:43:24 +01:00
Adam Tauber
52eba0c721 [fix] pep8 2020-07-08 00:46:03 +02:00
Markus Heiser
16f8ec894a [fix] revise google images engine
this commit is picked from #1985
2020-07-07 21:59:15 +02:00
Markus Heiser
410c2f903d [fix] revise google engine
this commit is picked from #1985
2020-07-07 21:50:59 +02:00
Markus Heiser
8d318ee142
Merge branch 'master' into gigablast 2020-06-29 16:09:59 +00:00
Sophie Tauchert
71db7b1238
Fix YaCy text results returned as images 2020-06-29 14:48:56 +02:00
Noémi Ványi
93cbd85b8a
Merge branch 'master' into duckduckgo_correction 2020-06-28 20:28:12 +02:00
Markus Heiser
5fac6cffa2
Merge branch 'master' into gigablast 2020-06-26 08:09:33 +00:00
Markus Heiser
5293e58032 [fix] yahoo engine - changed content_xpath
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-25 21:45:42 +02:00
Markus Heiser
223430ff30
Merge branch 'master' into gigablast 2020-06-16 07:36:44 +00:00
Adam Tauber
32f7877235 [fix] resolve flickr_noapi encoding issues 2020-06-15 19:15:24 +02:00
Gordon Quad
385e9b5c9e add correction support for duckduckgo 2020-06-13 22:43:10 +01:00
Markus Heiser
ee0da61cbb
Merge branch 'master' into gigablast 2020-06-10 04:54:57 +00:00
Adam Tauber
aa7c043ff4 [fix] resolve pep8 errors 2020-06-10 00:34:57 +02:00
Adam Tauber
29960aa1d9 [enh] add official site link to the top of the infobox - closes #1644 2020-06-09 23:49:13 +02:00
Adam Tauber
6c06286251 [enh] add length and author details to youtube videos
closes #775
2020-06-09 20:31:51 +02:00
Adam Tauber
2c6531b233 [enh] add routing directions to osm search - closes #254 2020-06-09 17:18:44 +02:00
Markus Heiser
74135007eb
Merge branch 'master' into gigablast 2020-06-01 06:23:45 +00:00
Noémi Ványi
e3282748d0 add display_error_messages option to engine settings
A new option is added to engines to hide error messages from users. It
is called `display_error_messages` and by default it is set to `True`.
If it is set to `False` error messages do not show up on the UI.

Keep in mind that engines are still suspended if needed regardless of
this setting.

Closes #1828
2020-05-31 19:17:48 +02:00
Markus Heiser
ee5d2b319b [fix] gigablast requires a random extra parameter
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-29 12:48:58 +02:00
Markus Heiser
a18760b322 [fix] revise of the gigablast engine (WIP)
The gigablast API has changed and seems to have some quirks, this is the first
revise.  More work (hacks) are needed.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-28 19:24:03 +02:00
Markus Heiser
57c7b90edd [fix] gigablast does no longer support *supported_languages_url*
Since there are zero results, we can remove it:

    $ make engines.languages
    fetch languages ..
    ...
    fetched 0 languages from engine gigablast

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-28 15:21:01 +02:00
Markus Heiser
de179ecc5b [fix] remove debug print from commit e5305f8
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-28 15:17:59 +02:00
Markus Heiser
9302d1fc17
Merge branch 'master' into master 2020-04-20 12:44:50 +00:00
Noémi Ványi
fcb44c6542
Merge branch 'master' into fix_startpage_ValueError_on_spanish_datetime 2020-04-15 23:33:16 +02:00
HLFH
3a26093c46
Remove discontinued faroo engine 2020-04-15 16:46:15 +01:00
Spühler Stefan
4f90fb6a92 [Fix] Startpage ValueError on Spanish date format
datetime.parser.parse() does not know the Spanish date format which
leads to a ValueError. Fixes #1870

Traceback (most recent call last):
  File "/usr/local/searx/searx/search.py", line 160, in search_one_http_request_safe
    search_results = search_one_http_request(engine, query, request_params)
  File "/usr/local/searx/searx/search.py", line 97, in search_one_http_request
    return engine.response(response)
  File "/usr/local/searx/searx/engines/startpage.py", line 102, in response
    published_date = parser.parse(date_string, dayfirst=True)
  File "/usr/local/searx/searx-ve/lib/python3.6/site-packages/dateutil/parser/_parser.py", line 1358, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)
  File "/usr/local/searx/searx-ve/lib/python3.6/site-packages/dateutil/parser/_parser.py", line 649, in parse
    raise ValueError("Unknown string format:", timestr)
ValueError: ('Unknown string format:', '24 Ene 2013')
2020-03-09 09:31:20 +01:00
Markus Heiser
ad7a6e6e10 bugfix(!biv) : bing-video do not like "older" User-Agents
When selecting other languages than 'en', bing-video did not handle the language
correct and gave very bad results.  Since User-Agent is normaly rotated in
searx, the behavior of a !biv search was unpredictable and paging was broken.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 17:28:59 +01:00
Markus Heiser
1c853f9573 bing_news: parital rollback of c89c05bc
The bing_news bug (discussed in #1838) was caused by wrong language tags, which
was fixed e0c99d9d / no need to change the bing_news search string.

closes: https://github.com/asciimoo/searx/issues/1838

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 11:07:59 +01:00
Markus Heiser
e0c99d9dcb bugfix: fetch_supported_languages bing, -news, -videos, -images
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 08:01:36 +01:00
Markus Heiser
c89c05bceb bugfix: google-news and bing-news has changed the language parameter
closes: https://github.com/asciimoo/searx/issues/1838

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-25 18:44:28 +01:00
piplongrun
f0684a5bb5
Add eTools engine 2020-02-12 23:58:50 +01:00