Commit graph

1531 commits

Author SHA1 Message Date
jazzzooo 74600c028d [fix] engine - Crossref
Crossref was broken on result types journal-issue and component .. The old code
had lots of assumptions, and broke during parsing.  Now the assumptions are more
explicit and checked them with the API.
2023-09-14 17:39:23 +02:00
Bnyro 3568a3cafb [feat] odysee: implement fetch_traits for language support 2023-09-13 21:41:33 +02:00
Bnyro 09c61dabc9 [mod] odysee: time range support 2023-09-13 21:41:33 +02:00
jazzzooo b98907e91f [fix] engine - piped.music incorrect timestamps 2023-09-13 21:39:37 +02:00
jazzzooo 6039dbf211 [fix] engine - invidious thumbnails 2023-09-13 11:37:42 +02:00
jazzzooo b2fd6304bf [fix] engine - openstreetmap currency rendering 2023-09-13 10:56:52 +02:00
jazzzooo 54a3e03b45 [fix] engine - openstreetmap currency matching 2023-09-12 20:57:05 +02:00
Bnyro 64d9587ac8 [feat] new engine: svgrepo 2023-09-12 20:38:36 +02:00
jazzzooo b189578b6b [fix] engine - brave 2023-09-12 11:31:43 +02:00
Bnyro f182abd6f8 [mod] library of congress: fix engine 2023-09-11 19:42:31 +02:00
Bnyro e73a6f5d14 [fix] engine deviantart: review of the result-scrapper
The deviantart site changed and hence deviantart is currently unusable.
2023-09-11 13:22:36 +02:00
Alexandre Flament d07c006aed Replace chompjs with pure Python code
The new implementation is good enough for the current usage (brave)
2023-09-09 13:02:36 +02:00
Bnyro 9e83c0dedc [feat] engine: implementation of Yummly
Co-authored-by: Markus Heiser <markus.heiser@damarit.de>
2023-09-08 11:47:13 +02:00
Bnyro a3d7e9c285 [mod] utils.py: add markdown_to_text helper function 2023-09-08 11:47:13 +02:00
Hackurei 1f21ac7d62 [feat] engine: implementation of bilibili https://www.bilibili.com 2023-09-05 22:53:03 +02:00
Markus Heiser 696c35d2c3 [fix] engine - duckduckgo_images / determination of vqd value incorrect
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-05 22:24:51 +02:00
bonswouar 4fb26cd96d [fix] engine duckduckgo weather api changes 2023-09-05 16:55:00 +02:00
Markus Heiser 01be9e0e20 [fix] engine: wikicommons - don't quoute ':|' in URL parameters
From [1]: It seems to be because of [2] For some reason it gets url encoded
twice, resulting in

- ``filetype%253Abitmap%257Cdrawing+birds`` instead of
- ``filetype:bitmap%7Cdrawing+birds``

[1] https://github.com/searxng/searxng/issues/2707
[2] https://github.com/searxng/searxng/blob/master/searx/engines/wikicommons.py#L43

Closes: #2707
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-03 18:45:39 +02:00
Markus Heiser 4f8895c6de [fix] follow-up of 4da7003ae / add missing review from @Bnyro
[1] https://github.com/searxng/searxng/pull/2656#pullrequestreview-1607956209

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-02 09:26:45 +02:00
Hackurei 4da7003ae0 [feat] engine: implementation of odysee 2023-09-02 09:14:12 +02:00
Bnyro 9c4e9d3814 [feat] implementation of Wikimedia commons for images 2023-09-01 18:39:24 +02:00
Alexandre Flament faa4280e1a [mod] bing: resolve redirect without additional requests
Remove the usage of searx.network.multi_requests
The results from Bing contains the target URL encoded in base64
See the u parameter, remove the first two character "a1", and done.

Also add a comment the check of the result_len / pageno
( from https://github.com/searx/searx/pull/1387 )
2023-08-29 07:39:06 +02:00
Markus Heiser b0d2cd5ca9 [doc] add documentation of Mwmbl engine & autocompleter
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-27 17:25:26 +02:00
Bnyro 19300a5659 [mod] engine mwmbl: add link to official api docs 2023-08-27 17:25:26 +02:00
Alexandre Flament e16c007c22 [fix] openstreetmap engine
It seems there is an API change:
extratags can be either a dictionnary or None.

This commit avoid crash when extratags is None

Test query "!osm gare du nord"
2023-08-27 11:49:16 +02:00
Markus Heiser 0647f83a3e [fix] google engine: don't overspecify the search query to Google
The method EngineTraits.get_region(..) returns engine's region string
that **best fits** to SearXNG's locale.  This means it returns a
region (country) if only a language is set in the locale.  By example the method
returns for a locale tag `es` a region `ES`.

Google's search parameter `cr` restricts search results to documents originating
in a particular country / in case of a locale tag (language) as described above,
this argument should be unset in the query send to Google.

Closes: https://github.com/searxng/searxng/issues/2672
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-26 07:47:07 +02:00
Markus Heiser 4b42644579 [fix] engine google_video: google has changed the layout of the rsponse
Closes: https://github.com/searxng/searxng/issues/2664
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-22 08:34:04 +02:00
Bnyro c59ae91b76 [feat] engine: implementation of mwmbl 2023-08-19 18:23:42 +02:00
Markus Heiser c741fc6f00 [mod] currency_convert: support for showing the answer source url
Show URL of the ddg-search page, not the URL of a (generic) Javascript.  The
latter one is not usefull for the user.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-18 19:07:14 +02:00
Markus Heiser e2744520f8 [mod] google: support for showing the answer source url
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-18 19:07:14 +02:00
Bnyro 5ec7df3480 [mod] engine duckduckgo definitions: support for answer source 2023-08-18 19:07:14 +02:00
Bnyro 64bc98b5fb [mod] brave: support for showing the answer source url 2023-08-18 19:07:14 +02:00
Markus Heiser 9100a48541 [mod] improve seekr engines and add documentation
Tis patch adds some more fields to the result items and changed paging to the
``nextResultSet`` given in seekr's JSON response.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-15 16:17:42 +02:00
Bnyro 2bab658d39 [feat] engine: implementation of seekr for news, images and videos 2023-08-15 16:17:42 +02:00
Bnyro e25d1c7288 [feat] engine: implementation of German news, Tagesschau
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 20:27:54 +02:00
Bnyro 834e1c3f12 [mod] engine lemmy: increase thumbnail quality to align with theme 2023-08-10 12:58:40 +02:00
Markus Heiser c381fc001f [mod] settings: remove lemmy from categ 'general' & enable by default
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 12:58:40 +02:00
Markus Heiser fda111c0c9 [mod] engine lemmy: add more info fields to the result items
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 12:58:40 +02:00
Bnyro 224f2250ae [feat] engine: support for lemmy communities, posts, comments and users 2023-08-10 12:58:40 +02:00
Bnyro 9f82c39610 [mod] engine google_play: raise error on unsupported category 2023-08-10 12:35:24 +02:00
Bnyro 0a99dc85b9 [mod] engine brave: raise error on unsupported category 2023-08-10 12:35:24 +02:00
allendema_searxng_pi c00c0c5434 [mod] remove discontinued petalsearch engines 2023-08-09 07:17:40 +02:00
Markus Heiser b8352eca0c [mod] brave engines: add fetch_traits() / improve language support
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 16:21:45 +02:00
Markus Heiser 460bbe5b81 [mod] implement brave (WEB) engine to replace XPath configuration
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 16:21:45 +02:00
Bnyro d151497db3 [feat] engine: brave - support for news 2023-08-08 16:21:45 +02:00
Bnyro cae06f2781 [feat] engine: brave - support for videos 2023-08-08 16:21:45 +02:00
Bnyro 73364e158e [feat] engine: brave - support for images 2023-08-08 16:21:45 +02:00
Markus Heiser 1d0abb7157 [doc] engine bt4g: add documentation to docs/dev/engines/online/
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-06 09:30:48 +02:00
Emilien Devos 0fc8f99ecc [feat] new engine: bt4g added & enabled and disable by default btdigg
Disable btdigg because on most SearXNG instances, SearXNG is blocked by btdigg
due to cloudflare too many requests.

This impementation did not parse the HTML page because there is an API in
XML (RSS).  The RSS feed provides fewer data like amount of seeders/leechers and
the files in the torrent file. It's a tradeoff for a "stable" engine as the XML
from RSS content will change way less than the HTML page.

Closes: https://github.com/searxng/searxng/issues/2553
2023-08-06 09:30:48 +02:00
Markus Heiser db522cf76d [mod] engine: wikimedia - improve results, add addition settings & doc
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-04 19:06:50 +02:00