Anna’s Archive has cleaned up their languages, available file extensions and
changed the HTML form.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Crossref was broken on result types journal-issue and component .. The old code
had lots of assumptions, and broke during parsing. Now the assumptions are more
explicit and checked them with the API.
Remove the usage of searx.network.multi_requests
The results from Bing contains the target URL encoded in base64
See the u parameter, remove the first two character "a1", and done.
Also add a comment the check of the result_len / pageno
( from https://github.com/searx/searx/pull/1387 )
It seems there is an API change:
extratags can be either a dictionnary or None.
This commit avoid crash when extratags is None
Test query "!osm gare du nord"
The method EngineTraits.get_region(..) returns engine's region string
that **best fits** to SearXNG's locale. This means it returns a
region (country) if only a language is set in the locale. By example the method
returns for a locale tag `es` a region `ES`.
Google's search parameter `cr` restricts search results to documents originating
in a particular country / in case of a locale tag (language) as described above,
this argument should be unset in the query send to Google.
Closes: https://github.com/searxng/searxng/issues/2672
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Show URL of the ddg-search page, not the URL of a (generic) Javascript. The
latter one is not usefull for the user.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Tis patch adds some more fields to the result items and changed paging to the
``nextResultSet`` given in seekr's JSON response.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Disable btdigg because on most SearXNG instances, SearXNG is blocked by btdigg
due to cloudflare too many requests.
This impementation did not parse the HTML page because there is an API in
XML (RSS). The RSS feed provides fewer data like amount of seeders/leechers and
the files in the torrent file. It's a tradeoff for a "stable" engine as the XML
from RSS content will change way less than the HTML page.
Closes: https://github.com/searxng/searxng/issues/2553
SearXNG does not allow a None value in the content field of a result item.
If the key (shortDescription, uploaderName) in the JSON response from piped
exists but is set to None, SearXNG ignores this result item::
DEBUG searx : result: invalid content: { .., 'content': None, ..}
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- re-enables z-library as the new domain zlibrary-global.se is now available
from the open web. The announcement of the domain:
https://www.reddit.com/r/zlibrary/comments/13whe08/mod_note_zlibraryglobalse_domain_is_officially/
It is an official domain, it requires to log in to the "personal" subdomain
only to download files, but the search works.
- changes the result template of zlibrary to paper.html, filling the appropriate fields
- implements language filtering for zlibrary
- implement zlibrary custom filters (engine traits)
- refactor and document the zlibrary engine
We have built up detailed documentation of the *settings* and the *engines* over
the past few years. However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.
This patch rearranges the Settings & Engines documentation for better
readability.
To review new ordered docs::
make docs.clean docs.live
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The renderuing of the WEB page is very strange; except the firts position all
other positions of Anna's result page are enclosed in SGML comments. These
cooments are *uncommented* by some JS code, see query of the class
'.js-scroll-hidden' in Anna's HTML template [1].
[1] https://annas-software.org/AnnaArchivist/annas-archive/-/blob/main/allthethings/templates/macros/md5_list.html
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- torznab engine using types and clearer code
- torznab option to hide torrent and magnet links.
- document the torznab engine
- add myself to authors
Closes: https://github.com/searxng/searxng/issues/1124
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
It seems that Google is rolling out a modified WEB API [1][2].
In the past there was only the UI language in the `hl` argument but nowadays it
seems a combination of the UI language and the "search region" is mixed in this
argument and the `gl` argument has been removed. I'm very surprised that google
is starting to mix the parameters of the UI with the parameters of the search
index.
This patch modifies the get_google_info(..) function. Beside Google-WEB this
function is also used by other Google services, here are some examples to test
region & language of ..
- Google-WEB: `!go dragon boat :en-CA`
- Google-News: `!gon dragon boat :en-CA`
- Google-Videos: `!gov bmw :en-CA`
- Goolge-Images `!goi bmw :en-CA`
- [1] https://github.com/searxng/searxng/issues/2515#issuecomment-1606294635
- [2] https://github.com/searxng/searxng/issues/2515#issuecomment-1607150817
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch fixes some quirks and issues related to the engines and the network.
Each engine has its own network and this network was broken for the following
engines[1]:
- archlinux
- bing
- dailymotion
- duckduckgo
- google
- peertube
- startpage
- wikipedia
Since the files have been touched anyway, the type annotaions of the engine
modules has also been completed so that error messages from the type checker are
no longer reported.
Related and (partial) fixed issue:
- [1] https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
- [2] https://github.com/searxng/searxng/issues/2513
- [3] https://github.com/searxng/searxng/issues/2515
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Wikipedia's zh-classical is not zh_Hant (see doc-string of engines.wikipedia).
Fixed the example in the doc-string of locales.get_engine_locale() to 'zh_TW'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
