Commit graph

2076 commits

Author SHA1 Message Date
Dalf
85b3723345 [mod] speed optimization
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
2019-11-15 09:33:15 +01:00
Mathieu Brunot
e9d90d06a9
Merge branch 'master' into feature/accessibility 2019-10-29 23:53:08 +01:00
Noémi Ványi
bf24b39fce
Merge branch 'master' into corrections_fix 2019-10-24 10:39:26 +02:00
Marc Abonce Seguin
dede46a2d5 fix out of range error in tracker remover plugin 2019-10-22 23:17:00 -07:00
Marc Abonce Seguin
b0f89ed477 [fix] preserve bangs in corrections 2019-10-22 21:41:50 -07:00
Mathieu Brunot
a51b2b6c20
Merge branch 'master' into feature/accessibility 2019-10-16 19:30:02 +02:00
Noémi Ványi
5796dc60c9 fix pep 8 check 2019-10-16 15:52:48 +02:00
Noémi Ványi
a6f20caf32 add initial support for offline engines && command engine 2019-10-16 15:52:48 +02:00
Adam Tauber
3c34e0c3a7 [mod] remove cloudflare protected digbt from engines list 2019-10-16 15:35:12 +02:00
Adam Tauber
7d8fd4b95e [fix] pep8 2019-10-16 15:32:03 +02:00
Adam Tauber
bbe4442a86 [fix] update gigablast engine 2019-10-16 15:27:37 +02:00
Adam Tauber
1057e42cfd [fix] update digg engine 2019-10-16 15:11:42 +02:00
Adam Tauber
72459b246b [fix] convert bytes type to string in language detection (fixes dictzone) 2019-10-16 14:52:57 +02:00
Adam Tauber
7177c9e12f [fix] update devianart engine 2019-10-16 14:42:31 +02:00
Adam Tauber
6ca1622378 [fix] update 1x engine 2019-10-16 13:27:05 +02:00
Adam Tauber
c98a2df36d [fix] enable paging support for arxiv engine 2019-10-16 13:12:17 +02:00
Adam Tauber
ed1c1bdb04 [fix] pep8 2019-10-14 15:09:39 +02:00
Adam Tauber
27ed31c7a2 [fix] add message to SearxParameterException - fixes #1722 2019-10-14 15:06:25 +02:00
Adam Tauber
28c75c187f [fix] update query params sanitization - closes #722 2019-10-14 14:58:20 +02:00
Adam Tauber
07a0a50e0d [fix] update voat xpaths - closes #1581 2019-10-14 14:25:49 +02:00
Adam Tauber
77a70fe541 [fix] update startpage engine - closes #1601 2019-10-14 14:18:41 +02:00
Adam Tauber
3d0c22c58d [fix] normalize time range parameter 2019-10-14 14:18:02 +02:00
Adam Tauber
94ea9d6622 [fix] duckduckgo paging - closes #1677 2019-10-14 13:52:15 +02:00
mathieu.brunot
4d17d453bf
Add aria labels for oscar buttons
Signed-off-by: mathieu.brunot <mathieu.brunot@monogramm.io>
2019-10-14 01:36:11 +02:00
Dalf
6fbba63c83 oscar and simple themes: no inline script (allow A+ Content-Security-Policy) 2019-08-06 12:14:11 +02:00
Frank de Lange
e4e8e6da4c Add image format and source information to display (#1567)
Add image format and source information to display - needs changes to engines to actually display something. 

Displays result.source (website from which the image was taken) and result.img_format (image type and size).

Result is styled with result-format and result-source classes. See PR #1566 for an example of an engine which has the necessary changes.

Strip <span class="highlight">...</span> in the oscar image template.
2019-08-06 09:31:24 +02:00
Léo Bourrel
88261e111c Fix bing engine results count (#1387)
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.

For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
2019-08-05 16:15:40 +02:00
Dalf
1cee2c1796 [fix] bing engine
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
2019-08-05 15:46:40 +02:00
Dalf
9a378ca8c6 [fix] etymonline, voat engines 2019-08-05 15:44:22 +02:00
Dalf
fcc9587ee9 [fix] fdroid engine 2019-08-05 15:44:02 +02:00
Dalf
fbf6b689dd [fix] dictzone engine 2019-08-05 15:43:21 +02:00
Dalf
9ff5001816 [fix] arxiv engine 2019-08-05 15:43:01 +02:00
Alexandre Flament
333e54943d
[fix] fix monkey patch in test_webapp.py (#1667)
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
2019-08-03 13:23:36 +02:00
Dalf
ba80a0ba38 [fix] fix paging for the oscar theme after PR #1640 2019-08-02 20:04:37 +02:00
Alexandre Flament
72029d27de
[enh] Add timeout limit per request (#1640)
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
2019-08-02 13:50:51 +02:00
Alexandre Flament
2179079a91
[fix] fix flickr_noapi decoding (#1655)
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
2019-08-02 13:37:13 +02:00
cy8aer
4dc792e1e2 [enh] add invidious engine. (#1657)
closes #1372
2019-08-02 13:25:25 +02:00
Dalf
0c032c8429 [fix] youtube_noapi engine: fix the title 2019-07-31 08:39:40 +02:00
Dalf
8b7ac56669 [fix] google_videos engine: some results don't a thumbnail 2019-07-31 08:38:45 +02:00
Dalf
d44677e226 [fix] dailymotion engine: remove HTML tags from the description 2019-07-31 08:37:51 +02:00
Dalf
6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection.
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
2019-07-29 07:39:39 +02:00
Frank de Lange
cbc5e13275 [enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. (#1571)
Fetch complete JSON data block, use legend to extract images. 
Unquote urlencoded strings.
Add image description as 'content'. 
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed). 
Show images which lack ownerid instead of discarding them.
2019-07-28 10:42:00 +02:00
Marc Abonce Seguin
59b1370824 [fix] small fixes in preferences view's text 2019-07-27 22:53:46 -07:00
Frank de Lange
204a2cbbf0 [fix] bing_videos (#1579)
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
2019-07-27 17:49:30 +02:00
Dalf
70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Dalf
23611897ec [fix] make sure then engine name is lower case
Minor fix: "%s engine initialized" display the right engine name
2019-07-27 08:52:30 +02:00
Dalf
8645ef717e [fix] settings.yml: make sure all engine names are lower case 2019-07-27 08:52:30 +02:00
Dalf
7d8000d965 [fix] "!wp !wp test" send only one request to wikipedia.
Note that "!general !wp test" is not fixed because the category for "!wp" is "none".
2019-07-27 08:52:30 +02:00
Dalf
2fab23ab9a [fix] fix the debug message "starting webserver on ip:port"
was "port:ip"
2019-07-27 08:52:30 +02:00
Marc Abonce Seguin
e1bb0e33f2 [fix] escape perecent signs in translation file (#1528) 2019-07-27 09:01:02 +02:00