Commit graph

34 commits

Author SHA1 Message Date
Mouse Reeve
45f2199c71 Gather and wait on async requests
This sends out the request tasks all at once and then aggregates the
results, instead of just running them one after another asynchronously.
2022-05-30 12:05:22 -07:00
Mouse Reeve
5e81ec75fb Set request headers in async search get request
Gotta ask for json
2022-05-30 11:19:16 -07:00
Mouse Reeve
9a9cef7766 Verify url before async search
The database lookup doesn't work during the asyn process, so this change
loops through the connectors and grabs the formatted urls before sending
it to the async handler.
2022-05-30 11:16:05 -07:00
Mouse Reeve
0adda36da7 Remove search endpoints from Connector
Instead of having individual search functions that make individual
requests, the connectors will always be searched asynchronously
together. The process_seach_response combines the parse and format
functions, which could probably be merged into one over-rideable
function.

The current to-do on this is to remove Inventaire search results that
are below the confidence threshhold after search, which used to happen
in the `search` function.
2022-05-30 10:37:24 -07:00
Mouse Reeve
9c03bf782e Make an async request to all search connectors
This is the untest first pass at re-arranging remote search to work in
parallel rather than sequence. It moves a couple functions around
(raise_not_valid_url, for example, needs to be in connector_manager.py
now to avoid circular imports). It adds a function to Connector objects
that generates a search result (either to the isbn endpoint or the free
text endpoint) based on the query, which was previously done as part of
the search.

I also lowered the timeout to 8 seconds by default.
2022-05-30 10:15:22 -07:00
Mouse Reeve
72d6a4ce52 Log info, not exception, for expected errors 2022-03-11 14:55:54 -08:00
Mouse Reeve
b18c69e186 Make search timeouts configurable 2022-01-07 07:42:05 -08:00
Mouse Reeve
5dd2aac600 Merge branch 'main' into search-refactor 2021-09-30 10:41:30 -07:00
Mouse Reeve
acfb1bb376 Updating string format synatx part 2 2021-09-18 11:32:00 -07:00
Mouse Reeve
fbe05623ff Updates first_search_result functionality 2021-09-16 11:07:36 -07:00
Mouse Reeve
1f06d1a1d8 Removes local connector 2021-09-14 15:26:36 -07:00
Mouse Reeve
aa91361fe4 Fixes celery kwarg for queue 2021-09-07 17:09:44 -07:00
Mouse Reeve
de3f18655c Set priorities on tasks 2021-09-07 16:33:43 -07:00
Mouse Reeve
9e5c7053e9 More pylint fixes 2021-06-18 14:29:24 -07:00
Mouse Reeve
cf3869ad32 Adds timeouts to get requests 2021-06-17 12:34:54 -07:00
Mouse Reeve
9b42bba236 Filter out inactive connectors 2021-05-11 11:34:58 -07:00
Mouse Reeve
df2c1f0723
Merge branch 'main' into fixes-search-display 2021-05-10 13:29:39 -07:00
Mouse Reeve
1844dd6b20 Only include result blobs with results in search results 2021-05-10 13:01:11 -07:00
Mouse Reeve
13dc5efe71 More comprehensive tests for connector search 2021-05-10 12:53:36 -07:00
Mouse Reeve
5cd974b78d Python formatting 2021-05-10 10:03:05 -07:00
Mouse Reeve
f2d985e583 Uses one set of search logic for all results or just first 2021-05-10 09:57:53 -07:00
Mouse Reeve
f2a6cfb4f3 Remove deduplication of external search results 2021-04-30 16:04:27 -07:00
Mouse Reeve
1edd00a0d1 Merge branch 'main' into list-fixes 2021-04-26 09:44:55 -07:00
Mouse Reeve
3ade2d3bb1 New version of black, new whitespace 2021-04-26 09:15:42 -07:00
Mouse Reeve
0f6b5cc6be Filter list search results to hide already added books 2021-04-26 08:02:30 -07:00
Mouse Reeve
954958b6f9 Handle arbitrary errors in isbn search 2021-04-07 10:54:00 -07:00
Mouse Reeve
f11d64f984 Handle all connector errors in search 2021-04-07 08:09:47 -07:00
Mouse Reeve
8ea60c66a3 Create connectors to federated bookwyrm servers
This got messed up when I refactored how connectors work! Poor
bookwyrm.social doesn't have a wyrms.de connector, but this will fix
that.
2021-04-01 17:02:45 -07:00
Mouse Reeve
66b7a3d193 Avoids error on empty search query 2021-03-31 12:03:58 -07:00
Mouse Reeve
414dd6bd20 Adds isbn search test to connector manager 2021-03-13 10:01:17 -08:00
Mouse Reeve
70296e760b Runs black 2021-03-08 08:49:10 -08:00
Henri Bourcereau
703ff60271 isbn search 2021-03-06 18:09:15 +01:00
Mouse Reeve
6a68fe9475 Search for books to add to lists 2021-01-31 11:11:26 -08:00
Mouse Reeve
d828b0ead9 Remove books manager at long last 2021-01-02 08:14:28 -08:00
Renamed from bookwyrm/books_manager.py (Browse further)