Commit graph

119 commits

Author SHA1 Message Date
Margaret Fero 91fe4ad535 Fix spacing for linter 2024-03-02 17:31:16 -08:00
Margaret Fero 9fa09d5ebe Add extra space required by linter 2024-03-02 17:30:37 -08:00
Margaret Fero 39da471f79 Disable Pylint Failure for imghdr deprecation for now 2024-03-02 15:59:17 -08:00
Joeri de Ruiter 2920973961 Some small improvements to annotations 2023-07-28 20:54:03 +02:00
Joeri de Ruiter f07d7b02f1 Type annotations and related changes for bookwyrm.connectors 2023-07-28 17:43:32 +02:00
Mouse Reeve cbb027c56c
Merge pull request #2778 from ranok/upstream_pr
Move the search request logic into the AbstractConnector
2023-04-25 16:20:24 -07:00
Jacob Torrey 84834eb5d3 Run bw-dev black to fix formatting
Signed-off-by: Jacob Torrey <jacob@jacobtorrey.com>
2023-04-17 15:06:41 +00:00
Josh Soref 7f8279fe54 spelling: format
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-04 20:02:54 -04:00
Josh Soref 8d4b69927b spelling: directly
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-04 20:02:54 -04:00
Jacob Torrey f9c75a43ae Fixing pylint issues
Signed-off-by: Jacob Torrey <jacob@jacobtorrey.com>
2023-04-04 16:46:32 +00:00
Jacob Torrey 797d339132 Move the search request logic into the AbstractConnector to allow for more flexibility
Signed-off-by: Jacob Torrey <jacob@jacobtorrey.com>
2023-04-04 16:03:37 +00:00
0x29a 22eeee7368 Urlencode search query 2023-02-02 21:02:57 +01:00
Hugh Rundle e8452011f7 handle get_data exceptions better
Makes exception handling more precise, only raising status for 401s.

Also fixes a string pylint was complaining about.
2023-01-20 19:55:38 +11:00
Hugh Rundle 4108238716 resolve SECURE_FETCH bugs
ERROR HANDLING FIXES

- use raise_for_status() to pass through response code
- handle exceptions where no response object is passed through

INSTANCE ACTOR

- models.User.objects.create_user function cannot take an ID
- allow instance admins to determine username and email for instance actor in settings.py
2023-01-20 16:32:17 +11:00
Mouse Reeve b37a4322de Change log level to info for connector exceptions
These errors in resolve_remote_id aren't really errors, they're
routine problems that we can expect from dealing with the outside world,
like a connection timeout, a server being down, a server being blocked,
et cetera. It's cluttering up the logs and causing unnecessary worry.
2022-11-17 12:35:19 -08:00
Hugh Rundle 1ee2ff4811 normalise isbn on local book search
- uppercase ISBN before checking it's a number to account for trailing 'x'
- check maybe_isbn for search_identifiers search. Without this we are only searching external connectors, not locally!
2022-08-30 20:00:09 +10:00
Hugh Rundle 18d3d2f85d linting 2022-08-28 17:30:46 +10:00
Hugh Rundle f219851f3a strip leading and following spaces from ISBN 2022-08-28 17:28:00 +10:00
Hugh Rundle da5fd32196 normalise isbn searching
ISBNs are always numeric except for when the check digit in ISBN-10s is a ten, indicated with a capital X.
These changes ensure that ISBNs are always upper-case so that a lower-case 'x' is not used when searching.

Additionally some ancient ISBNs have been printed without a leading zero (i.e. they only have 9 characters on the physical book). This change prepends a zero if something looks like an ISBN but only has 9 chars.
2022-08-28 11:05:40 +10:00
Mouse Reeve 83ee5a756f Filter intentaire results by confidence 2022-05-30 16:42:37 -07:00
Mouse Reeve 87fe984462 Combines search formatter and parser function
The parser was extracting the list of search results from the json
object returned by the search endpoint, and the formatter was converting
an individual json entry into a SearchResult object. This just merged
them into one function, because they are never used separately.
2022-05-30 12:52:31 -07:00
Mouse Reeve 0adda36da7 Remove search endpoints from Connector
Instead of having individual search functions that make individual
requests, the connectors will always be searched asynchronously
together. The process_seach_response combines the parse and format
functions, which could probably be merged into one over-rideable
function.

The current to-do on this is to remove Inventaire search results that
are below the confidence threshhold after search, which used to happen
in the `search` function.
2022-05-30 10:37:24 -07:00
Mouse Reeve 9c03bf782e Make an async request to all search connectors
This is the untest first pass at re-arranging remote search to work in
parallel rather than sequence. It moves a couple functions around
(raise_not_valid_url, for example, needs to be in connector_manager.py
now to avoid circular imports). It adds a function to Connector objects
that generates a search result (either to the isbn endpoint or the free
text endpoint) based on the query, which was previously done as part of
the search.

I also lowered the timeout to 8 seconds by default.
2022-05-30 10:15:22 -07:00
Mouse Reeve 72d6a4ce52 Log info, not exception, for expected errors 2022-03-11 14:55:54 -08:00
Mouse Reeve 3e635f497e Adds some simple url validation 2022-02-03 15:11:01 -08:00
Mouse Reeve 194c69f512 Fixes return values of null responses 2022-02-02 07:09:35 -08:00
Mouse Reeve 754e24812b Check image extensions before saving 2022-02-01 21:18:25 -08:00
Mouse Reeve b18c69e186 Make search timeouts configurable 2022-01-07 07:42:05 -08:00
Mouse Reeve 3545085a7d Fixes tests 2021-12-14 14:19:27 -08:00
Mouse Reeve 09f5218f9c Fixes accept header 2021-12-14 13:47:09 -08:00
Mouse Reeve 6e61e4d52c
Merge pull request #1578 from bookwyrm-social/improve-compatibility
Improve federation compability with Hubzilla and Zap
2021-12-09 11:06:04 -08:00
Mouse Reeve 071da7d4fb Handle various link generation needs 2021-12-05 13:38:15 -08:00
Mouse Reeve 4085714764 Update openlibrary author with ISNI 2021-12-05 13:26:22 -08:00
Mouse Reeve d7e4e6aa1e Adds openlibrary update for book 2021-12-05 13:02:42 -08:00
Mouse Reeve b824841cb3 Adds update logic to connectors 2021-12-05 12:47:27 -08:00
Mouse Reeve 8ba875af4a Improve federation compability with Hubzilla and Zap
Co-authored-by: hubzilla <redmatrix@users.noreply.github.com>
Fixes #1564
2021-10-26 14:41:06 -07:00
Mouse Reeve 5dd2aac600 Merge branch 'main' into search-refactor 2021-09-30 10:41:30 -07:00
Mouse Reeve d36ef2bcf1 Pylint change 2021-09-29 12:42:28 -07:00
Mouse Reeve 32391dd64d Python formatting 2021-09-29 12:38:31 -07:00
Mouse Reeve 0aef011258 Don't use the format detail if it maps directly 2021-09-29 12:29:17 -07:00
Mouse Reeve 123b23728f Infer format in openlibrary import 2021-09-29 12:21:19 -07:00
Mouse Reeve 08f6a97653 Python formatting 2021-09-18 11:33:43 -07:00
Mouse Reeve acfb1bb376 Updating string format synatx part 2 2021-09-18 11:32:00 -07:00
Mouse Reeve 18591c7b56 Fixes circular import 2021-09-16 11:30:04 -07:00
Mouse Reeve 1f06d1a1d8 Removes local connector 2021-09-14 15:26:36 -07:00
Mouse Reeve ad0fff7030 Prevent overwriting data on import form outside data source 2021-08-17 10:08:07 -07:00
Mouse Reeve 7a44a9b516 Use catch-all requests exception 2021-06-20 09:23:57 -07:00
Mouse Reeve 95a19a44f8 Don't error out when invalid urls are entered 2021-06-20 09:13:57 -07:00
Mouse Reeve 4479864008 Updates code for linter 2021-06-18 14:12:56 -07:00
Mouse Reeve cf3869ad32 Adds timeouts to get requests 2021-06-17 12:34:54 -07:00