Commit graph

239 commits

Author SHA1 Message Date
Mouse Reeve 5d3639e47f Minor simplications to search queries 2021-06-17 14:46:58 -07:00
Mouse Reeve cf3869ad32 Adds timeouts to get requests 2021-06-17 12:34:54 -07:00
Mouse Reeve 1e9211018d Handle openlibrary redirects 2021-05-20 15:55:50 -07:00
Mouse Reeve d7a8dd5e19 Removes outdated field form abstract connector 2021-05-11 13:20:17 -07:00
Mouse Reeve 9b42bba236 Filter out inactive connectors 2021-05-11 11:34:58 -07:00
Mouse Reeve e7c5b77b4c Removes unused connector fields and adds active boolean 2021-05-11 09:54:04 -07:00
Mouse Reeve 07c84a6e1a Adds confidence ratings to inventaire results
Fixes bad matches in csv imports
2021-05-10 15:35:46 -07:00
Mouse Reeve df2c1f0723
Merge branch 'main' into fixes-search-display 2021-05-10 13:29:39 -07:00
Mouse Reeve 1844dd6b20 Only include result blobs with results in search results 2021-05-10 13:01:11 -07:00
Mouse Reeve 13dc5efe71 More comprehensive tests for connector search 2021-05-10 12:53:36 -07:00
Mouse Reeve 5cd974b78d Python formatting 2021-05-10 10:03:05 -07:00
Mouse Reeve f2d985e583 Uses one set of search logic for all results or just first 2021-05-10 09:57:53 -07:00
Mouse Reeve c15358d715 Python formatting 2021-05-05 07:58:09 -07:00
Mouse Reeve 558c9c4d67 Sort edition list for better import results from inventaire 2021-05-05 07:53:10 -07:00
Mouse Reeve f2a6cfb4f3 Remove deduplication of external search results 2021-04-30 16:04:27 -07:00
Mouse Reeve a2621bce12 Fixes getting authors from work data 2021-04-30 15:48:52 -07:00
Mouse Reeve 485d20696b Fixes loading inventaire data by language code 2021-04-30 12:50:35 -07:00
Mouse Reeve 15790abc70 Don't show broken image previews when cover is absent 2021-04-29 13:03:56 -07:00
Mouse Reeve 095b60bff1 Show search result context for inventaire results 2021-04-29 11:22:09 -07:00
Mouse Reeve 7853610a20 Load descriptions correctly 2021-04-29 10:54:36 -07:00
Mouse Reeve b1c38d291c Set preferred language 2021-04-29 10:40:49 -07:00
Mouse Reeve ca86af22ce Get inventaire entity descriptions 2021-04-29 10:30:38 -07:00
Mouse Reeve cfd2c05ae2 Safely handle absent claims field 2021-04-29 10:12:56 -07:00
Mouse Reeve 8d38d1c9d1 Python formatting 2021-04-29 09:56:35 -07:00
Mouse Reeve f4800307b4 Adds isbn search 2021-04-29 09:54:42 -07:00
Mouse Reeve 95fce963d1 Images may already be absolute paths 2021-04-28 17:18:14 -07:00
Mouse Reeve 8eec3eca5b Don't spin out infinite load data tasks 2021-04-28 16:42:27 -07:00
Mouse Reeve 6551c59c45 Don't store default edition in the dataase 2021-04-28 16:05:17 -07:00
Mouse Reeve ccf10e8012 Fixes cover load logic 2021-04-28 13:50:47 -07:00
Mouse Reeve 3faacffaca Associated expanded editions with correct work 2021-04-28 13:28:07 -07:00
Mouse Reeve 7b65291a59 Python formatting for the new Black standard 2021-04-26 14:43:29 -07:00
Mouse Reeve 64c2313a5d Merge branch 'main' into inventaire 2021-04-26 14:22:05 -07:00
Mouse Reeve 1edd00a0d1 Merge branch 'main' into list-fixes 2021-04-26 09:44:55 -07:00
Mouse Reeve 3ade2d3bb1 New version of black, new whitespace 2021-04-26 09:15:42 -07:00
Mouse Reeve 0f6b5cc6be Filter list search results to hide already added books 2021-04-26 08:02:30 -07:00
Mouse Reeve 1903812b1d Class method for checking if urls are blocked 2021-04-10 11:44:29 -07:00
Mouse Reeve 954958b6f9 Handle arbitrary errors in isbn search 2021-04-07 10:54:00 -07:00
Mouse Reeve 63d37c281d Handle invalid authors when importing books 2021-04-07 08:59:33 -07:00
Mouse Reeve f11d64f984 Handle all connector errors in search 2021-04-07 08:09:47 -07:00
Mouse Reeve 922428cab7 Fixes error in reverse path 2021-04-06 18:51:43 -07:00
Mouse Reeve 29e7659b76 Expand inventaire book data 2021-04-06 18:34:55 -07:00
Mouse Reeve fec3d63e46 Python formatting 2021-04-06 18:17:33 -07:00
Mouse Reeve f21aca1211 Load remote keys 2021-04-06 18:10:42 -07:00
Mouse Reeve 4112862924 Fixes search data and new activitypub fields 2021-04-06 18:00:54 -07:00
Mouse Reeve 82c2f2eeb1 Adds more author identifier fields 2021-04-06 17:46:06 -07:00
Mouse Reeve e594cd0a36 Load simple fields from inventaire 2021-04-06 13:53:58 -07:00
Mouse Reeve 3158701075 Gets editions for works 2021-04-06 13:39:10 -07:00
Mouse Reeve 5149c7e8c2 Expands mappings for inventaire/wikidata properties 2021-04-06 13:03:22 -07:00
Mouse Reeve 22ebe60c0a Use custom data extractor for inventaire connector 2021-04-06 12:29:06 -07:00
Mouse Reeve fba44206ac Adds separate view and load links for book search results 2021-04-06 12:17:58 -07:00
Mouse Reeve d482c66ad4 Adds inventaire connector stub
And changes formatters to accept the key as well as value
2021-04-06 11:58:07 -07:00
Mouse Reeve f30d05acfc Update connector model to add new connector 2021-04-06 10:40:13 -07:00
Mouse Reeve 8ea60c66a3 Create connectors to federated bookwyrm servers
This got messed up when I refactored how connectors work! Poor
bookwyrm.social doesn't have a wyrms.de connector, but this will fix
that.
2021-04-01 17:02:45 -07:00
Mouse Reeve 66b7a3d193 Avoids error on empty search query 2021-03-31 12:03:58 -07:00
Mouse Reeve ad8257f8bd Formats changes 2021-03-13 14:04:41 -08:00
Mouse Reeve 05fcfbc66f Show more data about books in search 2021-03-13 13:55:20 -08:00
Mouse Reeve 78df37a861
Merge pull request #727 from mouse-reeve/connector-tests
Connector tests
2021-03-13 10:24:40 -08:00
Mouse Reeve 76f1643269 Tests ignore edition openlibrary connector code
also removes print statements oops
2021-03-13 10:11:13 -08:00
Mouse Reeve 414dd6bd20 Adds isbn search test to connector manager 2021-03-13 10:01:17 -08:00
Mouse Reeve 393ea625a2
Merge pull request #726 from mouse-reeve/search-tests
Search tests
2021-03-13 09:27:46 -08:00
Mouse Reeve d3b1941eaa Uses get_data helper in isbn search (plus test) 2021-03-13 09:07:19 -08:00
Mouse Reeve 2ab913018d Use get_data helper when executing searches 2021-03-13 08:52:36 -08:00
Mouse Reeve 919b166241 Catch error in serializing unknown boosts 2021-03-13 08:13:20 -08:00
Mouse Reeve 70296e760b Runs black 2021-03-08 08:49:10 -08:00
Henri Bourcereau 703ff60271 isbn search 2021-03-06 18:09:15 +01:00
Mouse Reeve cad19ee878 Merge branch 'main' into inbox-refactor 2021-02-24 09:51:34 -08:00
Mouse Reeve 726a8739a3 way too much logging coming from http errors 2021-02-22 11:42:45 -08:00
Mouse Reeve a9ca3a4290 Fixes calls to to_model, init with activitypub partially serialized 2021-02-16 20:17:38 -08:00
Mouse Reeve 3f61675a0a Updates usage of resolve_remote_id 2021-02-16 19:35:43 -08:00
Mouse Reeve 41cef9bdfd Catch expand book data load error 2021-02-10 17:54:49 -08:00
Mouse Reeve 4c2ec43fce Don't set default edition if it's already set 2021-02-10 16:53:51 -08:00
Mouse Reeve f46a708f7e Handle connector exceptions with sketchy book data from OL 2021-02-10 12:00:16 -08:00
Mouse Reeve 9a564a846a Better error handling and loggin in get_data 2021-02-10 11:56:08 -08:00
Mouse Reeve 6a68fe9475 Search for books to add to lists 2021-01-31 11:11:26 -08:00
Mouse Reeve 661d49d9cc Ignore openlibrary editions with little to no metadata
Also fixes the isbn problem
2021-01-30 17:19:01 -08:00
Mouse Reeve fe67f65307 Set default edition on bookwyrm import 2021-01-11 10:25:34 -08:00
Mouse Reeve 9a114e1736 Don't match arbitrary results on empty isbn queries 2021-01-07 09:26:05 -08:00
Mouse Reeve ef42d29bd9 Discovery landing page 2021-01-03 11:10:04 -08:00
Mouse Reeve 8afd111ff4
Merge pull request #466 from mouse-reeve/search-tweaks
Search tweaks
2021-01-02 16:30:28 -08:00
Mouse Reeve d3c181cacb Re-sorts search results after deduplication 2021-01-02 16:09:54 -08:00
Mouse Reeve a413c87963 Separate search for unique identifiers out from text search 2021-01-02 15:58:25 -08:00
Mouse Reeve a2e8cf1993 Return best matching edition instead of default in search 2021-01-02 15:15:25 -08:00
Mouse Reeve afa1921968 Deduplicates search results 2021-01-02 14:33:01 -08:00
Mouse Reeve 8c8aae2c92 Check if a book is already shelved after import 2021-01-02 12:08:30 -08:00
Mouse Reeve 3344eed3b9 Tests for goodreads import lookup 2021-01-02 11:35:56 -08:00
Mouse Reeve 86060f795d Updates tests 2021-01-02 08:55:42 -08:00
Mouse Reeve d828b0ead9 Remove books manager at long last 2021-01-02 08:14:28 -08:00
Mouse Reeve e169565e00 Makes expanding book data fully part of the connector 2021-01-02 07:45:45 -08:00
Mouse Reeve 03ed1cfa94 Small syntax changes in openlibrary connector 2020-12-31 11:03:30 -08:00
Mouse Reeve 171d09599c Fixes author path in openlibrary connector 2020-12-31 09:32:40 -08:00
Mouse Reeve c1e1bdac4b Fixes url formatting in openlibrary connector 2020-12-31 09:19:39 -08:00
Mouse Reeve 877e3356e5
Merge pull request #436 from mouse-reeve/search-errors
Catch error response decoding json in search connector
2020-12-30 12:18:43 -08:00
Mouse Reeve e5a914c3c2 Fixes API search causing 500 error 2020-12-30 11:37:26 -08:00
Mouse Reeve d7db6d50ba Log errors in conenctor search 2020-12-30 09:14:07 -08:00
Mouse Reeve 21f67c9e28 Catch error response decoding json in search connector 2020-12-30 09:11:00 -08:00
Renato "Lond" Cerqueira fb10cb35ad Add BookWyrm user-agent to http requests
This allows other software to identify BookWyrm in calls, as well as
will allow BookWyrm to differentiate between calls done from other
fediverse software and BookWyrm to answer with specific BookWyrm data.
2020-12-30 12:36:50 +01:00
Mouse Reeve ac261d7b1a Send connector with search result
also fix typo in get_work_from_edition_data function
2020-12-27 14:33:34 -08:00
Mouse Reeve 1fb728edbd
Merge pull request #415 from mouse-reeve/author_text
Generate author_text field dynamically
2020-12-21 13:09:40 -08:00
Mouse Reeve aac264c998 Generate author_text field dynamically 2020-12-21 12:10:42 -08:00
Mouse Reeve c3d0e8e7f7 Fixes openlibrary import to prefer editions with covers 2020-12-21 11:47:47 -08:00
Mouse Reeve 3e61f48da2 Fixes loading more data 2020-12-19 16:16:39 -08:00
Mouse Reeve d3671d981f Use author activitypub in OL connector 2020-12-19 15:20:31 -08:00
Mouse Reeve 2128219b05 Use activitypub to_model de-serializer in openlibrary connector 2020-12-19 14:56:03 -08:00
Mouse Reeve 31a407d74a Use name field only for author name
It feels janky to remove a more granular name designation, but all these
first/last name fields were algorithmically populated by a dubious
process of splitting the name by a space character. If it makes sense to
have first/last name fields, it should be re-added with some
consideration.
2020-12-12 09:53:21 -08:00
Mouse Reeve 5cf9e24ae5 Fixes name import in openlibrary 2020-12-12 09:43:07 -08:00
Mouse Reeve 27c45c0584 Catch json decode error in loading data 2020-12-03 13:42:02 -08:00
Mouse Reeve a85043b351 Updates to_model to use fields 2020-12-03 12:35:57 -08:00
Mouse Reeve 9ddd60ce16 Fixes broadcast tests 2020-11-29 09:40:15 -08:00
Mouse Reeve c9433a3c7e Simplify bookwyrm connector abstract 2020-11-28 18:56:28 -08:00
Mouse Reeve 9d84346d3c remove need for get_or_create_book 2020-11-28 18:46:50 -08:00
Mouse Reeve 7b6035898b Serialize activitypub authors from data in connector 2020-11-27 14:54:08 -08:00
Mouse Reeve 9b79eb5fc3 Removes outdated test of bookwyrm connector updater 2020-11-25 11:44:19 -08:00
Mouse Reeve 847014720e Refactors bookwyrm connector to use activitypub serializer 2020-11-24 16:05:00 -08:00
Mouse Reeve 28e1c9525c tweaks search rankings for better results 2020-11-13 11:03:39 -08:00
Mouse Reeve 86504989b4 fixes import matching with local books 2020-11-13 09:59:57 -08:00
Mouse Reeve 74236c4810 Handle errors tryingt o load covers from instances 2020-11-12 13:20:32 -08:00
Mouse Reeve 0a2d762d3b Fixes error state when there are author duplicates
there shouldn't be, but it shouldn't crash a goodreads import either
2020-11-12 13:16:26 -08:00
Mouse Reeve dd4e323e18 Fixes cover filesnames 2020-11-07 10:59:08 -08:00
Mouse Reeve 9238f4c74a Correclty parse ordered collection json for editions 2020-11-04 13:18:30 -08:00
Mouse Reeve 6bf9a7159b Load expanded book data from bookwyrm connector 2020-11-04 13:09:11 -08:00
Mouse Reeve 441220a547 Fixes edition/work author correction check 2020-11-04 12:12:29 -08:00
Mouse Reeve ff96835c2a Migrate default edition field to work model
Fixes #281
2020-11-04 11:27:49 -08:00
Mouse Reeve 2ac9a6fcee Fixes updating books from remote instances 2020-11-02 09:34:46 -08:00
Mouse Reeve 0393d81230 Fixes loading covers and authors 2020-10-30 17:18:25 -07:00
Mouse Reeve 203e526a83 Fixes loading remote books
- saves remote_id correctly
- loads remote books for incoming statuses
2020-10-30 17:04:10 -07:00
Mouse Reeve 6e97592518 Trying to catch more http request errors 2020-10-30 14:19:43 -07:00
Mouse Reeve 7ce0890a41 Stop assuming every book is Hamlet 2020-10-29 15:30:53 -07:00
Mouse Reeve a46d7f5dc7 Change how goodread import writes reviews
- adds published date
 - broadcasts review imports
 - completes review and shelve actions as it goes
 - some small connector fixes

 fixes #247
2020-10-29 14:31:06 -07:00
Mouse Reeve 7febcec229 convert between ibsn 10 and 13 2020-10-29 12:32:37 -07:00
Mouse Reeve 90cccc455e Another error check point in abstract connector 2020-10-28 17:16:16 -07:00
Mouse Reeve 6b5246f06a
Merge pull request #209 from mouse-reeve/connector-errors
Raise errors when connectors fail
2020-10-01 08:29:39 -07:00
Mouse Reeve 94d5986ff2 More error handling in connector/books manager 2020-09-30 20:09:25 -07:00
Mouse Reeve fe83f5d442 small style fixes and typo 2020-09-30 19:57:25 -07:00
Mouse Reeve 0b8f8e3659 Fixes celery media path 2020-09-30 19:43:42 -07:00
Mouse Reeve f72d59955e Raise errors when connectors fail 2020-09-30 10:37:29 -07:00
Mouse Reeve 56ebd6e7c0 Fixes bug in setting author text on books 2020-09-29 20:54:49 -07:00
Mouse Reeve 425da16fd6 Fixes linter issues 2020-09-21 10:25:26 -07:00
Mouse Reeve c7883cd615 Updates migrations
To get the app working again I ran resetdb, let it crash in initdb,
then ran the migration, then re-ran initdb
2020-09-21 08:10:37 -07:00
Mouse Reeve f77c156733 rename main code directory 2020-09-17 13:30:54 -07:00