Commit graph

7123 commits

Author SHA1 Message Date
Mouse Reeve
3aa159bc89
Merge branch 'main' into preview-generation-memory 2022-06-05 18:39:59 -07:00
Mouse Reeve
08231f52ff
Merge pull request #2150 from cincodenada/fix-pylint
Fix pylint config for pylint 2.14.0
2022-06-05 15:41:32 -07:00
Joel Bradshaw
6584cb6404 Go back to one requirements.txt, simplify workflow
The workflow can now use .pylintrc and the pylint req in
requirements.txt rather than having the options inline and installing it
separately
2022-06-05 14:57:42 -07:00
Joel Bradshaw
b3603c04c5 Add pylint to bw-dev
Because pylint requires the app to be fully parseable with all its
dependencies, we run it in the web container, and add pylint as a dev
dependency.
2022-06-05 14:49:21 -07:00
Joel Bradshaw
6d6ab9a531 Add .pylintrc with fixes for new pylint version 2022-06-05 14:38:03 -07:00
Joel Bradshaw
b744ff7836 Run black 2022-06-05 13:40:01 -07:00
Joel Bradshaw
482005f304 Update preview image generation to only query ids
Previously we were querying the full book objects just to get a list of
id's, which is much slower and also takes a lot more memory, which can
cause the process to be killed on memory-limited machines with a large
number of books.

Instead, since we're just dispatching jobs here, we can just ask for the
id's, which is faster and much more practical memory-wise.

The map is a little annoying, I didn't see a way to directly get just a
list of the value of one field, so we have to get a list of
dictionairies with one key and then pull that key out. Whatevs.
2022-06-05 13:07:44 -07:00
Mouse Reeve
d5bbb759e0
Merge pull request #2146 from bookwyrm-social/locales
Updates locales for stopped reading strings
2022-05-31 17:09:44 -07:00
Mouse Reeve
077c9bfe46 Updates locales for stopped reading strings 2022-05-31 16:53:46 -07:00
Mouse Reeve
9d5e113b92
Merge pull request #2145 from bookwyrm-social/about-layout
Clip column in about page
2022-05-31 14:05:43 -07:00
Mouse Reeve
4c050d0999
Merge pull request #2141 from bookwyrm-social/ol-search-rank
Use relative list order ranking in OpenLibrary search
2022-05-31 13:11:49 -07:00
Mouse Reeve
20f452ebf4 Clip column in about page
Text in the superlatives section can cause this column to expand outside
the container.
2022-05-31 12:23:59 -07:00
Mouse Reeve
374fdcf467 Use relative list order ranking in openlibrary search
Set OpenLibrary search condifidence based on the provided result order,
just using 1/(list index), so the first has rank 1, the second 0.5, the
third 0.33, et cetera.
2022-05-31 10:22:49 -07:00
Mouse Reeve
355e7039f0
Merge pull request #2139 from bookwyrm-social/search-refactor
Search refactor
2022-05-31 10:22:17 -07:00
Mouse Reeve
c3b35760a2 Updates test mocks for remote search 2022-05-31 09:37:54 -07:00
Mouse Reeve
969db13ff2 Safely return None in remote search return_first 2022-05-31 08:49:23 -07:00
Mouse Reeve
05fd30cfcf Pylint fixes in connector tests 2022-05-31 08:37:07 -07:00
Mouse Reeve
5e99002aad Raise priority for external connectors in initdb
By default, OpenLibrary and Inventaire were prioritzed below other
BookWyrm nodes. In practice, people have gotten better search results
from these connectors, hence the change. With the search refactor, this
has much less impact, but it will show these search results higher in
the list.

If the results page shows all the connectors' results integrated, this
field should be removed entirely.
2022-05-31 08:25:02 -07:00
Mouse Reeve
a053f20961 Re-implements return first option
Since we get all the results quickly now, this aggregates all the
results that came back and sorts them by confidence, and returns the
highest confidence result. The confidences aren't great on free text
search, but conceptually that's how it should work at least.

It may make sense to aggregate the search results in all contexts, but
I'll propose that in a separate PR.
2022-05-31 08:20:59 -07:00
Mouse Reeve
98ed03b6b4 Python formatting and test update 2022-05-30 17:00:34 -07:00
Mouse Reeve
83ee5a756f Filter intentaire results by confidence 2022-05-30 16:42:37 -07:00
Mouse Reeve
af19d728d2 Removes outdated unit tests 2022-05-30 16:16:10 -07:00
Mouse Reeve
87fe984462 Combines search formatter and parser function
The parser was extracting the list of search results from the json
object returned by the search endpoint, and the formatter was converting
an individual json entry into a SearchResult object. This just merged
them into one function, because they are never used separately.
2022-05-30 12:52:31 -07:00
Mouse Reeve
525e2a591d More error handing
Adds logging and error handling for some of the numerous ways a request
could fail (the remote site is down, the url is blocked, etc).

I also have the results boxes open by default, which makes it more
legible imo.
2022-05-30 12:40:13 -07:00
Mouse Reeve
45f2199c71 Gather and wait on async requests
This sends out the request tasks all at once and then aggregates the
results, instead of just running them one after another asynchronously.
2022-05-30 12:05:22 -07:00
Mouse Reeve
5e81ec75fb Set request headers in async search get request
Gotta ask for json
2022-05-30 11:19:16 -07:00
Mouse Reeve
9a9cef7766 Verify url before async search
The database lookup doesn't work during the asyn process, so this change
loops through the connectors and grabs the formatted urls before sending
it to the async handler.
2022-05-30 11:16:05 -07:00
Mouse Reeve
0adda36da7 Remove search endpoints from Connector
Instead of having individual search functions that make individual
requests, the connectors will always be searched asynchronously
together. The process_seach_response combines the parse and format
functions, which could probably be merged into one over-rideable
function.

The current to-do on this is to remove Inventaire search results that
are below the confidence threshhold after search, which used to happen
in the `search` function.
2022-05-30 10:37:24 -07:00
Mouse Reeve
9c03bf782e Make an async request to all search connectors
This is the untest first pass at re-arranging remote search to work in
parallel rather than sequence. It moves a couple functions around
(raise_not_valid_url, for example, needs to be in connector_manager.py
now to avoid circular imports). It adds a function to Connector objects
that generates a search result (either to the isbn endpoint or the free
text endpoint) based on the query, which was previously done as part of
the search.

I also lowered the timeout to 8 seconds by default.
2022-05-30 10:15:22 -07:00
Mouse Reeve
7905be7de2
Merge pull request #2138 from bookwyrm-social/ratings-query
Use general ratings rather than privacy filtered
2022-05-30 09:33:05 -07:00
Mouse Reeve
fb3c7205af Updates unit tests 2022-05-30 09:17:51 -07:00
Mouse Reeve
fc3b609ada Use general ratings rather than privacy filtered
The original system customized how a rating is displayed to every user
based on the privacy settings of the reviews and, relatedly, who the
user follows. This is cool, but the query is too complicated to load in
sessions, and the initial load, which isn't mitigated by caching, is too
much and causes timeouts for many users. Also the cache clearing wasn't
working correctly because I put in a wildcard, which does not work.
2022-05-30 08:42:48 -07:00
Mouse Reeve
4e3c346780
Merge pull request #2134 from bookwyrm-social/stopped-shelf-fixes
Stopped shelf fixes
2022-05-26 13:12:57 -07:00
Mouse Reeve
74925a379a Prettier 2022-05-26 12:54:31 -07:00
Mouse Reeve
4e0e6ed5a4 Tick javascript cache and version number 2022-05-26 12:49:04 -07:00
Mouse Reeve
09db4e48f4 Hide rather than remove current shelve list items 2022-05-26 12:46:34 -07:00
Mouse Reeve
c5f5d4d994 Only show "stop" option when a book is in progress 2022-05-26 12:27:44 -07:00
Mouse Reeve
4905652e22 Handle stopped reading special case in javascript
This should be refactored, but maybe not today
2022-05-26 12:23:13 -07:00
Mouse Reeve
4c5d2570ab Save and display stopped date in readthrough 2022-05-26 11:53:33 -07:00
Mouse Reeve
dfe0656eb4 Typo fix 2022-05-26 11:38:36 -07:00
Mouse Reeve
375c5a8789 Adds stopped date separate from finish date on readthrough 2022-05-26 11:36:37 -07:00
Mouse Reeve
1f6fbd8d29 Fixes stopped reading button logic
The stopped state is similar to finished
2022-05-26 11:28:54 -07:00
Mouse Reeve
9b4a498661 Don't show a button for the shelf a book is currently on
This will lead to nonsensical modal states
2022-05-26 11:19:49 -07:00
Mouse Reeve
92dbfec5f8 Adds status header for stopped reading statuses 2022-05-26 11:10:14 -07:00
Mouse Reeve
6848616ff1 Fixes reading status field in stop modal
The value of the reading status needs to match one of the database
options for `reading_status` in the `Comment` model
2022-05-26 11:09:11 -07:00
Mouse Reeve
007751c8cb Adds error logging to status views 2022-05-26 10:58:11 -07:00
Mouse Reeve
23c6019340 Adds merge migration 2022-05-26 10:23:32 -07:00
Mouse Reeve
77a7dfa924
Merge pull request #2133 from bookwyrm-social/activitypub-connection-erorr
Don't throw an error when unable to connect to remote data
2022-05-26 10:12:18 -07:00
Mouse Reeve
88b2cffcf2
Merge pull request #2035 from bookwyrm-social/stopped-shelf
Stopped shelf
2022-05-26 10:11:32 -07:00
Mouse Reeve
9d275db322 Updates ignore boost logic that no longer produces errors 2022-05-26 09:57:39 -07:00