Commit graph

248 commits

Author SHA1 Message Date
Mouse Reeve 458b258ad5
Merge pull request #3007 from hughrun/signed-get
create instance user on instance creation
2023-10-02 09:52:59 -07:00
Hugh Rundle b69031c01a
formatting 2023-09-24 12:31:01 +10:00
Joeri de Ruiter a5cf912ae8 Fix some annotations 2023-09-13 09:22:53 +02:00
Joeri de Ruiter 8f8587f79d Set **kwargs type to Any for ActivityObject.__init__ 2023-08-21 13:09:42 +02:00
Joeri de Ruiter f07d7b02f1 Type annotations and related changes for bookwyrm.connectors 2023-07-28 17:43:32 +02:00
Joeri de Ruiter 0354e53eea Type annotations for utils 2023-07-23 20:50:44 +02:00
Wesley Aptekar-Cassels 3e78e398c0 Switch from priority queues to function-based queues
Fixes: #2907
2023-07-20 12:25:30 -04:00
Mouse Reeve 290b740392
Merge pull request #2805 from WesleyAC/stop-ignoring-task-results
Stop ignoring task results
2023-04-15 09:03:43 -07:00
Wesley Aptekar-Cassels e9f26b7d50
Merge pull request #2800 from WesleyAC/fix-activitypub-request-accept-header
Fix Accept header for requesting ActivityPub objects
2023-04-08 11:02:27 -04:00
Wesley Aptekar-Cassels 1048638e30 Stop ignoring task results
This is essentially a revert of 9cbff312a. The commit was at the advice
of the Celery docs for optimization, but I've since decided that the
downsides in terms of making things harder to debug (it makes Flower
nearly useless, for instance) are bigger than the upsides in performance
gain (which seem extremely small in practice, given how long our tasks
take, and the number of tasks we have).
2023-04-07 21:51:44 -04:00
Wesley Aptekar-Cassels 912d0a0149 Fix Accept header for requesting ActivityPub objects
This is the header described in the ActivityPub spec, which should fix
some federation problems with GoToSocial and potentially other picky
services.

Related: #2794, superseriousbusiness/gotosocial#1676
2023-04-07 05:57:13 -04:00
Wesley Aptekar-Cassels b574a12fff Pass allow_external_connections through ignore_activity
Previously, ignore_activity could unexpectedly make a outgoing HTTP
connection, leading to unwanted latency, particularly when called via
ActivityObject.to_model, which had the allow_external_connections
parameter already.

Related: #2717
2023-04-06 23:37:49 -04:00
Mouse Reeve 12af5992a3
Merge pull request #2524 from chdorner/feature/tag-support
Initial hashtag support
2023-03-12 16:37:39 -07:00
Wesley Aptekar-Cassels 9cbff312a5 Ignore Celery task results
Since we don't use the results of our Celery tasks (all of them return
None implicitly), it's prudent to set the ignore_result flag, for a
potential performance improvement. See the Celery docs for details [1].

We could do this with the global CELERY_IGNORE_RESULT setting, but it
offers more flexibility if we want to use task results in the future to
set it on a per-task basis.

[1]: https://docs.celeryq.dev/en/stable/userguide/tasks.html#ignore-results-you-don-t-want
2023-03-08 02:12:13 -05:00
Christof Dorner 9ca9883e0b Enable finding existing hashtags case-insensitive
We should store hashtags case-sensitive, but ensures that an existing
hashtag with different case are found and re-used. for example,
an existing #BookWyrm hashtag will be found and used even if the
status content is using #bookwyrm.
2023-03-07 13:16:45 +01:00
Christof Dorner f3334b1550 Render hashtag links with data-mention="hashtag" attribute 2023-03-07 13:16:45 +01:00
Christof Dorner 276b255f32 Post-process status.content field to change hashtag URLs
Since the status content already contains rendered HTML when we receive an
ActivityPub inbox message it contains links to the mentioned hashtags on the
originating instance.

To fix this on the receiving instance we need to post-process the status content
after successfully storing the status and its many-to-many fields (the one we're
is the `mention_hashtags`). Post-processing means that we run a regex against the
content to find the anchor tags linking to the originating hashtag and replace the
`href` attribute with the URL to the hashtag page on the receiving (local) instance.
2023-03-07 13:16:45 +01:00
Christof Dorner 11640f986e Import hashtags from activitypub statuses 2023-03-07 12:59:31 +01:00
Christof Dorner c68304a99b Parse hashtags and store them in CreateStatus view 2023-03-07 12:55:36 +01:00
Mouse Reeve 779d2b0694 Attempt to complete inbox requests synchronously
When an inbox activity comes in from another fediverse instance, the
behavior prior to this commit was always to immediately give a 200
response to the external server and then create a celery activity
(usually in the MEDIUM_PRIORITY queue) to complete it.

Instead, this would receive a request and try to complete it without
making any http requests (which would make the request take too long to
process). If an external request is required to complete the activity, a
task is created and added to the queue.

Ideally, this will cause some tasks to happen very promptly, and reduce
the load on celery, which would help queued tasks happen more quickly as
well.

One downside is that this will make completing http requests from
external servers slowing (since it's doing a bunch of thinking before
responding).
2023-02-20 11:05:18 -08:00
Mouse Reeve f4de00088f
Merge branch 'main' into authorized-fetch 2023-01-26 06:22:03 -08:00
Hugh Rundle f0e1767bc9 black code 2023-01-22 16:10:30 +11:00
Hugh Rundle 803bba71a6 fix error handling
- when using raise_for_status we need to catch an HTTPError, not a ConnectionError
- simplify instance actor - use internal email address since it will never be used anyway, and make default username less likely to already be in use.
2023-01-22 15:59:19 +11:00
Hugh Rundle 317fa5cdfd black 2023-01-20 20:05:14 +11:00
Hugh Rundle e8452011f7 handle get_data exceptions better
Makes exception handling more precise, only raising status for 401s.

Also fixes a string pylint was complaining about.
2023-01-20 19:55:38 +11:00
Hugh Rundle f8c9df4aff pylint fixes 2023-01-20 18:20:18 +11:00
Hugh Rundle 4108238716 resolve SECURE_FETCH bugs
ERROR HANDLING FIXES

- use raise_for_status() to pass through response code
- handle exceptions where no response object is passed through

INSTANCE ACTOR

- models.User.objects.create_user function cannot take an ID
- allow instance admins to determine username and email for instance actor in settings.py
2023-01-20 16:32:17 +11:00
Hugh Rundle d66e2fe861 merge in latest changes 2023-01-18 14:12:01 +11:00
Jascha Urbach 54e0fdd5ae
Here we go again! 2023-01-15 09:41:39 +01:00
Mouse Reeve dd2b166382
Revert "Add Website field to author" 2023-01-11 19:21:40 -08:00
Andy Maloney e5539a6faf Fix spelling in various comments 2023-01-10 17:24:02 -05:00
Jascha Urbach 2d76595af6
Add Website field to author
Authors have Websites. that might be interesting for BookWyrm users. Now there is a field specify the website of an author and it is shown tu users under the wikipedia link

Adds max char for ISFDB ID (6) and ISNI (19 - 16 plus three -)
2022-12-28 15:41:48 +01:00
Jascha Urbach ac8b060d58
Isfdb ID for books and authors (#2482)
* New ID: Audible ASIN

Audible belongs to Amazon BUT they do not share the same IDs. The Audible ASIN of an audiobook is never the same as the Amazon ASIN.

Yeah, I know, Amazon is great. The fact that the ASIN is a good distinction for different works and editions bothers me more than I will ever be willing to admint.

* New ID "ISFDB"

Internet Speculative Ficiton Database ID for books and authors.
Links to the entry if set.

* Added aasin to test

Added aasin to test

* the answer expects more emptxy fields...
2022-12-11 11:33:33 -08:00
Fedor Indutny cdc833657f Fix serialization of JSON-LD subarrays
Properties like "tag" could be lists containing multiple subclasses of
ActivityObject. Make sure to serialize them recursively instead of
outputting them as they are, because otherwise we could get a bunch of
nulls in the resulting JSON and that wouldn't necessarily be a valid
JSON-LD object.

Fix: #2451
2022-12-04 18:12:39 -08:00
Mouse Reeve b37a4322de Change log level to info for connector exceptions
These errors in resolve_remote_id aren't really errors, they're
routine problems that we can expect from dealing with the outside world,
like a connection timeout, a server being down, a server being blocked,
et cetera. It's cluttering up the logs and causing unnecessary worry.
2022-11-17 12:35:19 -08:00
André Jaenisch 530d7de309
Use variable instead of string
Signed-off-by: André Jaenisch <andre.jaenisch@posteo.de>
2022-11-13 16:59:05 +01:00
Mouse Reeve 7eb2a82e5f
Merge branch 'main' into misc/add_signatures_to_requests_for_masto_compat 2022-07-26 15:46:18 -07:00
Mouse Reeve 77a7dfa924
Merge pull request #2133 from bookwyrm-social/activitypub-connection-erorr
Don't throw an error when unable to connect to remote data
2022-05-26 10:12:18 -07:00
Mouse Reeve 88b2cffcf2
Merge pull request #2035 from bookwyrm-social/stopped-shelf
Stopped shelf
2022-05-26 10:11:32 -07:00
Mouse Reeve 3e54a5f4a3 Python formatting 2022-05-26 09:00:45 -07:00
Mouse Reeve 0bfe1e9dfc Don't throw an error when unable to connect to remote data 2022-05-25 13:24:11 -07:00
Mouse Reeve b2775c5160 Check unsupported types before attempting to serialize 2022-05-16 10:21:54 -07:00
Mouse Reeve fd43b56d31 Fixes celery error encountering Article type activities 2022-05-16 10:17:21 -07:00
Mouse Reeve 819458e82a Improves error reporting on activitypub parser 2022-03-16 13:53:54 -07:00
Mouse Reeve d2dab0f2db
Merge branch 'main' into misc/add_signatures_to_requests_for_masto_compat 2022-03-01 10:20:39 -08:00
Mouse Reeve e90cb52f23 Add option to hide follows 2022-02-28 11:48:49 -08:00
Mouse Reeve a2c4dd4f9f Updates migration and database fields 2022-02-17 08:25:01 -08:00
Mouse Reeve 39691bed3a Merge branch 'main' into openlibrary-author-fields 2022-02-16 18:06:04 -08:00
Mouse Reeve c06817e9ff Python formatting 2022-02-09 12:23:01 -05:00
Mouse Reeve 6323b0e700 Use correct model type for federated links 2022-02-09 12:20:11 -05:00