The existing polling code had a few problems:
* It started the timer for a new request when the first request was
sent, rather than when a response was received.
* It increased the delay regardless of whether the response was a
success or a failure.
This commit changes it to a more standard exponential backoff system,
where it starts with a 5 minute ± 30 second delay, and uses that same
delay until it hits an error, at which point the delay is increased by
10%. Once it receives a successful response again, the delay is reset to
the default.
I suspect this should be nicer on the server, since it avoids the
initial sending of many requests. After about half an hour of leaving
the page open, the request rate for this new code will be higher than
that of the old code, so it's possible that this may cause problems, but
I think that a five-minute request frequency should be pretty reasonable.
We should store hashtags case-sensitive, but ensures that an existing
hashtag with different case are found and re-used. for example,
an existing #BookWyrm hashtag will be found and used even if the
status content is using #bookwyrm.
Since the status content already contains rendered HTML when we receive an
ActivityPub inbox message it contains links to the mentioned hashtags on the
originating instance.
To fix this on the receiving instance we need to post-process the status content
after successfully storing the status and its many-to-many fields (the one we're
is the `mention_hashtags`). Post-processing means that we run a regex against the
content to find the anchor tags linking to the originating hashtag and replace the
`href` attribute with the URL to the hashtag page on the receiving (local) instance.
This ensures that when an existing hashtag comes in through ActivityPub federation,
it correctly finds the local one, instead of creating duplicate hashtags.
This adds a new storage backend for Azure BlobStorage, I'm running this patch in Azure Kubernetes
hosting all my assets in Azure BlobStorage and it's been stable enough I thought I should commit
this back upstream.
This was suggested on Matrix a while ago but I only found the time now to move forward with it.
Signed-off-by: André Jaenisch <andre.jaenisch@posteo.de>
Expand TOTP validity window
This changes the default window to allow 2 codes (60 seconds) on either side. Admins can change this by setting a different `TWO_FACTOR_LOGIN_VALIDITY_WINDOW` value in `.env`
I think this will go a long way to solve the federation delay problems
we're seeing on b.s. I'm not sure at what point adding more queues will
create more problems than it solves, but I do think in this case the
queues are out of balance and moving broadcasts (which are the most
common type of `medium_priority` task at the moment) to their own queue
will be an improvement.
When an inbox activity comes in from another fediverse instance, the
behavior prior to this commit was always to immediately give a 200
response to the external server and then create a celery activity
(usually in the MEDIUM_PRIORITY queue) to complete it.
Instead, this would receive a request and try to complete it without
making any http requests (which would make the request take too long to
process). If an external request is required to complete the activity, a
task is created and added to the queue.
Ideally, this will cause some tasks to happen very promptly, and reduce
the load on celery, which would help queued tasks happen more quickly as
well.
One downside is that this will make completing http requests from
external servers slowing (since it's doing a bunch of thinking before
responding).
The migration uses `RESTRICT` instead of `PROTECT`, which is both more
correct, but also those values need to be identical, otherwise Django
thinks that there's a migration missing and will refuse to apply any
new migrations.
