This is still slow in some cases, despite #2723, so this information
should give useful data about how it could be optimized more.
This also adds some abstraction around getting the tracer, just to
follow the advice in the OpenTelemetry documentation not to use __name__
directly to set the tracer name. The advice is ignored in most of their
examples, so it probably doesn't matter, but IDK, seems reasonable to
try to follow it.
Related: #2720
This wasn't a problem in the past, since we only enabled automatic
instrumentation when this was set up, but it does cause errors when
trying to add manual instrumentation.
Many of these environment variables were probably not actually usable,
since they would be strings if set in the env file. Using the
typecasting functions fixes this, and generally shows the intention of
the code more clearly.
Splitting this into five separate queries avoids the large join that
prevents us from using indexes, and requires materializing to disk.
Fixes: #2157 (hopefully)
By default, Django doesn't run any context processors for server errors,
to make the error path as simple as possible. However, this has the
downside that our template does not load correctly. To fix this, I added
a custom 500 error handler, which will run the context processor.
Fixes: #2736
The queries as they previously existed required joining together 12
different tables, which is extremely expensive. Splitting it into four
queries means that the individual queries can effectively use the
indexes we have, and should be very fast no matter how many statuses are
in the database.
Removing the .distinct() call is fine, since we're adding them to a set
in Redis anyways, which will take care of the duplicates.
It's a bit ugly that we now make four separate calls to Redis (this
might result in things being slightly slower in cases where there are an
extremely small number of statuses), but doing things differently would
result in significantly more surgery to the existing code, so I've opted
to avoid that for the moment.
Fixes: #2725
This splits HomeStream.get_audience into two separate database queries,
in order to more effectively take advantage of the indexes we have.
Combining the user ID query and the user following query means that
Postgres isn't able to use the index we have on the userfollows table.
The query planner claims that the userfollows query should be about 20
times faster than it was previously, and the id query should take a
negligible amount of time, since it's selecting a single item by primary
key.
We don't need to worry about duplicates, since there is a constraint
preventing a user from following themself.
Fixes: #2720
Anywhere we have a user object, we can easily get the user ID in the
caller, and this will allow us more flexibility in the future to
implement optimizations that involve knowing a user ID without querying
the database for the user object.
The idea behind a streaming CSV export was to reduce the amount of
memory used, by avoiding building the entire CSV file in memory before
sending it to the client. However, it didn't work out this way in
practice: the query objects that were created to represent each line
caused Postgres to generate a very large (~200MB on bookwyrm.social)
temp file, not to mention the memory being used by the Query object
likely being similar to, if not larger than that used by the finalized
CSV row.
While we should in the long term run our CSV exports as a Celery task,
this change should allow CSV exports to work on large servers without
causing disk-space problems.
Fixes: #2157
get_data can return exceptions other than ConnectorException, and when
it does, we want to simply not show the update section, rather than
crashing.
Related: #2717
Since we don't use the results of our Celery tasks (all of them return
None implicitly), it's prudent to set the ignore_result flag, for a
potential performance improvement. See the Celery docs for details [1].
We could do this with the global CELERY_IGNORE_RESULT setting, but it
offers more flexibility if we want to use task results in the future to
set it on a per-task basis.
[1]: https://docs.celeryq.dev/en/stable/userguide/tasks.html#ignore-results-you-don-t-want
