Commit graph

1208 commits

Author SHA1 Message Date
kim
58af95a1d5
[chore] bump go-byteutil v1.2.0 -> v1.3.0 (#3356)
* bump go-byteutil v1.2.0 -> v1.3.0 which has safer (as in long-term API consistency) byte <-> string conversions

* fix test relying on byteutil exported type no longer existing
2024-09-26 19:50:08 +00:00
kim
53ee6aef08
[bugfix] s3 media uploaded without content-type (#3353)
* update go-storage dependency, for S3Storage manually call PutObject() so we can set content-type

* update calls to PutFile() to include the contentType
2024-09-26 14:43:10 +02:00
kim
b0fbc327f0
[chore] reduce number admin process workers (#3354) 2024-09-26 09:40:49 +02:00
tobi
713e73300c
[bugfix] Fix incorrect reply shown in interaction request (#3344) 2024-09-24 17:28:46 +00:00
kim
dc4059e9a2
[bugfix] add support for media with rotation contained in stream side data (#3335)
* add support for media with embedded rotation data in stream side data list

* *grumble grumble* linter
2024-09-23 15:13:01 +00:00
tobi
5bd6ad68e6
[bugfix/email] Don't use plainAuth when no smtp username/password provided (#3332)
* Do not use plainAuth when no user or password. Fixes #3320

* formatting

---------

Co-authored-by: Yonas Yanfa <yonas.y@gmail.com>
2024-09-23 16:07:13 +02:00
tobi
1ce854358d
[feature] Show info for pending replies, allow implicit accept of pending replies (#3322)
* [feature] Allow implicit accept of pending replies

* update wording
2024-09-23 14:42:19 +02:00
kim
2f13b72e2e
[chore] add nometrics build tagging to metrics API endpoint (#3331)
* add nometrics API endpoint code, to ensure prometheus NEVER compiled in unless wanted

* whoops, fix build tagging
2024-09-23 12:01:12 +00:00
kim
4592e29087
[chore] local instance count query caching, improved status context endpoint logging, don't log ErrHideStatus when timelining (#3330)
* ensure that errors checking status visibility / converting aren't dropped

* add some more context to error messages

* include calling function name in log entries

* don't error on timelining hidden status

* further code to ignore statusfilter.ErrHideStatus type errors

* remove unused error type

* add local instance status / domain / user counts

* add checks for localhost

* rename from InstanceCounts to LocalInstance

* improved code comment
2024-09-23 11:53:42 +00:00
kim
964262b169
[chore] header filter improvements (#3329)
* add error message to gin context on header blocked or not allowed

* remove the unused header filter tracking code (leaving OTEL TODOs in place)

* appease the linter
2024-09-23 11:36:56 +00:00
kim
77b095a8c3
[chore] ensure consistent caller name fetching regardless of compiler inlining (#3323)
* move logging levels into log package itself

* ensure inconsistent inlining doesn't mess with log calling function name

* remove unused global variable

* fix log level
2024-09-20 13:30:33 +00:00
tobi
c378ad2bb3
[feature] Allow users to submit interaction_policy on new statuses (#3314)
* [feature] Parse `interaction_policy` on status submission

* beep boop

* swagger? i barely know er
2024-09-18 16:35:35 +00:00
tobi
f819229988
[chore] make csv export ordering determinate (#3318) 2024-09-18 10:23:28 +00:00
kim
8effc77788
[chore] improved federatingdb logging in cases of unknown iri / types (#3313)
* improved federatingdb logging in cases of unknown iri / types, add new log methods

* whoops; forgot to wrap log argument in serialize{} !

* use debug instead of warn level

* switch last entry to Debug
2024-09-17 21:35:47 +02:00
tobi
e337aa83b8
[bugfix] Fix pending approval check (#3316) 2024-09-17 21:35:13 +02:00
tobi
95614fb298
[bugfix] Fix incorrect policy value parsing (#3315) 2024-09-17 21:22:45 +02:00
tobi
4bd5e68b2b
[bugfix] Ensure id set on outgoing Reject + Accept (#3312) 2024-09-16 22:41:04 +02:00
tobi
d4d6631435
[bugfix] Be more lenient when parsing mastodown following.csv (#3311)
* [bugfix] Be more lenient when parsing mastodown following.csv

* use follow.Notify
2024-09-16 20:39:15 +02:00
kim
84279f6a6a
[performance] cache more database calls, reduce required database calls overall (#3290)
* improvements to caching for lists and relationship to accounts / follows

* fix nil panic in AddToList()

* ensure list related caches are correctly invalidated

* ensure returned ID lists are ordered correctly

* bump go-structr to v0.8.9 (returns early if zero uncached keys to be loaded)

* remove zero checks in uncached key load functions (go-structr now handles this)

* fix issues after rebase on upstream/main

* update the expected return order of CSV exports (since list entries are now down by entry creation date)

* rename some funcs, allow deleting list entries for multiple follow IDs at a time, fix up more tests

* use returning statements on delete to get cache invalidation info

* fixes to recent database delete changes

* fix broken list entries delete sql

* remove unused db function

* update remainder of delete functions to behave in similar way, some other small tweaks

* fix delete user sql, allow returning on err no entries

* uncomment + fix list database tests

* update remaining list tests

* update envparsing test

* add comments to each specific key being invalidated

* add more cache invalidation explanatory comments

* whoops; actually delete poll votes from database in the DeletePollByID() func

* remove added but-commented-out field

* improved comment regarding paging being disabled

* make cache invalidation comments match what's actually happening

* fix up delete query comments to match what is happening

* rename function to read a bit better

* don't use ErrNoEntries on delete when not needed (it's only needed for a RETURNING call)

* update function name in test

* move list exclusivity check to AFTER eligibility check. use log.Panic() instead of panic()

* use the poll_id column in poll_votes for selecting votes in poll ID

* fix function name
2024-09-16 16:46:09 +00:00
tobi
0567b319c6
[chore] Refactor federatingDB.Undo, avoid 500 errors on Undo Like (#3310) 2024-09-16 15:49:40 +00:00
tobi
71261c62c2
[chore] Reject replies to rejected replies (#3291)
* [chore] Reject replies to rejected replies

* tweak

* don't set URI for implicit Rejects
2024-09-16 14:08:42 +02:00
tobi
efd1a4f717
[bugfix] Use better plaintext representation of status for filtering (#3301)
* [bugfix] Use better plaintext representation of status for filtering

* add new deps to readme

* lint

* update tests

* update regexes

* address review comments

* remove now unused xxhash

* whoops, wrong logger

* Merge branch 'main' into status_filtering_bugfix

* put cache in caches struct

* pain
2024-09-16 14:00:23 +02:00
kim
6dd936fbe1
[bugfix/chore] Always set the status sensitive if media + content-warning present (#3308)
* always set the status sensitive flag to true if it has a content-warning with media

* whoops use attachment ids instead of mention ids ... 🤦
2024-09-16 13:38:12 +02:00
Vyr Cossont
f3311d6273
[bugfix] Hoist filterable text field extraction out of loop (#3297)
This doesn't change within the loop, so there's no point in repeating it.
2024-09-15 10:42:04 +02:00
Daenney
4fa0d41202
[chore] Update go-sqlite3 to 0.18.3 (#3295)
* [chore] Update go-sqlite3 to 0.18.3

* [chore] Fix getting the sqlite3.Conn
2024-09-14 16:36:25 +02:00
tobi
20fe430ef9
[bugfix] Update home timeline query to ignore exclusive list entries (#3289)
* [bugfix] Update home timeline query to ignore exclusive list entries

* a
2024-09-11 10:55:25 +00:00
tobi
307d98e386
[feature] Process Reject of interaction via fedi API, put rejected statuses in the "sin bin" 😈 (#3271)
* [feature] Process `Reject` of interaction via fedi API, put rejected statuses in the "sin bin"

* update test

* move nil check back to `rejectStatusIRI`
2024-09-10 12:34:49 +00:00
kim
3254ef1923
[chore] status dereferencing improvements (#3255)
* search for mentions also by username,domain in status deref, handle deleted statuses in enrichStatusSafely()

* return d.enrichStatusSafely() directly
2024-09-10 12:33:32 +00:00
Vyr Cossont
540edef0c2
[feature] Implement exclusive lists (#3280)
Fixes #2616
2024-09-09 15:56:58 -07:00
tobi
5543fd5340
[feature/frontend] Add options to include Unlisted posts or hide all posts (#3272)
* [feature/frontend] Add options to include Unlisted posts or hide all posts

* finish up

* swagger

* move invalidate call into bundb package, avoid invalidating if not necessary

* rename show_web_statuses => web_visibility

* don't use ptr for webvisibility

* last bits
2024-09-09 18:07:25 +02:00
Vyr Cossont
100d660797
[bugfix] Swagger: add missing filter_action param for v2 filter PUT (#3281) 2024-09-09 09:52:49 +02:00
tobi
b17010cf17
[bugfix] Fix temp table deletion causing runaway allocations (#3278)
* [bugfix] Fix temp table deletion causing runaway allocations

* move some vars around

* small fixes

* rely on conn max age to recycle temp tables

* fackin' ell m8
2024-09-08 16:14:56 +02:00
Daenney
0e2ee857ac
[chore] More AI blocking (#3273) 2024-09-07 17:21:36 +02:00
kim
6ee540a503
fix new processing media / emoji not being added to dereferencer maps (#3269) 2024-09-03 13:30:41 +00:00
tobi
7b7659f1fa
[chore/performance] Further reduce nil uncached queries (#3267)
* [chore/performance] Further reduce nil uncached queries

* more checks
2024-09-02 18:15:12 +02:00
tobi
0560c5ce89
[chore] Don't try to select zero uncached filters (#3266) 2024-09-02 17:13:54 +02:00
tobi
25a815a8a4
[chore/performance] Avoid unnecessary "uncached" queries (#3265)
* [chore/performance] Avoid unnecessary "uncached" queries

* go fmt
2024-09-02 14:00:17 +02:00
kim
0a1555521d
[performance] use single-threaded image transforms (#3252)
* use single-threaded image resizing in native code so we have more control over goroutines

* implement parallel-free versions of image transform functions also

* remove debug code
2024-08-31 10:41:38 +02:00
tobi
88a81fbcaf
[chore] Close copied request body in SignDelivery (#3254) 2024-08-30 17:00:55 +00:00
tobi
cd93a5baf3
[security] Implement allowFiles fs for better isolation of ffmpeg / ffprobe (#3251)
* [chore] Implement readOneFile fs

* further isolation

* remove fmt call

* tweaks
2024-08-30 14:03:59 +02:00
tobi
e10aa76612
[chore] Only call imaging.Resize when necessary, use even tinier blurhashes (#3247)
* [chore] Use `imaging.Fit`, use even tinier blurhashes

* avoid calling resize if not necessary

* update blurhashes + thumb
2024-08-29 17:43:14 +02:00
Daenney
277b043633
[chore] Update robots.txt with more AI scrapers (#3248) 2024-08-29 17:42:48 +02:00
tobi
8a34e4c28f
[bugfix] Fix incorrect json-ld @context serialization (#3243) 2024-08-26 19:17:45 +02:00
tobi
db59ca36b1
[chore] Remove unused "env" module (#3235) 2024-08-25 15:37:20 +00:00
tobi
2bd31ab710
[chore] Tiny tweak to ApprovedByURI (#3234) 2024-08-25 15:44:08 +02:00
tobi
d3887bf6cc
[bugfix] Let prometheus client do its own compression handling (#3232) 2024-08-25 13:25:32 +02:00
tobi
bef0dfc66c
[bugfix] Carry-over ApprovedByURI to avoid marking already-approved remote statuses as pending approval (#3231)
* [bugfix] Carry-over pendingApproval + approvedByURI to avoid pending already-approved remote statuses

* don't carry over pending_approval, not necessary

* comment
2024-08-25 12:18:39 +02:00
tobi
da3fa2d4a2
[chore/bugfix] Fix missing insertion of preapproved interaction requests (#3228) 2024-08-24 12:17:55 +00:00
tobi
f23f04e0b1
[feature] Interaction requests client api + settings panel (#3215)
* [feature] Interaction requests client api + settings panel

* test accept / reject

* fmt

* don't pin rejected interaction

* use single db model for interaction accept, reject, and request

* swaggor

* env sharting

* append errors

* remove ErrNoEntries checks

* change intReqID to reqID

* rename "pend" to "request"

* markIntsPending -> mark interactionsPending

* use log instead of returning error when rejecting interaction

* empty migration

* jolly renaming

* make interactionURI unique again

* swag grr

* remove unnecessary locks

* invalidate as last step
2024-08-24 11:49:37 +02:00
kim
8e5a72ac5c
[performance] ffmpeg ffprobe wrapper improvements (#3225)
* use a single instance of wazero runtime and compiled modules

* remove test output 🤦

* undo process-{media,emoji} changes

* update test runner to include wazero compilation cache

* sign drone.yml

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-23 17:15:35 +02:00