* Implement conversations API
* Sort and page conversations by last status ID
* Appease linter
* Fix deleting conversations and statuses
* Refactor to make migrations automatic
* Lint
* Update tests post-merge
* Fixes from live-fire testing
* Linter caught a format problem
* Refactor tests, fix cache
* Negative test for non-DMs
* Run conversations advanced migration on testrig startup as well as regular server startup
* Document (lack of) side effects of API method for deleting a conversation
* Make not-found check less nested for readability
* Rename PutConversation to UpsertConversation
* Use util.Ptr instead of IIFE
* Reduce cache used by conversations
* Remove unnecessary TableExpr/ColumnExpr
* Use struct tags for both unique constraints on Conversation
* Make it clear how paging with GetDirectStatusIDsBatch should be used
* Let conversation paging skip conversations it can't render
* Use Bun NewDropTable
* Convert delete raw query to Bun
* Convert update raw query to Bun
* Convert latestConversationStatusesTempTable raw query partially to Bun
* Convert conversationStatusesTempTable raw query partially to Bun
* Rename field used to store result of MaxDirectStatusID
* Move advanced migrations to their own tiny processor
* Catch up util function name with main
* Remove json.… wrappers
* Remove redundant check
* Combine error checks
* Replace map with slice of structs
* Address processor/type converter comments
- Add context info for errors
- Extract some common processor code into shared methods
- Move conversation eligibility check ahead of populating conversation
* Add error context when dropping temp tables
* add flac support to the ffprobe format/stream -> filetype parser
* also add audio/flac for flac (not just x-flac)
* update tests
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* [feature/frontend] Respect `prefers-reduced-motion` for avatars, headers, and emojis
* go fmt
* fix tests
* use static version of instance thumbnail when appropriate
* use prefers-reduced-motion
* simplify account conversion a bit
* fix c&p error
* update to use webp for thumbnails
* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)
* update to use yuva colorspace, and use thumbnail=n=10 to select frame
* fix missing comma in ffmpeg args
* add links to appropriate ffmpeg docs
* update tests
* add file size tests for thumbnails
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* don't set emoji / media image paths on failed download, migrate FileType from string to integer
* fix incorrect uses of util.PtrOr, fix returned frontend media
* fix migration not setting arguments correctly in where clause
* fix not providing default with not null column
* whoops
* ensure a default gets set for media attachment file type
* remove the exclusive flag from writing files in disk storage
* rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match
* slight wording changes
* use singular / plural word forms (no parentheses), is better for screen readers
* update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests
* store first instance in ffmpeg wasm pool, fill remaining with closed instances
* [feature] Allow users to set default interaction policies
* use vars for default policies
* avoid some code repetition
* unfuck form binding
* avoid bonkers loop
* beep boop
* put policyValsToAPIPolicyVals in separate function
* don't bother with slices.Grow
* oops
* add more supported file types to our media processor that ffmpeg supports, update supported mime type lists
* add code comments to the supported mime types slice
* don't check for zero value string, just parse
* remove some unneeded consts which make the code a bit harder to read
* fix test expected instance media mime types, use compact ffprobe json, simple media processing by type
* final tweaks to media processing code
* don't use safe divide where we don't need to
* [feature/frontend] Audio player for audio media types
* use video preview images for previews instead of video itself
* don't preload
* update tests for new zork status
* collapse media gallery into single row when small
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg
* specify the video codec to use when generating static image from emoji
* update go-storage library (fixes incompatibility after updating go-iotools)
* maintain image aspect ratio when generating a thumbnail for it
* update readme to show go-ffmpreg
* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility
* remove extra debug from error message
* fix up incorrect function signatures
* update PutFile to just use regular file copy, as changes are file is on separate partition
* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe
* update more tests, add more code comments
* add utilities to generate processed emoji / media outputs
* fix remaining tests
* add test for opus media file, add license header to utility cmds
* limit the number of concurrently available ffmpeg / ffprobe instances
* reduce number of instances
* further reduce number of instances
* fix envparsing test with configuration variables
* update docs and configuration with new media-{local,remote}-max-size variables
* start updating media manager interface ready for storing attachments / emoji right away
* store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load()
* remove now unused media workers
* fix tests and issues
* fix another test!
* fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues
* fix more tests
* fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis
* whoops, rebase issue
* remove kim's whacky experiments
* do some reshuffling, ensure emoji uri gets set
* ensure marked as not cached on cleanup
* tweaks to media / emoji processing to handle context canceled better
* ensure newly fetched emojis actually get set in returned slice
* use different varnames to be a bit more obvious
* move emoji refresh rate limiting to dereferencer
* add exported dereferencer functions for remote media, use these for recaching in processor
* add check for nil attachment in updateAttachment()
* remove unused emoji and media fields + columns
* see previous commit
* fix old migrations expecting image_updated_at to exists (from copies of old models)
* remove freshness checking code (seems to be broken...)
* fix error arg causing nil ptr exception
* finish documentating functions with comments, slight tweaks to media / emoji deref error logic
* remove some extra unneeded boolean checking
* finish writing documentation (code comments) for exported media manager methods
* undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot
* move doesColumnExist() to util.go in migrations package
* [bugfix] Deref stats async, allow peek if handshaking
* don't return totalItems when handshaking or hiding collections
* use GetLimit()
* use StubAccountStats
* fix possible infinite recursion if moved accounts are self-referential
* adds a defensive check for a boost being a boost of a boost wrapper
* add checks on input for a boost of a boost
* remove unnecessary check
* add protections on account move to prevent move recursion loops
* separate status conversion without boost logic into separate function to remove risk of recursion
* move boost check to boost function itself
* formatting
* use error 422 instead of 500
* use gtserror not standard errors package for error creation
* User muting
* Address review feedback
* Rename uniqueness constraint on user_mutes to match convention
* Remove unused account_id from where clause
* Add UserMute to NewTestDB
* Update test/envparsing.sh with new and fixed cache stuff
* Address tobi's review comments
* Make compiledUserMuteListEntry.expired consistent with UserMute.Expired
* Make sure mute_expires_at is serialized as an explicit null for indefinite mutes
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* [feature] Email change
* frontend stuff for changing email
* docs
* tests etc
* differentiate more clearly between local user+account and account
* populate user
* do not uncache status / emoji media if attached status is bookmarked
* add status bookmark and bookmark IDs caches
* update status bookmark tests
* move IsStatusBookmarkedBy() to StatusBookmark{} interface, rely on cache
* fix envparsing.sh test
* Use correct entity name
* We support server-side filters now
* Document filter v1 methods that can throw a 409
* Validate v1 filter phrase as filter title
* Always check v1 filter API status codes in tests
* Document keyword minimum requirement on filter API v1
* Make it possible to specify filter keyword update columns per filter keyword
* Implement v2 filter API
* Fix lint and tests
* Update Swagger spec
* Fix filter update test
* Update Swagger spec *correctly*
* Update actual files Swagger spec was generated from
* Remove keywords_attributes and statuses_attributes
* Add test for serialization of empty filter
* More helpful messages when object is owned by wrong account
* Implement profile API
This Mastodon 4.2 extension provides capabilities missing from the existing Mastodon account update API: deleting an account's avatar or header.
See: https://docs.joinmastodon.org/methods/profile/
* Move profile media methods to media processor
* Remove check for moved account
* Remove dead code
* Filter statuses when converting to frontend representation
* status.filtered is an array
* Make matching case-insensitive
* Remove TODOs that don't need to be done now
* Add missing filter check for notification
* lint: rename ErrHideStatus
* APIFilterActionToFilterAction not used yet
* swaggerino docseroni
* Address review comments
* Add apimodel.FilterActionNone
---------
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* [feature/performance] Store account stats in separate table, get stats from remote
* test account stats
* add some missing increment / decrement calls
* change stats function signatures
* rejig logging a bit
* use lock when updating stats
* [feature] User sign-up form and admin notifs
* add chosen + filtered languages to migration
* remove stray comment
* chosen languages schmosen schmanguages
* proper error on local account missing
* [feature] User-selectable preset themes
* docs, more theme stuff
* lint, tests
* fix css name
* correct some little issues
* add another theme
* fix poll background
* okay last theme i swear
* make retrieval of apimodel themes more conventional
* preallocate stylesheet slices
* Implement client-side v1 filters
* Exclude linter false positives
* Update test/envparsing.sh
* Fix minor Swagger, style, and Bun usage issues
* Regenerate Swagger
* De-generify filter keywords
* Remove updating filter statuses
This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.
* Add a test for filter statuses specifically
* De-generify filter statuses
* Inline FilterEntry
* Use vertical style for Bun operations consistently
* Add comment on Filter DB interface
* Remove GoLand linter control comments
Our existing linters should catch these, or they don't matter very much
* Reduce memory ratio for filters
* update activity library so dereferencer returns full response and checks *final* link to allow for redirects
* temporarily add bodged fixed library
* remove unused code
* update getAccountFeatured() to use dereferenceCollectionPage()
* make sure to release map
* perform a 2nd decode to ensure reader is empty after primary decode
* add comment explaining choice of using Decode() instead of Unmarshal()
* update embedded activity library to latest matching https://github.com/superseriousbusiness/activity/pull/21
* add checks to look for changed URI and re-check database if redirected
* update max iteration count to 512, add checks during dereferenceAncestors() for indirect URLs
* remove doubled-up code
* fix use of status instead of current
* use URIs for checking equality for security
* use the latest known URI for boost_of_uri in case original was an indirect
* add dereferenceCollection() function for dereferenceAccountFeatured()
* pull in latest github.com/superseriousbusiness/activity version (and remove the bodge!!)
* fix typo in code comments
* update decodeType() to accept a readcloser and handle body closing
* switch to checking using BoostOfID and add note why not using BoostOfURI
* ensure InReplyTo gets unset when deleting status parent in case currently stubbed
* add tests for Collection and CollectionPage iterators
* [feature] serdes for moved/also_known_as
* document `alsoKnownAs` and `movedTo` properties
* only implicitly populate AKA uris from DB for local accounts
* don't let remotes store more than 20 AKA uris to avoid shenanigans
* tidy up account, status, webfingering logic a wee bit
* go fmt
* invert published check
* alter resp initialization
* get Published from account in typeutils
* don't instantiate error for no darn good reason
* shadow err
* don't repeat error codes in wrapped errors
* don't wrap error unnecessarily
* rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial
* use actual go-structr release version (not just commit hash)
* revert go toolchain changes (damn you go for auto changing this)
* fix go mod woes
* ensure %w is used in calls to errs.Appendf()
* fix error checking
* fix possible panic
* remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop
* fix copy-paste artifact... 😇
* fix all comment copy-paste artifacts
* remove dropID() function, now we can just use slices.DeleteFunc()
* use util.Deduplicate() instead of collate(), move collate to util
* move orderByIDs() to util package and "generify"
* add a util.DeleteIf() function, use this to delete entries on failed population
* use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol)
* add note about how collate differs from deduplicate
