* start replacing client + federator + media workers with new worker + queue types
* refactor federatingDB.Delete(), drop queued messages when deleting account / status
* move all queue purging to the processor workers
* undo toolchain updates
* code comments, ensure dereferencer worker pool gets started
* update gruf libraries in readme
* start the job scheduler separately to the worker pools
* reshuffle ordering or server.go + remove duplicate worker start / stop
* update go-list version
* fix vendoring
* move queue invalidation to before wipeing / deletion, to ensure queued work not dropped
* add logging to worker processing functions in testrig, don't start workers in unexpected places
* update go-structr to add (+then rely on) QueueCtx{} type
* ensure more worker pools get started properly in tests
* fix remaining broken tests relying on worker queue logic
* fix account test suite queue popping logic, ensure noop workers do not pull from queue
* move back accidentally shuffled account deletion order
* ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete()
* silently drop deletes from accounts not permitted to
* don't warn log on forwarded deletes
* make if else clauses easier to parse
* use getFederatorMsg()
* improved code comment
* improved code comment re: requesting account delete checks
* remove boolean result from worker start / stop since false = already running or already stopped
* remove optional passed-in http.client
* remove worker starting from the admin CLI commands (we don't need to handle side-effects)
* update prune cli to start scheduler but not all of the workers
* fix rebase issues
* remove redundant return statements
* i'm sorry sir linter
cmp.Or was introduced in Go 1.22 and picks the first value that's not
the zero value for the type. For a string, the zero value is the empty
string, which is what os.Getenv will return if the environment variable
is not set. That then results in "error" being returned instead.
This allows loading an environment variable with a default without
having to do the check and write out the conditional.
* [feature] User sign-up form and admin notifs
* add chosen + filtered languages to migration
* remove stray comment
* chosen languages schmosen schmanguages
* proper error on local account missing
* add delivery worker type that pulls from queue to httpclient package
* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller
* hook up queue deletion logic
* support deleting queued http requests by target ID
* don't index APRequest by hostname in the queue
* use gorun
* use the original context's values when wrapping msg type as delivery{}
* actually log in the AP delivery worker ...
* add uncommitted changes
* use errors.AsV2()
* use errorsv2.AsV2()
* finish adding some code comments, add bad host handling to delivery workers
* slightly tweak deliveryworkerpool API, use advanced sender multiplier
* remove PopCtx() method, let others instead rely on Wait()
* shuffle things around to move delivery stuff into transport/ subpkg
* remove dead code
* formatting
* validate request before queueing for delivery
* finish adding code comments, fix up backoff code
* finish adding more code comments
* clamp minimum no. senders to 1
* add start/stop logging to delivery worker, some slight changes
* remove double logging
* use worker ptrs
* expose the embedded log fields in httpclient.Request{}
* ensure request context values are preserved when updating ctx
* add delivery worker tests
* fix linter issues
* ensure delivery worker gets inited in testrig
* fix tests to delivering messages to check worker delivery queue
* update error type to use ptr instead of value receiver
* fix test calling Workers{}.Start() instead of testrig.StartWorkers()
* update docs for advanced-sender-multiplier
* update to the latest activity library version
* add comment about not using httptest.Server{}
* [chore] Log less output on failed test
This changes the testrig log level to be error by default instead of
info. This makes test failures a lot easier to read, as we don't have
the parade of info logs for each failure to scroll through.
It speeds up the test suite by a couple of seconds since we need
to buffer and flush a lot less messages. On a clean run, so no test
failures, it's about a 3s difference on my machine. Depending on the
amount of test failures, total time saved can vary.
This also introduces a GTS_TESTRIG_LOG_LEVEL environment variable that
we explicitly check for, making it easy to override the log level should
we have a need for it. This would be primarily for running locally, and
not so much as part of go test.
Lastly, it updates the syslog tests to use log.Error because if the log
level is set to error but we call log.Info no message is emitted and we
hang indefinitely on the channel read.
* [chore] Rename the testrig log level env var
* Implement client-side v1 filters
* Exclude linter false positives
* Update test/envparsing.sh
* Fix minor Swagger, style, and Bun usage issues
* Regenerate Swagger
* De-generify filter keywords
* Remove updating filter statuses
This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.
* Add a test for filter statuses specifically
* De-generify filter statuses
* Inline FilterEntry
* Use vertical style for Bun operations consistently
* Add comment on Filter DB interface
* Remove GoLand linter control comments
Our existing linters should catch these, or they don't matter very much
* Reduce memory ratio for filters
* add more stringent checks for remote status permissibility
* add check for inreplyto of a remote status being a boost
* do not permit inReplyTo boost wrapper statuses
* change comment wording
* fix calls to NewFederator()
* add code comments for NotPermitted() and SetNotPermitted()
* improve comment
* check that existing != nil before attempting delete
* ensure replying account isn't suspended
* use a debug log instead of info. check for boost using ID
* shorten log string length. make info level
* add note that replying to boost wrapper status shouldn't be able to happen anyways
* update to use onFail() function
This switches the documentation to use human sizes, so 50MiB instead of
an integer number of bytes. This makes it much easier to understand what
values are set, and less likely to accidentally set the wrong value.
* [feature] serdes for moved/also_known_as
* document `alsoKnownAs` and `movedTo` properties
* only implicitly populate AKA uris from DB for local accounts
* don't let remotes store more than 20 AKA uris to avoid shenanigans
* rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial
* use actual go-structr release version (not just commit hash)
* revert go toolchain changes (damn you go for auto changing this)
* fix go mod woes
* ensure %w is used in calls to errs.Appendf()
* fix error checking
* fix possible panic
* remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop
* fix copy-paste artifact... 😇
* fix all comment copy-paste artifacts
* remove dropID() function, now we can just use slices.DeleteFunc()
* use util.Deduplicate() instead of collate(), move collate to util
* move orderByIDs() to util package and "generify"
* add a util.DeleteIf() function, use this to delete entries on failed population
* use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol)
* add note about how collate differs from deduplicate
* [feature] Account alias / move API + db models
* go fmt
* fix little cherry-pick issues
* update error checking, formatting
* add and use new util functions to simplify alias logic
* [chore] Refactor HTML templates and CSS
* eslint
* ignore "Local"
* rss tests
* fiddle with OG just a tiny bit
* dick around with polls a bit more so SR stops saying "clickable"
* remove break
* oh lord
* don't lazy load avatar
* fix ogmeta tests
* clean up some cruft
* catch remaining calls to c.HTML
* fix error rendering + stack overflow in tag
* allow templating attributes
* fix indent
* set aria-hidden on status complementary content, since it's already present in the label anyway
* tidy up templating calls a little
* try to make styling a bit more consistent + readable
* fix up some remaining CSS issues
* fix up reports
* [feature] Render polls nicely on the web view
* use figure for poll, other small tweaks
* reverse share + count (lines up better)
* poll options list entries
* fix up some remaining things
* update go text, include text/display
* [feature] Set instance langs, show post lang on frontend
* go fmt
* WebGet
* set language for whole article, don't use FA icon
* mention instance languages + other optional config vars
* little tweak
* put languages in config properly
* warn log language parse
* change some naming around
* tidy up validate a bit
* lint
* rename LanguageTmpl in template
* [bugfix] process account delete side effects in serial, not in parallel
* StartWorkers / StartNoopWorkers for tests
* undo testrig trace logging
* log errors instead of immediately returning
* deinterface router, start messing about with deadlines
* weeeee
* thanks linter (thinter)
* write Connection: close when timing out requests
* update wording
* don't replace req
* don't bother with fancy Cause functions (I'll use them one day...)
* add db models + functions for keeping track of threads
* give em the old linty testy
* create, remove, check mutes
* swagger
* testerino
* test mute/unmute via api
* add info log about new index creation
* thread + allow muting of any remote statuses that mention a local account
* IsStatusThreadMutedBy -> IsThreadMutedByAccount
* use common processing functions in status processor
* set = NULL
* favee!
* get rekt darlings, darlings get rekt
* testrig please, have mercy muy liege
* update typeconverter to use state structure
* deinterface the typeutils.TypeConverter -> typeutils.Converter
* finish copying over old type converter code comments
* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
* love like winter! wohoah, wohoah
* domain allow side effects
* tests! logging! unallow!
* document federation modes
* linty linterson
* test
* further adventures in documentation
* finish up domain block documentation (i think)
* change wording a wee little bit
* docs, example
* consolidate shared domainPermission code
* call mode once
* fetch federation mode within domain blocked func
* read domain perm import in streaming manner
* don't use pointer to slice for domain perms
* don't bother copying blocks + allows before deleting
* admonish!
* change wording just a scooch
* update docs
* [feature] Don't emit timestamp in log lines
When running gotosocial with a service manager like systemd, or a
container runtime, the associated log driver usually emits timestamps
itself. In those cases, having the extra timestamp from our own log
lines ends up being a bit noisy and when centrally ingesting logs is
duplicate information.
This introduces a configuration flag that allows disabling emitting the
timestamp. It's only wired up for "daemonised" processes, meaning server
and testrig.
* [chore] Add docs for log-timestamp
* [feature] Simplify timestamp handling
Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
* [chore] Less escaped double-quotes
* [chore] Fix help string
---------
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
* init instance rules database model, admin api
* expose instance rules in public instance api
* public /api/v1/instance/rules route
* GET ruleById
* createRule route
* createRule auth check
* updateRule
* deleteRule
* list rules on about page
* ruleGet auth
* add about page ids for anchors
* process and store adding violated rules to reports
* admin api models for instance rules
* instance rule edit frontend
* change rule inputs to textareas
* database fixes after rebase (#2124)
* remove unused imports
* fix db migration column name
* fix tests
* fix more tests
* fix postgres error with wrongly used Ident
* add some tests, fiddle with rule model a bit, fix postgres migration
* swagger docs
---------
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* Add/update some DB functions.
* move async workers into subprocessor
* rename FromFederator -> FromFediAPI
* update home timeline check to include check for current status first before moving to parent status
* change streamMap to pointer to mollify linter
* update followtoas func signature
* fix merge
* remove errant debug log
* don't use separate errs.Combine() check to wrap errs
* wrap parts of workers functionality in sub-structs
* populate report using new db funcs
* embed federator (tiny bit tidier)
* flesh out error msg, add continue(!)
* fix other error messages to be more specific
* better, nicer
* give parseURI util function a bit more util
* missing headers
* use pointers for subprocessors
* update go-fed
* do the things
* remove unused columns from tags
* update to latest lingo from main
* further tag shenanigans
* serve stub page at tag endpoint
* we did it lads
* tests, oh tests, ohhh tests, oh tests (doo doo doo doo)
* swagger docs
* document hashtag usage + federation
* instanceGet
* don't bother parsing tag href
* rename whereStartsWith -> whereStartsLike
* remove GetOrCreateTag
* dont cache status tag timelineability
