Commit graph

91 commits

Author SHA1 Message Date
kim
23fc70f4e6
[feature] add support for receiving federated status edits (#3597)
* add support for extracting Updated field from Statusable implementers

* add support for status edits in the database, and update status dereferencer to handle them

* remove unused AdditionalInfo{}.CreatedAt

* remove unused AdditionalEmojiInfo{}.CreatedAt

* update new mention creation to use status.UpdatedAt

* remove mention.UpdatedAt, fixes related to NewULIDFromTime() change

* add migration to remove Mention{}.UpdatedAt field

* add migration to add the StatusEdit{} table

* start adding tests, add delete function for status edits

* add more of status edit migrations, fill in more of the necessary edit delete functionality

* remove unused function

* allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid`

* add StatusEdit{} test models

* fix new statusedits sql

* use model instead of table name

* actually remove the Mention.UpdatedAt field...

* fix tests now new models are added, add more status edit DB tests

* fix panic wording

* add test for deleting status edits

* don't automatically set `updated_at` field on updated statuses

* flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses

* remove media_attachments.updated_at column

* fix up more tests, further complete the dereferencer status edit tests

* update more status serialization tests not expecting 'updated' AS property

* gah!! json serialization tests!!

* undo some gtscontext wrapping changes

* more serialization test fixing 🥲

* more test fixing, ensure the edit.status_id field is actually set 🤦

* fix status edit test

* grrr linter

* add edited_at field to apimodel status

* remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...)

* ensure that status.updated_at always fits chronologically

* fix more serialization tests ...

* add more code comments

* fix envparsing

* update swagger file

* properly handle media description changes during status edits

* slight formatting tweak

* code comment
2024-12-05 13:35:07 +00:00
Victor Dyotte
9609c4550d
[feature] Add global instance CSS customization setting (#3352)
Allow instance admins to add custom CSS that will affect
every page of their instance.

This is done with a new CustomCSS instance setting that
works pretty much exactly like the Users CustomCSS property.
This custom CSS is then requested for every page load.
User styles/themes take precedence over this CSS.

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-12-02 12:24:48 +01:00
Jannis
9ace025da1
[bugfix] post counters should not include direct messages (#3554)
* [bugfix] post counters should not include direct messages #3504

The fix is relativly simple, it just adds a line to the relevant
function which excludes all private posts.

* Formating fix

* mb
2024-11-21 11:06:06 +01:00
kim
6f4cb2f14e
[bugfix] sets the max value placeholders to MaxInt32 instead of MaxInt (#3517)
* sets the max value placeholders to MaxInt32 instead of MaxInt

* update tests
2024-11-05 22:16:06 +00:00
kim
f3b2eca8b8
[feature] add support for hinting via api/v_/instance preferred image / video max sizes (#3505)
* add support for hinting via api/v_/instance endpoints a preferred image / video size limit

* fix tests expecting old default values
2024-11-04 15:00:10 +01:00
kim
8f288f1689
[bugfix] determine mime-type to use during ffprobe evaluation stage, don't bother checking against file extension (#3506)
* determine mime-type to use during ffprobe evaluation stage, don't bother rechecking by file extension

* set mjpeg content-type

* fix up tests expecting differing default values
2024-11-04 14:58:15 +01:00
kim
d8a83860bc
[bugfix] incorrect /api/v_/instance domain uri fields (#3477)
* update instance v1 / v2 endpoints to set uri / domain correctly (i.e. account domain)

* fix tests relying on old instance behaviour
2024-10-22 14:47:28 +00:00
tobi
8a93300ac4
[feature] Add image descriptions for default avatar + header; don't allow editing default desc (#3473) 2024-10-21 14:04:50 +02:00
tobi
602c858379
[chore] Thumbnail only first frame of animated media (#3448) 2024-10-18 15:44:08 +02:00
Markus Unterwaditzer
95a316236e
[bugfix] Account.last_status_at is a date, not datetime (#3419)
* [bugfix] Account.last_status_at is a date, not datetime

Fix #3418

* update swagger
2024-10-12 10:02:26 +02:00
kim
0a1555521d
[performance] use single-threaded image transforms (#3252)
* use single-threaded image resizing in native code so we have more control over goroutines

* implement parallel-free versions of image transform functions also

* remove debug code
2024-08-31 10:41:38 +02:00
tobi
e10aa76612
[chore] Only call imaging.Resize when necessary, use even tinier blurhashes (#3247)
* [chore] Use `imaging.Fit`, use even tinier blurhashes

* avoid calling resize if not necessary

* update blurhashes + thumb
2024-08-29 17:43:14 +02:00
kim
f77005128a
[performance] move thumbnail generation to go code where possible (#3183)
* wrap thumbnailing code to handle generation natively where possible

* more code comments!

* add even more code comments!

* add code comments about blurhash generation

* maintain image rotation if contained in exif data

* move rotation before resizing

* ensure pix_fmt actually selected by ffprobe, check for alpha layer with gifs

* use linear instead of nearest-neighbour for resizing

* work with image "orientation" instead of "rotation". use default 75% quality for both webp and jpeg generation

* add header to new file

* use thumb extension when getting thumb mime type

* update test models and tests with new media processing

* add suggested code comments

* add note about thumbnail filter count reducing memory usage
2024-08-08 17:12:13 +00:00
tobi
7b5917d6ae
[feature] Allow import of following and blocks via CSV (#3150)
* [feature] Import follows + blocks via settings panel

* test import follows
2024-08-02 12:41:46 +01:00
Vyr Cossont
fd837776e2
[feature] Implement Mastodon-compatible roles (#3136)
* Implement Mastodon-compatible roles

- `Account.role` should only be available through verify_credentials for checking current user's permissions
- `Account.role` now carries a Mastodon-compatible permissions bitmap and a marker for whether it should be shown to the public
- `Account.roles` added for *public* display roles (undocumented but stable since Mastodon 4.1)
- Web template now uses only public display roles (no user-visible change here, we already special-cased the `user` role)

* Handle verify_credentials case for default role

* Update JSON exact-match tests

* Address review comments

* Add blocks bit to admin permissions bitmap
2024-07-31 09:26:09 -07:00
kim
c635080a0b
[feature] add flac support (#3121)
* add flac support to the ffprobe format/stream -> filetype parser

* also add audio/flac for flac (not just x-flac)

* update tests

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-21 14:42:51 +02:00
tobi
027a93facc
[feature/frontend] Respect prefers-reduced-motion for avatars, headers, and emojis (#3118)
* [feature/frontend] Respect `prefers-reduced-motion` for avatars, headers, and emojis

* go fmt

* fix tests

* use static version of instance thumbnail when appropriate

* use prefers-reduced-motion

* simplify account conversion a bit

* fix c&p error
2024-07-21 14:22:08 +02:00
tobi
ad700c5029
[chore/frontend] Use smaller webp images for logo and default avatars (#3119) 2024-07-20 14:02:22 +01:00
kim
50c9b5498b
[feature] use webp for thumbnails (#3116)
* update to use webp for thumbnails

* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)

* update to use yuva colorspace, and use thumbnail=n=10 to select frame

* fix missing comma in ffmpeg args

* add links to appropriate ffmpeg docs

* update tests

* add file size tests for thumbnails

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-19 17:28:43 +02:00
kim
de45c0be60
[feature] more filetype support! (#3107)
* add more supported file types to our media processor that ffmpeg supports, update supported mime type lists

* add code comments to the supported mime types slice

* don't check for zero value string, just parse

* remove some unneeded consts which make the code a bit harder to read

* fix test expected instance media mime types, use compact ffprobe json, simple media processing by type

* final tweaks to media processing code

* don't use safe divide where we don't need to
2024-07-15 15:24:53 +01:00
tobi
9efb11d848
[feature/frontend] Add player for audio files; use thumbnail for poster (#3099)
* [feature/frontend] Audio player for audio media types

* use video preview images for previews instead of video itself

* don't preload

* update tests for new zork status

* collapse media gallery into single row when small
2024-07-15 10:47:57 +01:00
kim
cde2fb6244
[feature] support processing of (many) more media types (#3090)
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
tobi
ab2d063fcb
[feature] Process outgoing Move from clientAPI (#2750)
* prevent moved accounts from taking create-type actions

* update move logic

* federate move out

* indicate on web profile when an account has moved

* [docs] Add migration docs section

* lock while checking + setting move state

* use redirectFollowers func for clientAPI as well

* comment typo

* linter? i barely know 'er!

* Update internal/uris/uri.go

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* add a couple tests for move

* fix little mistake exposed by tests (thanks tests)

* ensure Move marked as successful

* attach shared util funcs to struct

* lock whole account when doing move

* move moving check to after error check

* replace repeated text with error func

* linterrrrrr!!!!

* catch self follow case

---------

Co-authored-by: Daenney <daenney@users.noreply.github.com>
2024-03-13 13:53:29 +01:00
Vyr Cossont
fc3741365c
[bugfix] Fix Swagger spec and add test script (#2698)
* Add Swagger spec test script

* Fix Swagger spec errors not related to statuses with polls

* Add API tests that post a status with a poll

* Fix creating a status with a poll from form params

* Fix Swagger spec errors related to statuses with polls (this is the last error)

* Fix Swagger spec warnings not related to unused definitions

* Suppress a duplicate list update params definition that was somehow causing wrong param names

* Add Swagger test to CI

- updates Drone config
- vendorizes go-swagger
- fixes a file extension issue that caused the test script to generate JSON instead of YAML with the vendorized version

* Put `Sample: ` on its own line everywhere

* Remove unused id param from emojiCategoriesGet

* Add 5 more pairs of profile fields to account update API Swagger

* Remove Swagger prefix from dummy fields

It makes the generated code look weird

* Manually annotate params for statusCreate operation

* Fix all remaining Swagger spec warnings

- Change some models into operation parameters
- Ignore models that already correspond to manually documented operation parameters but can't be trivially changed (those with file fields)

* Documented that creating a status with scheduled_at isn't implemented yet

* sign drone.yml

* Fix filter API Swagger errors

* fixup! Fix filter API Swagger errors

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-03-06 18:05:45 +01:00
tobi
9cadc764b3
[feature] Add experimental instance-federation-spam-filter option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
tobi
d5e3996a18
[feature] Parse instance descriptors as markdown, show T&C on /about (#2481)
* [feature] Parse instance descriptors as markdown, show T&C on /about

* lint

* remove unnecessary nullzero tags
2024-01-05 13:39:31 +01:00
tobi
d60edf7ec6
[bugfix] Ensure pre renders as expected, fix orderedCollectionPage (#2434) 2023-12-10 12:36:00 +01:00
kim
74700cc803
[performance] http response encoding / writing improvements (#2374) 2023-11-27 14:00:57 +00:00
tobi
fc02d3c6f7
[feature] Set/show instance language(s); show post language on frontend (#2362)
* update go text, include text/display

* [feature] Set instance langs, show post lang on frontend

* go fmt

* WebGet

* set language for whole article, don't use FA icon

* mention instance languages + other optional config vars

* little tweak

* put languages in config properly

* warn log language parse

* change some naming around

* tidy up validate a bit

* lint

* rename LanguageTmpl in template
2023-11-17 11:35:28 +01:00
tobi
4ee436e98a
[bugfix] process account delete side effects in serial, not in parallel (#2360)
* [bugfix] process account delete side effects in serial, not in parallel

* StartWorkers / StartNoopWorkers for tests

* undo testrig trace logging

* log errors instead of immediately returning
2023-11-14 15:57:25 +01:00
tobi
c7ecab9e6f
[chore/bugfix/horror] Allow expires_in and poll choices to be parsed from strings (#2346) 2023-11-10 16:42:48 +00:00
kim
e9e5dc5a40
[feature] add support for polls + receiving federated status edits (#2330) 2023-11-08 14:32:17 +00:00
kim
69ba9a79a1
[chore] de-interface{} the federator and dereferencer structs (#2285)
* de-interface{} the federator and dereferencer structs

* fix broken type signatures
2023-10-23 10:58:13 +01:00
kim
8f67dd583d
[chore] deinterface the typeutils.Converter and update to use state structure (#2217)
* update typeconverter to use state structure

* deinterface the typeutils.TypeConverter -> typeutils.Converter

* finish copying over old type converter code comments

* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-09-23 18:44:11 +02:00
f0x52
92de8fb396
[feature] Instance rules (#2125)
* init instance rules database model, admin api

* expose instance rules in public instance api

* public /api/v1/instance/rules route

* GET ruleById

* createRule route

* createRule auth check

* updateRule

* deleteRule

* list rules on about page

* ruleGet auth

* add about page ids for anchors

* process and store adding violated rules to reports

* admin api models for instance rules

* instance rule edit frontend

* change rule inputs to textareas

* database fixes after rebase (#2124)

* remove unused imports

* fix db migration column name

* fix tests

* fix more tests

* fix postgres error with wrongly used Ident

* add some tests, fiddle with rule model a bit, fix postgres migration

* swagger docs

---------

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-19 14:33:15 +02:00
Daenney
be3718f6e4
[chore] Use generic pointer function (#2080)
This replaces the different $TypePtr functions with a generic
implementation.
2023-08-07 18:38:11 +01:00
f0x52
8fb5a7e7f8
[Frontend] Settings for profile fields (#1885)
* get max emoji size from instance settings

* expose (hardcoded) max amount of profile fields in instance api

* basic profile field setting

* fix profile field hook structure for updates

* *twirls mustache* fix ze tests

---------

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-06-13 12:21:26 +02:00
kim
5faeb4de20
[chore] tidy up media manager, add calling func to errors, build-script improvements (#1835)
* media manager tidy-up: de-interface and remove unused PostDataFunc

Signed-off-by: kim <grufwub@gmail.com>

* remove last traces of media.Manager being an interface

Signed-off-by: kim <grufwub@gmail.com>

* update error to provide caller, allow tuneable via build tags

Signed-off-by: kim <grufwub@gmail.com>

* remove kim-specific build script changes

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* update build-script to support externally setting build variables

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-05-28 14:08:35 +02:00
tobi
f5c004d67d
[feature] Add List functionality (#1802)
* start working on lists

* further list work

* test list db functions nicely

* more work on lists

* peepoopeepoo

* poke

* start list timeline func

* we're getting there lads

* couldn't be me working on stuff... could it?

* hook up handlers

* fiddling

* weeee

* woah

* screaming, pissing

* fix streaming being a whiny baby

* lint, small test fix, swagger

* tidying up, testing

* fucked! by the linter

* move timelines to state like a boss

* add timeline start to tests using state

* invalidate lists
2023-05-25 10:37:38 +02:00
Daenney
b7dd32da42
[bugfix] Return languages in api/v1/instance (#1741)
It turns out that in Masto v2.3.0 the languages key was added to the V1
Instance and that it's effectively mandatory. Though in GtS we don't
really have this concept yet, some apps will explode if the languages
key is missing altogether. So at least return the empty array on V1 too
in the hopes that it makes things work well enough.

For history's sake, you can see the attributes that will get serialised
in
f877aa9d70/app/serializers/rest/v1/instance_serializer.rb (L6-L9).
Because the attribute does not have a conditional defined for it,
there isn't a filter that optionally omits it, or a def languages to
modify the behaviour the attribute is effectively always included and
serialised.

Fixes: #1662
2023-05-06 22:38:19 +02:00
Daenney
5e2bf0bdca
[chore] Improve copyright header handling (#1608)
* [chore] Remove years from all license headers

Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.

In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.

* [chore] Add license header check

This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.

* [chore] Add missing license headers

* [chore] Further updates to license header

* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
2023-03-12 16:00:57 +01:00
Vyr Cossont
e6cde25466
[feature] Advertise rich text formats, support content_type field (#1370)
* Advertise rich text formats, support content_type field

* Update JSON in instance patch tests

* Replace format with content_type everywhere

* update migration to work with both pg and sqlite

* regenerate swagger docs

* update instance serialization + tests

* fix up

* learn to code tobi please, i'm begging you

---------

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-02 12:06:40 +01:00
kim
baf933cb9f
[chore] move client/federator workerpools to Workers{} (#1575)
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly

* improve code comment

* change back testrig default log level

* un-comment-out TestAnnounceTwice() and fix

---------

Signed-off-by: kim <grufwub@gmail.com>
Reviewed-by: tobi
2023-03-01 18:26:53 +00:00
tobi
b6fbdc66c1
[chore] Deinterface processor and subprocessors (#1501)
* [chore] Deinterface processor and subprocessors

* expose subprocessors via function calls

* missing license header
2023-02-22 16:05:26 +01:00
tobi
e8a04b7ce1
[bugfix] Fix account roles (#1542)
* Change account role from string to object

* Update tests

* small fixes + swagger docs

---------

Co-authored-by: zowhoey <11893985+zowhoey@users.noreply.github.com>
2023-02-20 17:00:44 +01:00
tobi
b5993095fa
[bugfix] Set 'discoverable' properly on API accounts (#1511) 2023-02-16 13:20:23 +00:00
tobi
80c26d61f7
[bugfix] Allow instance thumbnail description to be set separately from image (#1417) 2023-02-04 15:53:11 +01:00
tobi
382512a5a6
[feature] Implement /api/v2/instance endpoint (#1409)
* interim: start adding /api/v2/instance

* finish up
2023-02-02 14:08:13 +01:00
f0x52
17eecfb6d9
[feature] Public list of suspended domains (#1362)
* basic rendered domain blocklist (unauthenticated!)

* style basic domain block list

* better formatting for domain blocklist

* add opt-in config option for showing suspended domains

* format/linter

* re-use InstancePeersGet for web-accessible domain blocklist

* reword explanation, border styling

* always attach blocklist handler, update error message

* domain blocklist error message grammar
2023-01-25 18:06:41 +01:00
tobi
eafd73c292
[chore] Remove omitempty on account source; refactor tests to use prettyprint json (#1337)
* remove omitEmpty tag on account source items

* update tests
2023-01-13 17:02:45 +01:00