Commit graph

1022 commits

Author SHA1 Message Date
tobi
d6487933c7
[feature] Implement Report database model and utility functions (#1310)
* implement report database model

* implement report cache + config changes

* implement report database functions

* report uri / regex functions

* update envparsing test

* remove unnecessary uri index

* remove unused function + cache lookup

* process error when storing report
2023-01-10 14:19:05 +00:00
kim
1bda6a2002
[bugfix] return early in websocket upgrade handler (#1315)
* launch websocket streaming in goroutine to allow upgrade handler to return

* don't send any message on ping, improved close check on failed read

* use context to signal wsconn close, ensure canceled in read goroutine

Signed-off-by: kim <grufwub@gmail.com>
2023-01-08 11:43:08 +00:00
kim
2bf9bfa24f
[bugfix] fix panic during status delete loop by breaking out early on len(statuses) == 0 (#1317)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2023-01-08 11:33:03 +01:00
tobi
0dbe6c514f
[chore] Update/add license headers for 2023 (#1304) 2023-01-05 12:43:00 +01:00
tobi
ff46dd485a
[chore] Fix emoji notnull constraint on initial gtsmodel (#1303) 2023-01-05 12:37:50 +01:00
tobi
90a14abb0c
[feature] HTTP request throttling middleware (#1297)
* [feature] Add throttling middleware to AP endpoints

* refactor a lil bit

* use config setting, start updating docs

* doc updates

* use relative links in faq doc

* small docs fixes

* return code 503 instead of 429 when throttled

* throttle other endpoints too

* simplify token channel prefills
2023-01-04 11:57:59 +01:00
kim
71dfea7e47
[chore] shuffle middleware to split rate limitting into client/s2s/fileserver, share gzip middleware globally (#1290)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2023-01-03 11:50:59 +01:00
tobi
941893a774
[chore] The Big Middleware and API Refactor (tm) (#1250)
* interim commit: start refactoring middlewares into package under router

* another interim commit, this is becoming a big job

* another fucking massive interim commit

* refactor bookmarks to new style

* ambassador, wiz zeze commits you are spoiling uz

* she compiles, we're getting there

* we're just normal men; we're just innocent men

* apiutil

* whoopsie

* i'm glad noone reads commit msgs haha :blob_sweat:

* use that weirdo go-bytesize library for maxMultipartMemory

* fix media module paths
2023-01-02 12:10:50 +00:00
tobi
1659f75ae6
[feature] For video attachments, store + return fps, bitrate, duration (#1282)
* start messing about with different mp4 metadata extraction

* heyyooo it works

* add test cow

* move useful multierror to gtserror package

* error out if video doesn't seem to be a real mp4

* test parsing mkv in disguise as mp4

* tidy up error handling

* remove extraneous line

* update framerate formatting

* use float32 for aspect

* fixy mctesterson
2022-12-22 11:48:28 +01:00
tobi
eabb906268
[bugfix] fix media create error not being checked (#1283) 2022-12-21 17:52:07 +01:00
tobi
6ebdc306ed
[bugfix] Close reader gracefully when streaming recache of remote media to fileserver api caller (#1281)
* close pipereader on failed data function

* gently slurp the bytes

* readability updates

* go fmt

* tidy up file server tests + add more cases

* start moving io wrappers to separate iotools package. Remove use of buffering while piping recache stream

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-12-21 11:17:43 +01:00
tobi
2bbc64be43
[feature] Enable basic video support (mp4 only) (#1274)
* [feature] basic video support

* fix missing semicolon

* replace text shadow with stacked icons

Co-authored-by: f0x <f0x@cthu.lu>
2022-12-17 04:38:56 +00:00
tobi
0f38e7c9b0
[chore] fix some little config whoopsies (#1272) 2022-12-16 22:38:41 +00:00
kim
da751c02fb
update go-cache to v3.2.0 with support for ignoring errors (#1273) 2022-12-16 22:36:52 +00:00
Autumn!
eb08529f35
[chore/bugfix] Switch markdown from blackfriday to goldmark (#1267)
Co-authored-by: Autumn! <autumnull@posteo.net>
2022-12-16 12:20:22 +01:00
tobi
1d24c1c283
[bugfix] Use null for empty api status language (#1268)
* [bugfix] Use null for empty api status `language`

* update swagger docs
2022-12-15 09:41:49 +01:00
Matthew Phillips
d10388cc28
[feature] support Sec-Websocket-Protocol in streaming API (#1254)
* [feature] support Sec-Websocket-Protocol in streaming API

* Fix lint problem

* Update based on reviews
2022-12-14 10:56:42 +01:00
kim
69dd5fed2c
[feature] domain block wildcarding (#1178)
* for domain block lookups, lookup along subdomain parts

Signed-off-by: kim <grufwub@gmail.com>

* only lookup up to a max of 5 domain parts to prevent DOS, limit inserted domains to max of 5 subdomains

Signed-off-by: kim <grufwub@gmail.com>

* add test for domain block wildcarding

Signed-off-by: kim <grufwub@gmail.com>

* check cached status first, increase cached domain time

Signed-off-by: kim <grufwub@gmail.com>

* fix domain wildcard part building logic

Signed-off-by: kim <grufwub@gmail.com>

* create separate domain.BlockCache{} type to hold all domain blocks in memory

Signed-off-by: kim <grufwub@gmail.com>

* remove unused variable

Signed-off-by: kim <grufwub@gmail.com>

* add docs and test to domain block cache, check for domain == host in domain block getter funcs

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>

* check order in which we check primary cache

Signed-off-by: kim <grufwub@gmail.com>

* add better documentation of how domain block checking is performed

Signed-off-by: kim <grufwub@gmail.com>

* change

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-12-14 10:55:36 +01:00
Christoph Voigt
8703933df4
[bugfix] fix unordered favorites (#1245)
* [bugfix] fix unordered favorites

* add test for favouritesget

* add license to new test files
2022-12-13 12:33:49 +01:00
kim
58c87bdd7f
[feature] allow uncaching of other media types (#1234)
* simplify pruneRemote, remove unncecessary media trace logging, update RemoteOlderThan() to include headers/avis

Signed-off-by: kim <grufwub@gmail.com>

* cleanup pruneallmeta, add remote header to pruneremote tests

Signed-off-by: kim <grufwub@gmail.com>

* fix olderthan duration additions

Signed-off-by: kim <grufwub@gmail.com>

* fix broken test now that test model header changed

Signed-off-by: kim <grufwub@gmail.com>

* instead use new remote test account for new header model

Signed-off-by: kim <grufwub@gmail.com>

* use newer generated ULID for remote_account_3 to ensure it is sorted last

Signed-off-by: kim <grufwub@gmail.com>

* reorganize serialized keys to match expected test account model order

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-12-12 12:22:19 +01:00
kim
cb2b2fd805
[feature] support configuring database caches (#1246)
* update config generator to support nested structs, add cache configuration options

* update envparsing test

* add cache configuration to config parse tests

* set cache configuration in testrig

* move caches to sub-cache "gts" namespace, update envparsing, add cache config docs to example config

Signed-off-by: kim <grufwub@gmail.com>
2022-12-11 13:03:15 +00:00
tobi
5e060d0bcb
[feature] Start implementing refetch of lost media files via /api/v1/admin/media_refetch (#1221)
* [chore] Move ShortcodeDomain to its own little util func

* [feature] Add RefetchEmojis function to media manager

* [feature] Expose admin media refresh via admin API

* update following review feedback

- change/fix log levels
- make sure not to try to refetch local emojis
- small style refactoring + comments

* log on emoji refetch start

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-12-10 21:43:11 +00:00
Christoph Voigt
199672e586
[bugfix] fix unordered favorites (#1236) 2022-12-09 11:46:52 +01:00
Matthew Phillips
477ae50933
[feature] Allow users to create + delete bookbarks, and view bookmarked statuses (#1168)
* Implement Bookmarks

* Update based on review comments

* Update swagger doc

* Fix argument passing to status.Bookmark

* Update changed test

* Updates based on latest PR review
2022-12-09 11:37:12 +01:00
kim
e58d2d8122
[chore] move caches to a separate State{} structure (#1078)
* move caches to a separate State{} structure

Signed-off-by: kim <grufwub@gmail.com>

* fix call to log.Panic not using formatted call

Signed-off-by: kim <grufwub@gmail.com>

* move caches to use interfaces, to make switchouts easier in future

Signed-off-by: kim <grufwub@gmail.com>

* fix rebase issue

Signed-off-by: kim <grufwub@gmail.com>

* improve code comment

Signed-off-by: kim <grufwub@gmail.com>

* fix further issues after rebase

Signed-off-by: kim <grufwub@gmail.com>

* heh

Signed-off-by: kim <grufwub@gmail.com>

* add missing license text

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-12-08 18:35:14 +01:00
Dominik Süß
199b685f43
[feature] overhaul the oidc system (#961)
* [feature] overhaul the oidc system

this allows for more flexible username handling and prevents account
takeover using old email addresses

* [feature] add migration path for old OIDC users

* [feature] nicer error reporting for users

* [docs] document the new OIDC flow

* [fix] return early on oidc error

* [docs]: add comments on the finalization logic
2022-12-06 14:15:56 +01:00
Sigrid Solveig Haflínudóttir
1a3f26fb5c
[feature] media: add webp support (#1155)
* media: add webp support

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>

* bump exif-terminator to v0.5.0

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>
2022-12-06 14:15:25 +01:00
Dominik Süß
611df7b22d
[bugfix]: Prevent extension of S3 presigned url TTL (#1208)
Thanks :)
2022-12-05 11:09:22 +01:00
Daniele Sluijters
847e7c7c3a
[chore] Fix a few possible cases of int truncation (#1207)
This fixes a couple of cases where due to int being platform dependent a
value could get truncated if running on 32bits.
2022-12-04 14:20:41 +01:00
Dominik Süß
bc917a4085
[performance]: make s3 urls cacheable (#1194)
Implements #864 and should speed up s3 based installations by a lot.

With more static urls, we can then also implement #1026 for even
better performance when used in conjunction with CDNs
2022-12-02 18:40:49 +00:00
tobi
281f712477
[bugfix] Fix admin account confirm on pre-confirmed account setting email address to empty string (#1203) 2022-12-02 17:41:10 +01:00
tobi
34716d7d7d
[bugfix] Add missing continues in emoji get funcs (#1200) 2022-12-02 16:41:05 +01:00
Daniele Sluijters
558b448ab2
[chore] Remove duplicate prefixes from sanitizer (#1195)
In the previous changes that expanded the IPv4 and IPv6 deny lists based
on the IANA registries we inadvertently added a number of duplicates.
This is unnecessary as they're already caught by larger prefixes and
means there's less entries to scan.

This change removes all prefixes that are subnets of other prefixes.
2022-12-01 20:34:55 +01:00
tobi
cf20397f26
[bugfix] Use case-insensitive selects when getting remote accounts by username/domain (#1191)
* [bugfix] Case-insensitive account selection

* don't lowercase cache key
2022-12-01 16:06:09 +01:00
Sigrid Solveig Haflínudóttir
5a0e418281
[feature] Support PKCS1 "RSA PUBLIC KEY" pem block type (#1179)
* ap: add support for PKCS1 "RSA PUBLIC KEY" pem block type

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>

* ap: report no PEM data or unknown pem block type

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>

Signed-off-by: Sigrid Solveig Haflínudóttir <sigrid@ftrv.se>
2022-11-30 23:13:13 +01:00
tobi
3a11861ac6
[bugfix] Don't call strings.ToLower() on usernames when selecting account by domain+username (#1190)
* don't lowercase account username when doing a select

* test getting remote user with uppercase username
2022-11-30 17:44:02 +01:00
tobi
927117d8e3
[bugfix] Avoid accidentally marking changed emojis as orphaned + pruning them (#1188)
* add predictable instance account to tests, use it in emoji urls + paths

* use static image url to select emojis when pruning orphaned
2022-11-30 16:20:57 +01:00
kim
cfa8d7900c
[bugfix] don't return error during account serialize on deleted emoji (#1177)
* only return error for emoji fetch if NOT errnoentries

Signed-off-by: kim <grufwub@gmail.com>

* reformat gts->api model slice conversion to standard error behaviours and reduce code reuse

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-29 17:59:59 +00:00
kim
d445c60a26
[bugfix] wrap the correct error on failed account update (#1176)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-29 18:40:30 +01:00
tobi
97f5453378
[chore] Tidy up some of the search logic (#1082)
* start refactoring some of the search + deref logic

* add tests for search api

* rename GetRemoteAccount + GetRemoteStatus

* make search function a bit simpler + clearer

* fix little fucky wucky uwu owo i'm just a little guy

* update faulty switch statements

* update test to use storage struct

* redo switches for clarity

* reduce repeated logic in search tests

* fastfail getstatus by uri

* debug log + trace log better

* add implementation note

* return early if no result for namestring search

* return + check on dereferencing error types

* errors hah what errors

* remove unneeded error type alias, add custom error text during stringification itself

* fix a woops recursion 🙈

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-11-29 09:24:55 +00:00
kim
0c1b1b01f8
fix missing lookup cache key for invalid domain block (#1158) 2022-11-27 14:11:49 +00:00
Daniele Sluijters
c534230600
[bugfix] Use Host domain for UA (#1156)
In d6f4d196c9 we swapped to use the
AccountDomain but that actually goes against the intent of the change.
This reverts that change and uses the host domain again.
2022-11-27 00:09:09 +00:00
Daniele Sluijters
c5ae88c51b
[chore] Set User-Agent header in transport (#1154)
Currently requests set their own User-Agent. This moves it down to set
it in the transport's do() method, to guarantee it's always set on all
requests.
2022-11-26 20:19:42 +00:00
Daniele Sluijters
d6f4d196c9
Tweak the User-Agent a bit (#1153)
* [bugfix] Use AccountDomain for user agent

By using the account domain we can pinpoint the source of the request
more accurately when looking at the User-Agent header.

* [chore] Align user-agent header with spec

Based on RFC 7231, our User-Agent header doesn't quite match. It seems
to always want Name [/ Version] pairs, with comments in parenthesis and
multiple comments separated by a semicolon.

Align our UA with that, using application name first by itself in case
someone has customised it with the source instance in a comment. Follow
that up with gotosocial/<version> and a comment pointing at the source
code.

This also drops the mention of gofed/activity since a fork is in use.
2022-11-26 20:15:19 +00:00
Daniele Sluijters
746f3fa4e6
Additional IP range validations (#1152)
* [bugfix] Ensure requests happen over TCP

It's possible for the network to be udp4 or udp6. This is rather
unlikely to occur, but since we're given the network anyway as part of
the Sanitize function getting called we might as well check for it.

* [chore] Align reserved v6 blocks to IANA registry

* [chore] Add test for ValidateIP

The net and netip packages diverge in that net.ParseIP will consider an
IPv4-mapped address to be an IPv4 address and as such it would get
caught by the IPv4Reserved list. However, netip considers it an IPv6
address, so we need to ensure the mapped range is in IPv6Reserved.

* [chore] Align reserved v4 blocks to IANA registry

This includes a number of tests for /32's explicitly called out in the
registry to ensure we always consider those invalid.
2022-11-26 11:09:55 +00:00
Daniele Sluijters
e6cd81babc
[bugfix]: Fix IPv6 validation (#1150)
* [bugfix]: Fix IPv6 validation

The current code considers ff00::/8 valid, but contrary to the comment
that's not the global unicast range. ff-prefixes in IPv6 denote
multicast.

This adapts the code to take the same approach as IPv4, explicitly
blacklisting reserved internal/private ranges.

* [chore] Add missing 4 in IPv4Reserved doc comment
2022-11-25 23:28:03 +00:00
tobi
13e9abd02a
[feature] Add admin media prune orphaned CLI command (#1146)
* add FilePath regex

* add `admin media prune orphaned` command

* add prune orphaned function to media manager

* don't mark flag as required

* document admin media prune orphaned cmd

* oh envparsing.sh you coy minx
2022-11-25 17:23:42 +00:00
tobi
b6dbe21026
[feature] PATCH /api/v1/admin/custom_emojis/{id} endpoint (#1061)
* start adding admin emoji PATCH stuff

* updating works OK, now how about copying

* allow emojis to be copied

* update swagger docs

* update admin processer to use non-interface storage driver

* remove shortcode updating for local emojis

* go fmt

Co-authored-by: f0x52 <f0x@cthu.lu>
2022-11-24 18:12:07 +00:00
tobi
3e82196d5e
[bugfix] Fix status boosts giving 404 (#1137)
We broke this at some point recently, and i'm not sure when.

In any case, i updated some of the logic in there + added a test for it.
2022-11-24 16:12:43 +01:00
tobi
da8954858a
[bugfix] Prevent future statuses entering timelines (#1134)
* [bugfix] Prevent future statuses entering timeline
Statuses created more than 5 minutes into the future are now rejected in the visibility package.

* Come on buddy
2022-11-24 13:54:49 +01:00
kim
fcb9c0bb8b
[chore] cleanup storage implementation, no need for multiple interface types (#1131)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-24 09:35:46 +01:00
tobi
c9d893fec1
[feature/performance] Fail fast when doing remote transport calls inside incoming request contexts (#1119)
* [feature/performance] Fail fast when doing remote transport calls inside incoming request contexts

* [chore] Reduce outgoing request timeout to 15s

* log error messages when fastfailing

* use context.Value() instead of wrapped context, wrap error with fastfail instead of extra log entry

* add fast-fail context key test

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-11-23 21:40:07 +00:00
tobi
1281a9fd4a
[chore/frontend] remove references to unused bundle.js (#1128) 2022-11-23 15:32:57 +01:00
tobi
50dc179d33
[feature] Prune timelines once per hour to plug memory leak (#1117)
* export highest/lowest ULIDs as proper const

* add stop + start to timeline manager, other small fixes

* unexport unused interface funcs + tidy up

* add LastGot func

* add timeline Prune function

* test prune

* update lastGot
2022-11-22 18:38:10 +00:00
kim
a898160b0c
[chore] use kv.KVStore also for S3 storage (#1113)
* replace s3 storage implementation to also use kv.KVStore

Signed-off-by: kim <grufwub@gmail.com>

* pull in latest `go-store` fix

Signed-off-by: kim <grufwub@gmail.com>

* pull-in go-store v2.0.9 fixes, update s3 put chunk size to 5MiB

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-22 14:28:55 +01:00
Norwin
f88cb3aebc
[chore] markdown: disable Smartypants rendering (#1111)
* [enhancement] markdown: disable SmartyPantsFractions

fixes #1028

* some fractions are still converted to unicode

to fix that, we might need to disable smartypants indeed in its entirety

* disable smartypants completely

for lack of a better simple solution
2022-11-22 11:08:50 +01:00
kim
4a9538593c
[chore] simplify logging to remove "safetywriter" given os.File has concurrency protection (#1083)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 17:40:15 +01:00
kim
2eea3d57e0
[chore] improve errors returned fetching account/statues from DB (#1084)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 17:38:58 +01:00
kim
5d55e8d920
[performance] add account block DB cache and remove block query joins (#1085)
* add account block DB cache and remove reliance on relational joins

* actually include cache key arguments...

* add a PutBlock() method which also updates the block cache, update tests accordingly

* use `PutBlock` instead of `Put(ctx, block)`

* add + use functions for deleting + invalidating blocks

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-11-20 16:33:49 +00:00
kim
9be16852f2
[bugfix] fix possible infinite loop on federated AP profile delete (#1091)
* refactor federator account statuses delete to better catch errors, ensure next maxID is always set

Signed-off-by: kim <grufwub@gmail.com>

* fix error statement missing 2nd format operator

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-20 15:57:19 +01:00
Michael Manfre
67106c9dc4
[feature] Support markdown format for Account bio/note (#1037)
* [feature] Status format also controls bio format

* test
2022-11-19 10:07:51 +01:00
kim
45ae719bd9
[performance] replace account emojis relational query with separate calls to emojiDB to rely on cache (#1074)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-18 18:29:25 +01:00
kim
dccc2eee81
[performance] replace status query relationals with separate calls in order to rely on caches more (#1073)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-18 18:28:33 +01:00
tobi
d98a48b446
[performance] don't use relations to select notification structs, use caches instead (#1072) 2022-11-18 17:23:14 +01:00
psa
c1824b034b
[chore] Replace blocking Floc with Topics. (#1068)
Google abandoned Floc in favour of Topics and changed the
necessary permissions policy. Currently, the Floc policy will
block Topics. This change includes switching to the updated
policy to be ahead of Google abandoning recognising the Floc
policy.

Update the function documentation to include the current relevant
documentation.
2022-11-17 15:04:06 +01:00
tobi
940abc279c
[chore] reversion: use specific columns for updating user again (#1059) 2022-11-16 11:27:08 +01:00
kim
8598dea98b
[chore] update database caching library (#1040)
* convert most of the caches to use result.Cache{}

* add caching of emojis

* fix issues causing failing tests

* update go-cache/v2 instances with v3

* fix getnotification

* add a note about the left-in StatusCreate comment

* update EmojiCategory db access to use new result.Cache{}

* fix possible panic in getstatusparents

* further proof that kim is not stinky
2022-11-15 18:45:15 +00:00
Mara Sophie Grosch
1f256e288b
[chore] refactor test/cliparsing.sh into a go test below internal/config (#1036)
Also adds AddGlobalFlags and AddServerFlags as methods on ConfigState,
very useful for testing.
2022-11-15 16:53:19 +01:00
ugla
52109776f6
[bugfix] Fix unicode-unaware word boundary check in hashtags (#1049)
* [bugfix] Fix unicode-unaware word boundary check in hashtag regex

Go `\b` does not care for Unicode, and without lookahead, the workarounds got
very ugly. So I replaced the regex with a parser.

The parser runs in O(n) time and performance should not be affected.

* [bugfix] Add back hashtag max length and add tests for it
2022-11-15 16:05:34 +01:00
tobi
a39a664525
[feature] Serialize local account role via API, and show it via web view (#1045)
* [feature] Add 'role' field to api serialization of local accounts

* [chore] Add a bit of license text while I'm here

* [frogend] render account role on same line as username in web view of profile

* style tweaking on role badges, general profile header layout

* profile stats wrapping

* don't render standard 'user' role on web view

Co-authored-by: f0x <f0x@cthu.lu>
2022-11-15 10:19:32 +01:00
tobi
4cd00d546c
[feature] Allow newly uploaded emojis to be placed in categories (#939)
* [feature] Add emoji categories GET
Serialize emojis in appropriate categories; make it possible to get categories via the admin API

* [feature] Create (or use existing) category for new emoji uploads

* fix lint issue

* update misleading line in swagger docs
2022-11-14 22:47:27 +00:00
Rob Sargant
d120743e8b
[feature] add instance-expose-public-timeline flag (#1039)
* Add instance-expose-public-timeline flag

Adds a config flag that allows unauthenticated access to /api/v1/timelines/public. Defaults to false to replicate existing behaviour.

* Update structure following review

* Add comment

* Fix linting
2022-11-14 10:30:01 +01:00
David Baker
96f11e757c
[bugfix] Fix login on Mastodon iOS app for users with no statuses (#1042)
* Fix login on Mastodon iOS app for users with no statuses

Mastodon for iOS can't cope with an empty string for a date and
expect a JSON `null` instead.

Fixes https://github.com/superseriousbusiness/gotosocial/issues/1010

* Fix expected values in tests to match
2022-11-13 21:38:01 +01:00
kim
e8c733da3f
[chore] bump go-cache to v3.1.7 to fix possible issues with zero value keys (#1038)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-13 14:02:07 +01:00
tobi
6fb47651c8
[chore] Fixes + updates in emoji dereferencing logic (#1022)
* fix incorrect static remote url use for emojis

* warn when emoji/attachment already exists

* defer emoji postdata execution

* rename ctx to innerCtx for clarity

* warn on emoji too large

* small efficiency fix in fetchRemoteAccountEmojis

* tidy up lock+load

* lock processing emojis

* fix little fucky wucky

* this wasn't go fmted for some reason
2022-11-11 19:27:37 +00:00
tobi
edcee14d07
[feature] Read + Write tombstones for deleted Actors (#1005)
* [feature] Read + Write tombstones for deleted Actors

* copyTombstone

* update to use resultcache instead of old ttl cache

Signed-off-by: kim <grufwub@gmail.com>

* update go-cache library to fix result cache capacity / ordering bugs

Signed-off-by: kim <grufwub@gmail.com>

* bump go-cache/v3 to v3.1.6 to fix bugs

Signed-off-by: kim <grufwub@gmail.com>

* switch on status code

* better explain ErrGone reasoning

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2022-11-11 12:18:38 +01:00
Mara Sophie Grosch
948e90b95a
[feature] S3: add config flag to proxy S3 media (#1014)
* S3: add config value "proxy" for not redirecting

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>

* S3: document new config value "proxy"

* S3: add new config value "proxy" to test scripts

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>
2022-11-11 12:03:18 +01:00
kim
3ce0e33f99
[chore] close in-storage media reader _before_ opening write, no need to leave it hanging around (#1016)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-11 12:01:53 +01:00
tobi
b4f7316a4c
[feature] Make instance thumbnail configurable via admin panel (#973)
* [feature] Make instance thumbnail configurable via admin panel

* log db errors in InstanceToAPIInstance

* only update instance in db if necessary

* start adding tests

* finish test
2022-11-08 17:11:06 +00:00
f0x52
eb25739c34
[frontend] Custom Emoji Deletion (#994)
* re-add eslint

* fix oauth url getting too long

* actually attach single emoji get and delete routes

* basic emoji details + deletion using rtk query

* refactor emoji upload to rtk query

* clean up old redux api+reducers for custom emoji

* fix validation order

* refactor custom emoji form fields

* remove unused requires

* cleanup, fix most eslint errors

* more small eslint fixes

* fix max emoji size

* tiny bit of function documentation
2022-11-08 17:51:44 +01:00
kim
0e57246083
[feature] various worker / request queue improvements (#995)
* greatly simplify httpclient request queuing

Signed-off-by: kim <grufwub@gmail.com>

* improved request queue mutex logic

Signed-off-by: kim <grufwub@gmail.com>

* use improved hashmap library

Signed-off-by: kim <grufwub@gmail.com>

* add warn logging when request queues are full

Signed-off-by: kim <grufwub@gmail.com>

* improve worker pool prefix var naming

Signed-off-by: kim <grufwub@gmail.com>

* improved worker pool error logging

Signed-off-by: kim <grufwub@gmail.com>

* move error message into separate field

Signed-off-by: kim <grufwub@gmail.com>

* remove old log statement

Signed-off-by: kim <grufwub@gmail.com>

* don't export worker message, it gets very spammy :')

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-08 10:35:24 +01:00
D
53a1e47791
[chore] Fix rel="nofollow" so that it only applies to external links (#991)
Resolves https://github.com/superseriousbusiness/gotosocial/issues/990
2022-11-07 14:25:36 +01:00
kim
05a8baa53a
[bugfix] KVStore doesn't like lost+found directory (#972)
* bump go-store version to v2.0.5, init kv.KVStore without initial clean (as we are using for storage, not as a key-value store)

Signed-off-by: kim <grufwub@gmail.com>

* remove newline

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-06 13:30:08 +01:00
tobi
4d66fb9603
[feature] Make rate limit requests amount configurable (#966)
* update rate limit documentation

* regenerate landingpage config helpers

* make rate limit rate configurable
2022-11-06 09:47:48 +00:00
tobi
15be35673e
[bugfix] Check media-description-min-chars on submission of new status (#960) 2022-11-05 12:33:38 +00:00
tobi
bcb80d3ff4
[chore] bump gruf/go-store to v2 (#953)
* [chore] bump gruf/go-store to v2

* no more boobs
2022-11-05 11:10:19 +00:00
tobi
a9addb59b6
[bugfix] Increase field size limits when registering apps (#958) 2022-11-05 11:06:50 +01:00
Adam Harvey
ee6a0a1e15
[bugfix] use the landing page user directly (#956)
If set, the landing page user configuration value is used as a Gin
context parameter, which seems incorrect, since a normal request isn't
going to have a parameter named after an arbitrarily configured user.
Instead, the user name should be used directly when building the
redirect URL.
2022-11-05 08:39:36 +01:00
tobi
1dfa7fe0d5
[bugfix] Wrap media in read closer (#941)
* use readcloser for content.Content

* call media postdata function no matter what

* return a readcloser from data func

* tidy of logic of readertostore

* fix whoopsie
2022-11-03 15:03:12 +01:00
tobi
bd05040133
[bugfix] Use []rune to check length of user-submitted text (#948) 2022-11-03 14:38:06 +01:00
Mia Heidenstedt
f3fc040c2e
[feature] Allow user to show instead of landing page on / (#922)
* configurable user instead of landing page

* rename DefaultUser to LandingPageUser

* code review changes and fix tests

* try to fix pipeline error

* code review changes

* code review changes

* code review changes

* remove unnecessary line
2022-11-03 13:15:31 +01:00
f0x52
c4c713988a
[frontend] change bundler to skulk (#942)
* replace web bundler with skulk

* upgrade skulk

* add license
2022-11-02 16:31:43 +01:00
Terin Stock
4a925e49b1
[bugfix] create admin_account_actions table in tx (#940)
The migration that adds the `admin_account_actions` table did so at the
same time as adding indexes onto the new table. This code was ran inside
a `RunInTx` function, but the table creation did not use the transaction
reference, while the creation of the indexes did. This could cause a
race between the table and index creations, depending on the scheduling
order. If the table creation did not win the race, then the migration
would fail.

This changeset corrects the table creation to also be done inside the
same transaction as the index creation.

Signed-off-by: Terin Stock <terinjokes@gmail.com>

Signed-off-by: Terin Stock <terinjokes@gmail.com>
2022-11-01 15:30:02 +01:00
Mia Heidenstedt
31f0f27bed
[bugfix] Fix imports for helpers.gen.go (#924) 2022-10-29 17:14:57 +02:00
tobi
c7ba195907
[bugfix] Prevent future statuses showing in timelines (#932) 2022-10-29 17:10:28 +02:00
tobi
ccd5b34c2c
[docs] Use correct form for status submit (#915) 2022-10-17 12:54:36 +02:00
tobi
f7416d6e94
[feature] Add emoji DELETE handler at /api/v1/admin/custom_emojis (#913)
* add emoji DELETE handler

* no need to process error (thanks kim)

* don't double check if user is admin

* add missing security annotation
2022-10-14 17:30:04 +02:00
tobi
6a95f5fa67
[feature] Add /api/v1/admin/custom_emojis/{id} endpoint for single emoji GET (#910)
* fix error in prev swagger docs

* add GET for single admin emoji
2022-10-13 16:37:55 +02:00
tobi
70d65b683f
[feature] Refetch emojis when they change on remote instances (#905)
* select emoji using image_static_url

* use updated on AP emojis

* allow refetch of updated emojis

* cheeky workaround for test

* clean up old files for refreshed emoji

* check error for originalPostData

* shorten GetEmojiByStaticImageURL

* delete kirby (sorry nintendo)
2022-10-13 15:16:24 +02:00
tobi
eb85ef7325
[feature] Add /api/v1/admin/custom_emojis endpoint (#902)
* add admin emojis get path + model + docs

* stub admin emojis get processor function

* add id + disabled fields to admin emoji

* add emoji -> api admin emoji converter

* tidy up a bit

* add GetEmojis function

* finish up get emojis function

* order by shortcodedomain

* ASC

* tidy up + explain

* update to allow paging

* make admin emojis pageable

* fix mixed case paging

* normalize emoji queries a bit better

* test emoji get paging

* make limit optional

* fix incorrect path in media cleanup tests

* i have bad coder syndrome

* don't trimspace

* rename -> GetUseableEmojis

* wrap emoji query in subquery
avoid selecting more than we need

* fix a bit of sillyness teehee

* fix subquery postgres woes
2022-10-12 15:01:42 +02:00
tobi
832befd727
[chore] Make paging logic more generic (#901)
* make paging logic more generic
not just for timelines!

* linty linterson
2022-10-10 15:52:49 +02:00
tobi
80663061d8
[feature] Add opt-in RSS feed for account's latest Public posts (#897)
* start adding rss functionality

* add gorilla/feeds dependency

* first bash at building rss feed
still needs work, this is an interim commit

* tidy up a bit

* add publicOnly option to GetAccountLastPosted

* implement rss endpoint

* fix test

* add initial user docs for rss

* update rss logo

* docs update

* add rssFeed to frontend

* feed -> feed.rss

* enableRSS

* increase rss logo size a lil bit

* add rss toggle

* move emojify to text package

* fiddle with rss feed formatting

* add Text field to test statuses

* move status to rss item to typeconverter

* update bun schema for enablerss

* simplify 304 checking

* assume account not rss

* update tests

* update swagger docs

* allow more characters in title, trim nicer

* update last posted to be more consistent
2022-10-08 14:00:39 +02:00
tobi
aa07750bdb
[chore] Standardize database queries, use bun.Ident() properly (#886)
* use bun.Ident for user queries

* use bun.Ident for account queries

* use bun.Ident for media queries

* add DeleteAccount func

* remove CaseInsensitive in Where+use Ident ipv Safe

* update admin db

* update domain, use ident

* update emoji, use ident

* update instance queries, use bun.Ident

* fix media

* update mentions, use bun ident

* update relationship + tests

* use tableexpr

* add test follows to bun db test suite

* update notifications

* updatebyprimarykey => updatebyid

* fix session

* prefer explicit ID to pk

* fix little fucky wucky

* remove workaround

* use proper db func for attachment selection

* update status db

* add m2m entries in test rig

* fix up timeline

* go fmt

* fix status put issue

* update GetAccountStatuses
2022-10-08 13:50:48 +02:00
kim
e58a6a2da3
[performance] cache domains after max retries in transport (#884) 2022-10-08 13:50:16 +02:00
tobi
3bb45b7179
[feature] oob oauth token support (#889)
* move helpful advice into oauth server

* rewrite HandleAuthorizeRequest to allow oob
2022-10-08 13:49:56 +02:00
tobi
5cf0f9950a
[bugfix] Fix new domain block date (#893) 2022-10-06 12:48:17 +02:00
tobi
f8528aa689
[feature] Add emoji image size to instance response (#892) 2022-10-06 12:00:53 +02:00
tobi
359ed1bcb5
[bugfix] Use background context instead of request context for async processing (#888)
Fixes an issue where async processing was not completing correctly.

In particular this applies to side effects of domain blocks: while the domain block was being entered and enforced correctly, side effects like deleting accounts and updating the instance entry for the blocked instance were not. This fixes that :)
2022-10-04 17:50:29 +02:00
f0x52
5249294a16
[chore] Bundler restructure (#880)
* re-structure bundler, settings panel files

* add more info logging

* tidy up CSS syntax errors

* split into lib/ files

* livereloading server

* fix factor function for production builds

* remove testing console.log

* default to production env, saves 300kb bundle size
2022-10-03 16:46:38 +02:00
tobi
56f53a2a6f
[performance] add user cache and database (#879)
* go fmt

* add + use user cache and database

* fix import

* update tests

* remove unused relation
2022-10-03 10:46:11 +02:00
tobi
deba75cad1
[chore] Use shorter timestamps in frontend for replies (#875)
* rename timestampShort -> timestampVague

* add ParseISO8601

* start fiddling with timestamp

* pad/margin a bit more consistently

* remove visibilty icon, change timestamp use

* update timestamp logic

* check + log errors

* properly cut-off long display- and usernames

Co-authored-by: f0x <f0x@cthu.lu>
2022-10-02 15:54:42 +02:00
tobi
196e474e43
fix emojis not showing in spoiler text (#878) 2022-10-02 13:01:47 +01:00
tobi
651303d970
[chore] Normalize domain blocks to punycode (#873)
* normalize domain blocks to punycode

* normalize -> normalizeDomain
2022-10-01 17:48:38 +02:00
kim
9fcfe61410
[chore] small logging cleanup (#869)
* handle more syslogging levels, use singular time format variable, add entry .New() function

* pass in calldepth to lower log functions to ensure correctly set

* update truncate length in syslog test
2022-10-01 16:36:08 +01:00
kim
1d999712e6
[feature] update config types to use bytesize.Size (#828)
* update config size types to use bytesize.Size

* submit unchecked-out file ... 🤦

* fix bytesize config var decoding

* bump bytesize version

* update kim's libraries in readme

* update envparse.sh to output more useful errors

* improve envparse.sh

* remove reliance on jq

* instead, use uint64 for bytesize flag types

* remove redundant type

* fix viper unmarshaling

* Update envparsing.sh

* fix envparsing test

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-09-29 21:50:43 +01:00
tobi
f0bf69d4d0
[chore] Add ipv6 localhost to trusted proxies by default (#868)
* add ipv6 localhost to trusted proxies by default

* test different trusted proxies
2022-09-29 17:08:56 +02:00
tobi
97b747b452
[bugfix] Fix incorrect use of bun.Ident (#866)
* remove misused bun.Ident

* test status delete via API

* fix test
2022-09-29 12:33:33 +02:00
tobi
dd83ad053c
[feature] Add meta robots tag; allow robots to index profile card if user is Discoverable (#842)
* rework robots.txt response

* don't let robots snippet from statuses/threads

* allow robots to index if user is Discoverable

* add license text
2022-09-29 12:03:17 +02:00
f0x52
938328cd07
[frontend] Unified panels (#812)
* settings panel restructuring

* clean up old Gin handlers

* colorscheme redesign, some other small css tweaks

* basic router layout, error boundary

* colorscheme redesign, some other small css tweaks

* kebab-case consistency

* superfluous padding on applist

* remove unused consts

* redux, whitespace changes..

* use .jsx extensions for components

* login flow up till app registration

* full redux oauth implementation, with basic error handling

* split oauth api functions

* oauth api revocation handling

* basic profile change submission

* move old dir

* profile overview

* fix keeping track of the wrong instance url (for different instance/api domains)

* use redux state for profile form

* delete old/index.js, old/basic.js, fully implemented

* implement old/user/profile.js

* implement password change

* remove debug logging

* support future api for removing files

* customize profile css

* remove unneeded wrapper components

* restructure form fields

* start on admin pages

* admin panel settings

* admin settings panel

* remove old/admin files

* add top-level redirect

* refactor/cleanup forms

* only do API checks on logged-in state

* admin-status based routing

* federation block routing

* federation blocks

* upgrade dependencies

* react 18 changes

* media cleanup

* fix useEffect hooks

* remove unused require

* custom emoji base

* emoji uploader

* delete last old panel files

* sidebar styling, remove unused page

* refactor submit functions

* fix sidebar boxshadow-border

* fix old css variables

* fix fake-toot avatar

* fix non-square emoji

* fix user settings redux keys

* properly get admin account contact from instance response

* Account.source default values

* source.status_format key

* mobile responsiveness

* mobile element tweaks

* proper redirect after removing block

* add redirects for old setting panel urls

* deletes

* fix mobile overflow

* clean up debug logging calls
2022-09-29 12:02:41 +02:00
kim
2f22780800
[chore] simplify generating log entry caller information (#863)
* vastly simplify logging caller information

Signed-off-by: kim <grufwub@gmail.com>

* fix failing test due to multiple calls to processor.Start()

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-29 12:02:12 +02:00
kim
a156188b3e
[chore] update dependencies, bump to Go 1.19.1 (#826)
* update dependencies, bump Go version to 1.19

* bump test image Go version

* update golangci-lint

* update gotosocial-drone-build

* sign

* linting, go fmt

* update swagger docs

* update swagger docs

* whitespace

* update contributing.md

* fuckin whoopsie doopsie

* linterino, linteroni

* fix followrequest test not starting processor

* fix other api/client tests not starting processor

* fix remaining tests where processor not started

* bump go-runners version

* don't check last-webfingered-at, processor may have updated this

* update swagger command

* update bun to latest version

* fix embed to work the same as before with new bun

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-28 18:30:40 +01:00
tobi
00d38855d4
[bugfix] Fix emphasis being added to emoji shortcodes with markdown parsing (#856)
* fix underscored emoji shortcodes being emphasized

* remove footnote parsing from md
2022-09-27 14:27:53 +02:00
tobi
429bb770e2
[performance] Update indexes that were causing slow db queries (#855)
* add status.url index, use id desc in some indexes

* test account last posted
2022-09-26 14:43:19 +02:00
tobi
c4a08292ee
[feature] Show + federate emojis in accounts (#837)
* Start adding account emoji

* get emojis serialized + deserialized nicely

* update tests

* set / retrieve emojis on accounts

* show account emojis in web view

* fetch emojis from db based on ids

* fix typo in test

* lint

* fix pg migration

* update tests

* update emoji checking logic

* update comment

* clarify comments + add some spacing

* tidy up loops a lil (thanks kim)
2022-09-26 11:56:01 +02:00
kim
15a67b7bef
[bugfix] more nil checks baybeeeeeeeeeeeeeeeeeeee (#854)
* more nil checks baybeeeeeeeeeeeeeeeeeeee

Signed-off-by: kim <grufwub@gmail.com>

* move item nil checks

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-26 10:50:14 +02:00
kim
7bea107608
[bugfix] add so, many, nil, checks (#853)
* add so, many, nil, checks.

* remove comment
2022-09-26 10:14:36 +02:00
kim
82061b1202
[bugfix] panic during child thread iteration (#852)
* *actually* start at top of stack loop on find remote child, fix iter indexing

Signed-off-by: kim <grufwub@gmail.com>

* add improved code comment

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-26 09:39:59 +02:00
kim
7a1aa04bbb
[bugfix] update thread iterators to not use recursion (#851)
* update thread iterators to not use recursion, rewrote both

Signed-off-by: kim <grufwub@gmail.com>

* fix endless descendant deref, don't error if fetching existing status

Signed-off-by: kim <grufwub@gmail.com>

* don't refetch remote ancestor statuses, improve descendant iter commenting

Signed-off-by: kim <grufwub@gmail.com>

* move collection page next logic so we capture first page of entities

Signed-off-by: kim <grufwub@gmail.com>

* improve log format argument quoting

Signed-off-by: kim <grufwub@gmail.com>

* improve code commenting of collection paging

Signed-off-by: kim <grufwub@gmail.com>

* only dereference announce's originating status if _not_ local. update DereferenceThread() signature. cleanup searchStatusByURI()

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-25 13:09:41 +02:00
tobi
78409f1985
[bugfix] Wrap media reader in length reader to determine length if no content-length given (#848)
* use lengthReader 2 determine fileSize if not given

* update tests

* small fixes

* go fmt
2022-09-24 11:11:47 +02:00
tobi
69a193dae5
[feature] Allow delivery to sharedInboxes where possible (#847)
* update Activity

* add instance-deliver-to-shared-inboxes setting

* update activity version again

* add SharedInboxURI field to accounts

* serdes for endpoints/sharedInbox

* deliver to sharedInbox if one is available

* update tests

* only assign shared inbox if shared domain

* look for shared inbox if currently nil

* go fmt

* finger to get params.RemoteAccountID if necessary

* make comments clearer

* compare dns more consistently
2022-09-23 20:27:35 +01:00
tobi
4cf76a2bfc
[chore] Tidy up status deletion, remove from cache too (#845)
* add func for deleting status from db + cache

* move deletes entirely back to processor
and also only do a delete if the requesting account owns the item being deleted

* tidy up unboost processing

* delete status more efficiently

* fix wrong account id on remote test attachments

* fix federator test
2022-09-21 19:55:52 +02:00
tobi
de26924a4a
don't error out if storage key already exists (#840) 2022-09-19 12:59:11 +01:00
tobi
3777f5c684
[bugfix] Server and closer bugfixes (#839)
* defer streaming from storage more forcefully

* shut down Server more gracefully

* use command context as server BaseContext
2022-09-19 12:43:22 +01:00
tobi
c1585d5f8a
[bugfix] Fix domains not being unblockable, log internal server errors from API (#833)
* log internal server errors from 500 api calls

* don't exec into nil dest

* don't exec into nil dest

* log error in router logger not api errorhandling

* update logging a tad

* linter
2022-09-17 19:12:12 +02:00
tobi
1149310673
[bugfix] Fix emojis, attachments, and mentions not being serialized correctly sometimes via AP (#829) 2022-09-15 11:29:05 +02:00
tobi
6a10123790
[docs] Swagger document /api/v1/custom_emojis (#824) 2022-09-13 13:30:07 +02:00
tobi
2b34a9b51c
[bug] fix unique constraint conflict (#820) 2022-09-12 13:58:31 +02:00
tobi
b42469e4e0
[feature] Allow users to set custom css for their profiles + threads (#808)
* add custom css account property + db func to fetch

* allow account to get/set custom css

* serve custom css for an account

* go fmt

* use monospace for customcss, add link

* add custom css to account cache

* fix broken field

* add custom css docs to user guide

* add `accounts-allow-custom-css` config flag

* add allow custom css to /api/v1/instance response

* only show/set custom css if allowed to do so

* only set/serve custom account css if enabled

* update swagger docs

* chain promise

* make bool a bit clearer

* use cache for GetAccountCustomCSSByUsername
2022-09-12 13:14:29 +02:00
tobi
268f252e0d
[feature] Fetch + display custom emoji in statuses from remote instances (#807)
* start implementing remote emoji fetcher

* update status where pk

* aaa

* tidy up a little

* check size limits for emojis

* thank you linter, i love you <3

* update swagger docs

* add emoji dereference test

* make emoji max sizes configurable

* normalize db.ErrAlreadyExists
2022-09-12 13:03:23 +02:00
tobi
a26280b31a
[bugfix] Fix preposterous characters reserved per URL amount (#809)
* clarify docs

* tidy up consts, set reserved chars much lower

* update tests
2022-09-08 12:36:42 +02:00
f0x52
d09ddb4769
[feature] opengraph meta tags (#806)
* f0x gitignore additions

* better meta title and descriptions

* user avatar icon for thread and profile meta tags

* use proper tag for image

* whitespace

* add noescapeAttr template function

* use ogMeta struct for opengraph

* maxOGDescriptionLength = 300

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-07 16:53:12 +02:00
tobi
a872ddebe6
[feature] Custom emoji updates (serve emoji via s2s api, tune db models) (#805)
* migrate emojis

* add get emoji to s2s (federation) API

* add new emoji db + cache functions

* add shortcodeDomain lookup for emojis

* check existing emojis w/cache, not w/constraints

* go fmt

* add putEmoji func

* use new db emoji funcs instead of where

* remove emojistringstotags func

* add unique constraint back in

* fix up broken migration

* update index
2022-09-06 12:42:55 +02:00
tobi
2db0c64738
[bugfix] Use 'Image' instead of unrecognized 'Gif' type for media attachments (#801)
* Store gifs as Image type

* remove Gif attachment type, add Gifv type

* update test
2022-09-04 14:58:58 +02:00
tobi
006c8b604b
[feature] Set default header and avatar for API accounts to GtS ones (#799)
* validate web-asset-base-dir

* move default icons into converter

* always ensure avatar + header on api accounts

* update tests

* add default header

* don't return error from web module creation anymore

* tidy a bit

* use pngs for default avatars rather than svgs
2022-09-04 14:41:42 +02:00
tobi
4e13408fd4
[bugfix] Fix status fields in_reply_to_id and in_reply_to_account_id not being nullable (#798)
* make reply status fields nullable pointers

* update tests
2022-09-02 17:00:11 +02:00
kim
d68c04a6c0
[performance] cache recently allowed/denied domains to cut down on db calls (#794)
* fetch creation and fetching domain blocks from db

Signed-off-by: kim <grufwub@gmail.com>

* add separate domainblock cache type, handle removing block from cache on delete

Signed-off-by: kim <grufwub@gmail.com>

* fix sentinel nil values being passed into cache

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-02 12:17:46 +02:00
Blackle Morisanchetto
bf9d146987
[feature] Federate custom emoji (outbound) (#791)
* Federate local custom emoji

* Add test for converting a status with tags to AP
2022-09-02 12:11:43 +02:00
kim
077e66381f
[performance] cache account db lookups by public key URI (#795)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-02 11:58:42 +02:00
kim
614ab12733
[performance] use GetAccountByUsernameDomain() for local account lookups to rely on cache (#793)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-02 11:56:33 +02:00
Blackle Morisanchetto
1e1cdee06a
[feature] Emojify spoiler and content in web templates (#785)
* Emojify spoiler and content in web templates

* Use more performance emojify code (thanks NyaaaWhatsUpDoc!)
2022-09-02 11:54:32 +02:00
Blackle Morisanchetto
ecb97f4e0b
[feature] Add support for the exclude_types[] parameter on the notifications endpoint (#784)
* Add support for the exclude_types[] parameter on the notifications endpoint

* Add swagger docs to notifications
2022-08-31 19:20:52 +02:00
Blackle Morisanchetto
f01492ae48
[bugfix] Use custom blackfriday renderer to only add mention/hashtag links in normal text (#787)
* Use custom blackfriday renderer to only add mention/hashtag links in normal text

* Add additional markdown tests
2022-08-31 17:40:11 +02:00
tobi
0245c606d7
[chore] Test fixes (#788)
* use 'test' value for testrig storage backend

* update test dependency

* add WaitFor func in testrig

* use WaitFor function instead of time.Sleep

* tidy up tests

* make SentMessages a sync.map

* go fmt
2022-08-31 17:31:21 +02:00
nya1
bee8458a2d
[feature] add rate limit middleware (#741)
* feat: add rate limit middleware

* chore: update vendor dir

* chore: update readme with new dependency

* chore: add rate limit infos to swagger.md file

* refactor: add ipv6 mask limiter option

Add IPv6 CIDR /64 mask

* refactor: increase rate limit to 1000

Address https://github.com/superseriousbusiness/gotosocial/pull/741#discussion_r945584800

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-08-31 12:06:14 +02:00
Blackle Morisanchetto
daec9ab10e
[feature] Sort follow requests, followers, and following by updated_at (#774)
* Sort follow requests, followers, and following by updated_at

* Add migration to regenerate indexes for follows and follow requests
2022-08-31 11:27:39 +02:00
Blackle Morisanchetto
c5c425b4e7
[bugfix] Fix tusky search issue by returning empty if offset is greater than zero (#786) 2022-08-31 09:57:50 +02:00
Blackle Morisanchetto
5d9c6b0e5a
[bugfix] Do not generate a notification when favouriting your own post (#780) 2022-08-30 11:43:29 +02:00
Blackle Morisanchetto
1f8ecf1c15
[bugfix] Fix issue where the 'favourited' field of a notification's status is always false (#779) 2022-08-30 11:42:52 +02:00
Blackle Morisanchetto
3ce26a60f8
[bugfix] Use reblogged status in notification, instead of wrapper status (#775) 2022-08-29 11:06:37 +02:00
tobi
969c194fcd
[bugfix] Relax outgoing http request queue (#760)
* add request queue trace logging

* fix  misleading wording

* implement request slots per host per method

* undo formatting change (?)

* remove gratuitous trace logging

* rename done -> release
avoids confusion with ctx.Done
2022-08-27 12:00:19 +02:00
Blackle Morisanchetto
54f6caed65
[bugfix] Status visibility + excludeReplies fixes (#769)
* Fix some bugs when viewing a user's posts: include their self-replies (threads) even when excludeReplies is set, and use in_reply_to_uri instead of in_reply_to_id to filter out replies

* Assign values to InReplyToURI when creating statuses. Add index and update old statuses with a migration
2022-08-27 11:35:31 +02:00
Blackle Morisanchetto
e9b5ba0502
[bugfix] Check the length of form.MediaIDs instead of just checking for null (#766) 2022-08-26 17:37:51 +02:00
tobi
79fb8bad04
[feature] Allow footnotes in markdown, use <br> instead of \n (#767)
* allow markdown footnotes + hard line breaks

* don't keep whitespace w/minify (unnecessary now)

* test markdown a bit more
2022-08-26 13:28:06 +02:00
tobi
2fe3a2b5b8
[bugfix] Fix boost of boost issue (#764) 2022-08-23 10:46:10 +02:00
tobi
b96533ca8f
[bugfix] Fix loss of account info on export/import, add tests (#759)
* start adding additional tests

* use random database address for in-memory sqlite

* add more fields to account export
2022-08-22 11:21:36 +02:00
tobi
570fa7c359
[bugfix] Fix potential dereference of accounts on own instance (#757)
* add GetAccountByUsernameDomain

* simplify search

* add escape to not deref accounts on own domain

* check if local + we have account by ap uri
2022-08-20 21:47:19 +01:00
tobi
586ebb5059
return after redirects to web representation (#755) 2022-08-16 19:50:26 +01:00
tobi
ac6ed3d939
[chore] Update bun / sqlite versions; update gtsmodels (#754)
* upstep bun and sqlite versions

* allow specific columns to be updated in the db

* only update necessary columns for user

* bit tidier

* only update necessary fields of media_attachment

* only update relevant instance fields

* update tests

* update only specific account columns

* use bool pointers on gtsmodels
includes attachment, status, account, user

* update columns more selectively

* test all default fields on new account insert

* updating remaining bools on gtsmodels

* initialize pointer fields when extracting AP emoji

* copy bools properly

* add copyBoolPtr convenience function + test it

* initialize false bool ptrs a bit more neatly
2022-08-15 11:35:05 +01:00
tobi
9a9702c964
[bugfix] Check orientation when reading width/height of images (#749) 2022-08-10 15:54:15 +02:00
tobi
91c8d5d20d
[bugfix] Fix thumbnails not taking exif rotation into account (#746)
* use disintegration/imaging instead of nfnt/resize

* update tests

* use disintegration lib for thumbing (if necessary)
2022-08-10 14:05:14 +02:00
tobi
117888cf59
[feature] Add first iteration of a user panel at /user (#736)
* start work on user panel

* parse source first before checking if empty form

* newline

* set avi + header nicely

* add posts settings

* render signin a bit nicer on mobile

* return OK json on successful change

* return unauthorized on bad password

* clarify message on insecure password

* make login a bit prettier

* add alt text + border round image previews

* add logout button

* add password change

* styling updates

* redirect /auth/edit to /user

* update tests

* fix validation tests

* better labels, link to more info

* make submit button generic component

* move submit button inside forms

* add autocomplete labels to password fields

* fix indentation (thx eslint)

* update eslintrc

* eslint: no-unescaped-entities

* initial deduplication between user and admin panel

* add default status/post format setting

* user panel styling for inputs

* update user panel styling, include normalize css

* add placeholder text

* input padding

Co-authored-by: f0x <f0x@cthu.lu>
2022-08-08 10:40:51 +02:00
tobi
879b4abde7
[bugfix] Markdown formatting updates (#743)
* add minify dependency specifically for markdown

* rearrange markdown formatting

* update markdown tests
2022-08-07 18:19:16 +02:00
tobi
f5689a9e5f
[feature] Let accounts set default status format, and use this when processing new statuses (#739)
* add post_format to acct & use it when making post

* update swagger docs

* add status_format updating to frontend

* fix up tests

* post_format => status_format

* add status_format to account validation
2022-08-06 12:09:21 +02:00
tobi
3ab3f58342
[bugfix] Parse source first before checking if empty form (#738) 2022-08-05 12:30:47 +02:00
Artémis
4fdbef04b4
[feature] Implemented notification clear (#720)
* Implemented notification clear

* Added the cache clear mechanism

* added multi user check test
2022-08-01 11:13:49 +02:00
tobi
8fdc9ed552
serve HEAD requests via the fileserver (#735) 2022-07-30 13:42:47 +01:00
tobi
fc81e6443a
[bugfix] remove <= 0 expires_in from oauth token response (#731)
* remove <= 0 expired_in from oauth token response

* go fmt
2022-07-28 16:43:42 +02:00
tobi
8106b69856
[feature] add 'state' oauth2 param to /oauth/authorize (#730) 2022-07-28 16:43:27 +02:00
tobi
7ca5bac7c6
[bugfix] Fix Toot CLI media attachments not working properly (#726) 2022-07-22 13:43:51 +02:00
tobi
73b8839c5d
[bugfix] Make /api/v2/media more compatible with masto API (#724)
* update docs

* make api version into a path param

* update tests

* workaround to unset URL if using v2 of api

* make some fields into pointers
2022-07-22 12:48:19 +02:00
tobi
40f91d052c
[chore] Update image/video size defaults to mastodon's (#723) 2022-07-20 12:25:06 +02:00
tobi
c84384e660
[bugfix] html escape special characters in text instead of totally removing them (#719)
* remove minify dependency

* tidy up some tests

* remove pre + postformat funcs

* rework sanitization + formatting

* update tests

* add some more markdown tests
2022-07-19 15:21:17 +02:00
kim
098dbe6ff4
[chore] use our own logging implementation (#716)
* first commit

Signed-off-by: kim <grufwub@gmail.com>

* replace logging with our own log library

Signed-off-by: kim <grufwub@gmail.com>

* fix imports

Signed-off-by: kim <grufwub@gmail.com>

* fix log imports

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>

* fix package import cycle between config and log package

Signed-off-by: kim <grufwub@gmail.com>

* fix empty kv.Fields{} being passed to WithFields()

Signed-off-by: kim <grufwub@gmail.com>

* fix uses of log.WithFields() with whitespace issues and empty slices

Signed-off-by: kim <grufwub@gmail.com>

* *linter related grumbling*

Signed-off-by: kim <grufwub@gmail.com>

* gofmt the codebase! also fix more log.WithFields() formatting issues

Signed-off-by: kim <grufwub@gmail.com>

* update testrig code to match new changes

Signed-off-by: kim <grufwub@gmail.com>

* fix error wrapping in non fmt.Errorf function

Signed-off-by: kim <grufwub@gmail.com>

* add benchmarking of log.Caller() vs non-cached

Signed-off-by: kim <grufwub@gmail.com>

* fix syslog tests, add standard build tags to test runner to ensure consistency

Signed-off-by: kim <grufwub@gmail.com>

* make syslog tests more robust

Signed-off-by: kim <grufwub@gmail.com>

* fix caller depth arithmatic (is that how you spell it?)

Signed-off-by: kim <grufwub@gmail.com>

* update to use unkeyed fields in kv.Field{} instances

Signed-off-by: kim <grufwub@gmail.com>

* update go-kv library

Signed-off-by: kim <grufwub@gmail.com>

* update libraries list

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter get nerfed

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-07-19 10:47:55 +02:00
tobi
59be7466f3
[bugfix] Markdown format fixes (#718)
* just sanitize markdown, don't minify or escape

* tidy tests, add one for inline code

* add another test, it works!
2022-07-19 10:41:16 +02:00
tobi
c3b6a5b0f9
[feature] Implement cache-control and etags for static assets (#711)
* start working on etag stuff

* add + use cache middleware

* generate etags on the fly

* remove unused field

* clean up filepath

* add license headers to cache files

* add attachgroup function to router interface

* move cache into web module

* rename a couple things

* remove attachStaticFS function from router

* rename + tidy up a few things

* mount assets filesystem

* create assetsFileInfoCache

* update comment

* simplify hash

* fix string fmt

* skip last mod chk, prefer strong etags w/long cache

* move base handler to its own file
this matches the modules in the api folder

* generate new etag if file was modified

* wrap strong etag in quotation marks as per spec

* clarify logic in avatar search

* make hashing a little niftier
2022-07-18 12:55:06 +02:00
tobi
ab03318b7a
[chore] move dialer inside new (#715)
* move dialer inside new, use default resolver

* instantiate resolver
2022-07-18 11:25:26 +02:00
tobi
839c4346ad
[performance] Add long cache-control max age to fileserver (#710) 2022-07-15 13:23:04 +02:00
tobi
6418307c64
[feature] Add back/next buttons to profiles for paging through statuses (#708)
* add GetAccountWebStatuses to db

* add WebStatusesGet func to processor

* don't add limit to next/prev links if 0

* take query params for next/prev statuses

* add separate next + prev links for convenience

* show 'nothing here' message if no statuses exist

* add back / next links to profiles

* allow paging down only

* go fmt ./...

* 'recent public toots' -> 'latest public toots'
2022-07-13 09:57:47 +02:00
kim
6934ae378a
[chore] improved router logging, recovery and error handling (#705)
* move panic recovery to logging middleware, improve logging + panic recovery logic

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* remove skip paths code

Signed-off-by: kim <grufwub@gmail.com>

* re-enable log quoting

Signed-off-by: kim <grufwub@gmail.com>

* use human-friendly bytesize in logging body size

Signed-off-by: kim <grufwub@gmail.com>

* only disable quoting in debug builds

Signed-off-by: kim <grufwub@gmail.com>

* use logrus level instead of debug.DEBUG() to enable/disable quoting

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* fix instance tests

Signed-off-by: kim <grufwub@gmail.com>

* fix gin test contexts created with missing engine HTML renderer

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding not logging query parameters

Signed-off-by: kim <grufwub@gmail.com>

* better explain 'DisableQuoting' logic

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>
2022-07-12 08:32:20 +01:00
tobi
a465cefb8c
[performance] Add new index to fix slow web profile queries (#706)
* start adding new index migration

* fix up index
2022-07-11 12:52:12 +01:00
kim
7cc40302a5
[chore] consolidate caching libraries (#704)
* add miekg/dns dependency

* set/validate accountDomain

* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test

* consolidate to all use same caching libraries

Signed-off-by: kim <grufwub@gmail.com>

* perform more caching in the database layer

Signed-off-by: kim <grufwub@gmail.com>

* remove ASNote cache

Signed-off-by: kim <grufwub@gmail.com>

* update cache library, improve db tracing hooks

Signed-off-by: kim <grufwub@gmail.com>

* return ErrNoEntries if no account status IDs found, small formatting changes

Signed-off-by: kim <grufwub@gmail.com>

* fix tests, thanks tobi!

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-10 17:18:21 +02:00
kim
e75d742afe
[chore] Improve panic handling (#700)
* add miekg/dns dependency

* set/validate accountDomain

* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test

* panics get logged at error level, now include stacktrace

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-08 12:01:44 +02:00
tobi
81dd4f3660
[bugfix] Fix footer info fields not wrapping (#694)
* flex-wrap footer items

* add testrig instance entry with more info set
2022-07-05 16:44:58 +02:00
tobi
b61b000e0a
[bugfix] Fix incorrect domain showing in profiles (#693)
* use instance account_domain in profile

* add instance account_domain field
2022-07-05 14:03:44 +02:00
tobi
4a69651a7c
[bugfix] Visibility fixes (#687)
* test self boost

* only CC to public when necessary
2022-07-04 15:41:20 +02:00
f0x52
c6d1b82f48
[chore] configure instance.Version for testrig (#659)
* configure instance.Version for testrig

* update instancepatch tests

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-03 12:27:32 +02:00
tobi
de4d75246c
[chore] Update pruneOneLocal to use ctx (#684)
* update pruneOneLocal to use ctx

* update test
2022-07-03 12:18:59 +02:00
Dominik Süß
9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi
664713ddd4
[bugfix] Make hashtag regex work with non-ascii characters (#682) 2022-07-03 11:03:03 +02:00
tobi
9e7d022a06
[feature] Cleanup unattached local media (#680)
* add localUnattached db function

* add parseOlderThan util function

* add pruneunusedlocalattachments to media manager

* add unusedlocal pruning to schedule + admin call

* set number of days to keep as a const

* fix test
2022-06-30 12:22:10 +02:00
tobi
07620acc0e
[feature] Use default instance thumbnail if instance account header not set (#672) 2022-06-26 12:33:11 +02:00
tobi
68736efd20
[feature] add configuration to /api/v1/instance response (#670)
* add configuration object to api instance model

* regenerate swagger docs

* add func to return all supported mimes for media

* add instance configuration to api serialization

* fix json tags

* update instance endpoint tests

* fix typeutils tests

* final regen of swagger docs

* omitempty instance configuration
2022-06-26 10:58:45 +02:00
ugla
3e4e57d554
[bugfix] disallow following or blocking yoursel (#667)
Closes #664
2022-06-25 11:14:05 +02:00
tobi
0846b76e93
[bugfix] Fix 404 on status delete redraft (#668)
* add unattach function to media processor

* call delete or unattach appropriately
unattach from client api, delete from federated api

* typo fix
2022-06-24 17:17:40 +02:00
tobi
7eacbd064b
[bugfix] allow setting empty email via instance patch (#665) 2022-06-24 10:43:21 +02:00
tobi
5f00d4980b
[feature] Implement /api/v1/instance/peers endpoint (#660)
* add missing license headers

* start adding instance peers get

* rename domainblock.go

* embed domain in domainblock so it can be reused

* update swagger docs

* add test instances to db

* update tests

* add/update instancepeersget

* update domain model

* add getinstancepeers to db

* instance-expose-peers, instance-expose-suspended

* add auth checks for both current filters

* attach endpoint to router

* include public comment

* obfuscate domain if required

* go mod tidy

* update swagger docs

* remove unnecessary comment

* return 'flat' peerlist if no query params provided
2022-06-23 16:54:54 +02:00
f0x52
7c6c0cd547
[frontend] Profile pages upgrade (#640)
* fix css indentation

* profile styling update

* update status styling to match profile

* empty header fix

* generate random avatars for thread views

* appease the linter gods

* upgrade deps

* turn profile accent into border + $bg background

* upgrade deps

* small accessibility tweaks

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-06-21 10:48:42 +02:00
tobi
8c7945fb78
[bugfix] Account self finger fix (#658)
* check only 1 of status/account when search by uri

* don't try to resolve local uris when searching
2022-06-20 14:11:36 +02:00
tobi
610395d5a5
[chore] make tests more cacheable by avoiding time.Now() (#656) 2022-06-19 17:10:24 +02:00
Mara Sophie Grosch
c48266c459
[bugfix] for the second accounts.note_raw migration (#653) 2022-06-16 19:39:57 +02:00
Mara Sophie Grosch
0e12ee0aa1
postgres locale: fix accounts.note_raw migration (#651)
Database migration 20220506110822_add_account_raw_note.go has some error
handling code to detect some error messages as "ok", but only done for
english error messages. This commit adds a check for the specific error
code, which should be locale agnostic.
2022-06-16 11:22:51 +02:00
tobi
13e4bbdbfa
[chore] Duplicated media cleanup (#649)
* add migration to clean up duplicated media

* use /tmp/gotosocial for testrig storage path

* defer remove storage tempdir

* skip if not attached to status or status not found

* log errors at error level

* only log delete as else clause if successful

* just return nil on down

* reword delete logic a little bit

* check if storage base path is defined

* check for status id more thoroughly

* don't log error if just no rows

* go fmt

* break statusIDLoop when found

* break currentlyUsedLoop when found
2022-06-14 17:00:57 +01:00
tobi
da2386bab1
[chore] Fix testrig emoji bug (#646)
* fix teeny tiny bug in testrig

* Update test case
2022-06-11 16:41:34 +02:00
tobi
7f9925afe5
[chore] Refactor thread dereference a bit for clarity (#647)
* refactor thread dereference a bit for clarity

* lint for the lint gods
2022-06-11 16:25:41 +02:00
tobi
cf5c6d724d
[chore] Validate/set account domain (#619)
* add miekg/dns dependency

* set/validate accountDomain
2022-06-11 11:09:31 +02:00
tobi
dfdc473cef
[chore] Webfinger rework (#627)
* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test
2022-06-11 11:01:34 +02:00
tobi
694a490589
[feature] Add created_at and error_description to /oauth/token endpoint (#645)
* start fiddling about with oauth server

* start returning more helpful errors from oauth

* test helpful(ish) token errors

* add missing license header
2022-06-11 10:39:39 +02:00
tobi
2385b51d58
[bugfix] Make accounts media_only query also work with pg (#643) 2022-06-10 10:56:49 +02:00
tobi
5864954e2e
[bugfix] Fix domain blocks get regression (#642)
* fix domain blocks get regression

* add missing license text headers
2022-06-09 13:28:15 +02:00
f0x52
b43f9ceca9
[frontend] Restructure Frontend Sources (#634)
* 🐸restructure frontend stuff, include admin and future user panel in main repo, properly deduplicate bundles for css+js across uses

* rename bundled to dist, caught by gitignore

* re-include status.css for profile template

* default to localhost

* serve frontend panels

* add todo message for abstraction

* refactor oauth registration flow

* oauth restructure

* update footer template

* change panel routes

* remove superfluous css imports

* write bundle to disk from test server, use forked budo-express

* wrap all page content in container

for robustness with addons etc injection other elements in body

* update documentation, goreleaser, Dockerfile

* update template meta tags

* add AGPL-3.0+ license header everywhere

* only attach update listener on EventEmitter

* cleaner config for various frontend bundles

* fix bundler script paths

* Merge commit 'd191931932b9293ce1be44ed08a1e69b9fcc1e25'

* fix up dockerfile, goreleaser

* go mod tidy

* add uglifyify

* move status hide/show js to frontend bundle

* fix stylesheet color( func regressions

* update contributing docs for new build path

* update goreleaser + docker building

* resolve dependency paths properly

* update package name

* use api errorhandler

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-06-09 12:51:19 +02:00
tobi
1ede54ddf6
[feature] More consistent API error handling (#637)
* update templates

* start reworking api error handling

* update template

* return AP status at web endpoint if negotiated

* start making api error handling much more consistent

* update account endpoints to new error handling

* use new api error handling in admin endpoints

* go fmt ./...

* use api error logic in app

* use generic error handling in auth

* don't export generic error handler

* don't defer clearing session

* user nicer error handling on oidc callback handler

* tidy up the sign in handler

* tidy up the token handler

* use nicer error handling in blocksget

* auth emojis endpoint

* fix up remaining api endpoints

* fix whoopsie during login flow

* regenerate swagger docs

* change http error logging to debug
2022-06-08 20:38:03 +02:00
kim
91c0ed863a
[bugfix] #621: add weak type handing to mapstructure decode (#625)
* Drone sig (#623)

* accept weakly typed input on mapstructure decode i.e. .UnmarshalMap()

Signed-off-by: kim <grufwub@gmail.com>

* add envparsing script to test for panics during environment variable parsing

Signed-off-by: kim <grufwub@gmail.com>

* add envparsing.sh script to drone commands

Signed-off-by: kim <grufwub@gmail.com>

* update drone signature

Co-authored-by: kim <grufwub@gmail.com>

* compare expected with output

* update expected output of envparsing

* update expected output to correct value

* use viper's unmarshal function instead
There were problems with marshalling
string slices from viper into the st.config
struct with the other function. Now, we
can use viper's unmarshal function and pass
in the custom decoder config that we need
as a hook. This ensures that we marshal
string slices from viper into our config
struct correctly.

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-06-08 20:28:28 +02:00
tobi
6f6e89e271
[feature] Add paging via Link header for notifications and account statuses (#629)
* test link headers

* page get account statuses properly

* page get notifications

* add util func for packaging timeline responses

* return timelined stuff from accountstatusesget

* rename timeline response

* use new convenience function

* go fmt
2022-06-08 20:22:49 +02:00
tobi
f3b44426f4
[bugfix] Update time marshalling format to provide 3 digits of ms (#630) 2022-06-04 20:03:01 +02:00
tobi
327d3f001f
[feature] Start adding advanced configuration options, starting with samesite (#628)
* fix incorrect port being used for db

* start adding advanced config flags

* use samesite lax by default
2022-06-03 15:40:38 +02:00
kim
43ac0cdb9c
[chore] Global server configuration overhaul (#575)
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct

Signed-off-by: kim <grufwub@gmail.com>

* improved code comment

Signed-off-by: kim <grufwub@gmail.com>

* linter

Signed-off-by: kim <grufwub@gmail.com>

* fix unmarshaling

Signed-off-by: kim <grufwub@gmail.com>

* remove kim's custom go compiler changes

Signed-off-by: kim <grufwub@gmail.com>

* generate setter and flag-name functions, implement these in codebase

Signed-off-by: kim <grufwub@gmail.com>

* update deps

Signed-off-by: kim <grufwub@gmail.com>

* small change

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter...

Signed-off-by: kim <grufwub@gmail.com>

* move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync

Signed-off-by: kim <grufwub@gmail.com>

* lint

Signed-off-by: kim <grufwub@gmail.com>

* update code comments

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* improved version string (removes time + go version)

Signed-off-by: kim <grufwub@gmail.com>

* fix version string build to pass test script + consolidate logic in func

Signed-off-by: kim <grufwub@gmail.com>

* add license text, update config.Defaults comment

Signed-off-by: kim <grufwub@gmail.com>

* add license text to generated config helpers file

Signed-off-by: kim <grufwub@gmail.com>

* defer unlock on config.Set___(), to ensure unlocked on panic

Signed-off-by: kim <grufwub@gmail.com>

* make it more obvious which cmd flags are being attached

Signed-off-by: kim <grufwub@gmail.com>
2022-05-30 14:41:24 +02:00
tobi
ae5402ada6
[chore] Mastodon api fixups (#617)
* don't omitempty on description

* don't omitempty on any fields

* add ms to timestamp format

* don't omitempty on text_url

* rearrange attachment fields a bit

* just give URL again as attachment text url

* update tests

* fix accidental replace
2022-05-28 18:59:55 +01:00
tobi
2b11d4b7b0
[bugfix] Add accept: application/activity+json to dereferencer (#611)
* add Accept application/activity+json to transport

* add application/activity+json comma-separated
2022-05-27 20:09:57 +01:00
tobi
73d5766572
[chore] Debug failed account get (#612) 2022-05-27 16:44:33 +02:00
tobi
dc8cc7e364
[chore] Add test for dereferencing Owncast service account (#613) 2022-05-27 16:35:35 +02:00
tobi
1cdc163276
[performance] Don't retry/backoff invalid http requests that will never succeed (#609)
* add httpguts (ew)

* add ValidateRequest err wrapping logic

* don't retry on unrecoverable errors

* i am very clever
2022-05-26 13:38:41 +02:00
tobi
0f01f72db0
[performance] Bump default workers to CPUs * 2 (#608)
* add license text to workers.go
* bump default workers to maxprocs*2
2022-05-26 11:51:59 +02:00
tobi
5668ce1ec7
[bugfix] Fix HTML escaping in instance title (#607)
* move caption sanitization -> sanitize.go

* use sanitizeplaintext rather than removehtml

* rename sanitizecaption to sanitizeplaintext

* avoid removing html twice from statuses

* unexport remoteHTML
it's no longer used outside the text package so this
makes it less confusing

* test instance PATCH
2022-05-26 11:37:13 +02:00
tobi
f848aaa81f
[security] Set SameSite to strict instead of browser default (#606) 2022-05-25 18:08:12 +02:00
tobi
a54efa09f9
[chore] Serialize times as UTC ISO8601 instead of RFC3339 (#602)
* add time util to mimic utc ISO8601

* use ISO8601 when serializing to frontend

* update test notification
2022-05-24 17:21:27 +01:00
tobi
196d542e4a
[bugfix] Don't serialize instance account if not set (#603)
* omit contactAccount from json if not set

* test instance serialization
2022-05-24 17:20:13 +01:00
tobi
2d748a68ae
[bugfix] Only search remote if protocol is http(s) (#601) 2022-05-24 13:38:11 +02:00
tobi
21557c92d9
[bugfix] Set refetch to true in iterateDescendants (#600) 2022-05-24 11:00:37 +02:00
tobi
f0c9f4169b
[bugfix] Fix multiple dereferences of boosted status causing media duplication (#589)
* add some announces to test models

* start on announce test logic

* test federatingDB.Announce

* change signature of GetRemoteStatus

* remove 'refresh' logic and replace it with refetch

* go fmt

* remove timeline manager from processor test

* make zork created at determinate

* test get account statuses

* test get + serialize zork

* make account keys determinate

* make admin accountCreate time determinate

* test account to as

* init test config before test log

* test status to frontend

* remove daft Within check

* hack around a bit

* use index of slice
2022-05-23 16:40:03 +01:00
tobi
f5a4f4321a
[bugfix] Fix error extracting status content: no content found (#598)
* don't return error if no content found in Activity

* add test for content extraction

* go fmt
2022-05-23 16:12:46 +01:00
tobi
a09e101931
[bugfix] If status URL is empty, use URI instead and don't log unnecessary error (#597)
* test parse status with no URL

* if no status URL is available, use the URI instead
2022-05-23 16:10:48 +01:00
tobi
469da93678
[security] Check all involved IRIs during block checking (#593)
* tidy up context keys, add otherInvolvedIRIs

* add ReplyToable interface

* skip block check if we own the requesting domain

* add block check for other involved IRIs

* use cacheable status fetch

* remove unused ContextActivity

* remove unused ContextActivity

* add helper for unique URIs

* check through CCs and clean slice

* add GetAccountIDForStatusURI

* add GetAccountIDForAccountURI

* check blocks on involved account

* add statuses to tests

* add some blocked tests

* go fmt

* extract Tos as well as CCs

* test PostInboxRequestBodyHook

* add some more testActivities

* deduplicate involvedAccountIDs

* go fmt

* use cacheable db functions, remove new functions
2022-05-23 11:46:50 +02:00
Adelie Paull
caa0cde0e0
[feature] implement custom_emojis endpoint (#563)
* implement custom_emojis api endpoint

* add tests for getting custom emoji out of the database and converting to api emoji

* change sort direction of emoji query

* change logging level and initialize array with known length as per kim's suggestions

* add continue to lessen risk of making a malformed struct during conversion from db to api emojis
2022-05-20 10:34:36 +02:00
tobi
62d4d756d3
[bugfix] Stop some statuses from being home timelined when they shouldn't be (#585)
* recursively check timelineability of parent status

* check following status creator

* add tests for hometimelineability (whew)

* add test with mix of public + unlocked vis
2022-05-18 22:23:49 +01:00
tobi
b2810fedf2
[bugfix] Clean up boosts of status when the status itself is deleted (#579)
* move status wiping logic to fromcommon.go

* delete reblogs of status when a status is deleted

* add admin boost of zork to test model

* update tests to make them more determinate

* Merge branch 'main' into status_reblog_cleanup

* move status wiping logic to fromcommon.go

* delete reblogs of status when a status is deleted

* add admin boost of zork to test model

* update tests to make them more determinate

* Merge branch 'main' into status_reblog_cleanup

* test status delete via client api

* go fmt
2022-05-18 22:13:03 +01:00
tobi
f4b0d76cd4
[performance] Add further indexes to mitigate laggy queries (#586)
* start adding more indexes as a migration

* update sqlite version
2022-05-18 15:58:26 +01:00
tobi
5ef41ba3f2
[chore] Timeline test updates (#578)
* add admin boost of zork to test model

* update tests to make them more determinate

* remove printf call
2022-05-16 18:48:59 +02:00
tobi
b915a41811
[feature] Basic config validation (#562)
* add optional config validation

* clarify that host and protocol are required

* add validation for host and protocol

* pass prerunArgs as a struct (validate by default)
2022-05-16 14:13:19 +02:00
tobi
b143877995
[feature] Unused avatar and header cleanup (#574)
* rename + tidy up remote pruning

* fix media attachment account join
see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships

* update logging to new function name

* add get avatars and headers to bun

* add pruneallmeta function

* don't set uncached since we're deleting anyway

* fix totalPruned being in wrong place

* test pruning meta

* go fmt ./...

* rename mediaprune

* add meta pruning to routine mediaprune

* tidy up cleanup job scheduling

* rename adminmediaremoteprune

* update mediacleanup to use renamed prune func

* update swagger docs a little bit

* reuse cancel + context
2022-05-15 15:45:04 +01:00
Sashanoraa
6e947ff266
[feature] Media cleanup endpoint (#560)
Adds an admin endpoint to trigger a remote media cleanup.

Fixed #348

Signed-off-by: Sashanoraa <sasha@noraa.gay>
2022-05-15 14:52:46 +02:00
kim
223025fc27
[security] transport.Controller{} and transport.Transport{} security and performance improvements (#564)
* cache transports in controller by privkey-generated pubkey, add retry logic to transport requests

Signed-off-by: kim <grufwub@gmail.com>

* update code comments, defer mutex unlocks

Signed-off-by: kim <grufwub@gmail.com>

* add count to 'performing request' log message

Signed-off-by: kim <grufwub@gmail.com>

* reduce repeated conversions of same url.URL object

Signed-off-by: kim <grufwub@gmail.com>

* move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue

Signed-off-by: kim <grufwub@gmail.com>

* fix security advisories regarding max outgoing conns, max rsp body size

- implemented by a new httpclient.Client{} that wraps an underlying
  client with a queue to limit connections, and limit reader wrapping
  a response body with a configured maximum size
- update pub.HttpClient args passed around to be this new httpclient.Client{}

Signed-off-by: kim <grufwub@gmail.com>

* add httpclient tests, move ip validation to separate package + change mechanism

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* use singular mutex in transport rather than separate signer mus

Signed-off-by: kim <grufwub@gmail.com>

* improved useragent string

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding missing test

Signed-off-by: kim <grufwub@gmail.com>

* remove useragent field from transport (instead store in controller)

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* reset other signing headers on each loop iteration

Signed-off-by: kim <grufwub@gmail.com>

* respect request ctx during retry-backoff sleep period

Signed-off-by: kim <grufwub@gmail.com>

* use external pkg with docs explaining performance "hack"

Signed-off-by: kim <grufwub@gmail.com>

* use http package constants instead of string method literals

Signed-off-by: kim <grufwub@gmail.com>

* add license file headers

Signed-off-by: kim <grufwub@gmail.com>

* update code comment to match new func names

Signed-off-by: kim <grufwub@gmail.com>

* updates to user-agent string

Signed-off-by: kim <grufwub@gmail.com>

* update signed testrig models to fit with new transport logic (instead uses separate signer now)

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter

Signed-off-by: kim <grufwub@gmail.com>
2022-05-15 11:16:43 +02:00
kim
4ac508f037
[chore] Update LE server to use copy of main http.Server{} to maintain server timeouts etc (#571)
* update LE server to use copy of main HTTP server to maintain server timeouts etc

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>
2022-05-15 11:10:55 +02:00
kim
9f2a2abe42
[bugfix] update syslog tests to listen/connect specifically to ipv4 loopback (#561)
Signed-off-by: kim <grufwub@gmail.com>
2022-05-13 13:35:41 +02:00
tobi
898d256511
[bugfix] Fix server trying to listen twice on same address when l.e. enabled (#557) 2022-05-12 11:35:36 +02:00
tobi
8e30671a62
[bugfix] Add account raw note fix (#556)
* hack the latest migration to fix psql issue

* add new migration to fix previous migration

* adjust query
2022-05-11 19:27:25 +02:00
Eamonn O'Brien-Strain
b24b71c0a4
[feature] Include password strength in error message when password strength is too low (#550)
* When password validation fails, return how close to enough entropy it has.

* Shorter version of low-strength password error message
2022-05-09 10:31:46 +02:00
tobi
5004e0a9da
[bugfix] Fix remote media pruning failing if media already gone (#548)
* fix error check of prune to allow missing files

* update go-store library, add test for pruning item with db entry but no file

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unneccessary error check

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: kim <grufwub@gmail.com>
2022-05-08 18:49:45 +01:00
kim
26b74aefaf
[bugfix] Fix existing bio text showing as HTML (#531)
* fix existing bio text showing as HTML

- updated replaced mentions to include instance
- strips HTML from account source note in Verify handler
- update text formatter to use buffers for string writes

Signed-off-by: kim <grufwub@gmail.com>

* go away linter

Signed-off-by: kim <grufwub@gmail.com>

* change buf reset location, change html mention tags

Signed-off-by: kim <grufwub@gmail.com>

* reduce FindLinks code complexity

Signed-off-by: kim <grufwub@gmail.com>

* fix HTML to text conversion

Signed-off-by: kim <grufwub@gmail.com>

* Update internal/regexes/regexes.go

Co-authored-by: Mina Galić <mina.galic@puppet.com>

* use improved html2text lib with more options

Signed-off-by: kim <grufwub@gmail.com>

* fix to produce actual plaintext from html

Signed-off-by: kim <grufwub@gmail.com>

* fix span tags instead written as space

Signed-off-by: kim <grufwub@gmail.com>

* performance improvements to regex replacements, fix link replace logic for un-html-ing in the future

Signed-off-by: kim <grufwub@gmail.com>

* fix tag/mention replacements to use input string, fix link replace to not include scheme

Signed-off-by: kim <grufwub@gmail.com>

* use matched input string for link replace href text

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code (to appease linter :sobs:)

Signed-off-by: kim <grufwub@gmail.com>

* improve hashtagFinger regex to be more compliant

Signed-off-by: kim <grufwub@gmail.com>

* update breakReplacer to include both unix and windows line endings

Signed-off-by: kim <grufwub@gmail.com>

* add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts

Signed-off-by: kim <grufwub@gmail.com>

* drop unnecessary code

Signed-off-by: kim <grufwub@gmail.com>

* update text package tests to fix logic changes

Signed-off-by: kim <grufwub@gmail.com>

* add raw note content testing to account update and account verify

Signed-off-by: kim <grufwub@gmail.com>

* remove unused modules

Signed-off-by: kim <grufwub@gmail.com>

* fix emoji regex

Signed-off-by: kim <grufwub@gmail.com>

* fix replacement of hashtags

Signed-off-by: kim <grufwub@gmail.com>

* update code comment

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: Mina Galić <mina.galic@puppet.com>
2022-05-07 17:55:27 +02:00
kim
08eb271a4c
[performance] improved logrus output switching performance (#544)
* improved logrus output switching performance

Signed-off-by: kim <grufwub@gmail.com>

* add license to test file

Signed-off-by: kim <grufwub@gmail.com>
2022-05-07 17:46:59 +02:00
kim
a561ef3541
[feature] Update media manager to use internal/worker package (#543)
* update media manager to use internal/worker package, update worker with better logging

Signed-off-by: kim <grufwub@gmail.com>

* fix Queue() trace log message format operators

Signed-off-by: kim <grufwub@gmail.com>

* update media manager comment to match updated worker implementation

Signed-off-by: kim <grufwub@gmail.com>
2022-05-07 17:36:01 +02:00
tobi
d7b46a4b63
[chore] Move initial MediaRemotePrune to cmd.server function (#536)
* move initial prune up to the server start function

* don't do initial prune when starting media manager

* add admin/mediaremoteprune in processor
2022-05-03 10:56:18 +01:00
kim
e09958e7f1
[feature] Run initial cache cleanup on gotosocial start in case max-age changed (#533)
Signed-off-by: kim <grufwub@gmail.com>
2022-05-03 11:05:22 +02:00
tobi
b28fd7433a
[chore] Fix inbox iris test to use 'contains' instead of index (#529) 2022-05-02 16:12:56 +02:00
tobi
3112949b6e
[bugfix] Fix panic when relative tmpl path given to router (#528) 2022-05-02 16:06:03 +02:00
Adelie Paull
9265a09a65
[bugfix] Allow self-boosting for any visibility but direct (#510)
* create visibility filter for boostability and allow self-boosting for any visbility but direct messages

* add a followers-only status to local_account_2

* fix typo in comment

* add license header, unwrap errors, be explicit about non-boostable visibility settings to avoid rogue boosting from miscoded clients, use ID compare for checking if self-boosting

* add tests for statusboostable filter

* fix tests that were affected by adding a new status to the test data

* fix the rest of tests affected by adding a status to the textrig data
2022-05-02 15:23:37 +02:00
kim
b56dae8120
[chore] Update all but bun libraries (#526)
* update all but bun libraries

Signed-off-by: kim <grufwub@gmail.com>

* remove my personal build script changes

Signed-off-by: kim <grufwub@gmail.com>
2022-05-02 15:05:18 +02:00
tobi
e06bf9cc9a
add extra indexes as a migration (#527) 2022-05-02 13:55:48 +01:00
tobi
a5852fd7e4
[performance] Speed up some of the slower db queries (#523)
* remove unnecessary LOWER() db calls

* warn during slow db queries

* use bundb built-in exists function

* add db block test

* update account block query

* add domain block db test

* optimize domain block query

* fix implementing wrong test

* exclude most columns when checking block

* go fmt

* remote more unnecessary use of LOWER()
2022-05-02 11:53:46 +01:00
kim
faae2505c0
Add logging to the new generic worker package (#516)
* add logging to generic worker type
2022-05-01 15:45:15 +01:00
tobi
d93e8ddf75
[chore] Add Federatingactor.Send() tests and log call (#509)
* expose testrig util functions

* add tests for federatingActor

* rename some suite vars
2022-04-29 15:53:04 +02:00
tobi
63f84da3e4
[chore] Inboxes for iri test (#508)
* tidy up some federation tests

* add missing license to following.go

* give zork some followers, as a treat

* test InboxesForIRI

* Go fmt

* update timeline tests
2022-04-29 15:05:13 +02:00
Terin Stock
f9b351438b
[feature] allow absolute path to templates (#507)
Only attempt to join the `WebTemplateBaseDir` with the current working
directory if the user has not configured an absolute path to the
template directory.

This changeset also makes a similar change to the testrig, allowing
tests to configure an alternative location for the templates directory.

Fixes #411

Signed-off-by: Terin Stock <terinjokes@gmail.com>
2022-04-29 11:00:25 +02:00
kim
69011d4901
Add support for running profiling when debug build-tags provided (#491)
* wrap root HTTP handler in debug.WithPprof(), rearrange router.Start() to support this
* remove unused code
* set debug buildtag in build script when $DEBUG set
* update go-debug version with fixed handler
* use clone of router.srv for LE cert manager, reset server timeouts in debug
* add kim's other libraries to README
2022-04-28 13:32:53 +01:00
kim
420e2fb22b
replace async client API / federator msg processing with worker pools (#497)
* replace async client API / federator msg processing with worker pools
* appease our lord-and-saviour, the linter
2022-04-28 13:23:11 +01:00
kim
cc5f2e98b7
[bugfix] Fix possible race condition in federatingdb (#490)
Signed-off-by: kim <grufwub@gmail.com>
2022-04-28 11:18:27 +02:00
Terin Stock
8e80f983b3
[bugfix] use Exec to rename media_attachments (#498)
The `database/sql` package in the Go stdlib has two methods that can run
queries against the database driver: `Query` and `Exec`. When the query
returns rows, such as "SELECT", the package expects the use of `Query`,
and returns `*sql.Rows`. When the query does not return rows, the
package expects the use of `Exec`, which returns `sql.Result`.

This changeset corrects the "media_attachments" migration to using
`ExecContext`, as "ALTER TABLE" does not return rows.

Signed-off-by: Terin Stock <terinjokes@gmail.com>
2022-04-28 08:43:41 +01:00
tobi
9cf66bf298
[chore] Return more useful errors from auth failure (#494)
* try rsa_sha256 sig algo first

* return more informative errors from auth

* adapt to reworked auth function
2022-04-26 18:10:11 +02:00
tobi
728c4a5e38
[bugfix] Trim log entries to 1700 chars before they enter syslog (#493)
* start implementing trimming hook

* add test with very long test

* test syslog w/ unix socket + long (trimmed) msg

* trim long entries with trimhook

* trim to 1700 chars instead
2022-04-26 17:55:24 +02:00
tobi
2259838108
[bugfix] Fix CWs not showing sometimes (#488)
* allow summaries that are parsed as iris

* test parsing a status with iri summary
2022-04-26 10:47:21 +02:00
tobi
eeb78bd141
[bugfix] Allow processing of .png files where checksum is not correct (#487)
* add png stripping code from google/wuffs

* experiment with stripping data from pngs

* add test images

* use StrippedPngDecode for pngs

* add StrippedPngDecode func

* update tests for (no)alphachannel pngs

* nolint on copied function
2022-04-25 14:45:44 +02:00
0hlov3
d5fe729cf4
[chore] Add two new test images for media processing (#424) 2022-04-24 14:49:37 +02:00
tobi
0d05bf473b
[bug] Fix status API / status web API being case sensitive (#481)
* make getStatus by id case-insensitive

* test get status case insensitive

* init config before log
2022-04-24 14:24:43 +02:00
tobi
9813a044c0
[feature] Implement media v2 endpoint to accommodate Tusky 17 (#480)
* serve v2 media api

* go fmt
2022-04-24 13:11:52 +02:00
tobi
8d34d5af3c
Revert "[chore] Tidy up federating db locks a tiny bit (#472)" (#479)
This reverts commit 31c1067a54.
2022-04-24 11:56:06 +02:00
tobi
31c1067a54
[chore] Tidy up federating db locks a tiny bit (#472) 2022-04-19 16:31:37 +02:00
tobi
1e3b38573d
[bugfix] Fix infinite domain block database loop (#467)
This fixes an issue where the domain block logic would go into an infinite loop.
2022-04-18 17:44:36 +02:00
tobi
c365863ea9
[bugfix] Use our own (Batch)Deliver implementation for federated messages (#466) 2022-04-18 17:17:05 +02:00
tobi
094f032f74
[feature] Add log-db-queries config option (#465) 2022-04-18 16:47:11 +02:00
tobi
7883dd5499
[bugfix] Convert IDNs to punycode before using as session name (#458)
* convert hostname to punycode for session name

* test punycode
2022-04-16 13:09:42 +02:00
tobi
af97d6bb7e
[bugfix] Use background ctx for domain block side effects (#457) 2022-04-16 12:56:41 +02:00
tobi
26683b3d49
[feature] Web profile pages for accounts (#449)
* add default avatars

* allow webModule to error

* return errWithCode from account get

* add AccountGetLocalByUsername

* check nil requesting account

* add timestampShort function for just month/year

* move loading logic to New + add default avatars

* add profile page view

* update swagger docs

* add excludeReblogs to GetAccountStatuses

* ignore casing when selecting local account by username

* appropriate redirects

* css fiddling

* add 'about' heading

* adjust thread page to work with routing

* return AP representation if requested + authorized

* simplify auth check

* go fmt

* golangci-lint ignore math/rand
2022-04-15 14:33:01 +02:00
tobi
a7e9dee33d
[bugfix] Fix broken only_media and only_public flags on /api/v1/accounts/:id/statuses (#448)
* bump superseriousbusiness/activity version

* fetch list of follower inbox iris

* add test for getting account statuses

* fix incorrect publicOnly parsing

* update media not null query
2022-04-10 15:11:12 +02:00
tobi
707de094a0
[dependency] Update superseriousbusiness/activity dependency (#446)
* bump superseriousbusiness/activity version

* fetch list of follower inbox iris
2022-04-05 18:05:24 +02:00
tobi
03d7c75ebf
[chore] Update Go version to 1.18 (#444)
* linting with new golangci-lint version

* update go to 1.18

* bump versions in drone.yml

* use new runtime/debug package for version info

* remove Commit build flag from goreleaser

* remove mock commit + version from build script

* go fmt

* add dummy version env flag to test container

* install git in golang container for testing

* only set versionString if Version is defined
2022-04-02 15:40:09 +02:00
tobi
37d310f981
[feature] Dereference remote mentions when the account is not already known (#442)
* remove mention util function from db

* add ParseMentionFunc to gtsmodel

* add parseMentionFunc to processor

* refactor search to simplify it a bit

* add parseMentionFunc to account

* add parseMentionFunc to status

* some renaming for clarity

* test dereference of unknown mentioned account
2022-03-29 11:54:56 +02:00
tobi
8cc70a14d6
[bugfix] Fix media uploads blocking after a failed upload #438
safer in case of failure
2022-03-22 12:42:34 +01:00
tobi
25cab0e1f4
[bugfix] Fix images not being processed correctly sometimes (#437)
* bump exif-terminator to latest version

* add and test giant turnip from turnip.farm

* don't error if content property is nil
2022-03-21 19:46:51 +01:00
tobi
36b2f2b4e6
don't error if content property is nil (#436) 2022-03-21 17:10:31 +01:00
tobi
73e9cca701
[bugfix] Close ReadClosers properly in the media package (#434)
* defer lock reader

* close readers when finished with them

* close the reader in the teereader when finished
2022-03-21 13:41:44 +01:00
tobi
55ad6dee71
[feature] Admin account actions (#432)
* add accountAction to the admin API

* model admin account action

* add admin account action to the processor

* add migration for new AdminAccountActions table

* fix accounts admin path

* Update swagger docs
2022-03-19 12:01:40 +01:00
tobi
532c4cc697
[feature] Federate local account deletion (#431)
* add account delete to API

* model account delete request

* add AccountDeleteLocal

* federate local account deletes

* add DeleteLocal

* update transport (controller) to allow shortcuts

* delete logic + testing

* update swagger docs

* more tests + fixes
2022-03-15 16:12:35 +01:00
tobi
e63b653199
[performance] Add dereference shortcuts to avoid making http calls to self (#430)
* update transport (controller) to allow shortcuts

* go fmt

* expose underlying sig transport to allow test sigs
2022-03-15 15:01:19 +01:00
tobi
4b4c935e02
[bugfix] Fix bug where admin panel could not be accessed at /admin (#427)
* clarify comments

* tidy up static serving + add /admin redirect
2022-03-13 18:35:26 +01:00
tobi
e306233166
[bugfix] Fix html-escaped characters in content warnings (#426)
* test status create with odd CWs

* use SanitizeCaption for content warning escaping
2022-03-13 16:37:45 +01:00
tobi
8de928b5e9
[performance] Database optimizations (#419)
* create first index on notifications

* tidy up + add tests

* log queries for trace, ops for debug

* index commonly used fields

* rearrange query

* add a few more indexes

* remove schema-breaking index
(add this back in later)

* re-add cleanup query index
2022-03-07 11:33:18 +01:00
tobi
07727753b9
[feature] Clean up/uncache remote media (#407)
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00
tobi
55b83bea7c
[feature] Add postDataCallbackFunc to allow cleanup (#408) 2022-02-22 13:50:33 +01:00
tobi
15d1e6b3a1
[bugfix] Fix thumbnail image type (#406)
* fix thumbnail content-type

* test fix thumbnail content-type
2022-02-21 11:26:26 +01:00
tobi
a089a98ea9
[feature] Gin enable gzip encoding (#405)
* add gin gzip dependency

* add gzip middleware to router

* go mod tidy
2022-02-19 12:12:41 +01:00
tobi
23034ec145
[feature] Stream files via reader (#404)
* serve files via reader rather than byte slice

* close readcloser when we're done with it

* cast reader to readcloser
2022-02-19 11:44:56 +01:00
Shadowfacts
09d6478d72
[bugfix] Only mark cookies as Secure on https (#398)
Fixes cookies not being stored/sent by Safari when serving over plain http
2022-02-15 16:00:07 +01:00
tsmethurst
8c0141d103 store and retrieve processState atomically 2022-02-08 13:38:44 +01:00
tsmethurst
dba9ad4348 hopefully fix potential race condition 2022-02-08 13:17:10 +01:00
tsmethurst
c4ece24654 log error on header/avatar fetch fail 2022-02-08 12:37:58 +01:00
tsmethurst
b42b0a667e go fmt 2022-02-08 12:19:54 +01:00
tsmethurst
3301148bb7 merge fixup 2022-02-08 12:17:53 +01:00
tsmethurst
4c294a596a Merge branch 'main' into media_refactor 2022-02-08 12:17:18 +01:00
tsmethurst
e811d03cc9 update outdated comment 2022-02-08 10:08:27 +01:00
Forest Johnson
6ed368cbeb
[feature] add authorization to the already-existing authentication (#365)
* add ensureUserIsAuthorizedOrRedirect to /oauth/authorize

* adding authorization (email confirm, account approve, etc) to TokenCheck

* revert un-needed changes to signin.go

* oops what happened here

* error css

* add account.SuspendedAt check

* remove redundant checks from oauth util Authed function

* wip tests

* tests passing

* stop stripping useful information from ErrAlreadyExists

* that feeling of scraping the dryer LINT off the screen

* oops I didn't mean to get rid of this NewTestRouter function

* make tests work with recorder

* re-add ConfigureTemplatesWithGin to handle template path err

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-02-07 12:04:31 +01:00
tobi
5c9d20cea3
don't bother negotiating Accept for webfinger (#381) 2022-02-06 15:00:57 +01:00
tobi
85b4f96263
if accountDomain isn't set, use Host value instead (#380) 2022-02-06 14:07:38 +01:00
tobi
1b36e85840
[feature] Rework timeline code to make it useful for more than just statuses (#373)
* add preparable and timelineable interfaces

* initialize timeline manager within the processor

* generic renaming

* move status-specific timeline logic into the processor

* refactor timeline to make it useful for more than statuses
2022-02-05 12:47:38 +01:00
tobi
18e7537393
[bug] Fix OIDC users requiring second approval (#371)
* tidy up NewSignup
* pre-approve users created via OIDC
2022-01-31 16:03:47 +01:00
Forest Johnson
5be8a7a7ea
[bug] Send plaintext emails to fix "message refused: Message is not RFC 2822 compliant" (#366)
* trying to fix "message refused: Message is not RFC 2822 compliant"

* fix "message refused: Message is not RFC 2822 compliant"

550 5.7.1 Delivery not authorized, message refused: Message is not RFC
2822 compliant

* remove silly regex

* lint

* fix tests

* we should use text/template instead of html/template now
2022-01-31 11:46:20 +01:00
tobi
959e38ac5c
[bug] Fix sqlite empty address issue (#370)
* error when empty address has been set for sqlite

* better explain sqlite db-address setting
2022-01-30 17:06:28 +01:00
tobi
66175c8ad9
[bug] Fix minimum description length check (#369)
* add/update tests

* don't check min description length on media create
2022-01-30 16:10:53 +01:00
tobi
e13718148d
[feature] Make Let's Encrypt integration an opt-in feature (#368) 2022-01-30 14:57:33 +01:00
tsmethurst
d9a778a795 put store lock file in base of storage 2022-01-29 12:16:12 +01:00
tsmethurst
c156602c66 ensure blocking calls to getRemoteAccount before showing stuff to client 2022-01-25 13:48:13 +01:00
tsmethurst
5d9e9e0e7f fix up some account conversion logic 2022-01-25 12:03:25 +01:00
tsmethurst
eba66d3a88 only update account in db if changed 2022-01-25 11:21:22 +01:00
tsmethurst
926b37e271 change getaccount function signature 2022-01-24 18:12:42 +01:00
tsmethurst
a6fb93ae2a ctx => innerctx 2022-01-24 18:12:28 +01:00
tsmethurst
c12520167d use background context w/deadline 2022-01-24 18:12:04 +01:00
tsmethurst
8e6ba1de52 start trying to figure out why this test is failing 2022-01-24 18:11:38 +01:00
tsmethurst
0e7f24ff26 test with disk storage as well 2022-01-24 18:11:27 +01:00
tsmethurst
667e7f112c update remote account get/deref logic 2022-01-24 13:12:17 +01:00
tsmethurst
3c1eb155e4 add file size checks 2022-01-23 15:52:18 +01:00
tsmethurst
c157b1b20b rework data function to provide filesize 2022-01-23 14:41:58 +01:00
tsmethurst
589bb9df02 pass reader around instead of []byte 2022-01-16 18:52:55 +01:00
tsmethurst
723bfe8944 lint, fmt 2022-01-15 17:41:18 +01:00
tsmethurst
6bf39d0fc1 emoji code passing muster 2022-01-15 17:36:15 +01:00
tsmethurst
c4a533db72 start fixing up emoji processing code 2022-01-15 14:33:58 +01:00
tsmethurst
33ca5513ad pin instanceAccountID to in-process emoji 2022-01-11 17:51:45 +01:00
tsmethurst
113f9d9ab4 pass a function into the manager, start work on emoji 2022-01-11 17:49:14 +01:00
tsmethurst
e0f9323b9a test the media manager a bit, add shutdown logic 2022-01-10 18:36:09 +01:00
tsmethurst
0ef478584c add async test 2022-01-09 18:52:28 +01:00
tsmethurst
dccf21dd87 tests are passing, but there's still much to be done 2022-01-09 18:41:22 +01:00
tsmethurst
f61c3ddcf7 compiling now 2022-01-08 17:17:01 +01:00
tsmethurst
c2ff8f392b further refinements 2022-01-08 13:45:42 +01:00
tsmethurst
7ebe0f6a15 start working on thumb + full funcs 2022-01-04 17:37:54 +01:00
tsmethurst
8abfa7751a return very partial image on first upload 2022-01-03 17:37:38 +01:00
tsmethurst
2f57eb5ece fiddle around with workers 2022-01-02 15:00:53 +01:00
tsmethurst
c4d63d125b more refactoring, media handler => manager 2021-12-28 16:36:00 +01:00
tsmethurst
6803c1682b start refactor of media package 2021-12-27 18:03:36 +01:00
embr
ed9158fa05
[fix] Make postgres connections magically work in common setups (#352)
* Don't use the system 'postgres' database by default

* Use postgres adapter defaults

The pgx code actually goes to great lengths to make postgres connections
Just Work(tm) out of the box, including supporting `~/.pg_service.conf`,
SSL certificates, UNIX sockets if it can find a socket at a common path,
and falling back to TCP to localhost if not.

(On Windows, it won't try to use UNIX sockets, but will read credentials
from %appdata% as is standard over there.)

By applying our flags as overrides only when they're specified, database
connections should Just Work(tm) anywhere `psql gotosocial` does.
2021-12-21 12:08:27 +01:00
tobi
ef5a9256a8
Extend license notices to 2022 (#354) 2021-12-20 18:42:19 +01:00
tobi
cb8688f429
Remove unnecessary storage config variables (#344)
* rewire config to not use extraneous serve vars

* rename 'file' to 'local' for consistency

* use Type and Size again
2021-12-20 15:19:53 +01:00
tobi
2582515b4d
return first offer when no accept header set (#351) 2021-12-20 11:08:54 +01:00
embr
86e8e7fd21
Log when listening (#350)
* Log when starting letsencrypt endpoint + clearer errors

Running `gotosocial server` with the default configuration will try to
bind to :80 and listen for letsencrypt challenges, which will fail if
running as non-root (w/o capabilities), or if eg. nginx hogs the port.

When that happens, this should make it more obvious what's wrong.

* Log what address/port we're listening on

Always nice not to have to guess. Also feels more consistent than just
doing it for the letsencrypt endpoint.
2021-12-20 10:34:57 +01:00
tobi
c111b239f7
Add optional syslog logrus hook (#343)
* add optional syslog logrus hook

* document syslog
2021-12-12 18:00:20 +01:00
tobi
67ac8db190
Upstep Go dependencies (#340)
* Upstep Go dependencies

* tiny linter fix

* Tidy
2021-12-12 15:47:51 +01:00
tobi
e2daf0f012
Add Accept header negotiation to relevant API endpoints (#337)
* start centralizing negotiation logic for API

* swagger document nodeinfo endpoint

* go fmt

* document negotiate function

* use content negotiation

* tidy up negotiation logic

* negotiate content throughout client api

* swagger

* remove attachment on Content

* add accept header to test requests
2021-12-11 17:50:00 +01:00
tobi
0884f89431
Implement Cobra CLI tooling, Viper config tooling (#336)
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
tobi
ce22e03f9d
Require confirmed email when checking oauth token (#332)
* move token checker to security package

* update tests with new security package

* add oauth token checking to security package

* check if user email confirmed when parsing token
2021-11-27 14:53:34 +01:00
Forest Johnson
ab316d2250
Fix broken defaults and broken flags (#314)
* start with a default config, not an empty config.

* some data structures were present on Empty config but not Default config

* the monkey patched CLIContext is working

* remove print debugging log

* make the behaviour of the flags consistent across all data types

Conflicts:
	internal/config/config.go

* try to fix accidentally broken test
2021-11-24 17:01:38 +01:00
tobi
3caae376e7
Fix streamed messages ending up in wrong timeline(s) (#325)
* define timeline consts

* remove double stream of status

* change test stream creation up a bit

* stream messages more selectively

* add test for streaming new status creation via clientAPI

* tidy code + comments a bit

* tidy up tests

* make sure new status isn't streamed to public
2021-11-22 19:03:21 +01:00
tobi
79ccd8fd8a
Fix mentioned accounts visibility bug (#323)
* update other tests

* set test status to followers_only

* add test dm

* fix mentioned accounts not being added to relevantAccounts

* add some visibility tests for statuses
2021-11-22 14:40:23 +01:00
tobi
b46e5fb65d
Fix incorrect target being used in CC prop for mentioning statuses (#322)
* hardcode time on admin_account_status_3

* add test for as status with mention

* fix incorrect target URI being used for mentions
2021-11-22 12:48:46 +01:00
tobi
24f9e11221
Fix image description unnecessarily html-escaping innocent characters (#321)
* implement SanitizeCaption function

* tidy up text test setup
2021-11-22 11:49:11 +01:00
tobi
1ded58b34b
add bindAddress configuration option (#320)
* add bindAddress configuration option

* clarify that bindAddress can be a hostname
2021-11-22 10:55:52 +01:00
tobi
6c0550e76f
properly initialize user client module (#319) 2021-11-22 09:04:16 +01:00
tobi
f8630348b4
Enable stricter linting with golangci-lint (#316)
* update golangci-lint

* add golangci config file w/ more linters

* correct issues flagged by stricter linters

* add more generous timeout for golangci-lint

* add some style + formatting guidelines

* move timeout to config file

* go fmt
2021-11-22 08:46:19 +01:00
tobi
38d73f0316
tidy up database creation logic a bit (#317) 2021-11-21 17:41:51 +01:00
tobi
f3e36de78a
fix broken GetInstanceAccount selector (#311)
* fix broken GetInstanceAccount selector

* patch tests
2021-11-17 14:40:39 +01:00
tobi
ff7c96caa3
Fix instance account not being created (#310)
* Rework 'CreateInstanceAccount' with better checking logic

* add test for create instance account
2021-11-17 13:40:09 +01:00
tobi
aee1932bd4
First db migration (#297)
* store first db schema

* md linting *screams*
2021-11-13 17:30:01 +01:00
tobi
09ef9e639e
move to ssb gofed fork (#298) 2021-11-13 17:29:43 +01:00
tobi
829a934d23
update dependencies (#296) 2021-11-13 12:29:08 +01:00
tobi
2aaec82732
smtp + email confirmation (#285)
* add smtp configuration

* add email confirm + reset templates

* add email sender to testrig

* flesh out the email sender interface

* go fmt

* golint

* update from field with more clarity

* tidy up the email formatting

* fix tests

* add email sender to processor

* tidy client api processing a bit

* further tidying in fromClientAPI

* pin new account to user

* send msg to processor on new account creation

* generate confirm email uri

* remove emailer from account processor again

* add processCreateAccountFromClientAPI

* move emailer accountprocessor => userprocessor

* add email sender to user processor

* SendConfirmEmail function

* add noop email sender

* use noop email sender in tests

* only assemble message if callback is not nil

* use noop email sender if no smtp host is defined

* minify email html before sending

* fix wrong email address

* email confirm test

* fmt

* serve web hndler

* add email confirm handler

* init test log properly on testrig

* log emails that *would* have been sent

* go fmt ./...

* unexport confirm email handler

* updatedAt

* test confirm email function

* don't allow tokens older than 7 days

* change error message a bit

* add basic smtp docs

* add a few more snippets

* typo

* add email sender to outbox tests

* don't use dutch wikipedia link

* don't minify email html
2021-10-31 15:46:23 +01:00
tobi
4b1d9d3780
Serve outbox for Actor (#289)
* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* bit more work on outbox paging

* wrapNoteInCreate function

* test + hook up the processor functions

* don't do prev + next links on empty reply

* test get outbox through api

* don't fail on no status entries

* add outbox implementation doc

* typo
2021-10-24 11:57:39 +02:00
kim
7e4c3fa5c7
fix mention extracting when no domain exists (usually intra-instance mentions) (#272)
* fix mention extracting when no domain exists (usually when intra-instance mentions)

Signed-off-by: kim <grufwub@gmail.com>

* fix search logic to match new mention matching logic

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter :p

Signed-off-by: kim <grufwub@gmail.com>
2021-10-17 14:19:49 +02:00
tobi
15621f5324
Follow request improvements (#282)
* tiny doc update

* add rejectfollowrequest to db

* add follow request reject to processor

* add reject handler

* tidy up follow request api

* tidy up federation call

* regenerate swagger docs

* api endpoint tests

* processor test

* add reject federatingdb handler

* start writing reject tests

* test reject follow request

* go fmt

* increase sleep for slow test setups

* more relaxed time.sleep
2021-10-16 13:27:43 +02:00
tobi
107685e22e
User password change (#280)
* start passwordChangeHandler

* add user scope

* add user module / api path

* add password change request

* make comment clearer

* add user to processor

* required true

* add processor call to handler

* don't pass tc or channel

* change password func + tests

* add some first docs about password management

* update swagger docs

* add api tests

* go fmt

* test fixes
2021-10-14 14:26:04 +02:00
tobi
a07e62e49e
go fmt (#278) 2021-10-13 13:50:24 +02:00
R. Aidan Campbell
083099a957
reference global logrus (#274)
* reference logrus' global logger instead of passing and storing a logger reference everywhere

* always directly use global logrus logger instead of referencing an instance

* test suites should also directly use the global logrus logger

* rename gin logging function to clarify that it's middleware

* correct comments which erroneously referenced removed logger parameter

* setting log level for tests now uses logrus' exported type instead of the string value, to guarantee error isn't possible
2021-10-11 14:37:33 +02:00
tobi
367bdca250
Handle forwarded messages (#273)
* correct path of foss_satan

* add APIri and notes

* test create forward note

* rename target => receiving account

* split up create into separate funcs

* update extractFromCtx

* tidy up from federator processing

* foss satan => http not https

* check if status in db

* mock dereference of status from IRI

* add forward message deref test

* update test with activities

* add remote_account_2 to test rig
2021-10-10 12:39:25 +02:00
tobi
3dc7644ae6
Derive visibility fixes (#271)
* use pub public const

* don't error on no summary

* move extract visibility to separate function

* extract visibility test

* add addressable interface
2021-10-06 18:18:02 +02:00
tobi
ff77bf1d11
fix logs not working properly (#264) 2021-10-04 16:00:35 +02:00
tobi
e04b187702
Refactor/tidy (#261)
* tidy up streaming

* cut down code duplication

* test get followers/following

* test streaming processor

* fix some test models

* add TimeMustParse

* fix uri / url typo

* make trace logging less verbose

* make logging more consistent

* disable quote on logging

* remove context.Background

* remove many extraneous mastodon references

* regenerate swagger

* don't log query on no rows result

* log latency first for easier reading
2021-10-04 15:24:19 +02:00
tobi
9ce4234b9f
Follow request auto approval (#259)
* start messing about

* fiddle more

* Tests & fiddling
2021-10-01 19:08:50 +02:00
tobi
365c3bf5d7
rework mention replacement func (#258) 2021-10-01 14:10:20 +02:00
tobi
28b6ce59d6
don't catch mentions within links (#257) 2021-09-30 18:11:57 +02:00
tobi
0cd2bd2960
allow dereferencing of groups (#256) 2021-09-30 12:27:42 +02:00
tobi
231075f28d
Golint (#255) 2021-09-30 11:16:23 +02:00
tobi
36a09dd0df
handle remote account deletion more systematically (#254) 2021-09-30 10:56:02 +02:00
tobi
08cb8a3385
skip account on error instead of returning error (#251)
* skip account on error instead of returning error

* still return error on a real error
2021-09-28 16:19:13 +02:00
tobi
b5a7e1ba32
Account update issue (#250)
* start poking around

* tests

* notes and fiddling
2021-09-28 15:21:59 +02:00
tobi
b3fd9c39a3
Weird notif issue (#248)
* start working on weird issue

* go fmt ./...

* more tests
2021-09-27 17:42:20 +02:00
tobi
c7cfbe2702
Unblock fix (#247)
* start tests for inbox posts

* go mod tidy

* rename transferContext

* test block/unblock

* improve logging

* improve logging

* fix comment typo
2021-09-24 17:56:48 +02:00
tobi
ddfd83d0fb
update media processing (#244) 2021-09-23 11:13:11 +02:00
tobi
7e6350b448
remove hardcoded friendica user agent block (#239) 2021-09-21 11:40:16 +02:00
tobi
ffc55e9b15
tweak db settings slightly (#238) 2021-09-20 18:20:21 +02:00
tobi
c7c9fff730
use timestamptz instead of timestamp (#237) 2021-09-20 17:41:52 +02:00
tobi
b315eee43b
Update webfingering a little, add tests (#236)
* Update webfingering a little, add tests

* fix broken tests oops
2021-09-20 16:46:45 +02:00
tobi
92186c8c6f
federated authentication better logging + tidying (#232)
* change trace logging in authenticator

* messing about

* lil changes

* go fmt

* error fix

* Fix broken test
2021-09-16 11:35:09 +02:00
tobi
2e5dcc2929
Fix mentions not notifying (#230)
* set default privacy for new accounts

* teshts

* found it

* tiny change

* aaaa
2021-09-14 12:23:56 +02:00
tobi
69bb56368d
set default privacy for new accounts (#225) 2021-09-13 17:35:33 +02:00
f0x52
026674bc2c
Thread views on the web (#207)
* Webviews for status threads

* fix up templates

* add ForkAwesome and gotosocial-styling into repo

* clean up gotosocial-styling, old styling

* update CONTRIBUTING with new css building, and nodemon recommendation

* update Dockerfile with new css bundling

* those weren't supposed to make it in

* upgrade gotosocial-styling deps

* update authorize template with main wrapper

* update css pipeline

* abstract status from thread to avoid copy-pasting

* basic CW implementation

* fix PR review suggestions

* fix no-image-desc icon alignment

* remove template loading println

* remove println

* remove changes to testmodels

* reset changes to testmodels
2021-09-13 14:45:33 +02:00
kim (grufwub)
6db57d1487 fix keys used to access storage items
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 20:28:57 +01:00
kim (grufwub)
7f36688643 rename KVStore uses as storage to differentiate between upcoming store package
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 20:18:06 +01:00
kim (grufwub)
e43a46e982 add git.iim.gay/grufwub/go-store for storage backend, replacing blob.Storage
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 20:12:47 +01:00
tobi
793f9af6e6
only store statuses in the AS cache, not accounts (#210) 2021-09-11 14:16:00 +02:00
tobi
9dc2255a8f
kim is a reply guy (#208)
* bun debug

* bun trace logging hooks

* more tests

* fix up some stuffffff

* drop the frontend cache until a proper fix is made

* go fmt
2021-09-11 13:19:06 +02:00
tobi
e681aac589
fixes + db changes (#204)
* fixes + db changes

* make duration more lenient
2021-09-10 14:36:10 +02:00
kim
446dbb7a72
remove boosted statuses from public (federated timeline) (#201)
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-10 10:09:15 +02:00
kim
4c4a622088
Prune unnecessary nullzeros, fixup db tags (#200)
* prune gtsmodel.Account bun tags, add note to gtsmodel dir

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* further database tag fixes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* more db tag fixups

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix removing nullzero for account timestamps...

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add nullzero back to accountid tag

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* rename gtsmodel readme

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-10 10:08:21 +02:00
kim
41ace19e0c
fix up status inreplyto visibility, + small format improvements (#199)
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-10 08:37:28 +02:00
tobi
555ea8edfb
Import export (#194)
* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* update bun library -> v1.0.4

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library -> v4.3.1-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* handle oauth token scope, fix user.SigninCount + token.UserID

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library --> v4.3.2-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update sqlite library -> v1.13.0

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* review changes

* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* review changes

Co-authored-by: kim (grufwub) <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2021-09-09 16:15:25 +02:00
kim
a027da0ac9
Merge pull request #198 from NyaaaWhatsUpDoc/update/sqlite-library
Update/sqlite library
2021-09-08 21:26:32 +01:00
kim (grufwub)
71a4f8667c update sqlite library -> v1.13.0
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 21:12:23 +01:00
kim (grufwub)
d2a85f2f5f handle oauth token scope, fix user.SigninCount + token.UserID
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:46:08 +01:00
tobi
2b14b20802
rework media processing a little bit (#191)
* rework media processing a little bit

* review changes
2021-09-04 14:02:01 +02:00
tobi
ff05046df7
tests + announce notification fix (#193) 2021-09-04 13:29:56 +02:00
kim
25edd57eaf
Merge pull request #186 from superseriousbusiness/struct_validation
Struct validation
2021-09-03 10:27:40 +01:00
tsmethurst
32fe2282c4 review changes 2021-09-03 11:12:19 +02:00
tsmethurst
a116676814 Review changes 2021-09-03 10:30:40 +02:00
tsmethurst
5d5327614d lint 2021-09-02 12:24:18 +02:00
tsmethurst
cc7a35ee92 token tests 2021-09-02 12:22:09 +02:00
tsmethurst
42892f91e1 client_test 2021-09-02 12:17:14 +02:00
tsmethurst
6f7da0ecdd account + application tests 2021-09-02 12:10:59 +02:00
tobi
ac7c5e8cd2
session name fix (#185) 2021-09-01 22:12:31 +02:00
tsmethurst
4696e1a7b3 moving stuff around 2021-09-01 18:29:25 +02:00
tsmethurst
684bd56528 move oauth models into gtsmodel 2021-09-01 11:45:01 +02:00
tsmethurst
55a9b8553f lil move 2021-09-01 11:13:01 +02:00
tsmethurst
7b01304dac more updates 2021-09-01 11:12:10 +02:00
tsmethurst
2786b5f887 change muchos things 2021-09-01 11:11:26 +02:00
tsmethurst
fffff93b75 go fmt 2021-09-01 11:11:26 +02:00
tsmethurst
356d28fef9 more work on struct validation 2021-09-01 11:11:26 +02:00
tsmethurst
dc2e1bf9ab more work on struct validation 2021-09-01 11:11:26 +02:00
tsmethurst
8ba816cd83 statusmute annotations 2021-09-01 11:11:26 +02:00
tsmethurst
d2276fc553 start working on struct validation for gtsmodel 2021-09-01 11:11:26 +02:00
kim
7d193de25f
Improve GetRemoteStatus and db.GetStatus() logic (#174)
* only fetch status parents / children if explicity requested when dereferencing

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Remove recursive DB GetStatus logic, don't fetch parent unless requested

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* StatusCache copies status so there are no thread-safety issues with modified status objects

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* remove sqlite test files

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix bugs introduced by previous commit

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix not continue on error in loop

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* use our own RunInTx implementation (possible fix for nested tx error)

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix cast statement to work with SQLite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* be less strict about valid status in cache

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add cache=shared ALWAYS for SQLite db instances

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix EnrichRemoteAccount when updating account fails

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add nolint tag

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* ensure file: prefixes the filename in sqlite addr

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add an account cache, add status author account from db

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix incompatible SQLite query

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* *actually* use the new getAccount() function in accountsDB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update cache tests to use test suite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add RelationshipTestSuite, add tests for methods with changed SQL

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-01 11:08:21 +02:00
kim
ed46224573
Add SQLite support, fix un-thread-safe DB caches, small performance f… (#172)
* Add SQLite support, fix un-thread-safe DB caches, small performance fixes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add SQLite licenses to README

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* appease the linter, and fix my dumbass-ery

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* make requested changes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add back comment

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-29 16:41:41 +02:00
tobi
53507ac2a3
Mention fixup (#167)
* rework mention creation a bit

* rework mention creation a bit

* tidy up status dereferencing

* start adding tests for dereferencing

* fixups

* fix

* review changes
2021-08-29 12:03:08 +02:00
kim
79afcdba3f
Replace federating DB locks map, add a cleanup goroutine (#166)
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-27 21:14:20 +02:00
tobi
add6eb6e2b
Gin logging (#165)
* start messing around with logger

* more messing about

* fix error with instance not created on startup (#156)

* set logger

* log user agent

* finalize logging

* go fmt

* start messing around with logger

* more messing about

* set logger

* log user agent

* finalize logging

* go fmt
2021-08-27 13:26:45 +02:00
kim
87328b6218
fix new tx within tx (#163)
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-27 11:38:24 +02:00
tobi
30f688dbe2
some more little fixes for all to enjoy (#161) 2021-08-26 22:06:34 +02:00
tobi
2162f21636
fix broken db queries in auth (#160) 2021-08-26 19:56:40 +02:00
tobi
1582cf8bad
fix null returned session (#159) 2021-08-26 18:55:39 +02:00
tobi
2eefaa0227
oops (#157) 2021-08-26 17:46:24 +02:00
tobi
14ebc94fd9
fix error with instance not created on startup (#156) 2021-08-26 17:22:41 +02:00
tobi
ddc120d5e6
fix public timeline bug (#150) 2021-08-26 11:28:16 +02:00
tobi
2dc9fc1626
Pg to bun (#148)
* start moving to bun

* changing more stuff

* more

* and yet more

* tests passing

* seems stable now

* more big changes

* small fix

* little fixes
2021-08-25 15:34:33 +02:00
tobi
071eca20ce
Manually approves followers (#146)
* update go-fed

* update go-fed

* manuallyapprovesfollowers

* serialize manuallyApprovesFollowers
2021-08-23 12:46:05 +02:00
tobi
4920229a3b
Database updates (#144)
* start moving some database stuff around

* continue moving db stuff around

* more fiddling

* more updates

* and some more

* and yet more

* i broke SOMETHING but what, it's a mystery

* tidy up

* vendor ttlcache

* use ttlcache

* fix up some tests

* rename some stuff

* little reminder

* some more updates
2021-08-20 12:26:56 +02:00
Tobi Smethurst
ce190d867c
Text/status parsing fixes (#141)
* aaaaaa

* vendor minify

* update + test markdown parsing
2021-08-16 19:17:56 +02:00
Tobi Smethurst
ff406be68f
Timeline loop fix (#140)
* uwu we made a fucky wucky

* uwu we made a fucky wucky

* work on timeline fixes a little

* fiddle with tests some more

* bleep bloop more tests

* more tests

* update drone yml

* update some sturf

* make the timeline code a bit lazier

* go fmt

* fix drone.yml
2021-08-15 18:43:08 +02:00
Tobi Smethurst
98263a7de6
Grand test fixup (#138)
* start fixing up tests

* fix up tests + automate with drone

* fiddle with linting

* messing about with drone.yml

* some more fiddling

* hmmm

* add cache

* add vendor directory

* verbose

* ci updates

* update some little things

* update sig
2021-08-12 21:03:24 +02:00
Tobi Smethurst
329a5e8144
Text duplication fix (#137)
* start testing text duplication

* tests

* fixes + tests
2021-08-11 16:54:54 +02:00
Tobi Smethurst
6acd410426
Bugfixerino (#133)
* fix some lil bugs

* fmt, lint
2021-08-10 16:56:59 +02:00
tsmethurst
93792d3f1e roll back to sha256 for signatures 2021-08-10 13:59:37 +02:00
Tobi Smethurst
0f2de6394a
Dereference remote replies (#132)
* decided where to put reply dereferencing

* fiddling with dereferencing threads

* further adventures

* tidy up some stuff

* move dereferencing functionality

* a bunch of refactoring

* go fmt

* more refactoring

* bleep bloop

* docs and linting

* start implementing replies collection on gts side

* fiddling around

* allow dereferencing our replies

* lint, fmt
2021-08-10 13:32:39 +02:00
Tobi Smethurst
0386a28b5a
Frodo swaggins (#126)
* more swagger fun

* document a whole bunch more stuff

* more swagger yayyyyyyy

* progress + go fmt
2021-08-02 19:06:44 +02:00
Tobi Smethurst
6bd26ff4c4
more swagger docs + other changes (#125)
* more swagger docs + other changes

* go fmt
2021-07-31 23:17:39 +02:00
Tobi Smethurst
58dddd86e0
Swagger (#124)
* start experimenting with swagger documentation

* further adventures in swagger

* do a few more api paths

* account paths documented

* go fmt

* fix up some models

* bit o lintin'
2021-07-31 17:49:59 +02:00
Tobi Smethurst
eb13faf54f
Password change (#123)
* add password change command

* document cli commands

* go fmt
2021-07-31 13:57:23 +02:00
Tobi Smethurst
a940a520d3
Link hashtag bug (#121)
* link + hashtag bug

* remove printlns

* tidy up some duplicated code
2021-07-29 13:18:22 +02:00
Tobi Smethurst
ea8ad8b346
Link parsing (#120)
* add link parsing + formatting functionality

* refinement + docs

* add missing test

* credit url library
2021-07-28 11:42:26 +02:00
Tobi Smethurst
2c2dbe318e
federating db updates (#118) 2021-07-27 10:45:22 +02:00
Tobi Smethurst
ad0e26dc04
Markdown Statuses (#116)
* parse markdown statuses if desired

* add some preliminary docs for writing posts
2021-07-26 20:25:54 +02:00
Tobi Smethurst
e2757ae676
add trusted proxy for parsing client IPs (#115) 2021-07-26 16:15:36 +02:00
Tobi Smethurst
5bf4fb49e0
Tiny fix (#114)
* allow custom ports for webserver and le

* fix teensy mistake not setting default port
2021-07-24 19:31:38 +02:00
Tobi Smethurst
bc1d8a9265
allow custom ports for webserver and le (#111) 2021-07-24 19:02:41 +02:00
Tobi Smethurst
05e9af089c
Oidc (#109)
* add oidc config

* inching forward with oidc idp

* lil webfingy fix

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* add oidc config

* inching forward with oidc idp

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* document

* docs + comments
2021-07-23 10:36:28 +02:00
Tobi Smethurst
113186ce4e
lil webfingy fix (#106)
* lil webfingy fix

* return requested content type from fede requests
2021-07-21 17:46:59 +02:00
Tobi Smethurst
29bdc41baa
Config bugfix (#104)
* fix some config bugs

* go fmt
2021-07-19 19:31:47 +02:00
Tobi Smethurst
b1a4f38e38
allow different host + accountDomain (#103)
* allow different host + accountDomain

* use accountDomain in tags
2021-07-19 18:42:08 +02:00
Tobi Smethurst
677490bc4e
Db tls (#102)
* go mod tidy

* complete example config

* add tls support for db connection

* add certpool to tlsConfig

* add some lil docker scripts
2021-07-19 18:03:07 +02:00
f0x52
bbc2494c58
Static fileserver improvements, optional admin panel route (#100)
* better asset serving, optional admin panel route

* linting
2021-07-14 17:22:51 +02:00
f0x52
f05b255511
Noescape (#88)
* disable html escaping for short description

* provide noescape function for templates

* move template functions before template loading

* go fmt

* linter fixes
2021-07-13 16:05:03 +02:00
Tobi Smethurst
bdba3ff9a9
sanitize html for statuses + instance (#97)
* sanitize html for statuses + instance

* sanitization
2021-07-13 16:03:51 +02:00
Tobi Smethurst
846057f0d6
Block/unblock (#96)
* remote + local block logic, incl. federation

* improve blocking stuff

* fiddle with display of blocked profiles

* go fmt
2021-07-11 16:22:21 +02:00
Tobi Smethurst
c7da64922f
favourites GET implementation (#95) 2021-07-09 18:32:48 +02:00
Tobi Smethurst
fe269cd641
fix up some of the instance patching stuff (#85) 2021-07-08 15:05:19 +02:00
Tobi Smethurst
5460271bb5
Auth flow fixes (#82)
* preliminary fixes to broken auth flow

* fix some auth/cookie weirdness

* fmt
2021-07-08 11:32:31 +02:00
Tobi Smethurst
c71e55ecc4
clean up some weirdness in the router (#80) 2021-07-07 15:46:42 +02:00
Tobi Smethurst
3568579218
Blocklist import (#77)
* first steps on importing blocklists

* unblock domains properly
2021-07-06 13:29:11 +02:00
Tobi Smethurst
d389e7b150
Domain block (#76)
* start work on admin domain blocking

* move stuff around + further work on domain blocks

* move + restructure processor

* prep work for deleting account

* tidy

* go fmt

* formatting

* domain blocking more work

* check domain blocks way earlier on

* progress on delete account

* delete more stuff when an account is gone

* and more...

* domain blocky block block

* get individual domain block, delete a block
2021-07-05 13:23:03 +02:00
Tobi Smethurst
4f3b3f5c0b
put version in binary properly (#73)
Addresses #71 :

    Set version on the CLI framework.
    Add a build.sh script that injects variables into the build tooling using git and a version file.
    Set version in config.
2021-06-28 12:17:20 +02:00
Tobi Smethurst
87cf621e21
Remote instance dereferencing (#70)
Remote instances are now dereferenced when they post to an inbox on a GtS instance.

    Dereferencing will be done first by checking the /api/v1/instance endpoint of an instance.
    If that doesn't work, /.well-known/nodeinfo will be checked.
    If that doesn't work, only a minimal representation of the instance will be stored.

A new field was added to the Instance database model. To create it:

alter table instances add column contact_account_username text;
2021-06-27 16:52:18 +02:00
tsmethurst
869a6c111c Go fmt 2021-06-27 13:58:59 +02:00
Tobi Smethurst
3e6aef00b2
fix the annoying infinite handshake bug (tested) (#69) 2021-06-27 11:46:07 +02:00
Tobi Smethurst
b71bbc86a7
remove regex hostname parsing (#67)
Drop regex validation for postgres hostname, because it was breaking when running in a docker-compose stack where hostnames can be just one word.

If necessary this can be added in again later, but it probably won't be necessary because it doesn't actually add anything useful!
2021-06-26 20:59:38 +02:00
Tobi Smethurst
b6c62309f2
separate public key handler (#64) 2021-06-26 16:21:40 +02:00
Tobi Smethurst
c1e107266f
nodeinfo compliance (#61) 2021-06-24 14:26:08 +02:00
Tobi Smethurst
16e486ad96
Timeline bugfix (#60)
* fix a stack overflow in the timeline

* go fmt
2021-06-23 18:42:20 +02:00
Tobi Smethurst
8c9a853343
Instance settings updates (#59)
Allow admins to set instance settings through a PATCH to /api/v1/instance

Update templates to reflect some of the new fields
2021-06-23 16:35:57 +02:00
f0x52
5a2f3b35ba
add 404 handler (#57) 2021-06-21 21:08:02 +02:00