Commit graph

1161 commits

Author SHA1 Message Date
tobi
689a10fe17
[bugfix] Fix deleted status causing issues when getting bookmark (#1551)
* [bugfix] Delete bookmark when status deleted

* [chore] Give bookmark processing func some love

* fix paging + embetter tests
2023-02-23 22:11:10 +00:00
Daenney
074f352709
[chore] improve opengraph descripiton tag (#1550)
This changes parseDescription to properly encode things to be safe for
usage without removing things like backslashes that may be relevant.

* text.SanitizePlaintext already calls html.UnescapeString so we don't
  have to do that
* Replace \n with space early
* Remove duplicate white-space by splitting on fields and joining
* HTML-escape the string we have
* For extra certainty, encode the backslash as \

Fixes #1549
2023-02-22 21:36:18 +00:00
tobi
b6fbdc66c1
[chore] Deinterface processor and subprocessors (#1501)
* [chore] Deinterface processor and subprocessors

* expose subprocessors via function calls

* missing license header
2023-02-22 16:05:26 +01:00
tobi
adb596600b
[bugfix] Remove initial storage cleanup (#1545) 2023-02-21 10:49:57 +01:00
f0x52
fa70ad757f
[docs] Add site_url for absolute 404 page assets (#1544) 2023-02-21 09:01:10 +01:00
Daenney
40eea3b409
[feature] Use Material theme for docs (#1535)
* [feature] Use Material theme for docs

This changes the theme to use Material with the slate (dark) theme and
the accent colour set to orange. It also replaces the swagger plugin
with one that works correctly with the Material theming.

* Add theme by @f0x52

Co-authored-by: f0x52 <f0x@cthu.lu>

* Enable social cards

The dependencies for cairosvg and pillow can be satisfied on RTD, so
this adds them to our requirements.txt and enables the social plugin.
This generates opengraph and twitter card tags as well as fancy preview
images in GTS colour style for platforms that show that as part of link
previews.

* Update Conda environment.yml

* update theme for swagger, basic light theme option

* dark mode mobile header bg color

---------

Co-authored-by: f0x52 <f0x@cthu.lu>
2023-02-20 23:09:17 +01:00
tobi
87b30a4f91
[docs] correct swagger docs (#1543) 2023-02-20 17:06:36 +01:00
tobi
e8a04b7ce1
[bugfix] Fix account roles (#1542)
* Change account role from string to object

* Update tests

* small fixes + swagger docs

---------

Co-authored-by: zowhoey <11893985+zowhoey@users.noreply.github.com>
2023-02-20 17:00:44 +01:00
f0x52
b6143c9ab8
[feature] About page (#1495)
* about page basics

* more info, styling

* update emoji sizing on about page contact card
2023-02-20 16:29:29 +01:00
dependabot[bot]
e824e14705
[chore]: Bump golang.org/x/crypto from 0.5.0 to 0.6.0 (#1541)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-20 16:28:09 +01:00
Sam Lade
f559d46261
[bugfix] Fix failure to look up remote profiles with duplicate emojis in some cases (#1534)
* Tidy up emoji parsing on profile submission

Don't bother reparsing for emoji unless one of the fields that can have
emoji in it has changed.
Deduplicate emoji between the display name and profile note - I'm not
sure whether this was hurting anything, but better safe.

* Deduplicate emoji when parsing remote accounts

Some servers - Misskey at least - don't deduplicate emoji, so it's
possible to get an account which has the same emoji used in both the
display name and note and therefore includes that emoji twice in its
metadata. When we start trying to put those into our database, we run
into a uniqueness constraint and fall over.

This change just deduplicates at the point of construction of an
account.
2023-02-20 16:27:41 +01:00
kim
2af33d3fb5
Update README.md (#1533) 2023-02-19 16:10:40 +00:00
Daenney
70398891b8
[chore] Move request validation earlier in client (#1531)
This moves checking if the request is valid as early as possible in the
chain. This should ensure that for an invalid request we never bother
acquiring the wait queue and taking up a spot in it.
2023-02-19 11:01:15 +00:00
kim
51c156cca1
pull in latest go-kv, go-cache (#1530)
Signed-off-by: kim <grufwub@gmail.com>
2023-02-19 10:36:38 +01:00
tobi
a0068e8915
[bugfix] In Postgres, drop shortcodedomain constraint before creating new emoji table (#1528) 2023-02-18 17:54:51 +01:00
kim
a684fc4628
[chore] transport improvements (#1524)
* improve error readability, mark "bad hosts" as fastFail

Signed-off-by: kim <grufwub@gmail.com>

* pull in latest go-byteutil version with byteutil.Reader{}

Signed-off-by: kim <grufwub@gmail.com>

* use rewindable body reader for post requests

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-02-18 17:02:19 +01:00
dependabot[bot]
3649b231c4
[chore]: Bump golang.org/x/net from 0.5.0 to 0.7.0 (#1523)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-02-18 16:52:33 +01:00
mushus
fe66a2aed2
[bugfix] Keep png transparency (#1522)
* keep png transparency

* rewrite to switch case
2023-02-18 16:52:15 +01:00
f0x52
59b2e10787
[bugfix] fix oob token route, update templates+css for oob and errors (#1519) 2023-02-18 16:47:42 +01:00
kim
83d8a44020
[chore] update gruf's libs on readme (#1520)
also groups them into a single block to improve readability
2023-02-18 16:45:43 +01:00
dependabot[bot]
a822198ebf
[chore]: Bump golang.org/x/image from 0.4.0 to 0.5.0 (#1516)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/golang/image/releases)
- [Commits](https://github.com/golang/image/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-18 14:02:49 +00:00
Daenney
b4d18887d3
[chore] Update more log calls to include context (#1517)
In #1476 we updated log.WithFields() but we forgot about
log.WithField(). Also updates a few explicit log.Entry{} creations.
2023-02-17 19:05:43 +00:00
Daenney
d39280ec33
[bug] Pass context in logging middleware (#1514)
This updates the middleware log.WithField calls that create new loggers
to include the context the first time around. Without it the requestID
does not get logged.

Fixup from #1476
2023-02-17 12:37:57 +01:00
Daenney
1ca5c62e25
[bug] maxprocs set logger to nil (#1512)
We want to set the Logger to nil, but that means we still need to wrap
it in `maxprocs.Logger()`.
2023-02-17 12:31:34 +01:00
Daenney
68e6d08c76
[feature] Add a request ID and include it in logs (#1476)
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.

Co-authored-by: kim <grufwub@gmail.com>
2023-02-17 12:02:29 +01:00
tobi
b5993095fa
[bugfix] Set 'discoverable' properly on API accounts (#1511) 2023-02-16 13:20:23 +00:00
tobi
6ee0dc8c7d
[bugfix] Set cache-control max-age dynamically for s3 (#1510)
* [bugfix] set cache-control max-age dynamically for s3

* woops

* double whoops

* time until, thank you linter, bless you, you're the best, no matter what kim says

* aa
2023-02-16 13:18:53 +00:00
Sam Lade
40b584c219
Fix 410 Gone race on account deletes (#1507) 2023-02-15 19:41:16 +00:00
f0x52
b8e1ab312d
use woff(2) fonts for Noto Sans (#1509) 2023-02-15 20:10:18 +01:00
Daenney
4cba90c0f6
[chore] Split the bug template in two (#1500)
Have a separate template for:
* Frontend, with labels bug and frontend and requiring the browser field
* Regular/backend, for other bugs with only the bug label and removal of
  the browser field
2023-02-15 14:37:08 +01:00
wioxjk
674646b79e
[docs] Update config.yaml (#1499)
Made it more clear how GTS accesses S3 buckets
2023-02-15 13:26:18 +01:00
Minecraftchest1
f3eb28a0f5
[docs] Suggest confirming host option in config (#1502)
Having the `Host` config option not match the domain that GTS is exposed on can cause a 400 bad request error on signin.
2023-02-15 13:04:48 +01:00
Daenney
fd62847c83
[docs] Fix nginx fileserver caching example (#1506)
* [bug] Fix nginx fileserver caching example

This updates the example to ensure the nginx proxies the request on to
GTS if the file is not found on disk. This can happen due to media
pruning.

* [chore] Set cache-control in nginx to private

This makes the header match with the backend. For things from the
fileserver it may not be appropriate for anything other than a private
cache (i.e the client) to cache things.
2023-02-15 11:44:30 +01:00
f0x52
700ed7769f
[chore] Webkit frontend fixes (#1492)
* fix webkit profile preview layout

* general page layout
2023-02-14 11:58:57 +01:00
tobi
6c6f042290
[bugfix] Return empty result rather than 500 error when searching for blocked domains (#1498)
* [bugfix] Return empty result when searching for blocked domains

* add tests
2023-02-14 11:55:02 +01:00
tobi
561ad71e58
[bugfix] Fix up error getting account avatar/header errors, other small fixes (#1496)
* start fiddling with media + account queries a little

* initialize state when pruning

* allow for unsetting remote media
make sure to wait til media loaded
fix silly tiny bug

* move comment a bit for readability

* slight reformat of fetchRemoteAccount{Avatar,Header}

* fix issue after rebase

* slightly neaten up logic of avatar/header media handling

* remove log prefix (callername log field handles this)

---------

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
2023-02-13 20:19:51 +00:00
kim
acc95923da
[performance] processing media and scheduled jobs improvements (#1482)
* replace media workers with just runners.WorkerPool, move to state structure, use go-sched for global task scheduling

* improved code comment

* fix worker tryUntil function, update go-runners/go-sched

* make preprocess functions package public, use these where possible to stop doubled up processing

* remove separate emoji worker pool

* limit calls to time.Now() during media preprocessing

* use Processor{} to manage singular runtime of processing media

* ensure workers get started when media manager is used

* improved error setting in processing media, fix media test

* port changes from processingmedia to processing emoji

* finish code commenting

* finish code commenting and comment-out client API + federator worker pools until concurrency worker pools replaced

* linterrrrrrrrrrrrrrrr

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-02-13 18:40:48 +00:00
f0x52
76d1b484d0
[docs] move federating with gotosocial documentation into single file (#1494) 2023-02-13 18:34:44 +01:00
tobi
041c8e695e
[chore] Do cache-control in a less silly way to avoid writing header twice (#1481)
* do cache-control in a less silly way to avoid writing header twice

* add comment back in
2023-02-13 12:58:22 +01:00
f0x52
eeca1988ce
[docs] Update user/admin settings docs (#1491)
* update user/admin settings docs

* trigger read-the-docs pr hook

* update menu links

* update images
2023-02-13 12:40:24 +01:00
dependabot[bot]
efbc5dab6a
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.47 to 7.0.48 (#1486)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.47 to 7.0.48.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.47...v7.0.48)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 07:46:15 +00:00
dependabot[bot]
33b77b3c1a
[chore]: Bump golang.org/x/image from 0.3.0 to 0.4.0 (#1485)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/golang/image/releases)
- [Commits](https://github.com/golang/image/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: golang.org/x/image
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 07:44:04 +00:00
dependabot[bot]
723175219b
[chore]: Bump modernc.org/sqlite from 1.20.3 to 1.20.4 (#1484)
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 07:43:16 +00:00
tobi
c223c7598d
[bugfix] Set appropriate cache-control when using presigned s3 links (#1480) 2023-02-12 14:42:28 +01:00
tobi
40bc03e717
[chore/performance] Update media prune logic, add extra CLI command (#1474)
* start updating media prune stuff a wee bit

* continue prune / uncache work

* more tidying + consistency stuff

* add prune CLI command

* docs

* arg
2023-02-11 12:48:38 +01:00
kim
70739d32cc
[performance] remove throttling timers (#1466)
* remove throttling timers, support setting retry-after, use retry-after in transport

* remove unused variables

* add throttling-retry-after to cmd flags

* update envparsing to include new throttling-retry-after

* update example config to include retry-after documentation

* also support retry-after formatted as date-time, ensure max backoff time

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-02-10 20:16:01 +00:00
kim
6ac1dda96f
[chore] small changes missed in previous dereferencer.GetAccount() PRs (#1467)
* small formatting changes, rewrite fetchRemoteMedia to use separate funcs + use mutex lock correctly

* move url parsing before acquiring mutex locks

* use wrapped mutexes to allow safe unlocking. (previously i did a fucky and passed mutex by value...)

* remove unused code

* use consistent map keying for dereferencing headers/avatars

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-02-10 20:15:23 +00:00
tobi
e5e257c259
[bugfix] Fix error on searching for account w/accountDomain by host (#1465) 2023-02-09 10:34:44 +01:00
tobi
95715f9251
[performance] Don't fetch avatar + header if uri hasn't changed (#1463) 2023-02-09 09:27:07 +01:00
f0x52
65b19411a4
[chore] Fix report username wrapping (#1464)
* fix report username wrapping

* move report status to separate line on portrait orientation
2023-02-09 09:26:35 +01:00