Commit graph

79 commits

Author SHA1 Message Date
tobi 6171dcbe51
[feature] Add HTTP header permission section to frontend (#2893)
* [feature] Add HTTP header filter section to frontend

* tweak naming a bit
2024-05-05 11:47:22 +00:00
Daenney 39b3a27c82
[docs] Remove last references to RPi (#2885)
This updates the documentation to remove the last stray references to
the copaganda Pi. It now uses the the term single-board computer. GtS
can run fine on all kinds of SBCs and isn't limited to that one
particular fruit version.
2024-04-30 14:27:44 +01:00
tobi 8b30709791
[chore] Turn accounts-registration-open false by default (#2839) 2024-04-15 14:41:15 +02:00
tobi 9fb8a78f91
[feature] New user sign-up via web page (#2796)
* [feature] User sign-up form and admin notifs

* add chosen + filtered languages to migration

* remove stray comment

* chosen languages schmosen schmanguages

* proper error on local account missing
2024-04-11 11:45:53 +02:00
kim a483bd9e38
[performance] massively improved ActivityPub delivery worker efficiency (#2812)
* add delivery worker type that pulls from queue to httpclient package

* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller

* hook up queue deletion logic

* support deleting queued http requests by target ID

* don't index APRequest by hostname in the queue

* use gorun

* use the original context's values when wrapping msg type as delivery{}

* actually log in the AP delivery worker ...

* add uncommitted changes

* use errors.AsV2()

* use errorsv2.AsV2()

* finish adding some code comments, add bad host handling to delivery workers

* slightly tweak deliveryworkerpool API, use advanced sender multiplier

* remove PopCtx() method, let others instead rely on Wait()

* shuffle things around to move delivery stuff into transport/ subpkg

* remove dead code

* formatting

* validate request before queueing for delivery

* finish adding code comments, fix up backoff code

* finish adding more code comments

* clamp minimum no. senders to 1

* add start/stop logging to delivery worker, some slight changes

* remove double logging

* use worker ptrs

* expose the embedded log fields in httpclient.Request{}

* ensure request context values are preserved when updating ctx

* add delivery worker tests

* fix linter issues

* ensure delivery worker gets inited in testrig

* fix tests to delivering messages to check worker delivery queue

* update error type to use ptr instead of value receiver

* fix test calling Workers{}.Start() instead of testrig.StartWorkers()

* update docs for advanced-sender-multiplier

* update to the latest activity library version

* add comment about not using httptest.Server{}
2024-04-11 11:45:35 +02:00
tobi e20c7c21e1
[docs] update deployment considerations docs with latest findings (#2821)
* [docs] update deployment considerations docs with latest findings

* simplify single-board computer section
2024-04-10 12:03:43 +02:00
Daenney e6e696ae22
[docs] Add IPv6 example for rate limit exceptions (#2761) 2024-03-15 13:13:57 +01:00
Daenney 40ba93654e
[docs] Use human sizes for size limitations (#2706)
This switches the documentation to use human sizes, so 50MiB instead of
an integer number of bytes. This makes it much easier to understand what
values are set, and less likely to accidentally set the wrong value.
2024-03-01 14:47:03 +00:00
9p4 9bf448be7a
[feature/oidc] Add support for very basic RBAC (#2642)
* Add support for very basic RBAC

* Add some small tests for allowedGroup and adminGroup

* Switch to table-driven tests
2024-02-27 16:07:29 +01:00
tobi 9cadc764b3
[feature] Add experimental instance-federation-spam-filter option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
tobi 4b0eefbcc9
[chore] Increase default max image description to 1500 chars, collapse cw char count into status (#2682)
* [chore] Make default max image description 1500 chars, collapse cw char count into status

* oops

* tests
2024-02-23 18:28:09 +00:00
Daenney dcb9fbfc03
[docs] Mounting config file in container (#2622)
* [docs] Mounting config file in container

This adds a small section clarifying how to mount a config.yaml in a
container.

* [docs] Work in GTS_CONFIG_PATH
2024-02-09 17:57:58 +01:00
Daenney e178a7b17e
[docs] Rework storage (#2571)
* Restructure the sections
* Mention how to clear out remote avatars and headers, related to #2328
* Add more links to authoritative documentation on the AWS side
* Show how to use the official AWS CLI
* Make a separate section for migration to S3 compatible object backends
2024-01-27 14:17:48 +01:00
Daenney c946d02c1f
[docs] Fix log-timestamp-format (#2572)
Supersedes: #2556

Closes: #2455
2024-01-25 18:31:57 +01:00
Daenney 5556767ff7
[docs] Change configuration creation instructions (#2408)
* [docs] Change configuration creation instructions

This changes the wording to push people towards creating their own
configuration, without copying the whole example configuration. This
makes it much easier to reconcile necessary configruation changes on
upgrades.

* [docs] Reword container version section

This changes the wording in the Version section to make it more clear
what the risks of a moving tag are. It pushes people to use an explicit
release tag instead.
2023-12-05 11:44:10 +01:00
Daenney c334df8f43
[docs] Explain how to secure metrics endpoints (#2382) 2023-11-26 15:53:53 +01:00
Daenney e9f636ea59
[docs] Add a warning about SQLite cache sizes (#2378)
* [docs] Add a warning about SQLite cache sizes

* [docs] Fix admonition text

* [docs] Lorde the indenting

* [docs] Rework the text a bit
2023-11-22 18:06:30 +00:00
Tsuribori 1ba3e14b36
[feature] Initial Prometheus metrics implementation (#2334)
* feat: Initial OTEL metrics

* docs: add metrics documentation

* fix: metrics endpoint conditional check

* feat: metrics endpoint basic auth

* fix: make metrics-auth-enabled default false

* fix: go fmt helpers.gen.go

* fix: add metric-related env vars to envparsing.sh

* fix: metrics docs

* fix: metrics related stuff in envparsing.sh

* fix: metrics docs

* chore: metrics docs wording

* fix: metrics stuff in envparsing?

* bump otel versions

---------

Co-authored-by: Tsuribori <user@acertaindebian>
Co-authored-by: Tsuribori <none@example.org>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-11-20 16:43:55 +01:00
tobi fc02d3c6f7
[feature] Set/show instance language(s); show post language on frontend (#2362)
* update go text, include text/display

* [feature] Set instance langs, show post lang on frontend

* go fmt

* WebGet

* set language for whole article, don't use FA icon

* mention instance languages + other optional config vars

* little tweak

* put languages in config properly

* warn log language parse

* change some naming around

* tidy up validate a bit

* lint

* rename LanguageTmpl in template
2023-11-17 11:35:28 +01:00
tobi 4dc0547dc0
[feature] Customizable media cleaner schedule (#2304) 2023-10-30 17:35:11 +00:00
tobi 183eaa5b29
[feature] Implement explicit domain allows + allowlist federation mode (#2200)
* love like winter! wohoah, wohoah

* domain allow side effects

* tests! logging! unallow!

* document federation modes

* linty linterson

* test

* further adventures in documentation

* finish up domain block documentation (i think)

* change wording a wee little bit

* docs, example

* consolidate shared domainPermission code

* call mode once

* fetch federation mode within domain blocked func

* read domain perm import in streaming manner

* don't use pointer to slice for domain perms

* don't bother copying blocks + allows before deleting

* admonish!

* change wording just a scooch

* update docs
2023-09-21 12:12:04 +02:00
Daenney 14ef098099
[feature] Support OTLP HTTP, drop Jaeger (#2184)
* [feature] Add http trace exporter, drop Jaeger

Jaeger supports ingesting traces using the OpenTelemetry gRPC or HTTP
methods. The Jaeger project has deprecated the old jaeger transport.

* Add support for submitting traces over HTTP
* Drop support for the old Jaeger protocol
* Upgrade the trace libraries to v1.17

Fixes: #2176
Fixes: #2179
2023-09-07 13:20:37 +02:00
tobi 8f38dc2e7f
[feature] Add rate limit exceptions option, use ISO8601 for rate limit reset (#2151)
* start updating rate limiting, add exceptions

* tests, comments, tidying up

* add rate limiting exceptions to example config

* envparsing

* nolint

* apply kimbediff

* add examples
2023-08-23 14:32:27 +02:00
Daenney 4ae16bce8c
[feature] Make log format configurable (#2130)
* [feature] Don't emit timestamp in log lines

When running gotosocial with a service manager like systemd, or a
container runtime, the associated log driver usually emits timestamps
itself. In those cases, having the extra timestamp from our own log
lines ends up being a bit noisy and when centrally ingesting logs is
duplicate information.

This introduces a configuration flag that allows disabling emitting the
timestamp. It's only wired up for "daemonised" processes, meaning server
and testrig.

* [chore] Add docs for log-timestamp

* [feature] Simplify timestamp handling

Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>

* [chore] Less escaped double-quotes

* [chore] Fix help string

---------

Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-08-21 19:07:55 +01:00
tobi 1e2db7a32f
[feature/bugfix] Probe S3 storage for CSP uri, add config flag for extra URIs (#2134)
* [feature/bugfix] Probe S3 storage for CSP uri, add config flag for extra URIs

* env parsing tests, my coy mistress
2023-08-20 13:35:55 +02:00
Max Rosin 7f1f2b80ea
[docs] fix postgres create database command (#2071)
The current command `create database gotosocial with locale C.UTF-8 template template0;` fails because the locale has to be quoted:
```
postgres=# create database gotosocial with locale C.UTF-8 template template0;
ERROR:  syntax error at or near "."
LINE 1: create database gotosocial with locale C.UTF-8 template temp...
```

Tested manually that it works with the quotes and the official postgres examples also use quotes around the locales: https://www.postgresql.org/docs/current/sql-createdatabase.html
2023-08-06 11:54:07 +02:00
tobi 2be83fdca5
[feature] Allow users to skip http client tls verification for testing purposes (with appropriately loud warnings) (#2052) 2023-08-01 18:50:17 +01:00
Daenney da8282081c
[feature] Lower remote media cache config duration (#2007)
The old default of 30d can lead to a lot of media getting cached and
significant disk usage, even on small or single person instances. A lot
of deployments decrease this value, to 15 or even less. This is less of
an issue when using object storage, but for local storage which is the
more popular deployment option running out of disk space is unpleasant.

With GoToSocial's aim to fit in small places, this changes the default
to a much more conservative 7 days. In all likelihood people aren't
scrolling that far back in their timeline so this change shouldn't
result in any issue. Existing deployments will only be affected by
this change if the admin hasn't already configured this value, or didn't
bootstrap from the example configuration.
2023-07-21 16:22:59 +02:00
firescry 8d92b2479f
[bugfix] Align default values in the configuration file with the code (#1971)
* Set default value of SMTPFrom to empty string

This parameter should contain proper e-mail address (to be provided by user during configuration).

* Update default values in example/config.yaml

Default values and related comments in example/config.yaml are aligned
with values defined in internal/config/defaults.go.
Small improvements to foramting of config.yaml file.

* Add default value for AdvancedThrottlingRetryAfter to internal/config/defaults.go

AdvancedThrottlingRetryAfter was introduced in 70739d3 (superseriousbusiness/gotosocial#1466).

* Update config.yaml snippets in documentation
2023-07-12 10:28:41 +02:00
tobi 2a99df0588
[feature] enable + document explicit IP dialer allowing/denying (#1950)
* [feature] enable + document explicit IP dialer allowing/denying

* lord have mercy

* allee jonge

* shortcut check ipv6 prefixes

* comment

* separate httpclient_test, export Sanitizer
2023-07-07 16:17:39 +02:00
Daenney c4cf6326d8
[docs] Clarify email requirement for OIDC (#1918) 2023-06-21 21:22:51 +02:00
tobi fc524f8cf1
[feature] Configurable custom css length (#1827)
* [feature] Make accounts custom css length configurable

* test custom css validation
2023-05-25 15:18:15 +02:00
Dominik Süß 6392e00653
feat: initial tracing support (#1623) 2023-05-09 18:19:48 +01:00
tobi cbb9e2d3f0
[chore/performance] Make sender multiplier configurable (#1750) 2023-05-08 18:03:38 +01:00
tobi 7db81cde44
[feature] Email notifications for new / closed moderation reports (#1628)
* start fiddling about with email sending to allow multiple recipients

* do some fiddling

* notifs working

* notify on closed report

* finishing up

* envparsing

* use strings.ContainsAny
2023-03-19 13:11:46 +01:00
Daenney a312238e79
[feature] Provide .well-known/host-meta endpoint (#1604)
* [feature] Provide .well-known/host-meta endpoint

This adds the host-meta endpoint as Mastodon clients use this to
discover the API domain to use when the host and account domains aren't
the same.

* Address review comments
2023-03-09 17:55:45 +00:00
Daenney d2f6de0185
[feature] Allow loading TLS certs from disk (#1586)
Currently, GtS only supports using the built-in LE client directly for
TLS. However, admins may still want to use GtS directly (so without a
reverse proxy) but with certificates provided through some other
mechanism. They may have some centralised way of provisioning these
things themselves, or simply prefer to use LE but with a different
challenge like DNS-01 which is not supported by autocert.

This adds support for loading a public/private keypair from disk instead
of using LE and reconfigures the server to use a TLS listener if we
succeed in doing so.

Additionally, being able to load TLS keypair from disk opens up the path
to using a custom CA for testing purposes avoinding the need for a
constellation of containers and something like Pebble or Step CA to
provide LE APIs.
2023-03-04 17:24:02 +00:00
Daenney 9cfb69f75d
[feature] Make OIDC admin groups configurable (#1555)
This removes the current default of checking for membership of the admin
or admins group and makes it required to explicitly configure which
groups should grant admin access, if any.

Relying on the implicit default of admin or admins is potentially
dangerous as that group may contain a different subset of people that we
may wish to grant admin access to GtS. This is probably not an issue for
a single-person instance, but for a community instance different admin
groups may exist in an OIDC provider for different applications.

I'm explicitly opting for not defaulting the value of oidc-admin-groups
to admin,admins because I think it's better for those things to be
explicitly configured.
2023-02-25 16:37:39 +00:00
Daenney 68e6d08c76
[feature] Add a request ID and include it in logs (#1476)
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.

Co-authored-by: kim <grufwub@gmail.com>
2023-02-17 12:02:29 +01:00
tobi 4ee4cd2da1
[chore/performance] use only 1 sqlite db connection regardless of multiplier (#1408) 2023-02-01 11:55:34 +01:00
tobi b80be48fed
[chore] Use 'immediate' lock for sqlite transactions (#1404)
* [chore] Use 'immediate' lock for sqlite transactions

* allow 1 connection regardless of cpu amount
2023-01-31 13:46:45 +01:00
tobi 8d18888db6
[chore/docs] add instance-expose-suspended-web to instance docs (#1391) 2023-01-27 09:23:54 +01:00
tobi 782169da76
[chore] set max open / idle conns + conn max lifetime for both postgres and sqlite (#1369)
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite

* reduce cache size default to 8MiB, reduce connections to 2 * cpu

* introduce max open conns multiplier, tune sqlite and pg separately

* go fmt
2023-01-26 15:12:48 +01:00
tobi 974ec80a20
[chore] Change default sqlite busy timeout to 5m (#1352) 2023-01-18 14:37:52 +01:00
tobi 627b8eeae6
[feature] Tune sqlite pragmas (#1349)
* sqlite pragma tuning

* use formatuint

* add sqlite busy timeout

* fix incorrect cache size format

* update envparsing test

* add sqlite tuning flags to cli

* set sqlite timeout to 30s default
2023-01-17 12:29:44 +00:00
Logan 9859a43b87
[docs] Add s3 ssl variable to storage docs (#1294)
* update storage docs

* add use ssl to example/config.yaml
2023-01-08 12:28:58 +01:00
Anant Shrivastava 2a1205ab32
[docs] AWS S3 config details added (#1300)
* AWS S3 config details added

It was interesting to note that since presigned urls are used buckets dont need to be exposed publically. this was an interesting change compared to other mastodon specific s3 bucket guides hence documented here for correct directions.

* Update storage.md

1. Added AWS identified to make it clear its aws specific.
2. Adjusted text around data migration

* updation as requested

Refining the doc as per request.
2023-01-06 14:32:40 +01:00
tobi 90a14abb0c
[feature] HTTP request throttling middleware (#1297)
* [feature] Add throttling middleware to AP endpoints

* refactor a lil bit

* use config setting, start updating docs

* doc updates

* use relative links in faq doc

* small docs fixes

* return code 503 instead of 429 when throttled

* throttle other endpoints too

* simplify token channel prefills
2023-01-04 11:57:59 +01:00
Andrea 923d333823
[docs] encourage using loopback bind address (#1166) 2022-12-07 21:50:37 +01:00
Dominik Süß 199b685f43
[feature] overhaul the oidc system (#961)
* [feature] overhaul the oidc system

this allows for more flexible username handling and prevents account
takeover using old email addresses

* [feature] add migration path for old OIDC users

* [feature] nicer error reporting for users

* [docs] document the new OIDC flow

* [fix] return early on oidc error

* [docs]: add comments on the finalization logic
2022-12-06 14:15:56 +01:00