Commit graph

124 commits

Author SHA1 Message Date
Dominik Süß
9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi
68736efd20
[feature] add configuration to /api/v1/instance response (#670)
* add configuration object to api instance model

* regenerate swagger docs

* add func to return all supported mimes for media

* add instance configuration to api serialization

* fix json tags

* update instance endpoint tests

* fix typeutils tests

* final regen of swagger docs

* omitempty instance configuration
2022-06-26 10:58:45 +02:00
tobi
5f00d4980b
[feature] Implement /api/v1/instance/peers endpoint (#660)
* add missing license headers

* start adding instance peers get

* rename domainblock.go

* embed domain in domainblock so it can be reused

* update swagger docs

* add test instances to db

* update tests

* add/update instancepeersget

* update domain model

* add getinstancepeers to db

* instance-expose-peers, instance-expose-suspended

* add auth checks for both current filters

* attach endpoint to router

* include public comment

* obfuscate domain if required

* go mod tidy

* update swagger docs

* remove unnecessary comment

* return 'flat' peerlist if no query params provided
2022-06-23 16:54:54 +02:00
f0x52
b43f9ceca9
[frontend] Restructure Frontend Sources (#634)
* 🐸restructure frontend stuff, include admin and future user panel in main repo, properly deduplicate bundles for css+js across uses

* rename bundled to dist, caught by gitignore

* re-include status.css for profile template

* default to localhost

* serve frontend panels

* add todo message for abstraction

* refactor oauth registration flow

* oauth restructure

* update footer template

* change panel routes

* remove superfluous css imports

* write bundle to disk from test server, use forked budo-express

* wrap all page content in container

for robustness with addons etc injection other elements in body

* update documentation, goreleaser, Dockerfile

* update template meta tags

* add AGPL-3.0+ license header everywhere

* only attach update listener on EventEmitter

* cleaner config for various frontend bundles

* fix bundler script paths

* Merge commit 'd191931932b9293ce1be44ed08a1e69b9fcc1e25'

* fix up dockerfile, goreleaser

* go mod tidy

* add uglifyify

* move status hide/show js to frontend bundle

* fix stylesheet color( func regressions

* update contributing docs for new build path

* update goreleaser + docker building

* resolve dependency paths properly

* update package name

* use api errorhandler

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-06-09 12:51:19 +02:00
tobi
1ede54ddf6
[feature] More consistent API error handling (#637)
* update templates

* start reworking api error handling

* update template

* return AP status at web endpoint if negotiated

* start making api error handling much more consistent

* update account endpoints to new error handling

* use new api error handling in admin endpoints

* go fmt ./...

* use api error logic in app

* use generic error handling in auth

* don't export generic error handler

* don't defer clearing session

* user nicer error handling on oidc callback handler

* tidy up the sign in handler

* tidy up the token handler

* use nicer error handling in blocksget

* auth emojis endpoint

* fix up remaining api endpoints

* fix whoopsie during login flow

* regenerate swagger docs

* change http error logging to debug
2022-06-08 20:38:03 +02:00
tobi
327d3f001f
[feature] Start adding advanced configuration options, starting with samesite (#628)
* fix incorrect port being used for db

* start adding advanced config flags

* use samesite lax by default
2022-06-03 15:40:38 +02:00
Martijn de Boer
d6abe105b3
[docs] Add Caddy instructions to the documentation (#594)
* Add instructions for working with Caddy 2

Add instructions for working with Caddy 2. Some texts are duplicated from the NGINX part (mainly the configuration part, about systemctl).

* Add new Caddy docs to mkdocs.yml

Adds the new Caddy document to the document tree

* Remove up and downstream headers from configuration

Removed the header statements for the proxy, as proxying those are transparant. Kept the flush_interval directive, disabling the response buffer completely so we can write without delays.

* Update caddy.md

Corrects the opening link

* Apply comments mentioned in PR

@igalic mentioned a few comments to improve these docs. I've processed those in this PR.
2022-05-23 11:45:55 +02:00
tobi
b143877995
[feature] Unused avatar and header cleanup (#574)
* rename + tidy up remote pruning

* fix media attachment account join
see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships

* update logging to new function name

* add get avatars and headers to bun

* add pruneallmeta function

* don't set uncached since we're deleting anyway

* fix totalPruned being in wrong place

* test pruning meta

* go fmt ./...

* rename mediaprune

* add meta pruning to routine mediaprune

* tidy up cleanup job scheduling

* rename adminmediaremoteprune

* update mediacleanup to use renamed prune func

* update swagger docs a little bit

* reuse cancel + context
2022-05-15 15:45:04 +01:00
Sashanoraa
6e947ff266
[feature] Media cleanup endpoint (#560)
Adds an admin endpoint to trigger a remote media cleanup.

Fixed #348

Signed-off-by: Sashanoraa <sasha@noraa.gay>
2022-05-15 14:52:46 +02:00
f0x52
00111666a0
[documentation] Admin Panel installation + usage (#552)
* admin panel documentation

* wrap link so it's clickable

* format img so it works with mkdocs serve

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-05-11 12:25:53 +02:00
tobi
8124dcf081
[Documentation] tidy docker-compose docs, add note about uid/gid (#551) 2022-05-09 11:15:46 +02:00
tobi
807a8e1cf6
[Documentation] Update glossary, expand entry for dereferencing (#542)
* Update glossary, expand entry for dereferencing

* Add glossary as separate file
2022-05-04 14:33:24 +02:00
tobi
d2b7c89d24
[documentation] Add known issues + project links in readme and docs (#541) 2022-05-04 11:36:20 +02:00
tobi
f040ad899d
[documentation] put single-quotes around passwords in install docs (#534) 2022-05-03 12:06:08 +02:00
tobi
4851ffe11b
[documentation] Recommend users take latest *stable* release in installation guides (#512)
* add note about release version number

* recommend users take latest stable release
2022-04-29 21:11:10 +02:00
Nivex
c36fbab86f
[documentaton] Fix port number in Reverse proxy section (#506) 2022-04-29 10:58:39 +02:00
tobi
4bace80fab
[documentation] NGINX docs updates (#471)
* start adjusting nginx documentation

* update NGINX docs

* add link to the websocket docs
2022-04-24 11:04:26 +02:00
snan
03bcd8a711
[documentation] Update index.md (#475)
This sentence kind of trai
2022-04-23 13:52:09 +02:00
tobi
61e60133ef
[documentation] Add WebSocket protocol documentation (#470) 2022-04-22 13:35:19 +02:00
tobi
0e2f31f4e3
[Documentation] Simplify docker-compose documentation + example (#468)
* simplify docker-compose docs + example

* Change note about reverse proxy, add traefik

* Linting
2022-04-19 12:45:54 +02:00
Mina Galić
721061b046
[docs] unify nginx explainers and add apache httpd (#455)
* docs: unify nginx explainers and add apache httpd

there are two places where nginx + certbot is explained, unify that into
one place.
Add apache httpd, following the same steps, but using mod_md for
LetsEncrypt
add a note about #453 in both guides.
Link to both, and call the section reverse proxy, instead of NGINX

* restore full nginx.conf from docker.md

* add installation_guide/apache-httpd.md to mkdocs
2022-04-18 17:45:43 +02:00
tobi
094f032f74
[feature] Add log-db-queries config option (#465) 2022-04-18 16:47:11 +02:00
tobi
e74ec70dbf
[documentation] add screenshots to docs (#459) 2022-04-16 13:32:17 +02:00
tobi
26683b3d49
[feature] Web profile pages for accounts (#449)
* add default avatars

* allow webModule to error

* return errWithCode from account get

* add AccountGetLocalByUsername

* check nil requesting account

* add timestampShort function for just month/year

* move loading logic to New + add default avatars

* add profile page view

* update swagger docs

* add excludeReblogs to GetAccountStatuses

* ignore casing when selecting local account by username

* appropriate redirects

* css fiddling

* add 'about' heading

* adjust thread page to work with routing

* return AP representation if requested + authorized

* simplify auth check

* go fmt

* golangci-lint ignore math/rand
2022-04-15 14:33:01 +02:00
tobi
906981a718
[documentation] Add third-party packaging to documentation (#443) 2022-03-29 15:36:04 +02:00
tobi
55ad6dee71
[feature] Admin account actions (#432)
* add accountAction to the admin API

* model admin account action

* add admin account action to the processor

* add migration for new AdminAccountActions table

* fix accounts admin path

* Update swagger docs
2022-03-19 12:01:40 +01:00
tobi
532c4cc697
[feature] Federate local account deletion (#431)
* add account delete to API

* model account delete request

* add AccountDeleteLocal

* federate local account deletes

* add DeleteLocal

* update transport (controller) to allow shortcuts

* delete logic + testing

* update swagger docs

* more tests + fixes
2022-03-15 16:12:35 +01:00
tobi
07727753b9
[feature] Clean up/uncache remote media (#407)
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00
0hlov3
100f1280a6
[documentation] Creates Docker documentation and docker-compose.yaml (#416)
* exludes docker-volume from exemple/docker-compose

* Adds an docker-compose.yaml example and adds some readme to the
docker.md

* Changes Docker-Compose Example

* Configures docker-compose.yaml and docker.md

* Adds some cosmetics to the Documentation

* Adds UserID:GroupID

* Describes how to create a new user
2022-03-01 14:17:36 +01:00
Phil Hagelberg
98341a1d4d
[docs] Fix documentation to show --config-path in the right position. (#375)
The current position gets rejected as an unknown argument.
2022-02-03 12:30:06 +01:00
Paul Woolcock
aa34ff1bd9
Fix start command invocation (#372)
When I tried to start the server using the command `./gotosocial server start --config-file ./config.yaml`, I got the following error:

```
root@gotosocial-experiment:/gotosocial# ./gotosocial server start --config-file ./config.yaml
Incorrect Usage: flag provided but not defined: -config-file

NAME:
   gotosocial server start - start the gotosocial server

USAGE:
   gotosocial server start [command options] [arguments...]

OPTIONS:
   --help, -h  show help (default: false)

FATA[0000] flag provided but not defined: -config-file
```

Putting the flag after the `./gotosocial` but before the `server start` seems to work properly
2022-02-01 16:54:26 +01:00
tobi
959e38ac5c
[bug] Fix sqlite empty address issue (#370)
* error when empty address has been set for sqlite

* better explain sqlite db-address setting
2022-01-30 17:06:28 +01:00
tobi
e13718148d
[feature] Make Let's Encrypt integration an opt-in feature (#368) 2022-01-30 14:57:33 +01:00
tobi
ec4357485e
[documentation] add ipv6 config for nginx (#367) 2022-01-30 14:35:23 +01:00
zoe-bat
14c472c1ab
[documentation] Add section for systemd service and nginx to installation guide (#353)
* add nginx and systemd section

* fix typso

* fix mkdoc and dots in config guide

* specify port

* Update nginx.md

* add section about configuring gts to nginx docs

* Update nginx.md

Co-authored-by: zoe <z8V54gHftevp7LeRVyys$8@DosKq^FeQKUWU!WGpTyiLaxGojNdgnWkaWiv673Twg$aPm&hU>
2021-12-26 13:58:15 +01:00
tobi
a01b178467
[documentation] update database docs (#355)
* update config to new db defaults
* update database docs
2021-12-21 12:24:19 +01:00
tobi
ef5a9256a8
Extend license notices to 2022 (#354) 2021-12-20 18:42:19 +01:00
tobi
cb8688f429
Remove unnecessary storage config variables (#344)
* rewire config to not use extraneous serve vars

* rename 'file' to 'local' for consistency

* use Type and Size again
2021-12-20 15:19:53 +01:00
tobi
80ec714117
document post settings (#346) 2021-12-14 12:42:20 +01:00
tobi
c111b239f7
Add optional syslog logrus hook (#343)
* add optional syslog logrus hook

* document syslog
2021-12-12 18:00:20 +01:00
tobi
e2daf0f012
Add Accept header negotiation to relevant API endpoints (#337)
* start centralizing negotiation logic for API

* swagger document nodeinfo endpoint

* go fmt

* document negotiate function

* use content negotiation

* tidy up negotiation logic

* negotiate content throughout client api

* swagger

* remove attachment on Content

* add accept header to test requests
2021-12-11 17:50:00 +01:00
tobi
0884f89431
Implement Cobra CLI tooling, Viper config tooling (#336)
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
Mara Sophie Grosch
a7882fabc7
Use IPv6 doc prefix for docs (#324)
IPv6 has a dedicated prefix for documentation and examples, this commit makes use of it for BindAddress config flag docs and examples.
2021-11-22 17:58:19 +01:00
tobi
1ded58b34b
add bindAddress configuration option (#320)
* add bindAddress configuration option

* clarify that bindAddress can be a hostname
2021-11-22 10:55:52 +01:00
tobi
d81a123473
Update docs with better config + installation instructions (#300)
* start reworking some documentation

* fuller documentation  + better docs structure
2021-11-14 16:54:23 +01:00
tobi
2aaec82732
smtp + email confirmation (#285)
* add smtp configuration

* add email confirm + reset templates

* add email sender to testrig

* flesh out the email sender interface

* go fmt

* golint

* update from field with more clarity

* tidy up the email formatting

* fix tests

* add email sender to processor

* tidy client api processing a bit

* further tidying in fromClientAPI

* pin new account to user

* send msg to processor on new account creation

* generate confirm email uri

* remove emailer from account processor again

* add processCreateAccountFromClientAPI

* move emailer accountprocessor => userprocessor

* add email sender to user processor

* SendConfirmEmail function

* add noop email sender

* use noop email sender in tests

* only assemble message if callback is not nil

* use noop email sender if no smtp host is defined

* minify email html before sending

* fix wrong email address

* email confirm test

* fmt

* serve web hndler

* add email confirm handler

* init test log properly on testrig

* log emails that *would* have been sent

* go fmt ./...

* unexport confirm email handler

* updatedAt

* test confirm email function

* don't allow tokens older than 7 days

* change error message a bit

* add basic smtp docs

* add a few more snippets

* typo

* add email sender to outbox tests

* don't use dutch wikipedia link

* don't minify email html
2021-10-31 15:46:23 +01:00
tobi
de1f90ee46
regenerate swagger docs (#293) 2021-10-24 16:26:20 +02:00
tobi
84a8a07f38
docs typo fix (#290) 2021-10-24 12:11:31 +02:00
tobi
4b1d9d3780
Serve outbox for Actor (#289)
* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* bit more work on outbox paging

* wrapNoteInCreate function

* test + hook up the processor functions

* don't do prev + next links on empty reply

* test get outbox through api

* don't fail on no status entries

* add outbox implementation doc

* typo
2021-10-24 11:57:39 +02:00
tobi
15621f5324
Follow request improvements (#282)
* tiny doc update

* add rejectfollowrequest to db

* add follow request reject to processor

* add reject handler

* tidy up follow request api

* tidy up federation call

* regenerate swagger docs

* api endpoint tests

* processor test

* add reject federatingdb handler

* start writing reject tests

* test reject follow request

* go fmt

* increase sleep for slow test setups

* more relaxed time.sleep
2021-10-16 13:27:43 +02:00
tobi
107685e22e
User password change (#280)
* start passwordChangeHandler

* add user scope

* add user module / api path

* add password change request

* make comment clearer

* add user to processor

* required true

* add processor call to handler

* don't pass tc or channel

* change password func + tests

* add some first docs about password management

* update swagger docs

* add api tests

* go fmt

* test fixes
2021-10-14 14:26:04 +02:00
tobi
27605f7c47
add new logo + credits (#277) 2021-10-13 13:19:10 +02:00
tobi
e04b187702
Refactor/tidy (#261)
* tidy up streaming

* cut down code duplication

* test get followers/following

* test streaming processor

* fix some test models

* add TimeMustParse

* fix uri / url typo

* make trace logging less verbose

* make logging more consistent

* disable quote on logging

* remove context.Background

* remove many extraneous mastodon references

* regenerate swagger

* don't log query on no rows result

* log latency first for easier reading
2021-10-04 15:24:19 +02:00
tobi
d515c9f1ec
Goreleaser (#241)
* add goreleaser tooling

* add files + hook

* update hooks

* allow passing build-dir using cli args

* build tweaks

* tweak more

* update drone and goreleaser

* chill out tests

* remove postgres

* docker push on snapshot

* update releaser
2021-09-24 13:14:20 +02:00
tobi
e0f36278a0
add ap logo to readme (#228) 2021-09-13 20:02:59 +02:00
tobi
ab32ce642b
documentation updates (#211) 2021-09-11 16:45:50 +02:00
tobi
64bd689e55
Test both dbs (#205)
* move scripts, allow testing both dbs with one cmd

* tidy + vendor

* update test.sh to ignore cache

* put test commands directly in drone.yml

* change CONTRIBUTING slightly

* go ham on the readme
2021-09-10 18:13:24 +02:00
tobi
555ea8edfb
Import export (#194)
* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* update bun library -> v1.0.4

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library -> v4.3.1-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* handle oauth token scope, fix user.SigninCount + token.UserID

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library --> v4.3.2-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update sqlite library -> v1.13.0

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* review changes

* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* review changes

Co-authored-by: kim (grufwub) <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2021-09-09 16:15:25 +02:00
tobi
4920229a3b
Database updates (#144)
* start moving some database stuff around

* continue moving db stuff around

* more fiddling

* more updates

* and some more

* and yet more

* i broke SOMETHING but what, it's a mystery

* tidy up

* vendor ttlcache

* use ttlcache

* fix up some tests

* rename some stuff

* little reminder

* some more updates
2021-08-20 12:26:56 +02:00
Tobi Smethurst
0f2de6394a
Dereference remote replies (#132)
* decided where to put reply dereferencing

* fiddling with dereferencing threads

* further adventures

* tidy up some stuff

* move dereferencing functionality

* a bunch of refactoring

* go fmt

* more refactoring

* bleep bloop

* docs and linting

* start implementing replies collection on gts side

* fiddling around

* allow dereferencing our replies

* lint, fmt
2021-08-10 13:32:39 +02:00
Tobi Smethurst
0386a28b5a
Frodo swaggins (#126)
* more swagger fun

* document a whole bunch more stuff

* more swagger yayyyyyyy

* progress + go fmt
2021-08-02 19:06:44 +02:00
Tobi Smethurst
6bd26ff4c4
more swagger docs + other changes (#125)
* more swagger docs + other changes

* go fmt
2021-07-31 23:17:39 +02:00
tsmethurst
65bf285637 requirements.txt for RtD 2021-07-31 17:54:50 +02:00
Tobi Smethurst
58dddd86e0
Swagger (#124)
* start experimenting with swagger documentation

* further adventures in swagger

* do a few more api paths

* account paths documented

* go fmt

* fix up some models

* bit o lintin'
2021-07-31 17:49:59 +02:00
Tobi Smethurst
eb13faf54f
Password change (#123)
* add password change command

* document cli commands

* go fmt
2021-07-31 13:57:23 +02:00
Tobi Smethurst
ea8ad8b346
Link parsing (#120)
* add link parsing + formatting functionality

* refinement + docs

* add missing test

* credit url library
2021-07-28 11:42:26 +02:00
Tobi Smethurst
ad0e26dc04
Markdown Statuses (#116)
* parse markdown statuses if desired

* add some preliminary docs for writing posts
2021-07-26 20:25:54 +02:00
Tobi Smethurst
05e9af089c
Oidc (#109)
* add oidc config

* inching forward with oidc idp

* lil webfingy fix

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* add oidc config

* inching forward with oidc idp

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* document

* docs + comments
2021-07-23 10:36:28 +02:00
Tobi Smethurst
c5180b3860
Docs (#94)
* play around with docs

* add example images

* try some image fiddling

* images

* more detail

* little tweaks

* add licenses
2021-07-09 13:14:34 +02:00
tsmethurst
bffbdeb534 try some doc changes 2021-03-01 11:52:49 +01:00
tsmethurst
294d6d174d Doc updates 2021-02-28 18:08:27 +01:00
tsmethurst
7867fdebd2 add theme to conda env 2021-02-28 17:22:48 +01:00
tsmethurst
73cb9a1bbe Update docs 2021-02-28 17:20:12 +01:00
tsmethurst
e68b72ddf9 add docs and a bunch of other stuff 2021-02-28 15:17:18 +01:00