Commit graph

352 commits

Author SHA1 Message Date
f0x52
eb25739c34
[frontend] Custom Emoji Deletion (#994)
* re-add eslint

* fix oauth url getting too long

* actually attach single emoji get and delete routes

* basic emoji details + deletion using rtk query

* refactor emoji upload to rtk query

* clean up old redux api+reducers for custom emoji

* fix validation order

* refactor custom emoji form fields

* remove unused requires

* cleanup, fix most eslint errors

* more small eslint fixes

* fix max emoji size

* tiny bit of function documentation
2022-11-08 17:51:44 +01:00
tobi
4d66fb9603
[feature] Make rate limit requests amount configurable (#966)
* update rate limit documentation

* regenerate landingpage config helpers

* make rate limit rate configurable
2022-11-06 09:47:48 +00:00
tobi
bcb80d3ff4
[chore] bump gruf/go-store to v2 (#953)
* [chore] bump gruf/go-store to v2

* no more boobs
2022-11-05 11:10:19 +00:00
tobi
a9addb59b6
[bugfix] Increase field size limits when registering apps (#958) 2022-11-05 11:06:50 +01:00
tobi
1dfa7fe0d5
[bugfix] Wrap media in read closer (#941)
* use readcloser for content.Content

* call media postdata function no matter what

* return a readcloser from data func

* tidy of logic of readertostore

* fix whoopsie
2022-11-03 15:03:12 +01:00
tobi
bd05040133
[bugfix] Use []rune to check length of user-submitted text (#948) 2022-11-03 14:38:06 +01:00
tobi
ccd5b34c2c
[docs] Use correct form for status submit (#915) 2022-10-17 12:54:36 +02:00
tobi
f7416d6e94
[feature] Add emoji DELETE handler at /api/v1/admin/custom_emojis (#913)
* add emoji DELETE handler

* no need to process error (thanks kim)

* don't double check if user is admin

* add missing security annotation
2022-10-14 17:30:04 +02:00
tobi
6a95f5fa67
[feature] Add /api/v1/admin/custom_emojis/{id} endpoint for single emoji GET (#910)
* fix error in prev swagger docs

* add GET for single admin emoji
2022-10-13 16:37:55 +02:00
tobi
70d65b683f
[feature] Refetch emojis when they change on remote instances (#905)
* select emoji using image_static_url

* use updated on AP emojis

* allow refetch of updated emojis

* cheeky workaround for test

* clean up old files for refreshed emoji

* check error for originalPostData

* shorten GetEmojiByStaticImageURL

* delete kirby (sorry nintendo)
2022-10-13 15:16:24 +02:00
tobi
eb85ef7325
[feature] Add /api/v1/admin/custom_emojis endpoint (#902)
* add admin emojis get path + model + docs

* stub admin emojis get processor function

* add id + disabled fields to admin emoji

* add emoji -> api admin emoji converter

* tidy up a bit

* add GetEmojis function

* finish up get emojis function

* order by shortcodedomain

* ASC

* tidy up + explain

* update to allow paging

* make admin emojis pageable

* fix mixed case paging

* normalize emoji queries a bit better

* test emoji get paging

* make limit optional

* fix incorrect path in media cleanup tests

* i have bad coder syndrome

* don't trimspace

* rename -> GetUseableEmojis

* wrap emoji query in subquery
avoid selecting more than we need

* fix a bit of sillyness teehee

* fix subquery postgres woes
2022-10-12 15:01:42 +02:00
tobi
832befd727
[chore] Make paging logic more generic (#901)
* make paging logic more generic
not just for timelines!

* linty linterson
2022-10-10 15:52:49 +02:00
tobi
80663061d8
[feature] Add opt-in RSS feed for account's latest Public posts (#897)
* start adding rss functionality

* add gorilla/feeds dependency

* first bash at building rss feed
still needs work, this is an interim commit

* tidy up a bit

* add publicOnly option to GetAccountLastPosted

* implement rss endpoint

* fix test

* add initial user docs for rss

* update rss logo

* docs update

* add rssFeed to frontend

* feed -> feed.rss

* enableRSS

* increase rss logo size a lil bit

* add rss toggle

* move emojify to text package

* fiddle with rss feed formatting

* add Text field to test statuses

* move status to rss item to typeconverter

* update bun schema for enablerss

* simplify 304 checking

* assume account not rss

* update tests

* update swagger docs

* allow more characters in title, trim nicer

* update last posted to be more consistent
2022-10-08 14:00:39 +02:00
tobi
3bb45b7179
[feature] oob oauth token support (#889)
* move helpful advice into oauth server

* rewrite HandleAuthorizeRequest to allow oob
2022-10-08 13:49:56 +02:00
tobi
f8528aa689
[feature] Add emoji image size to instance response (#892) 2022-10-06 12:00:53 +02:00
tobi
56f53a2a6f
[performance] add user cache and database (#879)
* go fmt

* add + use user cache and database

* fix import

* update tests

* remove unused relation
2022-10-03 10:46:11 +02:00
tobi
97b747b452
[bugfix] Fix incorrect use of bun.Ident (#866)
* remove misused bun.Ident

* test status delete via API

* fix test
2022-09-29 12:33:33 +02:00
tobi
dd83ad053c
[feature] Add meta robots tag; allow robots to index profile card if user is Discoverable (#842)
* rework robots.txt response

* don't let robots snippet from statuses/threads

* allow robots to index if user is Discoverable

* add license text
2022-09-29 12:03:17 +02:00
kim
2f22780800
[chore] simplify generating log entry caller information (#863)
* vastly simplify logging caller information

Signed-off-by: kim <grufwub@gmail.com>

* fix failing test due to multiple calls to processor.Start()

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-29 12:02:12 +02:00
kim
a156188b3e
[chore] update dependencies, bump to Go 1.19.1 (#826)
* update dependencies, bump Go version to 1.19

* bump test image Go version

* update golangci-lint

* update gotosocial-drone-build

* sign

* linting, go fmt

* update swagger docs

* update swagger docs

* whitespace

* update contributing.md

* fuckin whoopsie doopsie

* linterino, linteroni

* fix followrequest test not starting processor

* fix other api/client tests not starting processor

* fix remaining tests where processor not started

* bump go-runners version

* don't check last-webfingered-at, processor may have updated this

* update swagger command

* update bun to latest version

* fix embed to work the same as before with new bun

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-09-28 18:30:40 +01:00
tobi
c4a08292ee
[feature] Show + federate emojis in accounts (#837)
* Start adding account emoji

* get emojis serialized + deserialized nicely

* update tests

* set / retrieve emojis on accounts

* show account emojis in web view

* fetch emojis from db based on ids

* fix typo in test

* lint

* fix pg migration

* update tests

* update emoji checking logic

* update comment

* clarify comments + add some spacing

* tidy up loops a lil (thanks kim)
2022-09-26 11:56:01 +02:00
tobi
3777f5c684
[bugfix] Server and closer bugfixes (#839)
* defer streaming from storage more forcefully

* shut down Server more gracefully

* use command context as server BaseContext
2022-09-19 12:43:22 +01:00
tobi
c1585d5f8a
[bugfix] Fix domains not being unblockable, log internal server errors from API (#833)
* log internal server errors from 500 api calls

* don't exec into nil dest

* don't exec into nil dest

* log error in router logger not api errorhandling

* update logging a tad

* linter
2022-09-17 19:12:12 +02:00
tobi
6a10123790
[docs] Swagger document /api/v1/custom_emojis (#824) 2022-09-13 13:30:07 +02:00
tobi
b42469e4e0
[feature] Allow users to set custom css for their profiles + threads (#808)
* add custom css account property + db func to fetch

* allow account to get/set custom css

* serve custom css for an account

* go fmt

* use monospace for customcss, add link

* add custom css to account cache

* fix broken field

* add custom css docs to user guide

* add `accounts-allow-custom-css` config flag

* add allow custom css to /api/v1/instance response

* only show/set custom css if allowed to do so

* only set/serve custom account css if enabled

* update swagger docs

* chain promise

* make bool a bit clearer

* use cache for GetAccountCustomCSSByUsername
2022-09-12 13:14:29 +02:00
tobi
268f252e0d
[feature] Fetch + display custom emoji in statuses from remote instances (#807)
* start implementing remote emoji fetcher

* update status where pk

* aaa

* tidy up a little

* check size limits for emojis

* thank you linter, i love you <3

* update swagger docs

* add emoji dereference test

* make emoji max sizes configurable

* normalize db.ErrAlreadyExists
2022-09-12 13:03:23 +02:00
tobi
a26280b31a
[bugfix] Fix preposterous characters reserved per URL amount (#809)
* clarify docs

* tidy up consts, set reserved chars much lower

* update tests
2022-09-08 12:36:42 +02:00
tobi
a872ddebe6
[feature] Custom emoji updates (serve emoji via s2s api, tune db models) (#805)
* migrate emojis

* add get emoji to s2s (federation) API

* add new emoji db + cache functions

* add shortcodeDomain lookup for emojis

* check existing emojis w/cache, not w/constraints

* go fmt

* add putEmoji func

* use new db emoji funcs instead of where

* remove emojistringstotags func

* add unique constraint back in

* fix up broken migration

* update index
2022-09-06 12:42:55 +02:00
tobi
2db0c64738
[bugfix] Use 'Image' instead of unrecognized 'Gif' type for media attachments (#801)
* Store gifs as Image type

* remove Gif attachment type, add Gifv type

* update test
2022-09-04 14:58:58 +02:00
tobi
006c8b604b
[feature] Set default header and avatar for API accounts to GtS ones (#799)
* validate web-asset-base-dir

* move default icons into converter

* always ensure avatar + header on api accounts

* update tests

* add default header

* don't return error from web module creation anymore

* tidy a bit

* use pngs for default avatars rather than svgs
2022-09-04 14:41:42 +02:00
tobi
4e13408fd4
[bugfix] Fix status fields in_reply_to_id and in_reply_to_account_id not being nullable (#798)
* make reply status fields nullable pointers

* update tests
2022-09-02 17:00:11 +02:00
Blackle Morisanchetto
ecb97f4e0b
[feature] Add support for the exclude_types[] parameter on the notifications endpoint (#784)
* Add support for the exclude_types[] parameter on the notifications endpoint

* Add swagger docs to notifications
2022-08-31 19:20:52 +02:00
tobi
0245c606d7
[chore] Test fixes (#788)
* use 'test' value for testrig storage backend

* update test dependency

* add WaitFor func in testrig

* use WaitFor function instead of time.Sleep

* tidy up tests

* make SentMessages a sync.map

* go fmt
2022-08-31 17:31:21 +02:00
nya1
bee8458a2d
[feature] add rate limit middleware (#741)
* feat: add rate limit middleware

* chore: update vendor dir

* chore: update readme with new dependency

* chore: add rate limit infos to swagger.md file

* refactor: add ipv6 mask limiter option

Add IPv6 CIDR /64 mask

* refactor: increase rate limit to 1000

Address https://github.com/superseriousbusiness/gotosocial/pull/741#discussion_r945584800

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-08-31 12:06:14 +02:00
Blackle Morisanchetto
e9b5ba0502
[bugfix] Check the length of form.MediaIDs instead of just checking for null (#766) 2022-08-26 17:37:51 +02:00
tobi
586ebb5059
return after redirects to web representation (#755) 2022-08-16 19:50:26 +01:00
tobi
ac6ed3d939
[chore] Update bun / sqlite versions; update gtsmodels (#754)
* upstep bun and sqlite versions

* allow specific columns to be updated in the db

* only update necessary columns for user

* bit tidier

* only update necessary fields of media_attachment

* only update relevant instance fields

* update tests

* update only specific account columns

* use bool pointers on gtsmodels
includes attachment, status, account, user

* update columns more selectively

* test all default fields on new account insert

* updating remaining bools on gtsmodels

* initialize pointer fields when extracting AP emoji

* copy bools properly

* add copyBoolPtr convenience function + test it

* initialize false bool ptrs a bit more neatly
2022-08-15 11:35:05 +01:00
tobi
91c8d5d20d
[bugfix] Fix thumbnails not taking exif rotation into account (#746)
* use disintegration/imaging instead of nfnt/resize

* update tests

* use disintegration lib for thumbing (if necessary)
2022-08-10 14:05:14 +02:00
tobi
117888cf59
[feature] Add first iteration of a user panel at /user (#736)
* start work on user panel

* parse source first before checking if empty form

* newline

* set avi + header nicely

* add posts settings

* render signin a bit nicer on mobile

* return OK json on successful change

* return unauthorized on bad password

* clarify message on insecure password

* make login a bit prettier

* add alt text + border round image previews

* add logout button

* add password change

* styling updates

* redirect /auth/edit to /user

* update tests

* fix validation tests

* better labels, link to more info

* make submit button generic component

* move submit button inside forms

* add autocomplete labels to password fields

* fix indentation (thx eslint)

* update eslintrc

* eslint: no-unescaped-entities

* initial deduplication between user and admin panel

* add default status/post format setting

* user panel styling for inputs

* update user panel styling, include normalize css

* add placeholder text

* input padding

Co-authored-by: f0x <f0x@cthu.lu>
2022-08-08 10:40:51 +02:00
tobi
879b4abde7
[bugfix] Markdown formatting updates (#743)
* add minify dependency specifically for markdown

* rearrange markdown formatting

* update markdown tests
2022-08-07 18:19:16 +02:00
tobi
f5689a9e5f
[feature] Let accounts set default status format, and use this when processing new statuses (#739)
* add post_format to acct & use it when making post

* update swagger docs

* add status_format updating to frontend

* fix up tests

* post_format => status_format

* add status_format to account validation
2022-08-06 12:09:21 +02:00
tobi
3ab3f58342
[bugfix] Parse source first before checking if empty form (#738) 2022-08-05 12:30:47 +02:00
Artémis
4fdbef04b4
[feature] Implemented notification clear (#720)
* Implemented notification clear

* Added the cache clear mechanism

* added multi user check test
2022-08-01 11:13:49 +02:00
tobi
8fdc9ed552
serve HEAD requests via the fileserver (#735) 2022-07-30 13:42:47 +01:00
tobi
8106b69856
[feature] add 'state' oauth2 param to /oauth/authorize (#730) 2022-07-28 16:43:27 +02:00
tobi
7ca5bac7c6
[bugfix] Fix Toot CLI media attachments not working properly (#726) 2022-07-22 13:43:51 +02:00
tobi
73b8839c5d
[bugfix] Make /api/v2/media more compatible with masto API (#724)
* update docs

* make api version into a path param

* update tests

* workaround to unset URL if using v2 of api

* make some fields into pointers
2022-07-22 12:48:19 +02:00
tobi
40f91d052c
[chore] Update image/video size defaults to mastodon's (#723) 2022-07-20 12:25:06 +02:00
tobi
c84384e660
[bugfix] html escape special characters in text instead of totally removing them (#719)
* remove minify dependency

* tidy up some tests

* remove pre + postformat funcs

* rework sanitization + formatting

* update tests

* add some more markdown tests
2022-07-19 15:21:17 +02:00
kim
098dbe6ff4
[chore] use our own logging implementation (#716)
* first commit

Signed-off-by: kim <grufwub@gmail.com>

* replace logging with our own log library

Signed-off-by: kim <grufwub@gmail.com>

* fix imports

Signed-off-by: kim <grufwub@gmail.com>

* fix log imports

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>

* fix package import cycle between config and log package

Signed-off-by: kim <grufwub@gmail.com>

* fix empty kv.Fields{} being passed to WithFields()

Signed-off-by: kim <grufwub@gmail.com>

* fix uses of log.WithFields() with whitespace issues and empty slices

Signed-off-by: kim <grufwub@gmail.com>

* *linter related grumbling*

Signed-off-by: kim <grufwub@gmail.com>

* gofmt the codebase! also fix more log.WithFields() formatting issues

Signed-off-by: kim <grufwub@gmail.com>

* update testrig code to match new changes

Signed-off-by: kim <grufwub@gmail.com>

* fix error wrapping in non fmt.Errorf function

Signed-off-by: kim <grufwub@gmail.com>

* add benchmarking of log.Caller() vs non-cached

Signed-off-by: kim <grufwub@gmail.com>

* fix syslog tests, add standard build tags to test runner to ensure consistency

Signed-off-by: kim <grufwub@gmail.com>

* make syslog tests more robust

Signed-off-by: kim <grufwub@gmail.com>

* fix caller depth arithmatic (is that how you spell it?)

Signed-off-by: kim <grufwub@gmail.com>

* update to use unkeyed fields in kv.Field{} instances

Signed-off-by: kim <grufwub@gmail.com>

* update go-kv library

Signed-off-by: kim <grufwub@gmail.com>

* update libraries list

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter get nerfed

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-07-19 10:47:55 +02:00
tobi
839c4346ad
[performance] Add long cache-control max age to fileserver (#710) 2022-07-15 13:23:04 +02:00
tobi
6418307c64
[feature] Add back/next buttons to profiles for paging through statuses (#708)
* add GetAccountWebStatuses to db

* add WebStatusesGet func to processor

* don't add limit to next/prev links if 0

* take query params for next/prev statuses

* add separate next + prev links for convenience

* show 'nothing here' message if no statuses exist

* add back / next links to profiles

* allow paging down only

* go fmt ./...

* 'recent public toots' -> 'latest public toots'
2022-07-13 09:57:47 +02:00
kim
6934ae378a
[chore] improved router logging, recovery and error handling (#705)
* move panic recovery to logging middleware, improve logging + panic recovery logic

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* remove skip paths code

Signed-off-by: kim <grufwub@gmail.com>

* re-enable log quoting

Signed-off-by: kim <grufwub@gmail.com>

* use human-friendly bytesize in logging body size

Signed-off-by: kim <grufwub@gmail.com>

* only disable quoting in debug builds

Signed-off-by: kim <grufwub@gmail.com>

* use logrus level instead of debug.DEBUG() to enable/disable quoting

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* fix instance tests

Signed-off-by: kim <grufwub@gmail.com>

* fix gin test contexts created with missing engine HTML renderer

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding not logging query parameters

Signed-off-by: kim <grufwub@gmail.com>

* better explain 'DisableQuoting' logic

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>
2022-07-12 08:32:20 +01:00
kim
e75d742afe
[chore] Improve panic handling (#700)
* add miekg/dns dependency

* set/validate accountDomain

* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test

* panics get logged at error level, now include stacktrace

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-08 12:01:44 +02:00
tobi
81dd4f3660
[bugfix] Fix footer info fields not wrapping (#694)
* flex-wrap footer items

* add testrig instance entry with more info set
2022-07-05 16:44:58 +02:00
tobi
b61b000e0a
[bugfix] Fix incorrect domain showing in profiles (#693)
* use instance account_domain in profile

* add instance account_domain field
2022-07-05 14:03:44 +02:00
tobi
4a69651a7c
[bugfix] Visibility fixes (#687)
* test self boost

* only CC to public when necessary
2022-07-04 15:41:20 +02:00
f0x52
c6d1b82f48
[chore] configure instance.Version for testrig (#659)
* configure instance.Version for testrig

* update instancepatch tests

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-03 12:27:32 +02:00
Dominik Süß
9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi
07620acc0e
[feature] Use default instance thumbnail if instance account header not set (#672) 2022-06-26 12:33:11 +02:00
tobi
68736efd20
[feature] add configuration to /api/v1/instance response (#670)
* add configuration object to api instance model

* regenerate swagger docs

* add func to return all supported mimes for media

* add instance configuration to api serialization

* fix json tags

* update instance endpoint tests

* fix typeutils tests

* final regen of swagger docs

* omitempty instance configuration
2022-06-26 10:58:45 +02:00
ugla
3e4e57d554
[bugfix] disallow following or blocking yoursel (#667)
Closes #664
2022-06-25 11:14:05 +02:00
tobi
7eacbd064b
[bugfix] allow setting empty email via instance patch (#665) 2022-06-24 10:43:21 +02:00
tobi
5f00d4980b
[feature] Implement /api/v1/instance/peers endpoint (#660)
* add missing license headers

* start adding instance peers get

* rename domainblock.go

* embed domain in domainblock so it can be reused

* update swagger docs

* add test instances to db

* update tests

* add/update instancepeersget

* update domain model

* add getinstancepeers to db

* instance-expose-peers, instance-expose-suspended

* add auth checks for both current filters

* attach endpoint to router

* include public comment

* obfuscate domain if required

* go mod tidy

* update swagger docs

* remove unnecessary comment

* return 'flat' peerlist if no query params provided
2022-06-23 16:54:54 +02:00
tobi
610395d5a5
[chore] make tests more cacheable by avoiding time.Now() (#656) 2022-06-19 17:10:24 +02:00
tobi
dfdc473cef
[chore] Webfinger rework (#627)
* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test
2022-06-11 11:01:34 +02:00
tobi
694a490589
[feature] Add created_at and error_description to /oauth/token endpoint (#645)
* start fiddling about with oauth server

* start returning more helpful errors from oauth

* test helpful(ish) token errors

* add missing license header
2022-06-11 10:39:39 +02:00
tobi
5864954e2e
[bugfix] Fix domain blocks get regression (#642)
* fix domain blocks get regression

* add missing license text headers
2022-06-09 13:28:15 +02:00
tobi
1ede54ddf6
[feature] More consistent API error handling (#637)
* update templates

* start reworking api error handling

* update template

* return AP status at web endpoint if negotiated

* start making api error handling much more consistent

* update account endpoints to new error handling

* use new api error handling in admin endpoints

* go fmt ./...

* use api error logic in app

* use generic error handling in auth

* don't export generic error handler

* don't defer clearing session

* user nicer error handling on oidc callback handler

* tidy up the sign in handler

* tidy up the token handler

* use nicer error handling in blocksget

* auth emojis endpoint

* fix up remaining api endpoints

* fix whoopsie during login flow

* regenerate swagger docs

* change http error logging to debug
2022-06-08 20:38:03 +02:00
tobi
6f6e89e271
[feature] Add paging via Link header for notifications and account statuses (#629)
* test link headers

* page get account statuses properly

* page get notifications

* add util func for packaging timeline responses

* return timelined stuff from accountstatusesget

* rename timeline response

* use new convenience function

* go fmt
2022-06-08 20:22:49 +02:00
tobi
f3b44426f4
[bugfix] Update time marshalling format to provide 3 digits of ms (#630) 2022-06-04 20:03:01 +02:00
kim
43ac0cdb9c
[chore] Global server configuration overhaul (#575)
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct

Signed-off-by: kim <grufwub@gmail.com>

* improved code comment

Signed-off-by: kim <grufwub@gmail.com>

* linter

Signed-off-by: kim <grufwub@gmail.com>

* fix unmarshaling

Signed-off-by: kim <grufwub@gmail.com>

* remove kim's custom go compiler changes

Signed-off-by: kim <grufwub@gmail.com>

* generate setter and flag-name functions, implement these in codebase

Signed-off-by: kim <grufwub@gmail.com>

* update deps

Signed-off-by: kim <grufwub@gmail.com>

* small change

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter...

Signed-off-by: kim <grufwub@gmail.com>

* move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync

Signed-off-by: kim <grufwub@gmail.com>

* lint

Signed-off-by: kim <grufwub@gmail.com>

* update code comments

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* improved version string (removes time + go version)

Signed-off-by: kim <grufwub@gmail.com>

* fix version string build to pass test script + consolidate logic in func

Signed-off-by: kim <grufwub@gmail.com>

* add license text, update config.Defaults comment

Signed-off-by: kim <grufwub@gmail.com>

* add license text to generated config helpers file

Signed-off-by: kim <grufwub@gmail.com>

* defer unlock on config.Set___(), to ensure unlocked on panic

Signed-off-by: kim <grufwub@gmail.com>

* make it more obvious which cmd flags are being attached

Signed-off-by: kim <grufwub@gmail.com>
2022-05-30 14:41:24 +02:00
tobi
ae5402ada6
[chore] Mastodon api fixups (#617)
* don't omitempty on description

* don't omitempty on any fields

* add ms to timestamp format

* don't omitempty on text_url

* rearrange attachment fields a bit

* just give URL again as attachment text url

* update tests

* fix accidental replace
2022-05-28 18:59:55 +01:00
tobi
5668ce1ec7
[bugfix] Fix HTML escaping in instance title (#607)
* move caption sanitization -> sanitize.go

* use sanitizeplaintext rather than removehtml

* rename sanitizecaption to sanitizeplaintext

* avoid removing html twice from statuses

* unexport remoteHTML
it's no longer used outside the text package so this
makes it less confusing

* test instance PATCH
2022-05-26 11:37:13 +02:00
tobi
f0c9f4169b
[bugfix] Fix multiple dereferences of boosted status causing media duplication (#589)
* add some announces to test models

* start on announce test logic

* test federatingDB.Announce

* change signature of GetRemoteStatus

* remove 'refresh' logic and replace it with refetch

* go fmt

* remove timeline manager from processor test

* make zork created at determinate

* test get account statuses

* test get + serialize zork

* make account keys determinate

* make admin accountCreate time determinate

* test account to as

* init test config before test log

* test status to frontend

* remove daft Within check

* hack around a bit

* use index of slice
2022-05-23 16:40:03 +01:00
Adelie Paull
caa0cde0e0
[feature] implement custom_emojis endpoint (#563)
* implement custom_emojis api endpoint

* add tests for getting custom emoji out of the database and converting to api emoji

* change sort direction of emoji query

* change logging level and initialize array with known length as per kim's suggestions

* add continue to lessen risk of making a malformed struct during conversion from db to api emojis
2022-05-20 10:34:36 +02:00
tobi
b143877995
[feature] Unused avatar and header cleanup (#574)
* rename + tidy up remote pruning

* fix media attachment account join
see https://bun.uptrace.dev/guide/golang-orm.html#table-relationships

* update logging to new function name

* add get avatars and headers to bun

* add pruneallmeta function

* don't set uncached since we're deleting anyway

* fix totalPruned being in wrong place

* test pruning meta

* go fmt ./...

* rename mediaprune

* add meta pruning to routine mediaprune

* tidy up cleanup job scheduling

* rename adminmediaremoteprune

* update mediacleanup to use renamed prune func

* update swagger docs a little bit

* reuse cancel + context
2022-05-15 15:45:04 +01:00
Sashanoraa
6e947ff266
[feature] Media cleanup endpoint (#560)
Adds an admin endpoint to trigger a remote media cleanup.

Fixed #348

Signed-off-by: Sashanoraa <sasha@noraa.gay>
2022-05-15 14:52:46 +02:00
kim
223025fc27
[security] transport.Controller{} and transport.Transport{} security and performance improvements (#564)
* cache transports in controller by privkey-generated pubkey, add retry logic to transport requests

Signed-off-by: kim <grufwub@gmail.com>

* update code comments, defer mutex unlocks

Signed-off-by: kim <grufwub@gmail.com>

* add count to 'performing request' log message

Signed-off-by: kim <grufwub@gmail.com>

* reduce repeated conversions of same url.URL object

Signed-off-by: kim <grufwub@gmail.com>

* move worker.Worker to concurrency subpackage, add WorkQueue type, limit transport http client use by WorkQueue

Signed-off-by: kim <grufwub@gmail.com>

* fix security advisories regarding max outgoing conns, max rsp body size

- implemented by a new httpclient.Client{} that wraps an underlying
  client with a queue to limit connections, and limit reader wrapping
  a response body with a configured maximum size
- update pub.HttpClient args passed around to be this new httpclient.Client{}

Signed-off-by: kim <grufwub@gmail.com>

* add httpclient tests, move ip validation to separate package + change mechanism

Signed-off-by: kim <grufwub@gmail.com>

* fix merge conflicts

Signed-off-by: kim <grufwub@gmail.com>

* use singular mutex in transport rather than separate signer mus

Signed-off-by: kim <grufwub@gmail.com>

* improved useragent string

Signed-off-by: kim <grufwub@gmail.com>

* add note regarding missing test

Signed-off-by: kim <grufwub@gmail.com>

* remove useragent field from transport (instead store in controller)

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* reset other signing headers on each loop iteration

Signed-off-by: kim <grufwub@gmail.com>

* respect request ctx during retry-backoff sleep period

Signed-off-by: kim <grufwub@gmail.com>

* use external pkg with docs explaining performance "hack"

Signed-off-by: kim <grufwub@gmail.com>

* use http package constants instead of string method literals

Signed-off-by: kim <grufwub@gmail.com>

* add license file headers

Signed-off-by: kim <grufwub@gmail.com>

* update code comment to match new func names

Signed-off-by: kim <grufwub@gmail.com>

* updates to user-agent string

Signed-off-by: kim <grufwub@gmail.com>

* update signed testrig models to fit with new transport logic (instead uses separate signer now)

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter

Signed-off-by: kim <grufwub@gmail.com>
2022-05-15 11:16:43 +02:00
Eamonn O'Brien-Strain
b24b71c0a4
[feature] Include password strength in error message when password strength is too low (#550)
* When password validation fails, return how close to enough entropy it has.

* Shorter version of low-strength password error message
2022-05-09 10:31:46 +02:00
tobi
5004e0a9da
[bugfix] Fix remote media pruning failing if media already gone (#548)
* fix error check of prune to allow missing files

* update go-store library, add test for pruning item with db entry but no file

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unneccessary error check

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: kim <grufwub@gmail.com>
2022-05-08 18:49:45 +01:00
kim
26b74aefaf
[bugfix] Fix existing bio text showing as HTML (#531)
* fix existing bio text showing as HTML

- updated replaced mentions to include instance
- strips HTML from account source note in Verify handler
- update text formatter to use buffers for string writes

Signed-off-by: kim <grufwub@gmail.com>

* go away linter

Signed-off-by: kim <grufwub@gmail.com>

* change buf reset location, change html mention tags

Signed-off-by: kim <grufwub@gmail.com>

* reduce FindLinks code complexity

Signed-off-by: kim <grufwub@gmail.com>

* fix HTML to text conversion

Signed-off-by: kim <grufwub@gmail.com>

* Update internal/regexes/regexes.go

Co-authored-by: Mina Galić <mina.galic@puppet.com>

* use improved html2text lib with more options

Signed-off-by: kim <grufwub@gmail.com>

* fix to produce actual plaintext from html

Signed-off-by: kim <grufwub@gmail.com>

* fix span tags instead written as space

Signed-off-by: kim <grufwub@gmail.com>

* performance improvements to regex replacements, fix link replace logic for un-html-ing in the future

Signed-off-by: kim <grufwub@gmail.com>

* fix tag/mention replacements to use input string, fix link replace to not include scheme

Signed-off-by: kim <grufwub@gmail.com>

* use matched input string for link replace href text

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code (to appease linter :sobs:)

Signed-off-by: kim <grufwub@gmail.com>

* improve hashtagFinger regex to be more compliant

Signed-off-by: kim <grufwub@gmail.com>

* update breakReplacer to include both unix and windows line endings

Signed-off-by: kim <grufwub@gmail.com>

* add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts

Signed-off-by: kim <grufwub@gmail.com>

* drop unnecessary code

Signed-off-by: kim <grufwub@gmail.com>

* update text package tests to fix logic changes

Signed-off-by: kim <grufwub@gmail.com>

* add raw note content testing to account update and account verify

Signed-off-by: kim <grufwub@gmail.com>

* remove unused modules

Signed-off-by: kim <grufwub@gmail.com>

* fix emoji regex

Signed-off-by: kim <grufwub@gmail.com>

* fix replacement of hashtags

Signed-off-by: kim <grufwub@gmail.com>

* update code comment

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: Mina Galić <mina.galic@puppet.com>
2022-05-07 17:55:27 +02:00
tobi
a5852fd7e4
[performance] Speed up some of the slower db queries (#523)
* remove unnecessary LOWER() db calls

* warn during slow db queries

* use bundb built-in exists function

* add db block test

* update account block query

* add domain block db test

* optimize domain block query

* fix implementing wrong test

* exclude most columns when checking block

* go fmt

* remote more unnecessary use of LOWER()
2022-05-02 11:53:46 +01:00
tobi
63f84da3e4
[chore] Inboxes for iri test (#508)
* tidy up some federation tests

* add missing license to following.go

* give zork some followers, as a treat

* test InboxesForIRI

* Go fmt

* update timeline tests
2022-04-29 15:05:13 +02:00
kim
420e2fb22b
replace async client API / federator msg processing with worker pools (#497)
* replace async client API / federator msg processing with worker pools
* appease our lord-and-saviour, the linter
2022-04-28 13:23:11 +01:00
tobi
0d05bf473b
[bug] Fix status API / status web API being case sensitive (#481)
* make getStatus by id case-insensitive

* test get status case insensitive

* init config before log
2022-04-24 14:24:43 +02:00
tobi
9813a044c0
[feature] Implement media v2 endpoint to accommodate Tusky 17 (#480)
* serve v2 media api

* go fmt
2022-04-24 13:11:52 +02:00
tobi
26683b3d49
[feature] Web profile pages for accounts (#449)
* add default avatars

* allow webModule to error

* return errWithCode from account get

* add AccountGetLocalByUsername

* check nil requesting account

* add timestampShort function for just month/year

* move loading logic to New + add default avatars

* add profile page view

* update swagger docs

* add excludeReblogs to GetAccountStatuses

* ignore casing when selecting local account by username

* appropriate redirects

* css fiddling

* add 'about' heading

* adjust thread page to work with routing

* return AP representation if requested + authorized

* simplify auth check

* go fmt

* golangci-lint ignore math/rand
2022-04-15 14:33:01 +02:00
tobi
a7e9dee33d
[bugfix] Fix broken only_media and only_public flags on /api/v1/accounts/:id/statuses (#448)
* bump superseriousbusiness/activity version

* fetch list of follower inbox iris

* add test for getting account statuses

* fix incorrect publicOnly parsing

* update media not null query
2022-04-10 15:11:12 +02:00
tobi
37d310f981
[feature] Dereference remote mentions when the account is not already known (#442)
* remove mention util function from db

* add ParseMentionFunc to gtsmodel

* add parseMentionFunc to processor

* refactor search to simplify it a bit

* add parseMentionFunc to account

* add parseMentionFunc to status

* some renaming for clarity

* test dereference of unknown mentioned account
2022-03-29 11:54:56 +02:00
tobi
55ad6dee71
[feature] Admin account actions (#432)
* add accountAction to the admin API

* model admin account action

* add admin account action to the processor

* add migration for new AdminAccountActions table

* fix accounts admin path

* Update swagger docs
2022-03-19 12:01:40 +01:00
tobi
532c4cc697
[feature] Federate local account deletion (#431)
* add account delete to API

* model account delete request

* add AccountDeleteLocal

* federate local account deletes

* add DeleteLocal

* update transport (controller) to allow shortcuts

* delete logic + testing

* update swagger docs

* more tests + fixes
2022-03-15 16:12:35 +01:00
tobi
07727753b9
[feature] Clean up/uncache remote media (#407)
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00
tobi
15d1e6b3a1
[bugfix] Fix thumbnail image type (#406)
* fix thumbnail content-type

* test fix thumbnail content-type
2022-02-21 11:26:26 +01:00
tobi
23034ec145
[feature] Stream files via reader (#404)
* serve files via reader rather than byte slice

* close readcloser when we're done with it

* cast reader to readcloser
2022-02-19 11:44:56 +01:00
tsmethurst
3301148bb7 merge fixup 2022-02-08 12:17:53 +01:00
tsmethurst
4c294a596a Merge branch 'main' into media_refactor 2022-02-08 12:17:18 +01:00
Forest Johnson
6ed368cbeb
[feature] add authorization to the already-existing authentication (#365)
* add ensureUserIsAuthorizedOrRedirect to /oauth/authorize

* adding authorization (email confirm, account approve, etc) to TokenCheck

* revert un-needed changes to signin.go

* oops what happened here

* error css

* add account.SuspendedAt check

* remove redundant checks from oauth util Authed function

* wip tests

* tests passing

* stop stripping useful information from ErrAlreadyExists

* that feeling of scraping the dryer LINT off the screen

* oops I didn't mean to get rid of this NewTestRouter function

* make tests work with recorder

* re-add ConfigureTemplatesWithGin to handle template path err

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-02-07 12:04:31 +01:00
tobi
5c9d20cea3
don't bother negotiating Accept for webfinger (#381) 2022-02-06 15:00:57 +01:00
tobi
1b36e85840
[feature] Rework timeline code to make it useful for more than just statuses (#373)
* add preparable and timelineable interfaces

* initialize timeline manager within the processor

* generic renaming

* move status-specific timeline logic into the processor

* refactor timeline to make it useful for more than statuses
2022-02-05 12:47:38 +01:00
tobi
18e7537393
[bug] Fix OIDC users requiring second approval (#371)
* tidy up NewSignup
* pre-approve users created via OIDC
2022-01-31 16:03:47 +01:00
tobi
66175c8ad9
[bug] Fix minimum description length check (#369)
* add/update tests

* don't check min description length on media create
2022-01-30 16:10:53 +01:00
tsmethurst
5d9e9e0e7f fix up some account conversion logic 2022-01-25 12:03:25 +01:00
tsmethurst
8e6ba1de52 start trying to figure out why this test is failing 2022-01-24 18:11:38 +01:00
tsmethurst
6bf39d0fc1 emoji code passing muster 2022-01-15 17:36:15 +01:00
tsmethurst
c4a533db72 start fixing up emoji processing code 2022-01-15 14:33:58 +01:00
tsmethurst
113f9d9ab4 pass a function into the manager, start work on emoji 2022-01-11 17:49:14 +01:00
tsmethurst
e0f9323b9a test the media manager a bit, add shutdown logic 2022-01-10 18:36:09 +01:00
tsmethurst
f61c3ddcf7 compiling now 2022-01-08 17:17:01 +01:00
tsmethurst
c4d63d125b more refactoring, media handler => manager 2021-12-28 16:36:00 +01:00
tobi
ef5a9256a8
Extend license notices to 2022 (#354) 2021-12-20 18:42:19 +01:00
tobi
cb8688f429
Remove unnecessary storage config variables (#344)
* rewire config to not use extraneous serve vars

* rename 'file' to 'local' for consistency

* use Type and Size again
2021-12-20 15:19:53 +01:00
tobi
2582515b4d
return first offer when no accept header set (#351) 2021-12-20 11:08:54 +01:00
tobi
67ac8db190
Upstep Go dependencies (#340)
* Upstep Go dependencies

* tiny linter fix

* Tidy
2021-12-12 15:47:51 +01:00
tobi
e2daf0f012
Add Accept header negotiation to relevant API endpoints (#337)
* start centralizing negotiation logic for API

* swagger document nodeinfo endpoint

* go fmt

* document negotiate function

* use content negotiation

* tidy up negotiation logic

* negotiate content throughout client api

* swagger

* remove attachment on Content

* add accept header to test requests
2021-12-11 17:50:00 +01:00
tobi
0884f89431
Implement Cobra CLI tooling, Viper config tooling (#336)
* start pulling out + replacing urfave and config

* replace many many instances of config

* move more stuff => viper

* properly remove urfave

* move some flags to root command

* add testrig commands to root

* alias config file keys

* start adding cli parsing tests

* reorder viper init

* remove config path alias

* fmt

* change config file keys to non-nested

* we're more or less in business now

* tidy up the common func

* go fmt

* get tests passing again

* add note about the cliparsing tests

* reorganize

* update docs with changes

* structure cmd dir better

* rename + move some files around

* fix dangling comma
2021-12-07 13:31:39 +01:00
tobi
ce22e03f9d
Require confirmed email when checking oauth token (#332)
* move token checker to security package

* update tests with new security package

* add oauth token checking to security package

* check if user email confirmed when parsing token
2021-11-27 14:53:34 +01:00
Forest Johnson
ab316d2250
Fix broken defaults and broken flags (#314)
* start with a default config, not an empty config.

* some data structures were present on Empty config but not Default config

* the monkey patched CLIContext is working

* remove print debugging log

* make the behaviour of the flags consistent across all data types

Conflicts:
	internal/config/config.go

* try to fix accidentally broken test
2021-11-24 17:01:38 +01:00
tobi
f8630348b4
Enable stricter linting with golangci-lint (#316)
* update golangci-lint

* add golangci config file w/ more linters

* correct issues flagged by stricter linters

* add more generous timeout for golangci-lint

* add some style + formatting guidelines

* move timeout to config file

* go fmt
2021-11-22 08:46:19 +01:00
tobi
aee1932bd4
First db migration (#297)
* store first db schema

* md linting *screams*
2021-11-13 17:30:01 +01:00
tobi
09ef9e639e
move to ssb gofed fork (#298) 2021-11-13 17:29:43 +01:00
tobi
829a934d23
update dependencies (#296) 2021-11-13 12:29:08 +01:00
tobi
2aaec82732
smtp + email confirmation (#285)
* add smtp configuration

* add email confirm + reset templates

* add email sender to testrig

* flesh out the email sender interface

* go fmt

* golint

* update from field with more clarity

* tidy up the email formatting

* fix tests

* add email sender to processor

* tidy client api processing a bit

* further tidying in fromClientAPI

* pin new account to user

* send msg to processor on new account creation

* generate confirm email uri

* remove emailer from account processor again

* add processCreateAccountFromClientAPI

* move emailer accountprocessor => userprocessor

* add email sender to user processor

* SendConfirmEmail function

* add noop email sender

* use noop email sender in tests

* only assemble message if callback is not nil

* use noop email sender if no smtp host is defined

* minify email html before sending

* fix wrong email address

* email confirm test

* fmt

* serve web hndler

* add email confirm handler

* init test log properly on testrig

* log emails that *would* have been sent

* go fmt ./...

* unexport confirm email handler

* updatedAt

* test confirm email function

* don't allow tokens older than 7 days

* change error message a bit

* add basic smtp docs

* add a few more snippets

* typo

* add email sender to outbox tests

* don't use dutch wikipedia link

* don't minify email html
2021-10-31 15:46:23 +01:00
tobi
4b1d9d3780
Serve outbox for Actor (#289)
* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* add statusesvisible convenience function

* add minID + onlyPublic to account statuses get

* move swagger collection stuff to common

* start working on Outbox GETting

* move functions into federationProcessor

* outboxToASCollection

* bit more work on outbox paging

* wrapNoteInCreate function

* test + hook up the processor functions

* don't do prev + next links on empty reply

* test get outbox through api

* don't fail on no status entries

* add outbox implementation doc

* typo
2021-10-24 11:57:39 +02:00
tobi
15621f5324
Follow request improvements (#282)
* tiny doc update

* add rejectfollowrequest to db

* add follow request reject to processor

* add reject handler

* tidy up follow request api

* tidy up federation call

* regenerate swagger docs

* api endpoint tests

* processor test

* add reject federatingdb handler

* start writing reject tests

* test reject follow request

* go fmt

* increase sleep for slow test setups

* more relaxed time.sleep
2021-10-16 13:27:43 +02:00
tobi
107685e22e
User password change (#280)
* start passwordChangeHandler

* add user scope

* add user module / api path

* add password change request

* make comment clearer

* add user to processor

* required true

* add processor call to handler

* don't pass tc or channel

* change password func + tests

* add some first docs about password management

* update swagger docs

* add api tests

* go fmt

* test fixes
2021-10-14 14:26:04 +02:00
R. Aidan Campbell
083099a957
reference global logrus (#274)
* reference logrus' global logger instead of passing and storing a logger reference everywhere

* always directly use global logrus logger instead of referencing an instance

* test suites should also directly use the global logrus logger

* rename gin logging function to clarify that it's middleware

* correct comments which erroneously referenced removed logger parameter

* setting log level for tests now uses logrus' exported type instead of the string value, to guarantee error isn't possible
2021-10-11 14:37:33 +02:00
tobi
3dc7644ae6
Derive visibility fixes (#271)
* use pub public const

* don't error on no summary

* move extract visibility to separate function

* extract visibility test

* add addressable interface
2021-10-06 18:18:02 +02:00
tobi
e04b187702
Refactor/tidy (#261)
* tidy up streaming

* cut down code duplication

* test get followers/following

* test streaming processor

* fix some test models

* add TimeMustParse

* fix uri / url typo

* make trace logging less verbose

* make logging more consistent

* disable quote on logging

* remove context.Background

* remove many extraneous mastodon references

* regenerate swagger

* don't log query on no rows result

* log latency first for easier reading
2021-10-04 15:24:19 +02:00
tobi
36a09dd0df
handle remote account deletion more systematically (#254) 2021-09-30 10:56:02 +02:00
tobi
b5a7e1ba32
Account update issue (#250)
* start poking around

* tests

* notes and fiddling
2021-09-28 15:21:59 +02:00
tobi
c7cfbe2702
Unblock fix (#247)
* start tests for inbox posts

* go mod tidy

* rename transferContext

* test block/unblock

* improve logging

* improve logging

* fix comment typo
2021-09-24 17:56:48 +02:00
tobi
ddfd83d0fb
update media processing (#244) 2021-09-23 11:13:11 +02:00
tobi
7e6350b448
remove hardcoded friendica user agent block (#239) 2021-09-21 11:40:16 +02:00
tobi
b315eee43b
Update webfingering a little, add tests (#236)
* Update webfingering a little, add tests

* fix broken tests oops
2021-09-20 16:46:45 +02:00
tobi
92186c8c6f
federated authentication better logging + tidying (#232)
* change trace logging in authenticator

* messing about

* lil changes

* go fmt

* error fix

* Fix broken test
2021-09-16 11:35:09 +02:00
kim (grufwub)
e43a46e982 add git.iim.gay/grufwub/go-store for storage backend, replacing blob.Storage
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-11 20:12:47 +01:00
tobi
9dc2255a8f
kim is a reply guy (#208)
* bun debug

* bun trace logging hooks

* more tests

* fix up some stuffffff

* drop the frontend cache until a proper fix is made

* go fmt
2021-09-11 13:19:06 +02:00
tobi
e681aac589
fixes + db changes (#204)
* fixes + db changes

* make duration more lenient
2021-09-10 14:36:10 +02:00
tobi
555ea8edfb
Import export (#194)
* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* update bun library -> v1.0.4

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library -> v4.3.1-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* handle oauth token scope, fix user.SigninCount + token.UserID

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update oauth library --> v4.3.2-SSB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update sqlite library -> v1.13.0

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* review changes

* start with export/import code

* messing about with decoding/encoding

* some more fiddling

* stuff is WORKING

* working pretty alright!

* go fmt

* fix up tests, add docs

* start backup/restore doc

* tweaks

* credits

* update advancedVisibility settings

* review changes

Co-authored-by: kim (grufwub) <grufwub@gmail.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2021-09-09 16:15:25 +02:00
kim (grufwub)
d2a85f2f5f handle oauth token scope, fix user.SigninCount + token.UserID
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:46:08 +01:00
tsmethurst
4696e1a7b3 moving stuff around 2021-09-01 18:29:25 +02:00
tsmethurst
684bd56528 move oauth models into gtsmodel 2021-09-01 11:45:01 +02:00
kim
7d193de25f
Improve GetRemoteStatus and db.GetStatus() logic (#174)
* only fetch status parents / children if explicity requested when dereferencing

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Remove recursive DB GetStatus logic, don't fetch parent unless requested

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* StatusCache copies status so there are no thread-safety issues with modified status objects

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* remove sqlite test files

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix bugs introduced by previous commit

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix not continue on error in loop

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* use our own RunInTx implementation (possible fix for nested tx error)

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix cast statement to work with SQLite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* be less strict about valid status in cache

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add cache=shared ALWAYS for SQLite db instances

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix EnrichRemoteAccount when updating account fails

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add nolint tag

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* ensure file: prefixes the filename in sqlite addr

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add an account cache, add status author account from db

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix incompatible SQLite query

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* *actually* use the new getAccount() function in accountsDB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update cache tests to use test suite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add RelationshipTestSuite, add tests for methods with changed SQL

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-01 11:08:21 +02:00
kim
ed46224573
Add SQLite support, fix un-thread-safe DB caches, small performance f… (#172)
* Add SQLite support, fix un-thread-safe DB caches, small performance fixes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add SQLite licenses to README

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* appease the linter, and fix my dumbass-ery

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* make requested changes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add back comment

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-29 16:41:41 +02:00
tobi
2162f21636
fix broken db queries in auth (#160) 2021-08-26 19:56:40 +02:00
tobi
2dc9fc1626
Pg to bun (#148)
* start moving to bun

* changing more stuff

* more

* and yet more

* tests passing

* seems stable now

* more big changes

* small fix

* little fixes
2021-08-25 15:34:33 +02:00
tobi
4920229a3b
Database updates (#144)
* start moving some database stuff around

* continue moving db stuff around

* more fiddling

* more updates

* and some more

* and yet more

* i broke SOMETHING but what, it's a mystery

* tidy up

* vendor ttlcache

* use ttlcache

* fix up some tests

* rename some stuff

* little reminder

* some more updates
2021-08-20 12:26:56 +02:00
Tobi Smethurst
ce190d867c
Text/status parsing fixes (#141)
* aaaaaa

* vendor minify

* update + test markdown parsing
2021-08-16 19:17:56 +02:00
Tobi Smethurst
98263a7de6
Grand test fixup (#138)
* start fixing up tests

* fix up tests + automate with drone

* fiddle with linting

* messing about with drone.yml

* some more fiddling

* hmmm

* add cache

* add vendor directory

* verbose

* ci updates

* update some little things

* update sig
2021-08-12 21:03:24 +02:00
Tobi Smethurst
0f2de6394a
Dereference remote replies (#132)
* decided where to put reply dereferencing

* fiddling with dereferencing threads

* further adventures

* tidy up some stuff

* move dereferencing functionality

* a bunch of refactoring

* go fmt

* more refactoring

* bleep bloop

* docs and linting

* start implementing replies collection on gts side

* fiddling around

* allow dereferencing our replies

* lint, fmt
2021-08-10 13:32:39 +02:00
Tobi Smethurst
0386a28b5a
Frodo swaggins (#126)
* more swagger fun

* document a whole bunch more stuff

* more swagger yayyyyyyy

* progress + go fmt
2021-08-02 19:06:44 +02:00
Tobi Smethurst
6bd26ff4c4
more swagger docs + other changes (#125)
* more swagger docs + other changes

* go fmt
2021-07-31 23:17:39 +02:00
Tobi Smethurst
58dddd86e0
Swagger (#124)
* start experimenting with swagger documentation

* further adventures in swagger

* do a few more api paths

* account paths documented

* go fmt

* fix up some models

* bit o lintin'
2021-07-31 17:49:59 +02:00
Tobi Smethurst
a940a520d3
Link hashtag bug (#121)
* link + hashtag bug

* remove printlns

* tidy up some duplicated code
2021-07-29 13:18:22 +02:00
Tobi Smethurst
ad0e26dc04
Markdown Statuses (#116)
* parse markdown statuses if desired

* add some preliminary docs for writing posts
2021-07-26 20:25:54 +02:00
Tobi Smethurst
05e9af089c
Oidc (#109)
* add oidc config

* inching forward with oidc idp

* lil webfingy fix

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* add oidc config

* inching forward with oidc idp

* bit more progress

* further oidc

* oidc now working

* document dex config

* replace broken images

* add additional credits

* tiny doc update

* update

* document

* docs + comments
2021-07-23 10:36:28 +02:00
Tobi Smethurst
113186ce4e
lil webfingy fix (#106)
* lil webfingy fix

* return requested content type from fede requests
2021-07-21 17:46:59 +02:00
Tobi Smethurst
b1a4f38e38
allow different host + accountDomain (#103)
* allow different host + accountDomain

* use accountDomain in tags
2021-07-19 18:42:08 +02:00
Tobi Smethurst
846057f0d6
Block/unblock (#96)
* remote + local block logic, incl. federation

* improve blocking stuff

* fiddle with display of blocked profiles

* go fmt
2021-07-11 16:22:21 +02:00
Tobi Smethurst
c7da64922f
favourites GET implementation (#95) 2021-07-09 18:32:48 +02:00
Tobi Smethurst
fe269cd641
fix up some of the instance patching stuff (#85) 2021-07-08 15:05:19 +02:00
Tobi Smethurst
5460271bb5
Auth flow fixes (#82)
* preliminary fixes to broken auth flow

* fix some auth/cookie weirdness

* fmt
2021-07-08 11:32:31 +02:00
Tobi Smethurst
c71e55ecc4
clean up some weirdness in the router (#80) 2021-07-07 15:46:42 +02:00
Tobi Smethurst
3568579218
Blocklist import (#77)
* first steps on importing blocklists

* unblock domains properly
2021-07-06 13:29:11 +02:00
Tobi Smethurst
d389e7b150
Domain block (#76)
* start work on admin domain blocking

* move stuff around + further work on domain blocks

* move + restructure processor

* prep work for deleting account

* tidy

* go fmt

* formatting

* domain blocking more work

* check domain blocks way earlier on

* progress on delete account

* delete more stuff when an account is gone

* and more...

* domain blocky block block

* get individual domain block, delete a block
2021-07-05 13:23:03 +02:00
tsmethurst
869a6c111c Go fmt 2021-06-27 13:58:59 +02:00
Tobi Smethurst
b6c62309f2
separate public key handler (#64) 2021-06-26 16:21:40 +02:00
Tobi Smethurst
c1e107266f
nodeinfo compliance (#61) 2021-06-24 14:26:08 +02:00
Tobi Smethurst
8c9a853343
Instance settings updates (#59)
Allow admins to set instance settings through a PATCH to /api/v1/instance

Update templates to reflect some of the new fields
2021-06-23 16:35:57 +02:00
Tobi Smethurst
a5fd6f427b
Deletes+unboosts (#52)
* Status deletes properly streamed now.
* Unboosts now work locally and federated.
* Documentation updates.
2021-06-21 15:56:00 +02:00
Tobi Smethurst
aa8a0d0850
Streaming (#49)
Add new status and notification websocket streaming capabilities
2021-06-19 11:18:55 +02:00
Tobi Smethurst
b4288f3c47
Timeline manager (#40)
* start messing about with timeline manager

* i have no idea what i'm doing

* i continue to not know what i'm doing

* it's coming along

* bit more progress

* update timeline with new posts as they come in

* lint and fmt

* Select accounts where empty string

* restructure a bunch, get unfaves working

* moving stuff around

* federate status deletes properly

* mention regex better but not 100% there

* fix regex

* some more hacking away at the timeline code phew

* fix up some little things

* i can't even

* more timeline stuff

* move to ulid

* fiddley

* some lil fixes for kibou compatibility

* timelines working pretty alright!

* tidy + lint
2021-06-13 18:42:28 +02:00
Tobi Smethurst
6ac6f8d614
Tidy + timeline embetterment (#38)
* tidy up timelines a bit + stub out some endpoints

* who's faved and who's boosted, reblog notifs

* linting

* Update progress with new endpoints
2021-05-31 17:36:35 +02:00
Tobi Smethurst
3d77f81c7f
Move a lot of stuff + tidy stuff (#37)
Lots of renaming and moving stuff, some bug fixes, more lenient parsing of notifications and home timeline.
2021-05-30 13:12:00 +02:00
Tobi Smethurst
1fe5e36ac3
Search (#36)
First implementation of search functionality for remote account and status lookups.
2021-05-29 19:39:43 +02:00
Tobi Smethurst
87177d840b
Announce/boost (#35)
Remote boosts incoming/outgoing now working.
2021-05-28 19:57:04 +02:00
Tobi Smethurst
40add68691
Notifications (#34)
Notifications working for:

* Mentions
* Faves
* New follow requests
* New followers
2021-05-27 16:06:24 +02:00
Tobi Smethurst
ee65d19ff3
status deletes, profile updates (#30)
1. Proper DELETE of federated statuses (not yet deleting all the media and stuff -- i still have to implement this -- but the actual status is toast).
2. Proper UPDATE of profiles. When you change your profile picture on your remote instance, that will now register properly in GoToSocial.
3. Scrolling down the home timeline - it no longer just sort of ends, and will keep loading older statuses as you scroll.
4. Little bugfixes -- still had some nil pointer errors when dereferencing remote accounts.
2021-05-23 18:07:04 +02:00
tsmethurst
aeb665df55 small fiddling to allow whalebird to work (a bit) 2021-05-22 15:51:20 +02:00
Tobi Smethurst
0df2e18cc0
Home timeline (#28)
* v. basic implementation of home timeline

* Go fmt ./...
2021-05-21 23:04:59 +02:00
Tobi Smethurst
d839f27c30
Follows and relationships (#27)
* Follows -- create and undo, both remote and local
* Statuses -- federate new posts, including media, attachments, CWs and image descriptions.
2021-05-21 15:48:26 +02:00
Tobi Smethurst
6cd033449f
Refine statuses (#26)
Remote media is now dereferenced and attached properly to incoming federated statuses.
    Mentions are now dereferenced and attached properly to incoming federated statuses.
    Small fixes to status visibility.
    Allow URL params for filtering statuses:

	// ExcludeRepliesKey is for specifying whether to exclude replies in a list of returned statuses by an account.
      	// PinnedKey is for specifying whether to include pinned statuses in a list of returned statuses by an account.
      	// MaxIDKey is for specifying the maximum ID of the status to retrieve.
      	// MediaOnlyKey is for specifying that only statuses with media should be returned in a list of returned statuses by an account.

    Add endpoint for fetching an account's statuses.
2021-05-17 19:06:58 +02:00
Tobi Smethurst
cc48294c31
Inbox post (#22)
Inbox POST from federated servers now working for statuses and follow requests.
    Follow request client API added.
    Start work on federating outgoing messages.
    Other fixes and changes/tidying up.
2021-05-15 11:58:11 +02:00
Tobi Smethurst
742f985d5b
Mediahandler (#21)
Media GET and media PUT handlers
2021-05-10 16:29:05 +02:00
Tobi Smethurst
dc338dc881
Webfinger + Small fixes (#20) 2021-05-09 20:34:27 +02:00
Tobi Smethurst
3363e0ebdd
add api/v1/instance info handler + instance model (#18) 2021-05-09 14:06:06 +02:00
Tobi Smethurst
3c539cdfd6
status boosts (#16) 2021-05-08 15:16:24 +02:00
Tobi Smethurst
6f5c045284
Ap (#14)
Big restructuring and initial work on activitypub
2021-05-08 14:25:55 +02:00
Tobi Smethurst
aa9ce272dc
Oauth/token (#7)
* add host and protocol options

* some fiddling

* tidying up and comments

* tick off /oauth/token

* tidying a bit

* tidying

* go mod tidy

* allow attaching middleware to server

* add middleware

* more user friendly

* add comments

* comments

* store account + app

* tidying

* lots of restructuring

* lint + tidy
2021-03-22 22:26:54 +01:00
tsmethurst
1b11884121 auth flow working for code 2021-03-18 23:27:43 +01:00
tsmethurst
6eab00e05e getting there....... 2021-03-17 16:01:31 +01:00
tsmethurst
9d5fb0785f fiddling 2021-03-17 13:14:52 +01:00
tsmethurst
eb2ff2ab23 Some more messing around with oauth2 2021-03-17 11:33:06 +01:00
tsmethurst
4e281f31b0 whole buncha stuff 2021-03-15 23:05:24 +01:00
tsmethurst
cca676dcb3 tests 2021-03-15 18:59:38 +01:00
tsmethurst
1eecc2688c bit of experimenting and tidying 2021-03-15 16:15:14 +01:00
tsmethurst
9a79d176c9 moving stuff around, stubbing interfaces 2021-03-09 17:03:40 +01:00
tsmethurst
052783db66 tidying up here and there 2021-03-04 14:38:18 +01:00
tsmethurst
9ffbb353d0 bits n bobs 2021-03-01 15:41:43 +01:00