Commit graph

15 commits

Author SHA1 Message Date
Dominik Süß 9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi cf5c6d724d
[chore] Validate/set account domain (#619)
* add miekg/dns dependency

* set/validate accountDomain
2022-06-11 11:09:31 +02:00
tobi 1cdc163276
[performance] Don't retry/backoff invalid http requests that will never succeed (#609)
* add httpguts (ew)

* add ValidateRequest err wrapping logic

* don't retry on unrecoverable errors

* i am very clever
2022-05-26 13:38:41 +02:00
kim b56dae8120
[chore] Update all but bun libraries (#526)
* update all but bun libraries

Signed-off-by: kim <grufwub@gmail.com>

* remove my personal build script changes

Signed-off-by: kim <grufwub@gmail.com>
2022-05-02 15:05:18 +02:00
tobi 88979b35d4
[chore] Update bun and sqlite dependencies (#478)
* update bun + sqlite versions

* step bun to v1.1.3
2022-04-24 12:26:22 +02:00
tobi 07727753b9
[feature] Clean up/uncache remote media (#407)
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00
tobi 67ac8db190
Upstep Go dependencies (#340)
* Upstep Go dependencies

* tiny linter fix

* Tidy
2021-12-12 15:47:51 +01:00
tobi 182b4eea73
Update dependencies (#333) 2021-11-27 15:26:58 +01:00
tobi 8b7c3507fe
upstep bun to v1.0.14 (#291) 2021-10-24 13:14:37 +02:00
tobi 9a53b1a8d1
upstep bun to v1.0.9 (#252) 2021-09-29 15:09:45 +02:00
tobi f2e5bedea6
migrate go version to 1.17 (#203)
* migrate go version to 1.17

* update contributing
2021-09-10 14:42:14 +02:00
kim (grufwub) bdcc090851 update bun library -> v1.0.4
Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-08 20:05:26 +01:00
kim ed46224573
Add SQLite support, fix un-thread-safe DB caches, small performance f… (#172)
* Add SQLite support, fix un-thread-safe DB caches, small performance fixes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add SQLite licenses to README

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* appease the linter, and fix my dumbass-ery

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* make requested changes

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add back comment

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-08-29 16:41:41 +02:00
tobi 2dc9fc1626
Pg to bun (#148)
* start moving to bun

* changing more stuff

* more

* and yet more

* tests passing

* seems stable now

* more big changes

* small fix

* little fixes
2021-08-25 15:34:33 +02:00
Tobi Smethurst 98263a7de6
Grand test fixup (#138)
* start fixing up tests

* fix up tests + automate with drone

* fiddle with linting

* messing about with drone.yml

* some more fiddling

* hmmm

* add cache

* add vendor directory

* verbose

* ci updates

* update some little things

* update sig
2021-08-12 21:03:24 +02:00