Commit graph

319 commits

Author SHA1 Message Date
tobi
898d256511
[bugfix] Fix server trying to listen twice on same address when l.e. enabled (#557) 2022-05-12 11:35:36 +02:00
tobi
8e30671a62
[bugfix] Add account raw note fix (#556)
* hack the latest migration to fix psql issue

* add new migration to fix previous migration

* adjust query
2022-05-11 19:27:25 +02:00
Eamonn O'Brien-Strain
b24b71c0a4
[feature] Include password strength in error message when password strength is too low (#550)
* When password validation fails, return how close to enough entropy it has.

* Shorter version of low-strength password error message
2022-05-09 10:31:46 +02:00
tobi
5004e0a9da
[bugfix] Fix remote media pruning failing if media already gone (#548)
* fix error check of prune to allow missing files

* update go-store library, add test for pruning item with db entry but no file

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unneccessary error check

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: kim <grufwub@gmail.com>
2022-05-08 18:49:45 +01:00
kim
26b74aefaf
[bugfix] Fix existing bio text showing as HTML (#531)
* fix existing bio text showing as HTML

- updated replaced mentions to include instance
- strips HTML from account source note in Verify handler
- update text formatter to use buffers for string writes

Signed-off-by: kim <grufwub@gmail.com>

* go away linter

Signed-off-by: kim <grufwub@gmail.com>

* change buf reset location, change html mention tags

Signed-off-by: kim <grufwub@gmail.com>

* reduce FindLinks code complexity

Signed-off-by: kim <grufwub@gmail.com>

* fix HTML to text conversion

Signed-off-by: kim <grufwub@gmail.com>

* Update internal/regexes/regexes.go

Co-authored-by: Mina Galić <mina.galic@puppet.com>

* use improved html2text lib with more options

Signed-off-by: kim <grufwub@gmail.com>

* fix to produce actual plaintext from html

Signed-off-by: kim <grufwub@gmail.com>

* fix span tags instead written as space

Signed-off-by: kim <grufwub@gmail.com>

* performance improvements to regex replacements, fix link replace logic for un-html-ing in the future

Signed-off-by: kim <grufwub@gmail.com>

* fix tag/mention replacements to use input string, fix link replace to not include scheme

Signed-off-by: kim <grufwub@gmail.com>

* use matched input string for link replace href text

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code (to appease linter :sobs:)

Signed-off-by: kim <grufwub@gmail.com>

* improve hashtagFinger regex to be more compliant

Signed-off-by: kim <grufwub@gmail.com>

* update breakReplacer to include both unix and windows line endings

Signed-off-by: kim <grufwub@gmail.com>

* add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts

Signed-off-by: kim <grufwub@gmail.com>

* drop unnecessary code

Signed-off-by: kim <grufwub@gmail.com>

* update text package tests to fix logic changes

Signed-off-by: kim <grufwub@gmail.com>

* add raw note content testing to account update and account verify

Signed-off-by: kim <grufwub@gmail.com>

* remove unused modules

Signed-off-by: kim <grufwub@gmail.com>

* fix emoji regex

Signed-off-by: kim <grufwub@gmail.com>

* fix replacement of hashtags

Signed-off-by: kim <grufwub@gmail.com>

* update code comment

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: Mina Galić <mina.galic@puppet.com>
2022-05-07 17:55:27 +02:00
kim
08eb271a4c
[performance] improved logrus output switching performance (#544)
* improved logrus output switching performance

Signed-off-by: kim <grufwub@gmail.com>

* add license to test file

Signed-off-by: kim <grufwub@gmail.com>
2022-05-07 17:46:59 +02:00
kim
a561ef3541
[feature] Update media manager to use internal/worker package (#543)
* update media manager to use internal/worker package, update worker with better logging

Signed-off-by: kim <grufwub@gmail.com>

* fix Queue() trace log message format operators

Signed-off-by: kim <grufwub@gmail.com>

* update media manager comment to match updated worker implementation

Signed-off-by: kim <grufwub@gmail.com>
2022-05-07 17:36:01 +02:00
tobi
d7b46a4b63
[chore] Move initial MediaRemotePrune to cmd.server function (#536)
* move initial prune up to the server start function

* don't do initial prune when starting media manager

* add admin/mediaremoteprune in processor
2022-05-03 10:56:18 +01:00
kim
e09958e7f1
[feature] Run initial cache cleanup on gotosocial start in case max-age changed (#533)
Signed-off-by: kim <grufwub@gmail.com>
2022-05-03 11:05:22 +02:00
tobi
b28fd7433a
[chore] Fix inbox iris test to use 'contains' instead of index (#529) 2022-05-02 16:12:56 +02:00
tobi
3112949b6e
[bugfix] Fix panic when relative tmpl path given to router (#528) 2022-05-02 16:06:03 +02:00
Adelie Paull
9265a09a65
[bugfix] Allow self-boosting for any visibility but direct (#510)
* create visibility filter for boostability and allow self-boosting for any visbility but direct messages

* add a followers-only status to local_account_2

* fix typo in comment

* add license header, unwrap errors, be explicit about non-boostable visibility settings to avoid rogue boosting from miscoded clients, use ID compare for checking if self-boosting

* add tests for statusboostable filter

* fix tests that were affected by adding a new status to the test data

* fix the rest of tests affected by adding a status to the textrig data
2022-05-02 15:23:37 +02:00
kim
b56dae8120
[chore] Update all but bun libraries (#526)
* update all but bun libraries

Signed-off-by: kim <grufwub@gmail.com>

* remove my personal build script changes

Signed-off-by: kim <grufwub@gmail.com>
2022-05-02 15:05:18 +02:00
tobi
e06bf9cc9a
add extra indexes as a migration (#527) 2022-05-02 13:55:48 +01:00
tobi
a5852fd7e4
[performance] Speed up some of the slower db queries (#523)
* remove unnecessary LOWER() db calls

* warn during slow db queries

* use bundb built-in exists function

* add db block test

* update account block query

* add domain block db test

* optimize domain block query

* fix implementing wrong test

* exclude most columns when checking block

* go fmt

* remote more unnecessary use of LOWER()
2022-05-02 11:53:46 +01:00
kim
faae2505c0
Add logging to the new generic worker package (#516)
* add logging to generic worker type
2022-05-01 15:45:15 +01:00
tobi
d93e8ddf75
[chore] Add Federatingactor.Send() tests and log call (#509)
* expose testrig util functions

* add tests for federatingActor

* rename some suite vars
2022-04-29 15:53:04 +02:00
tobi
63f84da3e4
[chore] Inboxes for iri test (#508)
* tidy up some federation tests

* add missing license to following.go

* give zork some followers, as a treat

* test InboxesForIRI

* Go fmt

* update timeline tests
2022-04-29 15:05:13 +02:00
Terin Stock
f9b351438b
[feature] allow absolute path to templates (#507)
Only attempt to join the `WebTemplateBaseDir` with the current working
directory if the user has not configured an absolute path to the
template directory.

This changeset also makes a similar change to the testrig, allowing
tests to configure an alternative location for the templates directory.

Fixes #411

Signed-off-by: Terin Stock <terinjokes@gmail.com>
2022-04-29 11:00:25 +02:00
kim
69011d4901
Add support for running profiling when debug build-tags provided (#491)
* wrap root HTTP handler in debug.WithPprof(), rearrange router.Start() to support this
* remove unused code
* set debug buildtag in build script when $DEBUG set
* update go-debug version with fixed handler
* use clone of router.srv for LE cert manager, reset server timeouts in debug
* add kim's other libraries to README
2022-04-28 13:32:53 +01:00
kim
420e2fb22b
replace async client API / federator msg processing with worker pools (#497)
* replace async client API / federator msg processing with worker pools
* appease our lord-and-saviour, the linter
2022-04-28 13:23:11 +01:00
kim
cc5f2e98b7
[bugfix] Fix possible race condition in federatingdb (#490)
Signed-off-by: kim <grufwub@gmail.com>
2022-04-28 11:18:27 +02:00
Terin Stock
8e80f983b3
[bugfix] use Exec to rename media_attachments (#498)
The `database/sql` package in the Go stdlib has two methods that can run
queries against the database driver: `Query` and `Exec`. When the query
returns rows, such as "SELECT", the package expects the use of `Query`,
and returns `*sql.Rows`. When the query does not return rows, the
package expects the use of `Exec`, which returns `sql.Result`.

This changeset corrects the "media_attachments" migration to using
`ExecContext`, as "ALTER TABLE" does not return rows.

Signed-off-by: Terin Stock <terinjokes@gmail.com>
2022-04-28 08:43:41 +01:00
tobi
9cf66bf298
[chore] Return more useful errors from auth failure (#494)
* try rsa_sha256 sig algo first

* return more informative errors from auth

* adapt to reworked auth function
2022-04-26 18:10:11 +02:00
tobi
728c4a5e38
[bugfix] Trim log entries to 1700 chars before they enter syslog (#493)
* start implementing trimming hook

* add test with very long test

* test syslog w/ unix socket + long (trimmed) msg

* trim long entries with trimhook

* trim to 1700 chars instead
2022-04-26 17:55:24 +02:00
tobi
2259838108
[bugfix] Fix CWs not showing sometimes (#488)
* allow summaries that are parsed as iris

* test parsing a status with iri summary
2022-04-26 10:47:21 +02:00
tobi
eeb78bd141
[bugfix] Allow processing of .png files where checksum is not correct (#487)
* add png stripping code from google/wuffs

* experiment with stripping data from pngs

* add test images

* use StrippedPngDecode for pngs

* add StrippedPngDecode func

* update tests for (no)alphachannel pngs

* nolint on copied function
2022-04-25 14:45:44 +02:00
0hlov3
d5fe729cf4
[chore] Add two new test images for media processing (#424) 2022-04-24 14:49:37 +02:00
tobi
0d05bf473b
[bug] Fix status API / status web API being case sensitive (#481)
* make getStatus by id case-insensitive

* test get status case insensitive

* init config before log
2022-04-24 14:24:43 +02:00
tobi
9813a044c0
[feature] Implement media v2 endpoint to accommodate Tusky 17 (#480)
* serve v2 media api

* go fmt
2022-04-24 13:11:52 +02:00
tobi
8d34d5af3c
Revert "[chore] Tidy up federating db locks a tiny bit (#472)" (#479)
This reverts commit 31c1067a54.
2022-04-24 11:56:06 +02:00
tobi
31c1067a54
[chore] Tidy up federating db locks a tiny bit (#472) 2022-04-19 16:31:37 +02:00
tobi
1e3b38573d
[bugfix] Fix infinite domain block database loop (#467)
This fixes an issue where the domain block logic would go into an infinite loop.
2022-04-18 17:44:36 +02:00
tobi
c365863ea9
[bugfix] Use our own (Batch)Deliver implementation for federated messages (#466) 2022-04-18 17:17:05 +02:00
tobi
094f032f74
[feature] Add log-db-queries config option (#465) 2022-04-18 16:47:11 +02:00
tobi
7883dd5499
[bugfix] Convert IDNs to punycode before using as session name (#458)
* convert hostname to punycode for session name

* test punycode
2022-04-16 13:09:42 +02:00
tobi
af97d6bb7e
[bugfix] Use background ctx for domain block side effects (#457) 2022-04-16 12:56:41 +02:00
tobi
26683b3d49
[feature] Web profile pages for accounts (#449)
* add default avatars

* allow webModule to error

* return errWithCode from account get

* add AccountGetLocalByUsername

* check nil requesting account

* add timestampShort function for just month/year

* move loading logic to New + add default avatars

* add profile page view

* update swagger docs

* add excludeReblogs to GetAccountStatuses

* ignore casing when selecting local account by username

* appropriate redirects

* css fiddling

* add 'about' heading

* adjust thread page to work with routing

* return AP representation if requested + authorized

* simplify auth check

* go fmt

* golangci-lint ignore math/rand
2022-04-15 14:33:01 +02:00
tobi
a7e9dee33d
[bugfix] Fix broken only_media and only_public flags on /api/v1/accounts/:id/statuses (#448)
* bump superseriousbusiness/activity version

* fetch list of follower inbox iris

* add test for getting account statuses

* fix incorrect publicOnly parsing

* update media not null query
2022-04-10 15:11:12 +02:00
tobi
707de094a0
[dependency] Update superseriousbusiness/activity dependency (#446)
* bump superseriousbusiness/activity version

* fetch list of follower inbox iris
2022-04-05 18:05:24 +02:00
tobi
03d7c75ebf
[chore] Update Go version to 1.18 (#444)
* linting with new golangci-lint version

* update go to 1.18

* bump versions in drone.yml

* use new runtime/debug package for version info

* remove Commit build flag from goreleaser

* remove mock commit + version from build script

* go fmt

* add dummy version env flag to test container

* install git in golang container for testing

* only set versionString if Version is defined
2022-04-02 15:40:09 +02:00
tobi
37d310f981
[feature] Dereference remote mentions when the account is not already known (#442)
* remove mention util function from db

* add ParseMentionFunc to gtsmodel

* add parseMentionFunc to processor

* refactor search to simplify it a bit

* add parseMentionFunc to account

* add parseMentionFunc to status

* some renaming for clarity

* test dereference of unknown mentioned account
2022-03-29 11:54:56 +02:00
tobi
8cc70a14d6
[bugfix] Fix media uploads blocking after a failed upload #438
safer in case of failure
2022-03-22 12:42:34 +01:00
tobi
25cab0e1f4
[bugfix] Fix images not being processed correctly sometimes (#437)
* bump exif-terminator to latest version

* add and test giant turnip from turnip.farm

* don't error if content property is nil
2022-03-21 19:46:51 +01:00
tobi
36b2f2b4e6
don't error if content property is nil (#436) 2022-03-21 17:10:31 +01:00
tobi
73e9cca701
[bugfix] Close ReadClosers properly in the media package (#434)
* defer lock reader

* close readers when finished with them

* close the reader in the teereader when finished
2022-03-21 13:41:44 +01:00
tobi
55ad6dee71
[feature] Admin account actions (#432)
* add accountAction to the admin API

* model admin account action

* add admin account action to the processor

* add migration for new AdminAccountActions table

* fix accounts admin path

* Update swagger docs
2022-03-19 12:01:40 +01:00
tobi
532c4cc697
[feature] Federate local account deletion (#431)
* add account delete to API

* model account delete request

* add AccountDeleteLocal

* federate local account deletes

* add DeleteLocal

* update transport (controller) to allow shortcuts

* delete logic + testing

* update swagger docs

* more tests + fixes
2022-03-15 16:12:35 +01:00
tobi
e63b653199
[performance] Add dereference shortcuts to avoid making http calls to self (#430)
* update transport (controller) to allow shortcuts

* go fmt

* expose underlying sig transport to allow test sigs
2022-03-15 15:01:19 +01:00
tobi
4b4c935e02
[bugfix] Fix bug where admin panel could not be accessed at /admin (#427)
* clarify comments

* tidy up static serving + add /admin redirect
2022-03-13 18:35:26 +01:00