Commit graph

10 commits

Author SHA1 Message Date
kim
077e66381f
[performance] cache account db lookups by public key URI (#795)
Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-09-02 11:58:42 +02:00
tobi
570fa7c359
[bugfix] Fix potential dereference of accounts on own instance (#757)
* add GetAccountByUsernameDomain

* simplify search

* add escape to not deref accounts on own domain

* check if local + we have account by ap uri
2022-08-20 21:47:19 +01:00
tobi
ac6ed3d939
[chore] Update bun / sqlite versions; update gtsmodels (#754)
* upstep bun and sqlite versions

* allow specific columns to be updated in the db

* only update necessary columns for user

* bit tidier

* only update necessary fields of media_attachment

* only update relevant instance fields

* update tests

* update only specific account columns

* use bool pointers on gtsmodels
includes attachment, status, account, user

* update columns more selectively

* test all default fields on new account insert

* updating remaining bools on gtsmodels

* initialize pointer fields when extracting AP emoji

* copy bools properly

* add copyBoolPtr convenience function + test it

* initialize false bool ptrs a bit more neatly
2022-08-15 11:35:05 +01:00
tobi
f5689a9e5f
[feature] Let accounts set default status format, and use this when processing new statuses (#739)
* add post_format to acct & use it when making post

* update swagger docs

* add status_format updating to frontend

* fix up tests

* post_format => status_format

* add status_format to account validation
2022-08-06 12:09:21 +02:00
tobi
c3b6a5b0f9
[feature] Implement cache-control and etags for static assets (#711)
* start working on etag stuff

* add + use cache middleware

* generate etags on the fly

* remove unused field

* clean up filepath

* add license headers to cache files

* add attachgroup function to router interface

* move cache into web module

* rename a couple things

* remove attachStaticFS function from router

* rename + tidy up a few things

* mount assets filesystem

* create assetsFileInfoCache

* update comment

* simplify hash

* fix string fmt

* skip last mod chk, prefer strong etags w/long cache

* move base handler to its own file
this matches the modules in the api folder

* generate new etag if file was modified

* wrap strong etag in quotation marks as per spec

* clarify logic in avatar search

* make hashing a little niftier
2022-07-18 12:55:06 +02:00
kim
7cc40302a5
[chore] consolidate caching libraries (#704)
* add miekg/dns dependency

* set/validate accountDomain

* move finger to dereferencer

* totally break GetRemoteAccount

* start reworking finger func a bit

* start reworking getRemoteAccount a bit

* move mention parts to namestring

* rework webfingerget

* use util function to extract webfinger parts

* use accountDomain

* rework finger again, final form

* just a real nasty commit, the worst

* remove refresh from account

* use new ASRepToAccount signature

* fix incorrect debug call

* fix for new getRemoteAccount

* rework GetRemoteAccount

* start updating tests to remove repetition

* break a lot of tests
Move shared test logic into the testrig,
rather than having it scattered all over
the place. This allows us to just mock
the transport controller once, and have
all tests use it (unless they need not to
for some other reason).

* fix up tests to use main mock httpclient

* webfinger only if necessary

* cheeky linting with the lads

* update mentionName regex
recognize instance accounts

* don't finger instance accounts

* test webfinger part extraction

* increase default worker count to 4 per cpu

* don't repeat regex parsing

* final search for discovered accountDomain

* be more permissive in namestring lookup

* add more extraction tests

* simplify GetParseMentionFunc

* skip long search if local account

* fix broken test

* consolidate to all use same caching libraries

Signed-off-by: kim <grufwub@gmail.com>

* perform more caching in the database layer

Signed-off-by: kim <grufwub@gmail.com>

* remove ASNote cache

Signed-off-by: kim <grufwub@gmail.com>

* update cache library, improve db tracing hooks

Signed-off-by: kim <grufwub@gmail.com>

* return ErrNoEntries if no account status IDs found, small formatting changes

Signed-off-by: kim <grufwub@gmail.com>

* fix tests, thanks tobi!

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2022-07-10 17:18:21 +02:00
kim
26b74aefaf
[bugfix] Fix existing bio text showing as HTML (#531)
* fix existing bio text showing as HTML

- updated replaced mentions to include instance
- strips HTML from account source note in Verify handler
- update text formatter to use buffers for string writes

Signed-off-by: kim <grufwub@gmail.com>

* go away linter

Signed-off-by: kim <grufwub@gmail.com>

* change buf reset location, change html mention tags

Signed-off-by: kim <grufwub@gmail.com>

* reduce FindLinks code complexity

Signed-off-by: kim <grufwub@gmail.com>

* fix HTML to text conversion

Signed-off-by: kim <grufwub@gmail.com>

* Update internal/regexes/regexes.go

Co-authored-by: Mina Galić <mina.galic@puppet.com>

* use improved html2text lib with more options

Signed-off-by: kim <grufwub@gmail.com>

* fix to produce actual plaintext from html

Signed-off-by: kim <grufwub@gmail.com>

* fix span tags instead written as space

Signed-off-by: kim <grufwub@gmail.com>

* performance improvements to regex replacements, fix link replace logic for un-html-ing in the future

Signed-off-by: kim <grufwub@gmail.com>

* fix tag/mention replacements to use input string, fix link replace to not include scheme

Signed-off-by: kim <grufwub@gmail.com>

* use matched input string for link replace href text

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code (to appease linter :sobs:)

Signed-off-by: kim <grufwub@gmail.com>

* improve hashtagFinger regex to be more compliant

Signed-off-by: kim <grufwub@gmail.com>

* update breakReplacer to include both unix and windows line endings

Signed-off-by: kim <grufwub@gmail.com>

* add NoteRaw field to Account to store plaintext account bio, add migration for this, set for sensitive accounts

Signed-off-by: kim <grufwub@gmail.com>

* drop unnecessary code

Signed-off-by: kim <grufwub@gmail.com>

* update text package tests to fix logic changes

Signed-off-by: kim <grufwub@gmail.com>

* add raw note content testing to account update and account verify

Signed-off-by: kim <grufwub@gmail.com>

* remove unused modules

Signed-off-by: kim <grufwub@gmail.com>

* fix emoji regex

Signed-off-by: kim <grufwub@gmail.com>

* fix replacement of hashtags

Signed-off-by: kim <grufwub@gmail.com>

* update code comment

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: Mina Galić <mina.galic@puppet.com>
2022-05-07 17:55:27 +02:00
tobi
03d7c75ebf
[chore] Update Go version to 1.18 (#444)
* linting with new golangci-lint version

* update go to 1.18

* bump versions in drone.yml

* use new runtime/debug package for version info

* remove Commit build flag from goreleaser

* remove mock commit + version from build script

* go fmt

* add dummy version env flag to test container

* install git in golang container for testing

* only set versionString if Version is defined
2022-04-02 15:40:09 +02:00
tobi
f8630348b4
Enable stricter linting with golangci-lint (#316)
* update golangci-lint

* add golangci config file w/ more linters

* correct issues flagged by stricter linters

* add more generous timeout for golangci-lint

* add some style + formatting guidelines

* move timeout to config file

* go fmt
2021-11-22 08:46:19 +01:00
kim
7d193de25f
Improve GetRemoteStatus and db.GetStatus() logic (#174)
* only fetch status parents / children if explicity requested when dereferencing

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Remove recursive DB GetStatus logic, don't fetch parent unless requested

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* StatusCache copies status so there are no thread-safety issues with modified status objects

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* remove sqlite test files

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix bugs introduced by previous commit

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix not continue on error in loop

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* use our own RunInTx implementation (possible fix for nested tx error)

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* fix cast statement to work with SQLite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* be less strict about valid status in cache

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add cache=shared ALWAYS for SQLite db instances

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix EnrichRemoteAccount when updating account fails

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add nolint tag

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* ensure file: prefixes the filename in sqlite addr

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add an account cache, add status author account from db

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* Fix incompatible SQLite query

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* *actually* use the new getAccount() function in accountsDB

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* update cache tests to use test suite

Signed-off-by: kim (grufwub) <grufwub@gmail.com>

* add RelationshipTestSuite, add tests for methods with changed SQL

Signed-off-by: kim (grufwub) <grufwub@gmail.com>
2021-09-01 11:08:21 +02:00