Commit graph

548 commits

Author SHA1 Message Date
Dessalines 55f84dd38a
Fixing proxy images (#4722)
* Adding an image_details table to store image dimensions.

- Adds an image_details table, which stores the height,
  width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
  the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes #3328
- Also fixes #4703

* Running sql format.

* Running fmt.

* Don't fetch metadata in background for local API requests.

* Dont export remote_image table to typescript.

* Cleaning up validate.

* Dont proxy url.

* Fixing tests, fixing issue with federated thumbnails.

* Fix tests.

* Updating corepack, fixing issue.

* Refactoring image inserts to use transactions.

* Use select exists again.

* Fixing imports.

* Fix test.

* Removing pointless backgrounded metadata generation version.

* Removing public pictrs details route.

* Fixing clippy.

* Fixing proxy image fetching. Fixes #4703

- This extracts only the proxy image fixes from #4704, leaving off
  thumbnails.

* Fix test.

* Addressing PR comments.

* Address PR comments 2.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-05-22 10:28:47 +02:00
Nutomic 99aac07714
Mark database fields as sensitive so they dont show up in logs (#4720)
* Mark database fields as sensitive so they dont show up in logs

* add file

* fix test

* Update crates/apub/src/objects/person.rs

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* Update crates/apub/src/objects/community.rs

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* Update crates/apub/src/objects/instance.rs

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-05-16 20:41:57 -04:00
Nutomic 93c9a5f2b1
Dont federate post locking via Update activity (#4717)
* Dont federate post locking via Update activity

* cleanup

* add missing mod log entries

* update assets
2024-05-15 07:36:00 -04:00
Nutomic 9a9d518153
Fix import blocked objects (#4712)
* Allow importing partial backup (fixes #4672)

* Fetch blocked objects if not known locally (fixes #4669)

* extract helper fn

* add comment

* cleanup

* remove test

* fmt

* remove .ok()
2024-05-14 23:03:43 -04:00
Nutomic 723cb549d4
Allow importing partial backup (fixes #4672) (#4705)
* Allow importing partial backup (fixes #4672)

* Dont throw error on empty LocalUser::update

* fix tests
2024-05-14 22:37:30 -04:00
Nutomic 866d752a3c
Instance.preferred_username should be optional (fixes #4701) (#4713) 2024-05-08 08:01:04 -04:00
Nutomic e0b1d0553d
Add timeout for processing incoming activities (#4708)
* Add timeout for processing incoming activities

* move to const
2024-05-08 08:00:55 -04:00
Nutomic 7c146272c3
Federate with wordpress, improvements for NodeBB, Discourse federation (#4692)
* Federate with wordpress

* upgrade apub lib with fix

* Also read post's community from `audience`

* cleanup

* cargo update

* upgrade apub lib

* add wordpress test activity
2024-05-07 16:20:43 -04:00
Nutomic 7746db4169
Testing and minor fix for federation with Discourse (#4628)
* Testing and minor fix for federation with Discourse

* prettier
2024-05-02 07:49:19 -04:00
Nutomic 12163701e7
Avoid crash when handling urls without domain (#4676)
* Avoid crash when handling urls without domain

* Add some extra checks
2024-04-30 06:33:37 -04:00
Nutomic beec080274
Testing for federation with NodeBB, make community.followers_url optional (#4629)
* Testing for federation with NodeBB, make community.followers_url optional

* clippy
2024-04-29 12:34:11 +02:00
Dessalines 492d8f1b01
Fix communities with broken outboxes, and use PostView. Fixes #4658 (#4668)
* Fix communities with broken outboxes, and use PostView. Fixes #4658

* Fixing tests.

* Dont pass ref and clone.
2024-04-29 12:22:00 +02:00
Nutomic 31829b6c05
Untangle thumbnail generation logic (ref #4604) (#4615)
* Untangle thumbnail generation logic (ref #4604)

* prettier

* test cleanup

* fix tests

* also consider opengraph image for local thumbnail generation
2024-04-17 10:36:45 -04:00
TechVest b0370ae2fd
chore: fix some comments (#4637)
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines d075acce43
Make all single-fetch database calls return an Option. (#4617)
- Diesel ordinarily throws an error when no results are returned for a
  single fetch, which is a bit confusing. This PR ensures that the
  missing value cases are all caught, and wrapped with new LemmyErrors,
  rather than diesel errors.
- Fixes #4601
2024-04-16 14:48:15 +02:00
Nutomic 3a0c1dca90
Avoid overwriting local objects via federation (#4611)
* Dont allow federation to overwrite local objects

* is_local check in apub lib

* use imports

* fix check, update lib

* use verify_is_remote_object()

* submodule
2024-04-11 10:05:49 -04:00
Dessalines 5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 (#4614)
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613

* Fixing clippy.
2024-04-10 10:14:11 -04:00
Nutomic 0203b62a6d
Ignore old federated post edits (ref #4529) (#4586)
* Ignore old federated post edits (ref #4529)

* use filter on insert

* coalesce(updated, published)

* avoid comment conflict clause

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-04-10 10:03:51 -04:00
Nutomic 1d0a6ac08f
Avoid breaking api change, reduce api cache duration (#4610)
* Dont mark site.public_key as `serde(skip)` to avoid breaking change (fixes #4605)

* Reduce cache duration for api
2024-04-09 10:10:20 -04:00
Dessalines 8e54a4a6cc
Fixing bug where comment replies wouldn't be sent to blocked instances. (#4595)
* Fixing bug where comment replies wouldn't be sent to blocked instances.

- Instance blocks should only affect communities, not comments.
- Fixes #4590

* Revert "Fixing bug where comment replies wouldn't be sent to blocked instances."

This reverts commit 1349aa351a.

* Only block replies from the community's instance id.

- Also refactor send_local_notifs slightly, since it has to fetch the
  community now.
- Fixes #4590

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-04-08 16:26:24 +02:00
Nutomic 087684658a
Cache result of LocalSite::read to avoid unnecessary db calls (#4585)
* Cache result of LocalSite::read to avoid unnecessary db calls

* single const for cache duration

* clippy

* revert apub send changes

* clippy

* fmt
2024-04-03 17:38:31 -04:00
Nutomic a4b79ca610
Generate post thumbnail/metadata in background (ref #4529) (#4564)
* Generate post thumbnail/metadata in background (ref #4529)

* fix api test

* Apply suggestions from code review

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* fix test

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-27 10:54:42 -04:00
Dessalines 95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
Nutomic 846848c4f6
On registration, automatically set content languages from accept-language header (#4550)
* On registration, automatically set content languages from accept header

* no need to set site language or default language for new user anymore

* fix test

* fix langs

* avoid duplicate writing of new user languages
2024-03-25 16:02:12 -04:00
Nutomic d06ef2c47e
Migrate apub block activity to standard endTime property and deprecate expires (fixes #2316) (#4541)
* Migrate apub block activity to standard `endTime` property (fixes #2316)

* add todo
2024-03-25 08:10:09 -04:00
Nutomic ef4bb3cc40
Add delete user field removeData to apub assets (fixes #4544) (#4549) 2024-03-22 18:41:59 -04:00
Nutomic 21547dedf7
Fix handling of apub downvote (fixes #4545) (#4551)
* Fix handling of apub downvote (fixes #4545)

* fmt

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-22 18:41:09 -04:00
Nutomic 9d4299aaac
Dont require leading ! or @ for webfinger resolve (#4513)
* Dont require leading ! or @ for webfinger resolve

* fmt

* clippy
2024-03-15 08:42:09 -04:00
flamingos-cant 19a1a077c5
Add a blocklist for URLs. (#4515)
* Add a blocklist for URLs.

* Fix SQL format

* Make clippy happy.

* Use regex for URL matching.

* Escape regex chars in URLs.

* Use post for modification.

* Make URL block regex static and remove API routes.

* Add date fields to table and use transaction.

* Use Cache for blocklist.

* Rename check_links + move list to parameters of process_markdown.

* SQL format.

* Format, again.

* Remove println.

* Add API test.

* Set a shorter lifetime for regex in debug mode.

* Add missing macro.

* Update lemmy-js-client

* Update api_test/pnpm-lock.yaml

* Don't break other tests

* Use different URL for test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2024-03-15 07:03:29 -04:00
Nutomic 0e7080337b
Dont allow admins to post in community with posting_restricted_to_mods (fixes #3571) (#4534)
* Dont allow admins to post in community with `posting_restricted_to_mods` (fixes #3571)

* fmt
2024-03-14 17:31:54 -04:00
Nutomic 835d329134
Fix longstanding bug that breaks initial community view (fixes #3529) (#4535) 2024-03-14 16:57:56 -04:00
Nutomic f1de7b7590
Automatically include apub hashtag with posts (fixes #3906) (#4533) 2024-03-14 12:16:45 -04:00
Dessalines 15f02f00a9
Add a vote_display_mode local_user setting. (#4450)
* Add a vote_display_mode local_user setting.

- Fixes #4449

* Changing HideDownvotes to Score.

* Adding ScoreAndDownvote display mode.

* Adding upvote and downvote mode.

* Extracting vote_display_mode to another table.

* Fixing fmt.

* Remove published and updated columns.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-13 12:10:58 -04:00
Nutomic 5859502a2a
Fix missing private key for signed fetch (#4516)
* Fix missing private key for signed fetch (fixes #4451)

* clippy

* instance actor name and webfinger

* better webfinger handling

* upgrade lib

* update test asset
2024-03-08 10:23:15 -05:00
battmdpkq f228f9d7a9
fix some typos (#4519)
Signed-off-by: battmdpkq <cmaker@163.com>
2024-03-08 10:38:20 +01:00
Nutomic fed6b61eaf
Upgrade apub lib, correct webfinger content-type (#4498)
* Upgrade apub lib, correct webfinger content-type

* fmt

* fix test by avoiding network fetch
2024-03-06 11:21:46 -05:00
Dessalines 7f9950fe85
Add alt_text for posts. Fixes #1086 (#4477)
* Add alt_text for posts. Fixes #1086

* Moving alt_text to attachment name.

* Cleaning up mod action line.

* Addressing PR comments

* Addressing PR comments.

* Fixing clones.
2024-03-05 11:34:57 +01:00
Nutomic 3c358e5b0b
Mods and admins can comment in locked posts (fixes #4116) (#4488)
* Mods and admins can comment in locked posts (fixes #4116)

* fmt

* fix

* fix test
2024-03-04 09:15:21 -05:00
Nutomic 08b01a377d
Support listing type for person (fixes #4146) (#4487)
* Support listing type for person (fixes #4146)

* add test
2024-03-01 11:53:20 -05:00
Dessalines 87b577467b
Adding ability to hide posts. (#4480)
* Adding ability to hide posts.

- Adds an post/hide API route.
- Adds a `show_hidden` (default false) to `GetPosts`.
- Adds a `hidden` field to `PostView`.
- Removes the single `post_id` from MarkPostAsRead.
- Fixes #1403

* Add a check to make sure hidden field is true.

* Fixing test.

* Add back semicolon
2024-02-29 10:42:34 -05:00
Nutomic 328a48c9f5
Remove error-type feature from lemmy-utils (#4474)
* Remove error-type feature from lemmy-utils

* fixes

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-02-27 11:13:52 -05:00
Dessalines f3d48f2c2c
Adding some recommended fixes from nightly clippy. (#4473) 2024-02-26 09:47:10 -05:00
Nutomic 7316dd281a
Correctly synchronize collection of community featured posts (fixes #3867) (#4475) 2024-02-26 09:45:23 -05:00
SleeplessOne1917 f42420809b
Expose LemmyErrorType in lemmy_api_common (#4439)
* Expose LemmyErrorType in lemmy_api_common

* Make conditional compilation gates for utils

* Make it so api_common doesn't pull in unnecessary deps

* Make error type non exhaustive

* Fix formatting

* Format toml

* Add some convenience derives to LemmyError

* Simplify features

* Fix CI compile error

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
2024-02-24 19:54:27 -05:00
Dessalines 5d551e6da5
Adding an instance-level default_sort_type (#4454)
* Adding an instance-level default_sort_type

- Fixes #3796

* Fixing comment.

* Put user sort before site sort.
2024-02-16 09:36:46 -05:00
Nutomic 86b44c2a4d
Add site.content_warning, local_site.default_post_listing_mode (#4393)
* Include local_site.content_warning setting for showing nsfw by default

* Add community setting `only_followers_can_vote`

* clippy

* add auto_expand_images site setting

* cleanup

* add missing api params

* postquery/communityquery changes

* clippy

* change error

* replace auto_expand_images with default_site_post_listing_mode

* change post/community query params

* get rid of only_followers_can_vote

* machete

* fix

* clippy

* revert remaining vote changes

* remove dead code

* remove unused var

* fmt
2024-02-16 07:24:35 -05:00
Dessalines a3bf2f1cf1
Auto resolve reports on removing a comment or post. Fixes #4390 (#4402)
* Automatically resolve report when post/comment is removed (#3850)

* Automatically resolve report when post/comment is removed

* also handle apub removes

* Removing auto-resolve report triggers.

* Dont allow creating reports for deleted / removed items.

* Running pgformat.

* Fixing test.

* Addressing PR comments.

* Forgot comment report.

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-02-15 08:52:04 -05:00
Nutomic 0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
Nutomic dadf8f28f9
Send purges to federated instances (fixes #4119) (#4398)
* Send purges to federated instances (fixes #4119)

* clippy

* review

* remove unused function

* clippy
2024-01-25 09:24:09 -05:00
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00