Commit graph

5428 commits

Author SHA1 Message Date
Dessalines
a5b0475436
Fast history no background (#5873)
* Finishing up post and comment actions

* Adding community_actions.

* instance and person actions

* Fixing person_actions.

* Fixing down migrations

* Adding person_content_combined.

* Search combined.

* Aggregates tables.

* Dont rename old tables

* Fixing some constraints.

* Convert bigints to ints

* Forgot a bigint.

* Rest of i64 -> i32

* Adding actions id columns.

* Fixing connection config.

* Formatting smoosh up.sql

* Use current_date instead of now

* Adding history tables for liked_combined

* Upping wal size

* Fix clippy

* Fixing clippy.

* Fixing i64

* Getting rid of let mut conn

* Adding the history status table.

* Adding published indexes to speed up history.

* Adding comment like history scanning.

* Fixing schema options setup.

* post_read history

* person_post_aggregates / read_comments history

* post_like history

* Fixing conflicts.

* Fixing clippy.

* Use constant batch_size.

* person content combined post and comment history.

* Fixing id scanning.

* post/comment_actions -> saved_combined history

* search history

* Post and comment aggregates -> post / comment history.

* Uncommenting full history building.

* Changing DB_BATCH_SIZE to i64

* Fixing clippy.

* Fix index names.

* Fixing diff check by removing indexes.

* Adding the uplete ignore actions::id columns.

* Fixing merge imports.

* Fixing submodule update

* Try trigger disabling.

* Fix clippy

* Remove history table, do faster bulk inserts. Smoosh first.

Comments about how this fast insert is done is within smoosh comments.

* Adding some timings.

* Fast person content combined history

* Adding search_combined

* Fix person saved combined unique names

* person_liked_combined

* Remove-aggregates

* Fixing up.sql issues

* Re-building schema.rs

* Fixing down migrations.

* Removing history updating.

* Format sql.

* Move postgres logging to customPostgresql.conf

* Try using postgres 16-alpine in CI

* Speeding up add_report_count.

* speed up inbox_combined

* Speeding up remove_post_sort_type_enums

* Fixing post_sort_type

* Speeding up person votes

* Fixing wrong conn.

* Fixing broken migrations

* Remove comment.

* Make sure to re-index table after re-enabling indexes.

* Removing id columns from actions tables.

* Fixing down migrations.

* Using create table as for smoosh migration

* create_table as for person_content_combined.

* Fixing person_content_combined uniques

* create table as for search_combined

* create table as for liked_combined

* create table as for inbox_combined.

* Fixing a few score types.

* Fixing id positions.
2025-07-23 17:43:23 -04:00
dullbananas
b3e5e6c76a
Prevent stack overflow when fetching nested comment (#5787)
* attempt future wrapper

* Revert "attempt future wrapper"

This reverts commit ce95422228.

* use spawn

* remove `lazy` and change comment

* temporary change for test

* change 5000 back to 50

* fix comment about async laziness
2025-07-22 15:37:50 -04:00
Nutomic
6e292141cc
Allow remote groups to follow Lemmy communities (fixes #5354) (#5864)
* Allow remote groups to follow Lemmy communities (fixes #5354)

* index

* rename columns

* review changes

* primary key
2025-07-22 15:37:15 -04:00
Nutomic
40c89f99ca
Fix foreign key indexes migration (#5875) 2025-07-21 11:02:51 -04:00
dullbananas
de7d6ef148
Require indexes for all foreign keys (#5872)
* add test

* add migration with missing indexes
2025-07-21 09:56:58 +02:00
Nutomic
f65875c27a
Remove unused items (#5870)
* Remove some unused functions

* shear

* remove more

* full feature

* remove more

* some more

* mroe
2025-07-18 10:21:43 -04:00
Nutomic
72d254b4db
Notifications rewrite and post following (fixes #3069) (#5604)
* Add post_actions.disable_notifications (fixes #3042)

* Split up logic for send_local_notifs()

* refactor

* fmt

* add api endpoint, check

* dont call send_local_notifs from comment delete/remove

* move scrape_text_for_mentions() inside send_local_notifs()

* nullable

* simplify

* handle parent notification first

* cleanup

* remove `CommentResponse.recipient_ids`

* post notifications enum

* Follow posts (fixes #3069)

* use references

* cleanup

* new file

* db migration to merge mention, reply tables

* code adjustments

* adjust test code

* adjust enum case

* wip: get rid of inbox_combined table

* add table local_user_notification

* tests compiling

* get rid of inbox_combined, only use notification table

* rename view

* notify private messages

* clippy

* copy existing data

* wip: tests

* move tests

* clippy

* notify post subscribers

* community subscribe

* test fixes

* migration fix

* rename routes

* separate struct for api

* rename api params

* merge migrations

* separate notification modes for post/community

* fix

* down migration copy data

* test fix

* update api tests

* only single notification table

* clippy

* use local user id for recipient

* fix comments

* rename table

* recipient local user

* add indices

* keep local user id

* renames and cleanup

* NotificationDataType

* change notification response

* fix api tests

* test fix

* remove private_message.read

* fixes

* test fix

* fix
2025-07-17 19:04:09 -04:00
Dessalines
31dd9719b0
Fixing username-instance duplicate migration. (#5867)
Context: #5045 , #5858
2025-07-17 09:32:54 +02:00
Nutomic
cbc8f703b7
Extism release version (#5865) 2025-07-15 09:53:24 -04:00
Nutomic
2705728dbf
Delete report views, add missing report fields (#5863)
* Delete report views

* cleanup

* Add missing report fields

* simplify report trait

* simplify sized traits

* remove unnecessary traits with only one impl

* fix
2025-07-15 04:58:38 -04:00
Nutomic
cf0e021129
Add unique constraint for person(name, instance) (fixes #5045) (#5858)
* Add unique constraint for person(name, instance) (fixes #5045)

* fix

* not local

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2025-07-14 17:14:05 -04:00
dullbananas
21c1784a01
Use a new separate crate for uplete (#5848)
* replace lemmy_db_schema::utils::uplete with in-progress crate

* use published diesel-uplete crate

* remove unused tuplex dependency

* move diesel-uplete to dev-dependencies for lemmy_db_views_post

* import table modules in table_impls.rs

* update to new diesel-uplete with renamed stuff
2025-07-14 16:37:11 -04:00
Dessalines
fe30f30c4e
Don't export inbox combined view internal. (#5860)
Noticed this in the js-client. It's only used internally.
2025-07-11 17:15:59 -04:00
Nutomic
b5dee35b84
Update extism (#5859) 2025-07-11 16:52:53 -04:00
dullbananas
df4a79f4b6
Allow running all migrations with minimal dependencies (#5841)
* move schema_setup to separate crate

* remove lemmy_utils dependency

* add db_schema_setup/src/main.rs

* fix diesel.toml

* run diesel print-schema

* fix replaceable_schema paths

* remove unneeded dependencies

* taplo format

* fix duplicated cfg(test) attribute

* move replaceable_schema to crates/db_schema_setup/replaceable_schema

* warn if lemmy_db_schema_setup can be used instead of lemmy_server
2025-07-11 09:36:42 +02:00
Nutomic
a8a407ca6d
Federation refactoring (#5855)
* refactor federation http responses

* Rename Activity

* test fixes

* library release
2025-07-10 23:31:33 -04:00
Nutomic
7fb11c36fa
Fix comment depth off by one (#5851)
* Fix comment depth off by one

* fix

* test and comment
2025-07-10 23:28:06 -04:00
SleeplessOne1917
fd1e0b01e5
Add missing TS derives that were causing errors when generating Typescript bindings (#5853) 2025-07-09 09:48:14 +02:00
SleeplessOne1917
e66d0a3d61
Add creator_banned to VoteView (#5849) 2025-07-08 09:55:34 +02:00
Nutomic
8cf14f407f
Move local site user count updates to scheduled task (#5822)
* Move local site stats updates to scheduled task (fixes #5816, fixes #3213)

* only user count via scheduled task

* join to local site
2025-07-04 09:31:13 +02:00
Nutomic
3922f0f325
Revert " Add post_actions.disable_notifications (fixes #3042) (#5826)" (#5843)
This reverts commit 76fbb29079.
2025-07-02 11:29:42 +02:00
Nutomic
0d149dd613
Add missing comment depth check (#5840) 2025-07-02 09:37:20 +02:00
renovate[bot]
568e93ebef
Update docker (#5838)
* Update docker

* Update .woodpecker.yml

* Update .woodpecker.yml

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2025-07-01 03:58:18 -04:00
renovate[bot]
9029338e59
Update Rust crate diesel to v2.2.11 (#5839)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-07-01 00:23:59 -04:00
Nutomic
76fbb29079
Add post_actions.disable_notifications (fixes #3042) (#5826)
* Add post_actions.disable_notifications (fixes #3042)

* Split up logic for send_local_notifs()

* refactor

* fmt

* add api endpoint, check

* dont call send_local_notifs from comment delete/remove

* move scrape_text_for_mentions() inside send_local_notifs()

* nullable

* simplify

* handle parent notification first

* cleanup

* remove `CommentResponse.recipient_ids`

* post notifications enum

* adjust enum case
2025-06-30 09:52:19 +02:00
Dessalines
dc56863b75
Use vec_into (#5833) 2025-06-30 09:34:50 +02:00
Dessalines
899c87f21f
Adding GDPR-style data export (#5801)
* Adding GDPR-style data export

- Fixes #4540

Still need to:

- [ ] Figure out limits

* Fixing format

* Adding no_limit overrides.

* Slimming down export.

* Cleaning up types some more.

* Addressing PR comments.

* Embedding settings in backup.

* Fixing comment.
2025-06-27 13:14:59 -04:00
Nutomic
e3d36d4f9a
Revert "Use same federation keypair for all new users and communities (#5709)" (#5830)
This reverts commit b4180a57bf.
2025-06-26 13:15:12 -04:00
Nutomic
d77c9a0544
Drop table person_ban (fixes #5828) (#5831) 2025-06-26 13:13:39 -04:00
Nutomic
35f48c5a95
Better way to check if we should accept activity in remote community (#5823)
* Better way to check if we should accept activity in remote community (fixes #5596)

* fix test
2025-06-26 12:52:17 -04:00
Dessalines
a6b03fabdb
Keep totals of upvotes and downvotes given to each user. (#5786)
* Keep totals of upvotes and downvotes given to each user.

- Adds a `voted_at`, `upvotes` and `downvotes` to person_actions.
- Didn't use triggers, because I couldn't figure out how to get the
  voter, and also because it should only be for local users anyway.
- Fixes #2370

* Clippy

* Adding history filling.

* Try to fix postgres alias error 1

* Update crates/db_schema/src/impls/person.rs

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* Addressing PR comments

* Removing commented line.

* Fixing type errors.

* Fixing test

* Fixing like score 0 and api test errors.

* Some clippy fixes

* Update migrations/2025-06-14-141408_person_votes/up.sql

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* Update migrations/2025-06-14-141408_person_votes/up.sql

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* Update migrations/2025-06-14-141408_person_votes/up.sql

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* Formatting sql

* Fixing migration.

* Cleaning up merge

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2025-06-26 09:58:10 +02:00
Dessalines
d274d2bb1d
Remove votes when ban + remove data for a site or community. (#5810)
* Remove votes when ban + remove data for a site or community.

- Fixes #5569

* Starting to revert cleanup.

* Finishing up.
2025-06-26 09:52:13 +02:00
Dessalines
9ccd647e02
Fixing docker_update.sh script to not add context. (#5820)
* Fixing docker_update.sh script to not add context.

Use docker build from a higher level to not load massive context.

* Adding a custom postgresql for testing.

* Fixing docker update

* Fixing dir.

* Fix rust log

* Add bash / shell file format checking.

* Adding shfmt package.

* Running bash format.

* Fixing bash fmt.

* Fixing bash fmt 2.

* Adding pgtune note.
2025-06-26 09:48:39 +02:00
Dessalines
78fc703ebd
Optimizing person_content_combined migration. (#5819) 2025-06-26 09:40:21 +02:00
Nutomic
cc61b70b29
Send Undo/Dislike activity (fixes #4465) (#5825) 2025-06-24 10:29:01 -04:00
Dessalines
7e55a39677
Optimizing a few more combined migrations (#5821)
* Add indexes after insert to search_combined.

* Add indexes after for inbox_combined.

* Move index after for modlog combined.

* Fixing unique.
2025-06-24 10:22:17 -04:00
Nutomic
99ec0b7116
Rename rate limit columns (fixes #5805) (#5824)
* Rename rate limit columns (fixes #5805)

* update js client
2025-06-24 10:21:50 -04:00
Nutomic
497ec72f26
Optimize migrations (#5800)
* Fix migration local_image_person

* Make constraints deferrable (ref #5798)

* Some more adjustments

* fix down migrations

* set not deferrable once done

* fix

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2025-06-23 08:05:21 -04:00
dullbananas
6accb4110d
Fix db_perf.sh (#5815)
* db_perf.sh

* get_option function
2025-06-22 11:56:10 -04:00
Nutomic
0a22fac562
Remove creator_*_actions fields from api (ref #5532) (#5808)
* Dont return action structs in api (ref #5532)

* rename methods

* Remove creator instance fields, add missing check

* update js client

* Remove remaining creator actions

* update api tests
2025-06-20 11:58:41 -04:00
Nutomic
06e7ea00b8
Move config updated check to unit test, use diffy (#5803)
* Move config updated check to test

* use diffy

* clippy

* Fixes for "Move config updated check to unit test, use diffy" PR (#5812)

* replace diffy with unified_diff in test_config_defaults_updated

* revert changes to lemmy_db_schema_file

* use unified_diff in diff_check

* simplify

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2025-06-20 08:42:37 -04:00
Dessalines
e07e251ada
Removing an existing deferrable constraint, and fail test if any constraint is deferrable. (#5806)
* Removing an existing deferrable constraint.

- Also adding a check to make sure the final dump contains no DEFERs.

* Spelling error
2025-06-20 09:49:44 +02:00
dullbananas
fb762a3f9b
Remove outdated comment about deferrable constraints in triggers.sql (#5813) 2025-06-20 09:49:07 +02:00
Nutomic
9e3395dc22
Switch to library for rate limit (fixes #5550, fixes #5548) (#5731)
* Switch to library for rate limit (fixes #5550, fixes #5548)

* move debug rate limit logic

* move to file

* various rate limit methods

* move to utils crate

* use new code

* delete old rate limit code

* live updates

* live updates working

* clippy

* use binary keys

* clippy

* remove error

* cleanup
2025-06-19 18:52:56 -04:00
Nutomic
7cda94ca97
Set Accept: Vary header (fixes #5632) (#5809) 2025-06-19 18:44:48 -04:00
Nutomic
1f190e015e
Add incoming activity hook for plugins (#5792)
* Add incoming activity hook

* async working

* remove generic

* update with references

* remove insert_received_activity

* remove todo

* use crates.io dep
2025-06-18 16:28:25 +02:00
Nutomic
9a84180b5a
Fix migration local_image_person (#5799) 2025-06-18 08:56:01 -04:00
Anton Boitsov
070d049c49
Test migrations improvement (#5788)
* some test data in migration tests after initial setup (40 migrations)

* - fixed migration rollbacks creating NOT NULL fields without DEFAULT value
- added couple of checks

* fix sql fmt

* fix sql fmt

* allow indexing and unwrapping in tests
2025-06-18 09:39:08 +02:00
Nutomic
66540f1fe7
Delete scripts/ts_bindings_check.sh (#5797) 2025-06-17 11:20:58 -04:00
Dessalines
aef564f098
Better ntfy notifs. (#5793) 2025-06-17 11:37:18 +02:00