Commit graph

87 commits

Author SHA1 Message Date
Dessalines
355c8f6bb5 Merge remote-tracking branch 'origin/main' into auto_mark_posts_as_read 2024-11-07 11:38:41 -05:00
Nutomic
ad90cd77f9
Implement private communities (#5076)
* add private visibility

* filter private communities in post_view.rs

* also filter in comment_view

* community follower state

* remove unused method

* sql fmt

* add CommunityFollower.approved_by

* implement api endpoints

* api changes

* only admins can create private community for now

* add local api tests

* fix api tests

* follow remote private community

* use authorized fetch for content in private community

* federate community visibility

* dont mark content in private community as public

* expose ApprovalRequired in api

* also check content fetchable for outbox/featured

* address private community content to followers

* implement reject activity

* fix tests

* add files

* remove local api tests

* dont use delay

* is_new_instance

* single query for is_new_instance

* return subscribed type for pending follow

* working

* need to catch errors in waitUntil

* clippy

* fix query

* lint for unused async

* diesel.toml comment

* add comment

* avoid db reads

* rename approved_by to approver_id

* add helper

* form init

* list pending follows should return items for all communities

* clippy

* ci

* fix down migration

* fix api tests

* references

* rename

* run git diff

* ci

* fix schema check

* fix joins

* ci

* ci

* skip_serializing_none

* fix test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-11-07 05:49:05 -05:00
Dessalines
879b8e9206 Merge remote-tracking branch 'origin/main' into auto_mark_posts_as_read 2024-11-06 11:24:41 -05:00
Dessalines
917e408735
Fix postgres connection options causing slow query speed. (#5150)
* Adding a query speed check.

* Fixing slow queries due to connection config options.

* Remove pointless set_config sql function.

* Removing pointless bool.

* Removing comment

* Removing test.sh changes.

* Add analyze to speed up query

* Trying to fix DB perf connection try #1

* Try encoding option

* Fix woodpecker

* Try to use path character.

* Fixing lemmy config location.

* Removing pointless connection options.

* Use OnceLock to create a once-init psql connection.

* Fixing comment.

* Fix host encoding for dev DB.

* Address PR comments.

* Revert query mut change.
2024-11-06 10:58:40 -05:00
Dessalines
df664d9d9a
Upgrading ts_rs to 10.0.0 (#5163)
* Upgrading ts_rs to 10.0.0

* Adding ts_option directives, and woodpecker test.

* Fixing ts_options.
2024-11-06 15:50:13 +01:00
Dessalines
8997119c4e Moving db_perf to before federation tests. 2024-11-04 15:40:10 -05:00
renovate[bot]
02ba54c589
chore(deps): update node.js to v22 (#5155)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-11-01 09:57:33 -04:00
Dessalines
30951a37b6
Revert "chore(deps): update docker (#5154)" (#5156)
This reverts commit 556191ef16.
2024-11-01 09:34:12 -04:00
renovate[bot]
556191ef16
chore(deps): update docker (#5154)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-10-31 23:02:50 -04:00
Dessalines
d6d01a3b62
Adding cargo shear (#5139)
* Adding cargo shear (fail check)

* Running cargo shear.

* Fixing another missing.

* Removing cargo-machete in favor of cargo-shear
2024-10-28 11:41:09 -04:00
renovate[bot]
b009d45729
Update tmknom/prettier Docker tag to v3.2.5 (#5102)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-10-06 04:02:18 -04:00
renovate[bot]
a611a232ea
Update tamasfe/taplo Docker tag to v0.9.3 (#5101)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-10-06 01:17:46 -04:00
Dessalines
5a722146b5
Upgrading to rust 1.81 (#5032) 2024-09-19 17:00:20 -04:00
Dessalines
ef5e2d96cd
Fixing woodpecker warnings 1. (#5011) 2024-09-10 17:42:20 -04:00
Nutomic
c90ee3094d
Remove opentelemetry (#4741)
* Remove opentelemetry

* remove unused deps, use backtrace

* always print db migration messages regardless of log level (fixes #4725)

* fix ci

* Remove useless root span builder

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-09-10 13:36:03 -04:00
Dessalines
0a6186d7aa
Fixing apt-key deprecation, and using apt-get. (#4931)
* Fixing apt-key deprecation, and using apt-get.

* Try 2

* Try 3

* Try 4

* Try 5

* Try 6

* Try 7

* Try 8

* Try 9

* Try 10

* Try 11

* Try 12

* Try 13

* Try 14

* Try 15

* Try 16

* Try 17
2024-07-31 18:54:44 -04:00
SleeplessOne1917
1ef375e5c3
Replace 3rd party once_cell crate with recently stabilized standard library API (#4919)
* Replace 3rd party once_cell crate with recently stabilized standard library API

* Bump CI rust version
2024-07-30 10:11:39 -04:00
phiresky
a08642f813
federation: parallel sending per instance (#4623)
* federation: parallel sending

* federation: some comments

* lint and set force_write true when a request fails

* inbox_urls return vec

* split inbox functions into separate file

* cleanup

* extract sending task code to separate file

* move federation concurrent config to config file

* off by one issue

* improve msg

* fix both permanent stopping of federation queues and multiple creation of the same federation queues

* fix after merge

* lint fix

* Update crates/federate/src/send.rs

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* comment about reverse ordering

* remove crashable, comment

* comment

* move comment

* run federation tests twice

* fix test run

* prettier

* fix config default

* upgrade rust to 1.78 to fix diesel cli

* fix clippy

* delay

* add debug to make localhost urls not valid in ap crate, add some debug logs

* federation tests: ensure server stop after test and random activity id

* ci fix

* add test to federate 100 events

* fix send 100 test

* different data every time so activities are distinguishable

* allow out of order receives in test

* lint

* comment about https://github.com/LemmyNet/lemmy/pull/4623#discussion_r1565437391

* move sender for clarity, add comment

* move more things to members

* update test todo comment, use same env var as worker test but default to 1

* remove else below continue

* some more cleanup

* handle todo about smooth exit

* add federate inboxes collector tests

* lint

* actor max length

* don't reset fail count if activity skipped

* fix some comments

* reuse vars

* format

* Update .woodpecker.yml

* fix recheck time

* fix inboxes tests under fast mode

* format

* make i32 and ugly casts

* clippy

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2024-07-21 11:50:50 -04:00
Daniel Lo Nigro
8abbd56400
Switch PostgreSQL container to use pgautoupgrade (#4892)
* Switch PostgreSQL container to use pgautoupgrade

This handles automatically upgrading the data files to newer versions of PostgreSQL.

* Fixing other uses of postgres:16-alpine image.

* Simplifying upgrade scripts.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2024-07-17 09:59:21 +02:00
phiresky
96b7afc0b1
upgrade rust to 1.78 to fix diesel cli (#4761) 2024-05-31 08:39:45 -04:00
Dessalines
1a4aa3eaba
Stop using a diesel_cli docker image, use cargo-install in woodpecker. (#4723)
* Stop using a diesel_cli docker image, use cargo-binstall in woodpecker.

- The diesel_cli image is 500MB, and rebuilt daily. Much easier to use
  binstall to install it.

* Trying out a multiline var.

* Try sequence merges 1

* Try removing features.

* Try path fix.

* Abstracting diesel cli install.

* Try installing mysql and postgres.

* Try installing mysql and postgres 2.

* Try installing mysql and postgres 3.

* Try installing mysql and postgres 4.

* Try installing mysql and postgres 5.

* Try installing mysql and postgres 6.

* Try installing mysql and postgres 7.

* Try installing mysql and postgres 8.

* Try installing mysql and postgres 9.

* Try installing mysql and postgres 10.

* Try installing mysql and postgres 11.

* Try installing mysql and postgres 12.

* Try installing mysql and postgres 13.

* Add logging line.

* Add logging line 2.

* Add logging line 3.

* Add logging line 4.

* Removing binstall.

* Extract taplo into its own image, ignore binstall.

* Use a smaller taplo.

* taplo is the same image.
2024-05-16 16:41:36 -04:00
SleeplessOne1917
b05f221565
Remove login step from publish to crates.io (#4674) 2024-04-30 00:07:03 +00:00
dullbananas
4ba6221e04
Move SQL triggers from migrations into reusable sql file (#4333)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* Create down.sql

* Create up.sql

* Create replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update utils.rs

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Create dump_schema.sh

* Update start_dev_db.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* stuff

* Update replaceable_schema.sql

* Update .pg_format

* fmt

* stuff

* stuff (#21)

* Update replaceable_schema.sql

* Update up.sql

* Update replaceable_schema.sql

* fmt

* update cargo.lock

* stuff

* Update replaceable_schema.sql

* Remove truncate trigger because truncate is already restricted by foreign keys

* Update replaceable_schema.sql

* fix some things

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update .woodpecker.yml

* stuff

* fix TG_OP

* Psql env vars

* try to fix combine_transition_tables parse error

* Revert "try to fix combine_transition_tables parse error"

This reverts commit 75d00a4626.

* refactor combine_transition_tables

* try to fix create_triggers

* fix some things

* try to fix combined_transition_tables

* fix sql errors

* update comment count in post trigger

* fmt

* Revert "fmt"

This reverts commit a5bcd0834b.

* Revert "update comment count in post trigger"

This reverts commit 0066a4b42b.

* fix everything

* Update replaceable_schema.sql

* actually fix everything

* refactor create_triggers

* fix

* add semicolons

* add is_counted function and fix incorrect bool operator in update_comment_count_from_post

* refactor comment trigger

* refactor post trigger

* fix

* Delete crates/db_schema/src/utils/series.rs

* subscribers_local

* edit migrations

* move migrations

* remove utils::series module declaration

* fix everything

* stuff

* Move sql to schema_setup dir

* utils.sql

* delete .pg_format

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update .woodpecker.yml

* Merge remote-tracking branch 'upstream/main' into bliss

* fmt

* Create main.rs

* Update lib.rs

* Update main.rs

* Update .woodpecker.yml

* Update main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* YAY

* Update mod.rs

* Update Cargo.toml

* a

* Update Cargo.toml

* Update Cargo.toml

* Delete crates/db_schema/src/main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update utils.sql

* Update utils.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update down.sql

* Update up.sql

* Update triggers.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* Update down.sql

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update mod.rs

* Update Cargo.toml

* Update mod.rs

* make dump_schema.sh executable

* fix dump_schema.sh

* defer

* diff dumps

* fmt

* Update utils.sql

* Update .woodpecker.yml

* use correct version for pg_dump

* Update .woodpecker.yml

* Update .woodpecker.yml

* change migration date

* atomic site_aggregates insert

* temporarily repeat tests in CI

* drop r schema in CI migration check

* show ReceivedActivity::create error

* move check_diesel_migration CI step

* Update .woodpecker.yml

* Update scheduled_tasks.rs

* Update scheduled_tasks.rs

* update cargo.lock

* move sql files

* move rank functions

* filter post_aggregates update

* fmt

* cargo fmt

* replace post_id with id

* update cargo.lock

* avoid locking rows that need no change in up.sql

* only run replaceable_schema if migrations were run

* debug ci test failure

* make replaceable_schema work in CI

* Update .woodpecker.yml

* remove println

* Use migration revert and git checkout

* Update schema_setup.rs

* Fix

* Update schema_setup.rs

* Update schema_setup.rs

* Update .woodpecker.yml

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-04-17 20:58:44 -04:00
Dessalines
705e86eb4c
Fixing docker release (#4592)
* Re-add notif on tag failure.

* Upping rust version.

* Version 0.19.4-beta.1a

* Try again.

* Version 0.19.4-beta.1b

* Removing unstable inspect.

* Version 0.19.4-beta.1c

* Remove use release cache.

* Trying to fix cargo publish 1.

* Version 0.19.4-beta.1d

* Re-adding publish release

* Version 0.19.4-beta.2

* Fixing workspace for lemmy_federate
2024-04-04 16:14:59 +02:00
Dessalines
a1d632e582
Re-add notif on tag failure. (#4591) 2024-04-04 10:21:31 +02:00
Dessalines
aaaa362b98
Remove latest tag for pgformatter. (#4589) 2024-04-03 17:29:24 -04:00
Dessalines
067332553d
Fixing woodpecker. (#4575) 2024-03-27 11:09:44 -04:00
Dessalines
95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
Richard Schwab
80bfd23b4d
Ensure rustfmt is installed in cargo fmt CI job (#4466) 2024-02-26 11:22:22 +01:00
Dessalines
890565ca14
Deleting denied local_users older than a week. Fixes #4434 (#4448)
* Deleting denied local_users older than a week. Fixes #4434

* Addressing PR comments.

* Upping rust to 1.76

* Delete the person rows also.
2024-02-15 07:50:53 -05:00
Dessalines
9367cbdb00
Upgrading from postgres 15 -> 16-alpine. (#4426)
- Includes an upgrade script.
- Fixes #4406
2024-02-07 11:21:02 +01:00
Dessalines
eb0dc2fda4
Moving from yarn to pnpm. (#4414)
* Moving from yarn to pnpm.

* Prettier check.
2024-01-29 11:38:39 +01:00
dullbananas
f481a607d0
Show server output if federation tests fail in CI (#4389)
* Show server output if federation tests fail in CI

* dummy failure

* Revert dummy failure

* Update private_message.rs

* Fix errors

* Update private_message.rs

* Update private_message.rs

* Update .woodpecker.yml

* correct exit code

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update private_message.rs

* Update lib.rs

* Update lib.rs

* Update session_middleware.rs

* Update session_middleware.rs

* Update .woodpecker.yml
2024-01-26 10:38:15 +01:00
dullbananas
ade1cb1495
Remove outdated comment from .woodpecker.yml (#4404) 2024-01-26 10:33:03 +01:00
dullbananas
759f6d8a9a
Better query plan viewing experience (#4285)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* replace get(0) with first()

* as_slice

* Update series.rs

* Update db_perf.sh

* Update and rename crates/db_schema/src/utils/series.rs to crates/db_perf/src/series.rs

* Update utils.rs

* Update main.rs

* Update main.rs

* Update .woodpecker.yml

* fmt main.rs

* Update .woodpecker.yml

* Instance::delete at end

* Update main.rs

* Update Cargo.toml

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-01-24 10:22:33 -05:00
Dessalines
e78fe5a34c
Removing group from woodpecker, as its deprecated. (#4387)
* Removing group from woodpecker, as its deprecated.

* Removing meltwater drone cache.
2024-01-22 10:57:07 +01:00
Nutomic
92b49dea3a
Increase CI timeout for restore cache (#4358) 2024-01-09 12:37:48 -05:00
Dessalines
3d9dda4677
Woodpecker rust 1.75 (#4355)
* Upgrading woodpecker rust to 1.75

* Trying to get publish working, try 1.

* Version 0.19.2-rc.3.publish1

* Temporarily disable arm builds.

* Version 0.19.2-rc.3.publish2

* Comment out arm blocks.

* Version 0.19.2-rc.3.publish3

* Moving back publish block to below.

* Upgrade to rasky lemmy-arm v0.2.0

* Version 0.19.2-rc.3.publish4

* Test bump for upgraded woodpecker.

* Revert "Test bump for upgraded woodpecker."

This reverts commit e2a5a99f33.

* Adding link to raskys repo.
2024-01-09 11:48:29 -05:00
Nutomic
952c162dac
Fix CI cache (#4276)
* Minor CI improvements (second attempt)

* test slow check condition

* remove rebuild cache condition for testing

* trigger quick ci check

* mkdir

* ls, remove steps

* exclude cargo home from prettier

* ci

* increase timeout

* more timeout

* even higher just for testing

* disable compression

* also increase restore timeout

* cleanup

* try again

* Update .woodpecker.yml

* Update .woodpecker.yml

* cleanup

* rerun ci

* cleanup
2024-01-04 11:40:41 -05:00
Nutomic
2b9d89057d
Reenable crates.io releases (fixes #4291) (#4294)
* Reenable crates.io releases (fixes #4291)

* disable docker release for now

* fixes

* fix indent

* move to top

* install libssl

* -y

* apt update

* debian version

* install cargo, git

* use cargo install

* move to orig position
2023-12-20 08:18:54 -05:00
Dessalines
e4b97ad8a9
Revert "Minor CI improvements (#4266)" (#4271)
This reverts commit af4d008ce4.
2023-12-14 18:36:33 -05:00
Nutomic
af4d008ce4
Minor CI improvements (#4266)
* Speed up SQL formatting in CI with parallel processing

* mess up formatting

* check permissions

* make repo world writable

* need git

* use debian image

* apt update

* perl image for faster install

* fix format

* run restore cache in parallel to format

* add drone-cache exit code

* revert sql format changes
2023-12-14 11:18:15 -05:00
Nutomic
4a55d4f871
Quick fix for Peertube federation (fixes #4261) (#4264)
* Quick fix for Peertube federation (fixes #4261)

* prettier

* apub assets should be included in slow check paths
2023-12-14 07:25:47 -05:00
Nutomic
95130370f0
Add CI check for ignored files (#4252)
* Add CI check for ignored files

* remove ignored files
2023-12-12 08:54:06 -05:00
Kroese
fc07ba2d3b
Fix entrypoint in Dockerfile (#4202)
* Fix entrypoint

* Delete docker/builders/lemmy-builder-arm64/docker-build.sh

* Delete docker/builders directory

* Remove exception for builder

* Remove publish_builder_arm64
2023-11-26 16:50:31 -05:00
Enzo Nocera
8a05c8f8be
fix: Cross-compilation to ARM64 (#4142)
* feat(docker/docs): explain how building lemmy works

Signed-off-by: Enzo NOCERA <enzo@nocera.eu>

* feat: add arm build

* review: rename script & fix typo

* feat(ci): allow cross platform compilation

* feat(ci): prettier

* fix(docker): fix base image name

* fix: add dockerfile in CI path

Signed-off-by: Enzo Nocera <enzo@nocera.eu>

* fix(docker): fix runner name

* fix(docker): fix builder base image

* fix(docker): fix builder base image platform

* fix(docker): avoid using the wrapper adduser/addgroup

* feat: avoid adding the whole docker directory in the build context

---------

Signed-off-by: Enzo NOCERA <enzo@nocera.eu>
Signed-off-by: Enzo Nocera <enzo@nocera.eu>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-11-24 12:52:19 -05:00
Nutomic
934fe7d1cb
Remove pictrs config section old_db (#4199)
* Remove pictrs config section `old_db` (ref #4194)

* ci

* remove cargo config comment

* ci

* Revert "remove cargo config comment"

This reverts commit 9389e93649.

* rename .cargo to .cargo_home
2023-11-24 11:26:21 -05:00
dullbananas
30f7135f32
Use rust 1.74.0 in CI and move clippy lints to Cargo.toml (#4170)
* ci rust 1.74.0

* do it

* Update .woodpecker.yml

* Update Cargo.toml

* Update lint.sh

* Update lib.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-21 08:51:22 -05:00
Nutomic
9007d49324
Remove unused deps (#4111)
* Remove unused deps

* Adding cargo-machete to woodpecker.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-31 11:11:12 +01:00
Dessalines
645bf21d54
Moving to debian-based docker image. Fixes #3972 (#4004)
* Moving to debian-based docker image. Fixes #3972

* Fix cargo.toml format.

* Try nightly rust for cargo_fmt.

* Trying to fix cargo build.

* Trying to fix cargo build 2.

* Trying to fix federation tests.

* Trying to fix federation tests 2.

* Trying to fix federation tests 3.

* Try to fix aarch 1.

* Try to fix aarch 2.

* Try to fix aarch 3.

* Removing aarch

* Adding tests back in.

* Adding ca-certs, and debian non-root user

* Commenting arm image building.

* Fix platform on runner.

* Version 0.19.0-beta.6
2023-10-11 11:57:05 -04:00