* Renaming person_mention to person_comment_mention.
* Finishing up post body mentions.
* Combined tables try 2
* Finishing up combined report table.
* Fix ts optionals.
* Adding tests, triggers, and history updates for report_combined.
* Adding profile.
* Add cursor pagination to report_combined view (#5244)
* add pagination cursor
* store timestamp instead of id in cursor (partial)
* Revert "store timestamp instead of id in cursor (partial)"
This reverts commit 89359dde4b.
* use paginated query builder
* Fixing migration and paged API.
* Using dullbananas trigger procedure
* Removing pointless list routes, reorganizing tests.
* Fixing column XOR check.
* Forgot to remove list report actions.
* Cleanup.
* Use internal tagging.
* Fixing api tests.
* Adding a few indexes.
* Fixing migration name.
* Fixing unique constraints.
* Addressing PR comments.
* Start working on profile combined
* Adding views and replaceable schema.
* A few changes to profile view.
- Separating the profile fetch from its combined content fetch.
- Starting to separate saved_only into its own combined view.
* Finishing up combined person_saved and person_content.
* Fixing api tests.
* Moving to api-v4 routes.
* Fixing imports.
* Update crates/db_views/src/report_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update crates/db_views/src/report_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update crates/db_views/src/report_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update migrations/2024-12-02-181601_add_report_combined_table/up.sql
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update migrations/2024-12-02-181601_add_report_combined_table/up.sql
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Fixing import and fmt.
* Fixing null types in postgres.
* Comment out err.
* Fixing TS issues.
* Adding types, fixing allow and blocklist crud.
* Starting to work on combined views.
* Using dullbananas trigger procedure
* Adding the full combined view queries.
* Adding tests.
* taplo fmt.
* Upgrading package.json deps.
* Updating pnpm
* Most of the bulk work done, need to add tests yet.
* Finishing up inbox.
* Using assert_length
* Fixing sql_format.
* Running fmt.
* Fixing cargo shear.
* Fixing clippy.
* Addressing PR comments.
* Starting to work on search combined.
* Fix
* Removing serialization
* Removing serialization
* Moving db_views_actor and _moderator into db_views.
- This is necessary because the combined views use both, and that
separation was arbitrary to begin with. db_schema has no such crate
separation.
* Adding search combined view, need to write tests yet.
* Filters done, working on tests.
* Adding tests for person, post, and community.
* Finishing up tests.
* Fixing duped trigger.
* Remove saved_only test.
* Remove pointless post_tags types.
* Remove pointless index.
* Changing published to saved for person_saved_combined.
* Removing comment.
* Renaming modlog when_ columns to published.
- Fixes#5312
* Adding strum and simplifying imports.
* Avoiding clone in map_to_enum
* Changing modded_person to other_person.
* Update crates/db_views_moderator/src/modlog_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update crates/db_views_moderator/src/modlog_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Update crates/db_views_moderator/src/modlog_combined_view.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Addressing PR comments.
* Fixing split.
* Revert "Adding strum and simplifying imports."
This reverts commit 15f1671107.
* Running fmt.
* Using assert + matches instead of filter_map.
* Adding listPersonContent check.
* Updating lemmy-js-client
* Fixing mark all as read route, changing mark read to SuccessResponse.
* Adding post body mention api test, fixing api tests.
* Fixing route locations, and api tests.
* Formatting sql.
* Formatting sql 2.
* Fixing search result, running clippy.
* Fixing ts_option.
* Adding search_combined.score column, and DB triggers.
* Fixing API tests.
* Adding an index for score.
* Update crates/db_schema/src/newtypes.rs
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Avoiding inner joins for up.sql
* Adding person_aggregates.published column.
---------
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
* Removing a few SuccessResponses for PostHide and MarkPostAsRead.
- This also removes the pointless multiple post_ids. These can be done
as individual calls on the front end anyway.
- Fixes#4755
* Fixing federation tests.
* Upgrading lemmy-js-client deps.
* Add ability to mark several posts as read.
Context:
- https://github.com/LemmyNet/lemmy/pull/5043
- https://github.com/LemmyNet/lemmy/issues/4755
- https://github.com/LemmyNet/lemmy/pull/5160
* Simplifying forms.
* Fixing forms.
* Cleanup post action forms by using derive_new defaults.
- Fixes#5195
* Fix ntfy to notify on success builds also.
* Removing pointless naive_now function.
* Running taplo fmt.
* add private visibility
* filter private communities in post_view.rs
* also filter in comment_view
* community follower state
* remove unused method
* sql fmt
* add CommunityFollower.approved_by
* implement api endpoints
* api changes
* only admins can create private community for now
* add local api tests
* fix api tests
* follow remote private community
* use authorized fetch for content in private community
* federate community visibility
* dont mark content in private community as public
* expose ApprovalRequired in api
* also check content fetchable for outbox/featured
* address private community content to followers
* implement reject activity
* fix tests
* add files
* remove local api tests
* dont use delay
* is_new_instance
* single query for is_new_instance
* return subscribed type for pending follow
* working
* need to catch errors in waitUntil
* clippy
* fix query
* lint for unused async
* diesel.toml comment
* add comment
* avoid db reads
* rename approved_by to approver_id
* add helper
* form init
* list pending follows should return items for all communities
* clippy
* ci
* fix down migration
* fix api tests
* references
* rename
* run git diff
* ci
* fix schema check
* fix joins
* ci
* ci
* skip_serializing_none
* fix test
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* move code to new file
* rewrite markdown links (fixes#2987)
* add missing file
* add helper fn
* also convert post.url
* simplify search.rs
* clippy
* also rewrite user/community links in markdown
* Call from apub handlers, cleanup
* no network requests in test
* clippy
* fix tests
* serial
* test
* no mut
* add api test
* fix api test
* Add LemmyResult to session_middleware tests
* Add LemmyResult to inboxes tests
* Add LemmyResult to slurs tests
* Add LemmyResult to markdown tests
* Add LemmyResult to rate_limiter tests
* Add LemmyResult to error tests
* Add LemmyResult to api_common utils tests
* Add LemmyResult to request tests
* Add LemmyResult to claims tests
* Propagate registration_applications errors
* Remove clippy::unwrap_used from community tests
* Add LemmyResult to community_view tests
* Add LemmyResult to db_schema post tests
* Add LemmyResult to site_aggregates tests
* Add LemmyResult to private_message tests
* Add LemmyResult to activity tests
* Add LemmyResult to federation_allowlist tests
* Add LemmyResult to comment_aggregates tests
* Add LemmyResult to post_report tests
* Add LemmyResult to moderator tests
* Add LemmyResult to community_aggregates tests
* Add LemmyResult to person_aggregates tests
* Add LemmyResult to language tests
* Add LemmyResult to post_aggregates tests
* Add LemmyResult to db_schema comment tests
* Add LemmyResult to actor_language tests
* Add LemmyResult to vote_view tests
* Add LemmyResult to registration_application_view tests
* Add LemmyResult to private_message_view tests
* Add LemmyResult to private_message_report_view tests
* Add LemmyResult to post_report_view tests
* Add LemmyResult to comment_report_view tests
* Add LemmyResult to sitemap tests
* Replace .expect() with .unwrap()
* Format code
* Remove clippy::unwrap_used from activity tests
* Add diesel result in db_schema tests
* Format code
* Map to_bytes() error to LemmyErrorType
* Remove clippy::unwrap_used from error tests
* Removing a few more unwraps, and cleaning up language code.
* Replace map_err with unwrap_or_default
* Replace ok_or with and_then
---------
Co-authored-by: Dessalines <tyhou13@gmx.com>
* Adding a URL max length error.
- Also increasing the post.url max length to 2000 (seems standard)
- I ran into this when fixing torrent support, which often use longer
urls.
* Fixing sql_format.
* Allow empty string to clear URL-type DB fields.
- To address difficulties with clearing URL-type fields like
avatars, banners, site icons, this PR turns the URL type form
fields into strings.
- This allows an empty string to be used as a "clear data", as
in the case with the regular text form fields.
- Also includes various cleanups.
- Fixes#4777
- Context: #2287
* Fixing comment.
* Use Option<&str> and deref.
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* Adding an image_details table to store image dimensions.
- Adds an image_details table, which stores the height,
width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes#3328
- Also fixes#4703
* Running sql format.
* Running fmt.
* Don't fetch metadata in background for local API requests.
* Dont export remote_image table to typescript.
* Cleaning up validate.
* Dont proxy url.
* Fixing tests, fixing issue with federated thumbnails.
* Fix tests.
* Updating corepack, fixing issue.
* Refactoring image inserts to use transactions.
* Use select exists again.
* Fixing imports.
* Fix test.
* Removing pointless backgrounded metadata generation version.
* Removing public pictrs details route.
* Fixing clippy.
* Fixing proxy image fetching. Fixes#4703
- This extracts only the proxy image fixes from #4704, leaving off
thumbnails.
* Fix test.
* Addressing PR comments.
* Address PR comments 2.
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
- Diesel ordinarily throws an error when no results are returned for a
single fetch, which is a bit confusing. This PR ensures that the
missing value cases are all caught, and wrapped with new LemmyErrors,
rather than diesel errors.
- Fixes#4601
* Dont allow federation to overwrite local objects
* is_local check in apub lib
* use imports
* fix check, update lib
* use verify_is_remote_object()
* submodule
* Generate post thumbnail/metadata in background (ref #4529)
* fix api test
* Apply suggestions from code review
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* fix test
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* Add a blocklist for URLs.
* Fix SQL format
* Make clippy happy.
* Use regex for URL matching.
* Escape regex chars in URLs.
* Use post for modification.
* Make URL block regex static and remove API routes.
* Add date fields to table and use transaction.
* Use Cache for blocklist.
* Rename check_links + move list to parameters of process_markdown.
* SQL format.
* Format, again.
* Remove println.
* Add API test.
* Set a shorter lifetime for regex in debug mode.
* Add missing macro.
* Update lemmy-js-client
* Update api_test/pnpm-lock.yaml
* Don't break other tests
* Use different URL for test
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
* Add support for local only community (fixes#1576)
* add filters and tests to db views
* dont federate local only community
* test get apub community http
* tests
* more checks
* wip
* api test
* fix tests
* change community.local_only column to visibility enum
(for private communities)
* sql fmt
* rename vars
* clippy
* fix tests
* update lib
* review
* fix js client version
* update client
* Add markdown rule to add rel=nofollow for all links
* Add markdown image rule to add local image proxy (fixes#1036)
* comments
* rewrite markdown image links working
* add comment
* perform markdown image processing in api/apub receivers
* clippy
* add db table to validate proxied links
* rewrite link fields for avatar, banner etc
* sql fmt
* proxy links received over federation
* add config option
* undo post.url rewriting, move http route definition
* add tests
* proxy images through pictrs
* testing
* cleanup request.rs file
* more cleanup (fixes#2611)
* include url content type when sending post over apub (fixes#2611)
* store post url content type in db
* should be media_type
* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.
* fix tests
* add setting disable_external_link_previews
* federate post url as image depending on mime type
* change setting again
* machete
* invert
* support custom emoji
* clippy
* update defaults
* add image proxy test, fix test
* fix test
* clippy
* revert accidental changes
* address review
* clippy
* Markdown link rule-dess (#4356)
* Extracting opengraph_data to its own type.
* A few additions for markdown-link-rule.
---------
Co-authored-by: Nutomic <me@nutomic.com>
* fix setting
* use enum for image proxy setting
* fix test configs
* add config backwards compat
* clippy
* machete
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Handle invalid ban expires values (fixes#4045)
* Adding a few missing expire time checks. Fixing up time conversions. (#4051)
* Adding a few missing expire time checks. Fixing up time conversions.
* Increase settings export wait time.
* get rid of RemoveCommunity.expires
* fmt
* tests
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* HTML sanitization in apub code
* Sanitize API inputs
* fmt
* Dont allow html a, img tags
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Try using drone cache plugin
* Try another path
* Include volume
* Fix formatting
* Include fmt
* Exclude cargo dir from prettier
* Don't override cargo
* Just do check
* Add cache key
* Use different cache plugin
* Add clippy
* Try minio
* Add quotes
* Try adding secrets
* Try again
* Again
* Use correct secret formation
* Add back clippy
* Use secret for the root bucket name
* Try drone cache instead
* Add region
* Add path-style option
* Include cargo clippy
* Include everything again
* Fix formatting
* Don't run clippy twice
* Add `allow` statements for tests to pass
* Adjust endpoint to be a secret
* Fix prettier
* Merge & fix tests
* Try to restart the woodpecker test
* Change the ENV var name
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* a lot
* merge
* Fix stuff broken by merge
* Get rid of repetitive `&mut *context.conn().await?`
* Add blank lines under each line with `conn =`
* Fix style mistakes (partial)
* Revert "Fix style mistakes (partial)"
This reverts commit 48a033b87f.
* Revert "Add blank lines under each line with `conn =`"
This reverts commit 773a6d3beb.
* Revert "Get rid of repetitive `&mut *context.conn().await?`"
This reverts commit d2c6263ea1.
* Use DbConn for CaptchaAnswer methods
* DbConn trait
* Remove more `&mut *`
* Fix stuff
* Re-run CI
* try to make ci start
* fix
* fix
* Fix api_common::utils
* Fix apub::activities::block
* Fix apub::api::resolve_object
* Fix some things
* Revert "Fix some things"
This reverts commit 2bf8574bc8.
* Revert "Fix apub::api::resolve_object"
This reverts commit 3e4059aabb.
* Revert "Fix apub::activities::block"
This reverts commit 3b02389abd.
* Revert "Fix api_common::utils"
This reverts commit 7dc73de613.
* Revert "Revert "Fix api_common::utils""
This reverts commit f740f115e5.
* Revert "Revert "Fix apub::activities::block""
This reverts commit 2ee206af7c.
* Revert "Revert "Fix apub::api::resolve_object""
This reverts commit 96ed8bf2e9.
* Fix fetch_local_site_data
* Fix get_comment_parent_creator
* Remove unused perma deleted text
* Fix routes::feeds
* Fix lib.rs
* Update lib.rs
* rerun ci
* Attempt to create custom GetConn and RunQueryDsl traits
* Start over
* Add GetConn trait
* aaaa
* Revert "aaaa"
This reverts commit acc9ca1aed.
* Revert "Revert "aaaa""
This reverts commit 443a2a00a5.
* still aaaaaaaaaaaaa
* Return to earlier thing
Revert "Add GetConn trait"
This reverts commit ab4e94aea5.
* Try to use DbPool enum
* Revert "Try to use DbPool enum"
This reverts commit e4d1712646.
* DbConn and DbPool enums (db_schema only fails to compile for tests)
* fmt
* Make functions take `&mut DbPool<'_>` and make db_schema tests compile
* Add try_join_with_pool macro and run fix-clippy on more crates
* Fix some errors
* I did it
* Remove function variants that take connection
* rerun ci
* rerun ci
* rerun ci
With this change only http(s) schemes are allowed for post.url
field. This is checked for incoming api and federation requests.
Existing posts in database which are sent to clients are not
checked. Neither does it check urls in markdown.
