lemmy/crates/routes/src/images.rs

289 lines
8.1 KiB
Rust
Raw Normal View History

2022-03-02 18:05:48 +00:00
use actix_web::{
body::BodyStream,
error,
http::{
header::{HeaderName, ACCEPT_ENCODING, HOST},
StatusCode,
},
web,
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
web::Query,
2022-03-02 18:05:48 +00:00
Error,
HttpRequest,
HttpResponse,
};
use futures::stream::{Stream, StreamExt};
use lemmy_api_common::context::LemmyContext;
use lemmy_db_schema::source::{
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
images::{LocalImage, LocalImageForm, RemoteImage},
local_site::LocalSite,
};
use lemmy_db_views::structs::LocalUserView;
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
use lemmy_utils::{error::LemmyResult, rate_limit::RateLimitCell, REQWEST_TIMEOUT};
use reqwest::Body;
use reqwest_middleware::{ClientWithMiddleware, RequestBuilder};
use serde::{Deserialize, Serialize};
use std::time::Duration;
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
use url::Url;
use urlencoding::decode;
pub fn config(
cfg: &mut web::ServiceConfig,
client: ClientWithMiddleware,
rate_limit: &RateLimitCell,
) {
cfg
2022-03-02 18:05:48 +00:00
.app_data(web::Data::new(client))
.service(
web::resource("/pictrs/image")
.wrap(rate_limit.image())
.route(web::post().to(upload)),
)
2020-10-14 16:48:10 +00:00
// This has optional query params: /image/{filename}?format=jpg&thumbnail=256
.service(web::resource("/pictrs/image/{filename}").route(web::get().to(full_res)))
.service(web::resource("/pictrs/image/delete/{token}/{filename}").route(web::get().to(delete)));
}
#[derive(Debug, Serialize, Deserialize)]
struct Image {
file: String,
delete_token: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct Images {
msg: String,
files: Option<Vec<Image>>,
}
2020-10-14 16:48:10 +00:00
#[derive(Deserialize)]
struct PictrsParams {
2020-10-14 16:48:10 +00:00
format: Option<String>,
thumbnail: Option<i32>,
2020-10-14 16:48:10 +00:00
}
fn adapt_request(
request: &HttpRequest,
client: &ClientWithMiddleware,
url: String,
) -> RequestBuilder {
// remove accept-encoding header so that pictrs doesn't compress the response
const INVALID_HEADERS: &[HeaderName] = &[ACCEPT_ENCODING, HOST];
let client_request = client
.request(request.method().clone(), url)
.timeout(REQWEST_TIMEOUT);
request
.headers()
.iter()
.fold(client_request, |client_req, (key, value)| {
if INVALID_HEADERS.contains(key) {
client_req
} else {
client_req.header(key, value)
}
})
}
async fn upload(
req: HttpRequest,
body: web::Payload,
// require login
local_user_view: LocalUserView,
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
client: web::Data<ClientWithMiddleware>,
context: web::Data<LemmyContext>,
) -> Result<HttpResponse, Error> {
// TODO: check rate limit here
let pictrs_config = context.settings().pictrs_config()?;
let image_url = format!("{}image", pictrs_config.url);
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
let mut client_req = adapt_request(&req, &client, image_url);
if let Some(addr) = req.head().peer_addr {
client_req = client_req.header("X-Forwarded-For", addr.to_string())
};
let res = client_req
.timeout(Duration::from_secs(pictrs_config.upload_timeout))
.body(Body::wrap_stream(make_send(body)))
.send()
.await
.map_err(error::ErrorBadRequest)?;
let status = res.status();
let images = res.json::<Images>().await.map_err(error::ErrorBadRequest)?;
if let Some(images) = &images.files {
for uploaded_image in images {
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
let form = LocalImageForm {
local_user_id: Some(local_user_view.local_user.id),
pictrs_alias: uploaded_image.file.to_string(),
pictrs_delete_token: uploaded_image.delete_token.to_string(),
};
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
LocalImage::create(&mut context.pool(), &form)
.await
.map_err(error::ErrorBadRequest)?;
}
}
Ok(HttpResponse::build(status).json(images))
}
async fn full_res(
filename: web::Path<String>,
2020-10-14 16:48:10 +00:00
web::Query(params): web::Query<PictrsParams>,
req: HttpRequest,
client: web::Data<ClientWithMiddleware>,
context: web::Data<LemmyContext>,
local_user_view: Option<LocalUserView>,
) -> Result<HttpResponse, Error> {
// block access to images if instance is private and unauthorized, public
Make functions work with both connection and pool (#3420) * a lot * merge * Fix stuff broken by merge * Get rid of repetitive `&mut *context.conn().await?` * Add blank lines under each line with `conn =` * Fix style mistakes (partial) * Revert "Fix style mistakes (partial)" This reverts commit 48a033b87f4fdc1ce14ff86cc019e1c703cd2741. * Revert "Add blank lines under each line with `conn =`" This reverts commit 773a6d3beba2cf89eac75913078b40c4f5190dd4. * Revert "Get rid of repetitive `&mut *context.conn().await?`" This reverts commit d2c6263ea13710177d49b2791278db5ad115fca5. * Use DbConn for CaptchaAnswer methods * DbConn trait * Remove more `&mut *` * Fix stuff * Re-run CI * try to make ci start * fix * fix * Fix api_common::utils * Fix apub::activities::block * Fix apub::api::resolve_object * Fix some things * Revert "Fix some things" This reverts commit 2bf8574bc8333d8d34ca542d61a0a5b50039c24d. * Revert "Fix apub::api::resolve_object" This reverts commit 3e4059aabbe485b2ff060bdeced8ef958ff62832. * Revert "Fix apub::activities::block" This reverts commit 3b02389abd780a7b1b8a2c89e26febdaa6a12159. * Revert "Fix api_common::utils" This reverts commit 7dc73de613a5618fa57eb06450f3699bbcb41254. * Revert "Revert "Fix api_common::utils"" This reverts commit f740f115e5457e83e53cc223e48196a2c47a9975. * Revert "Revert "Fix apub::activities::block"" This reverts commit 2ee206af7c885c10092cf209bf4a5b1d60327866. * Revert "Revert "Fix apub::api::resolve_object"" This reverts commit 96ed8bf2e9dcadae760743929498312334e23d2e. * Fix fetch_local_site_data * Fix get_comment_parent_creator * Remove unused perma deleted text * Fix routes::feeds * Fix lib.rs * Update lib.rs * rerun ci * Attempt to create custom GetConn and RunQueryDsl traits * Start over * Add GetConn trait * aaaa * Revert "aaaa" This reverts commit acc9ca1aed10c39efdd91cefece066e035a1fe80. * Revert "Revert "aaaa"" This reverts commit 443a2a00a56d152bb7eb429efd0d29a78e21b163. * still aaaaaaaaaaaaa * Return to earlier thing Revert "Add GetConn trait" This reverts commit ab4e94aea5bd9d34cbcddf017339131047e75344. * Try to use DbPool enum * Revert "Try to use DbPool enum" This reverts commit e4d1712646a52006b865a1fbe0dcf79976fdb027. * DbConn and DbPool enums (db_schema only fails to compile for tests) * fmt * Make functions take `&mut DbPool<'_>` and make db_schema tests compile * Add try_join_with_pool macro and run fix-clippy on more crates * Fix some errors * I did it * Remove function variants that take connection * rerun ci * rerun ci * rerun ci
2023-07-11 13:09:59 +00:00
let local_site = LocalSite::read(&mut context.pool())
2022-11-09 10:05:00 +00:00
.await
.map_err(error::ErrorBadRequest)?;
if local_site.private_instance && local_user_view.is_none() {
return Ok(HttpResponse::Unauthorized().finish());
}
2020-10-14 16:48:10 +00:00
let name = &filename.into_inner();
2020-10-14 16:48:10 +00:00
// If there are no query params, the URL is original
let pictrs_config = context.settings().pictrs_config()?;
2020-10-14 16:48:10 +00:00
let url = if params.format.is_none() && params.thumbnail.is_none() {
format!("{}image/original/{}", pictrs_config.url, name,)
2020-10-14 16:48:10 +00:00
} else {
// Take file type from name, or jpg if nothing is given
let format = params
.format
.unwrap_or_else(|| name.split('.').last().unwrap_or("jpg").to_string());
2020-10-14 16:48:10 +00:00
let mut url = format!("{}image/process.{}?src={}", pictrs_config.url, format, name,);
2020-10-14 16:48:10 +00:00
if let Some(size) = params.thumbnail {
url = format!("{url}&thumbnail={size}",);
2020-10-14 16:48:10 +00:00
}
url
};
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
image(url, req, &client).await
}
async fn image(
url: String,
req: HttpRequest,
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
client: &ClientWithMiddleware,
) -> Result<HttpResponse, Error> {
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
let mut client_req = adapt_request(&req, client, url);
if let Some(addr) = req.head().peer_addr {
client_req = client_req.header("X-Forwarded-For", addr.to_string());
}
if let Some(addr) = req.head().peer_addr {
client_req = client_req.header("X-Forwarded-For", addr.to_string());
}
let res = client_req.send().await.map_err(error::ErrorBadRequest)?;
if res.status() == StatusCode::NOT_FOUND {
return Ok(HttpResponse::NotFound().finish());
}
let mut client_res = HttpResponse::build(res.status());
for (name, value) in res.headers().iter().filter(|(h, _)| *h != "connection") {
client_res.insert_header((name.clone(), value.clone()));
}
Ok(client_res.body(BodyStream::new(res.bytes_stream())))
}
async fn delete(
components: web::Path<(String, String)>,
req: HttpRequest,
client: web::Data<ClientWithMiddleware>,
context: web::Data<LemmyContext>,
// require login
_local_user_view: LocalUserView,
) -> Result<HttpResponse, Error> {
let (token, file) = components.into_inner();
let pictrs_config = context.settings().pictrs_config()?;
let url = format!("{}image/delete/{}/{}", pictrs_config.url, &token, &file);
let mut client_req = adapt_request(&req, &client, url);
if let Some(addr) = req.head().peer_addr {
client_req = client_req.header("X-Forwarded-For", addr.to_string());
}
let res = client_req.send().await.map_err(error::ErrorBadRequest)?;
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
LocalImage::delete_by_alias(&mut context.pool(), &file)
.await
.map_err(error::ErrorBadRequest)?;
Ok(HttpResponse::build(res.status()).body(BodyStream::new(res.bytes_stream())))
}
Rewrite images to use local proxy (#4035) * Add markdown rule to add rel=nofollow for all links * Add markdown image rule to add local image proxy (fixes #1036) * comments * rewrite markdown image links working * add comment * perform markdown image processing in api/apub receivers * clippy * add db table to validate proxied links * rewrite link fields for avatar, banner etc * sql fmt * proxy links received over federation * add config option * undo post.url rewriting, move http route definition * add tests * proxy images through pictrs * testing * cleanup request.rs file * more cleanup (fixes #2611) * include url content type when sending post over apub (fixes #2611) * store post url content type in db * should be media_type * get rid of cache_remote_thumbnails setting, instead automatically take thumbnail from federation data if available. * fix tests * add setting disable_external_link_previews * federate post url as image depending on mime type * change setting again * machete * invert * support custom emoji * clippy * update defaults * add image proxy test, fix test * fix test * clippy * revert accidental changes * address review * clippy * Markdown link rule-dess (#4356) * Extracting opengraph_data to its own type. * A few additions for markdown-link-rule. --------- Co-authored-by: Nutomic <me@nutomic.com> * fix setting * use enum for image proxy setting * fix test configs * add config backwards compat * clippy * machete --------- Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 14:22:11 +00:00
#[derive(Deserialize)]
pub struct ImageProxyParams {
url: String,
}
pub async fn image_proxy(
Query(params): Query<ImageProxyParams>,
context: web::Data<LemmyContext>,
) -> LemmyResult<HttpResponse> {
let url = Url::parse(&decode(&params.url)?)?;
// Check that url corresponds to a federated image so that this can't be abused as a proxy
// for arbitrary purposes.
RemoteImage::validate(&mut context.pool(), url.clone().into()).await?;
let pictrs_config = context.settings().pictrs_config()?;
let url = format!("{}image/original?proxy={}", pictrs_config.url, &params.url);
let image_response = context.client().get(url).send().await?;
Ok(HttpResponse::Ok().streaming(image_response.bytes_stream()))
}
fn make_send<S>(mut stream: S) -> impl Stream<Item = S::Item> + Send + Unpin + 'static
where
S: Stream + Unpin + 'static,
S::Item: Send,
{
// NOTE: the 8 here is arbitrary
let (tx, rx) = tokio::sync::mpsc::channel(8);
// NOTE: spawning stream into a new task can potentially hit this bug:
// - https://github.com/actix/actix-web/issues/1679
//
// Since 4.0.0-beta.2 this issue is incredibly less frequent. I have not personally reproduced it.
// That said, it is still technically possible to encounter.
actix_web::rt::spawn(async move {
while let Some(res) = stream.next().await {
if tx.send(res).await.is_err() {
break;
}
}
});
SendStream { rx }
}
struct SendStream<T> {
rx: tokio::sync::mpsc::Receiver<T>,
}
impl<T> Stream for SendStream<T>
where
T: Send,
{
type Item = T;
fn poll_next(
mut self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
std::pin::Pin::new(&mut self.rx).poll_recv(cx)
}
}