Commit graph

631 commits

Author SHA1 Message Date
asonix c17a8722c6 Simplify object and file path generation 2024-02-26 15:43:30 -06:00
asonix 7c6112e631 ffprobe: read from file, simplify input-file access 2024-02-25 13:05:47 -06:00
asonix 277b47af46 Fix imagemagick ffmpeg delegate, strip metadata while converting videos 2024-02-25 11:40:03 -06:00
asonix b139190663 clippy 2024-02-24 19:36:29 -06:00
asonix 03bd3cbe2f Even more piping 2024-02-24 19:27:34 -06:00
asonix 13fc0df31a More piping 2024-02-24 19:07:48 -06:00
asonix 2074334131 Switch more commands to be driven via STDIN 2024-02-24 14:02:41 -06:00
asonix eabd7ea228 Delete unused functions 2024-02-24 13:21:35 -06:00
asonix ad51e6cd9f Avoid writing blurhashed images to tmp 2024-02-24 13:16:25 -06:00
asonix 0fd19a5682 Start work on blurhash 2024-02-23 22:12:19 -06:00
asonix 6a6c61058a Split save_stream for object storage to early-drop stack-pinned stream 2024-02-22 18:55:12 -06:00
asonix d73e683d48 Remove unused From 2024-02-22 18:23:39 -06:00
asonix 16bf18bda4 Remove some Unpin and 'static bounds 2024-02-22 18:05:04 -06:00
asonix 227e9cc3a7 BytesReader: use built-in state tracking 2024-02-22 17:49:16 -06:00
asonix 59b03d548d Fix is_empty 2024-02-22 17:09:03 -06:00
asonix f3e455a1c3 Increase buffer size for AsyncRead -> Stream conversion 2024-02-22 16:25:03 -06:00
asonix 00a08a8bc9 Improve AyncRead impl 2024-02-22 16:21:31 -06:00
asonix de356c1f12 Remove MessageBody impl 2024-02-22 16:15:32 -06:00
asonix 3a7d5b7bfb Re-use try_from_stream where possible 2024-02-22 16:10:34 -06:00
asonix 0ebee2a07c cargo fix 2024-02-22 16:03:49 -06:00
asonix c1e651c01a Use BytesStream in more places 2024-02-22 16:03:04 -06:00
asonix 9db5cc82f8 Add special handling for FileSize errors 2024-02-19 12:55:44 -06:00
asonix 4a7a51d147 Add more imagemagick security configuration
Remove default security policy, since it is overridden anyway
Update default value for max_area
Inherit animation frame count configuration for list-length
Add release document
2024-02-05 14:30:01 -06:00
asonix 71d81681d5 It compiles 2024-02-04 15:45:47 -06:00
asonix d8c4a22ba9 Start describing metrics 2024-02-04 14:26:18 -06:00
asonix 8869b82065 Update rustls for actix-web 2024-02-03 21:58:08 -06:00
asonix 85f6356025 Fix images with trailing bytes failing to upload 2024-02-03 19:50:00 -06:00
asonix 835647d290 Unite launch-with-store fns 2024-02-03 18:42:34 -06:00
asonix c176e4c686 clippy 2024-02-03 18:30:52 -06:00
asonix 880dfc20ee Remove unneeded struct, import 2024-02-03 18:20:17 -06:00
asonix 7eba45f37e cargo fix & cargo fmt 2024-02-03 18:19:14 -06:00
asonix 50e31f96da It compiles again 2024-02-03 18:18:13 -06:00
asonix f2410a9283 use State 2024-02-03 13:32:20 -06:00
asonix cf7c753e65 Pass a State value around rather than a bunch of arguments 2024-02-03 13:31:54 -06:00
asonix c3e322f2c9 typo 2024-02-02 14:00:54 -06:00
asonix 3a6faef5ef Add logs for pict-rs binds 2024-02-01 21:55:31 -06:00
asonix 5805eb0aed Enable configuring imagemagick security policy from pictrs 2024-01-31 21:32:01 -06:00
asonix 50fd3e6182 Enable serving over TLS 2024-01-31 17:47:42 -06:00
asonix 08472eabeb Log console address, prometheus address with tracing 2024-01-30 15:21:15 -06:00
asonix e7357cbdaf Count spawned tasks in cleanup, debug log queue length 2024-01-30 14:57:48 -06:00
asonix 34cadb86b8 Don't propogate pop span into blocking closure 2024-01-30 14:30:24 -06:00
asonix ee00fafee4 Yield more often in queue, simplify traces, log job_id in higher span 2024-01-30 14:18:07 -06:00
asonix dc12f6f79d replace remaining web::block with spawn_blocking 2024-01-30 14:16:05 -06:00
asonix 7282ee9312 Give background jobs more opportunity to yield to runtime 2024-01-25 16:59:46 -06:00
asonix 819b83bab7 Use a far smaller pool to listen for db notifications 2024-01-25 16:50:36 -06:00
asonix c65b49339f Take at most 2048 variants and proxies during each cleanup job 2024-01-25 16:26:29 -06:00
asonix 574b12b23f Fix postgres commandline, add default for use_tls 2024-01-25 08:55:51 -06:00
asonix 2f588aa613 Ignore tokio-uring test in docs 2024-01-24 17:41:42 -06:00
asonix fe1132aec1 Enable at-most-once queueing for some cleanup jobs 2024-01-24 17:14:31 -06:00
asonix fee2294901 clippy 2024-01-15 18:21:36 -05:00
asonix 19147e2035 postgres: allow connecting to TLS-enabled databases 2024-01-15 18:11:08 -05:00
asonix f6087d65be Use async fn in trait for Store 2024-01-15 15:51:20 -05:00
asonix 9ed90efed4 Add release document, use .extend rather than .join 2024-01-11 16:48:18 -06:00
asonix 708c25f20a Join identifier onto public_endpoint, rather than overwriting the existing path 2024-01-11 16:06:44 -06:00
asonix 9ceb26bb5c Clippy 2024-01-07 10:09:06 -06:00
asonix 65e165af49 Set default audio codecs if none are specified 2024-01-07 10:03:40 -06:00
asonix ca34a15cd3 Debug-log command arguments 2024-01-07 10:01:02 -06:00
asonix 004a54c4ae Improve cooperation from background jobs 2024-01-06 17:47:44 -06:00
asonix bcf73eb4e4 Improve cooperation from from_iterator to prevent task starvation 2024-01-05 19:50:10 -06:00
asonix 4145637a33 Add a few more metrics, increase repo disconnect sleep 2024-01-05 19:35:52 -06:00
asonix 688c797082 Remove LIMIT 10, single_value is already in use 2024-01-04 11:13:29 -06:00
asonix 6514487ea9 Index on queue_time for job queue, limit to 10 jobs in ID query 2024-01-04 11:01:25 -06:00
asonix 0fd6c459d0 Clippy 2024-01-04 10:42:34 -06:00
asonix 7d4f44a540 Set worker-id to None when marking a job as runnable after expired heartbeats 2024-01-04 10:29:37 -06:00
asonix 7e215c691b Support RUST_LOG 2023-12-28 12:40:59 -06:00
asonix bfc2410552 Add trace-level logs to every loop (paranoid) 2023-12-28 11:58:38 -06:00
asonix dc90e00324 Update metrics to 0.22 (plus mechanical changes) 2023-12-26 18:06:38 -06:00
asonix bd3cdc925e Remove special-case for animated avif 2023-12-23 21:44:11 -06:00
asonix 6d259a0eea Don't kill already-exited processes, don't log tmp_dir when fetching details 2023-12-23 13:29:30 -06:00
asonix c2e53766c2 Don't -ping when extracting file details (workaround for imagemagick json bug) 2023-12-23 12:48:32 -06:00
asonix df8fc00ad5 Try to clean up properly even if command errors 2023-12-23 12:11:15 -06:00
asonix 89a4e3995f Add warn logs when performing blocking removes due to dropped files 2023-12-23 12:10:25 -06:00
asonix 6fa79b9188 Do more cleanup inline 2023-12-23 11:58:20 -06:00
asonix e8380c31c1 Fix remaining warnings 2023-12-22 21:03:51 -06:00
asonix c2b1b3db5d Apply cargo fix 2023-12-22 21:01:19 -06:00
asonix c544f167fb Remove unused code 2023-12-22 21:00:37 -06:00
asonix b938b09901 Apply cargo fix 2023-12-22 20:54:02 -06:00
asonix 5624671cbf Rework ProcessRead to require a closure for using the internal AsyncRead 2023-12-22 20:52:58 -06:00
asonix df88660b9e Lower priority of some object storage spans 2023-12-22 13:58:05 -06:00
asonix 4369283534 More tracing cleaning 2023-12-22 13:30:17 -06:00
asonix aa8fe7f3e8 Clean unused imports 2023-12-22 13:20:36 -06:00
asonix 0dbe722524 Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read 2023-12-22 13:20:21 -06:00
asonix b94ba5fcfc Inline process background future, clean tracing a bit 2023-12-22 13:12:19 -06:00
asonix db43392a3b Simplify process future by removing explicit channel, don't lock mutex as often 2023-12-22 12:03:05 -06:00
asonix c9155f7ce7 Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration 2023-12-17 23:15:27 -06:00
asonix bd0b89b165 Clean spans a bit 2023-12-15 22:34:45 -06:00
asonix 2544cb478f Add static bound to appease 1.74.1 2023-12-12 17:17:16 -06:00
asonix f1c5a56353 Port prune_missing from 0.4.x 2023-12-12 16:54:41 -06:00
asonix 3ef4097ceb Enable building without tokio_unstable 2023-12-12 16:23:02 -06:00
asonix 6daa745446 Add optional ability to log spans 2023-12-11 14:58:53 -06:00
asonix cd80ab16c3 Downgrade retry warnings to debug 2023-12-11 13:08:10 -06:00
asonix b452a577a7 Add object ID to not found error 2023-12-11 13:05:29 -06:00
asonix 7293628148 put timeout on collecting payloads 2023-12-10 19:26:16 -06:00
asonix 961de20a00 Add timeout and metrics to media processing 2023-12-10 19:11:36 -06:00
asonix 2523b6644b Add AggregateTimeout, don't aggregate with permit 2023-12-04 15:39:34 -06:00
asonix 6b2ccbf975 Allow configuring pict-rs' temporary directory 2023-11-24 11:48:45 -06:00
asonix a30044278c Improve documentation for using pict-rs as a library 2023-11-24 11:38:14 -06:00
asonix 71f8f52b58 Tweak error reporting 2023-11-13 17:24:56 -06:00
asonix 1538389caf Clippy 2023-11-11 14:28:05 -06:00
asonix 210af5d7d9 Add danger_dummy_mode 2023-11-11 14:22:12 -06:00