Commit graph

636 commits

Author SHA1 Message Date
asonix
25ef3861f1 Remove more unneeded 'static 2024-02-27 20:46:04 -06:00
asonix
6ee7c5c4a0 Remove unneeded 'static 2024-02-27 20:41:39 -06:00
asonix
04dcc9a0c8 Include file extension in identifiers 2024-02-27 20:41:25 -06:00
asonix
d13f7fe969 Include a few random bytes in the directory structure 2024-02-27 20:18:35 -06:00
asonix
16890eaa45 Just keep timestamp component of v7 for directory 2024-02-26 15:58:06 -06:00
asonix
c17a8722c6 Simplify object and file path generation 2024-02-26 15:43:30 -06:00
asonix
7c6112e631 ffprobe: read from file, simplify input-file access 2024-02-25 13:05:47 -06:00
asonix
277b47af46 Fix imagemagick ffmpeg delegate, strip metadata while converting videos 2024-02-25 11:40:03 -06:00
asonix
b139190663 clippy 2024-02-24 19:36:29 -06:00
asonix
03bd3cbe2f Even more piping 2024-02-24 19:27:34 -06:00
asonix
13fc0df31a More piping 2024-02-24 19:07:48 -06:00
asonix
2074334131 Switch more commands to be driven via STDIN 2024-02-24 14:02:41 -06:00
asonix
eabd7ea228 Delete unused functions 2024-02-24 13:21:35 -06:00
asonix
ad51e6cd9f Avoid writing blurhashed images to tmp 2024-02-24 13:16:25 -06:00
asonix
0fd19a5682 Start work on blurhash 2024-02-23 22:12:19 -06:00
asonix
6a6c61058a Split save_stream for object storage to early-drop stack-pinned stream 2024-02-22 18:55:12 -06:00
asonix
d73e683d48 Remove unused From 2024-02-22 18:23:39 -06:00
asonix
16bf18bda4 Remove some Unpin and 'static bounds 2024-02-22 18:05:04 -06:00
asonix
227e9cc3a7 BytesReader: use built-in state tracking 2024-02-22 17:49:16 -06:00
asonix
59b03d548d Fix is_empty 2024-02-22 17:09:03 -06:00
asonix
f3e455a1c3 Increase buffer size for AsyncRead -> Stream conversion 2024-02-22 16:25:03 -06:00
asonix
00a08a8bc9 Improve AyncRead impl 2024-02-22 16:21:31 -06:00
asonix
de356c1f12 Remove MessageBody impl 2024-02-22 16:15:32 -06:00
asonix
3a7d5b7bfb Re-use try_from_stream where possible 2024-02-22 16:10:34 -06:00
asonix
0ebee2a07c cargo fix 2024-02-22 16:03:49 -06:00
asonix
c1e651c01a Use BytesStream in more places 2024-02-22 16:03:04 -06:00
asonix
9db5cc82f8 Add special handling for FileSize errors 2024-02-19 12:55:44 -06:00
asonix
4a7a51d147 Add more imagemagick security configuration
Remove default security policy, since it is overridden anyway
Update default value for max_area
Inherit animation frame count configuration for list-length
Add release document
2024-02-05 14:30:01 -06:00
asonix
71d81681d5 It compiles 2024-02-04 15:45:47 -06:00
asonix
d8c4a22ba9 Start describing metrics 2024-02-04 14:26:18 -06:00
asonix
8869b82065 Update rustls for actix-web 2024-02-03 21:58:08 -06:00
asonix
85f6356025 Fix images with trailing bytes failing to upload 2024-02-03 19:50:00 -06:00
asonix
835647d290 Unite launch-with-store fns 2024-02-03 18:42:34 -06:00
asonix
c176e4c686 clippy 2024-02-03 18:30:52 -06:00
asonix
880dfc20ee Remove unneeded struct, import 2024-02-03 18:20:17 -06:00
asonix
7eba45f37e cargo fix & cargo fmt 2024-02-03 18:19:14 -06:00
asonix
50e31f96da It compiles again 2024-02-03 18:18:13 -06:00
asonix
f2410a9283 use State 2024-02-03 13:32:20 -06:00
asonix
cf7c753e65 Pass a State value around rather than a bunch of arguments 2024-02-03 13:31:54 -06:00
asonix
c3e322f2c9 typo 2024-02-02 14:00:54 -06:00
asonix
3a6faef5ef Add logs for pict-rs binds 2024-02-01 21:55:31 -06:00
asonix
5805eb0aed Enable configuring imagemagick security policy from pictrs 2024-01-31 21:32:01 -06:00
asonix
50fd3e6182 Enable serving over TLS 2024-01-31 17:47:42 -06:00
asonix
08472eabeb Log console address, prometheus address with tracing 2024-01-30 15:21:15 -06:00
asonix
e7357cbdaf Count spawned tasks in cleanup, debug log queue length 2024-01-30 14:57:48 -06:00
asonix
34cadb86b8 Don't propogate pop span into blocking closure 2024-01-30 14:30:24 -06:00
asonix
ee00fafee4 Yield more often in queue, simplify traces, log job_id in higher span 2024-01-30 14:18:07 -06:00
asonix
dc12f6f79d replace remaining web::block with spawn_blocking 2024-01-30 14:16:05 -06:00
asonix
7282ee9312 Give background jobs more opportunity to yield to runtime 2024-01-25 16:59:46 -06:00
asonix
819b83bab7 Use a far smaller pool to listen for db notifications 2024-01-25 16:50:36 -06:00
asonix
c65b49339f Take at most 2048 variants and proxies during each cleanup job 2024-01-25 16:26:29 -06:00
asonix
574b12b23f Fix postgres commandline, add default for use_tls 2024-01-25 08:55:51 -06:00
asonix
2f588aa613 Ignore tokio-uring test in docs 2024-01-24 17:41:42 -06:00
asonix
fe1132aec1 Enable at-most-once queueing for some cleanup jobs 2024-01-24 17:14:31 -06:00
asonix
fee2294901 clippy 2024-01-15 18:21:36 -05:00
asonix
19147e2035 postgres: allow connecting to TLS-enabled databases 2024-01-15 18:11:08 -05:00
asonix
f6087d65be Use async fn in trait for Store 2024-01-15 15:51:20 -05:00
asonix
9ed90efed4 Add release document, use .extend rather than .join 2024-01-11 16:48:18 -06:00
asonix
708c25f20a Join identifier onto public_endpoint, rather than overwriting the existing path 2024-01-11 16:06:44 -06:00
asonix
9ceb26bb5c Clippy 2024-01-07 10:09:06 -06:00
asonix
65e165af49 Set default audio codecs if none are specified 2024-01-07 10:03:40 -06:00
asonix
ca34a15cd3 Debug-log command arguments 2024-01-07 10:01:02 -06:00
asonix
004a54c4ae Improve cooperation from background jobs 2024-01-06 17:47:44 -06:00
asonix
bcf73eb4e4 Improve cooperation from from_iterator to prevent task starvation 2024-01-05 19:50:10 -06:00
asonix
4145637a33 Add a few more metrics, increase repo disconnect sleep 2024-01-05 19:35:52 -06:00
asonix
688c797082 Remove LIMIT 10, single_value is already in use 2024-01-04 11:13:29 -06:00
asonix
6514487ea9 Index on queue_time for job queue, limit to 10 jobs in ID query 2024-01-04 11:01:25 -06:00
asonix
0fd6c459d0 Clippy 2024-01-04 10:42:34 -06:00
asonix
7d4f44a540 Set worker-id to None when marking a job as runnable after expired heartbeats 2024-01-04 10:29:37 -06:00
asonix
7e215c691b Support RUST_LOG 2023-12-28 12:40:59 -06:00
asonix
bfc2410552 Add trace-level logs to every loop (paranoid) 2023-12-28 11:58:38 -06:00
asonix
dc90e00324 Update metrics to 0.22 (plus mechanical changes) 2023-12-26 18:06:38 -06:00
asonix
bd3cdc925e Remove special-case for animated avif 2023-12-23 21:44:11 -06:00
asonix
6d259a0eea Don't kill already-exited processes, don't log tmp_dir when fetching details 2023-12-23 13:29:30 -06:00
asonix
c2e53766c2 Don't -ping when extracting file details (workaround for imagemagick json bug) 2023-12-23 12:48:32 -06:00
asonix
df8fc00ad5 Try to clean up properly even if command errors 2023-12-23 12:11:15 -06:00
asonix
89a4e3995f Add warn logs when performing blocking removes due to dropped files 2023-12-23 12:10:25 -06:00
asonix
6fa79b9188 Do more cleanup inline 2023-12-23 11:58:20 -06:00
asonix
e8380c31c1 Fix remaining warnings 2023-12-22 21:03:51 -06:00
asonix
c2b1b3db5d Apply cargo fix 2023-12-22 21:01:19 -06:00
asonix
c544f167fb Remove unused code 2023-12-22 21:00:37 -06:00
asonix
b938b09901 Apply cargo fix 2023-12-22 20:54:02 -06:00
asonix
5624671cbf Rework ProcessRead to require a closure for using the internal AsyncRead 2023-12-22 20:52:58 -06:00
asonix
df88660b9e Lower priority of some object storage spans 2023-12-22 13:58:05 -06:00
asonix
4369283534 More tracing cleaning 2023-12-22 13:30:17 -06:00
asonix
aa8fe7f3e8 Clean unused imports 2023-12-22 13:20:36 -06:00
asonix
0dbe722524 Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read 2023-12-22 13:20:21 -06:00
asonix
b94ba5fcfc Inline process background future, clean tracing a bit 2023-12-22 13:12:19 -06:00
asonix
db43392a3b Simplify process future by removing explicit channel, don't lock mutex as often 2023-12-22 12:03:05 -06:00
asonix
c9155f7ce7 Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration 2023-12-17 23:15:27 -06:00
asonix
bd0b89b165 Clean spans a bit 2023-12-15 22:34:45 -06:00
asonix
2544cb478f Add static bound to appease 1.74.1 2023-12-12 17:17:16 -06:00
asonix
f1c5a56353 Port prune_missing from 0.4.x 2023-12-12 16:54:41 -06:00
asonix
3ef4097ceb Enable building without tokio_unstable 2023-12-12 16:23:02 -06:00
asonix
6daa745446 Add optional ability to log spans 2023-12-11 14:58:53 -06:00
asonix
cd80ab16c3 Downgrade retry warnings to debug 2023-12-11 13:08:10 -06:00
asonix
b452a577a7 Add object ID to not found error 2023-12-11 13:05:29 -06:00
asonix
7293628148 put timeout on collecting payloads 2023-12-10 19:26:16 -06:00
asonix
961de20a00 Add timeout and metrics to media processing 2023-12-10 19:11:36 -06:00
asonix
2523b6644b Add AggregateTimeout, don't aggregate with permit 2023-12-04 15:39:34 -06:00