mirrors/pict-rs
1
0
Fork 0
mirror of https://git.asonix.dog/asonix/pict-rs.git synced 2024-12-04 07:16:29 +00:00
Code Issues Projects Releases Packages Wiki Activity
848 commits 27 branches 313 tags 6 MiB
Commit graph

388 commits

Author SHA1 Message Date
asonix
8d35c2449d Simplify the cancel-safe processor 2023-07-22 10:46:16 -05:00
asonix
c1127b4117 Set content-length in PutObject, CompleteMultipart 2023-07-21 19:45:22 -05:00
asonix
2f0a3618d8 Replace awc with reqwest 2023-07-21 16:58:31 -05:00
asonix
fc14ac2038 Clippy 2023-07-18 21:59:27 -05:00
asonix
3d93300ba9 Add access repos for alias & identifier 2023-07-18 21:56:13 -05:00
asonix
127dd6cdad Make max file count configurable 2023-07-18 20:32:17 -05:00
asonix
eeac900d7e Add quality value to process method, move -coalesce earlier in pipelines 2023-07-18 16:18:01 -05:00
asonix
558605381d Fix dangling unprocessed uploads 
Adds error boundary around backgrounded ingest
 2023-07-17 22:32:08 -05:00
asonix
496bcc5dc1 Only include -b:v 0 for VP9 
also simplify building arguments
 2023-07-17 18:13:50 -05:00
asonix
3aecef3a9e Remove println 2023-07-17 17:47:48 -05:00
asonix
b1bbc6b159 Attempt adding quality settings to pict-rs 2023-07-17 17:45:26 -05:00
asonix
c4c920191f Add read-only mode 2023-07-17 14:24:49 -05:00
asonix
dd1d509bb1 Enable configuring download client timeout 2023-07-17 13:44:31 -05:00
asonix
5e8ab7856d Extract Status errors into command-specific errors 2023-07-17 13:30:08 -05:00
asonix
2961aae6e3 Port migration changes from 0.4.1 2023-07-16 22:07:42 -05:00
asonix
a751d92436 Port error changes to 0.5 2023-07-16 21:51:14 -05:00
asonix
9871ad2ee0 Clippy 2023-07-14 23:16:57 -05:00
asonix
4d78ce7689 Fix store delegation for public_url 2023-07-14 15:23:07 -05:00
asonix
786f583d98 Add public_url to Store for optional redirects to public object store urls 
Add details generation to file migration to set the content-type for uploads
 2023-07-14 14:53:37 -05:00
asonix
4e58388b41 Clean alias before checking hash option 2023-07-13 19:58:31 -05:00
asonix
afeac8294a Fix Alias cleanup ordering 
Update dev.toml
 2023-07-13 19:57:09 -05:00
asonix
ec6c9aac1e Better discovery validation, proper invoking of imagemagick 2023-07-13 19:21:57 -05:00
asonix
9b1dd1f8c3 Copy process from 0.4.x branch 2023-07-13 19:21:28 -05:00
asonix
2c22f2ee3a It compiles and runs, but doesn't work 2023-07-13 17:42:21 -05:00
asonix
27451971a6 Fix tests, add animated_webp test 2023-07-13 14:34:40 -05:00
asonix
33b83f97f2 It compiles 
Need to reinstate media limits
 2023-07-13 13:48:59 -05:00
asonix
ad1837f9dd VERY BROKEN: start replacing parts of pict-rs 2023-07-12 22:12:21 -05:00
asonix
58d9765594 WIP: Rework format detection and conversions 
- Add a bunch of types & methods that aren't used yet. The idea is
  creating the output from the known input parameters is pure, and works
  for any uploaded media
- Introduce a new kind of media "animation" which is neither an image
  nor a video. It is likely that animations can be processed properly at
  the /image/process.{ext} endpoint with a little massaging

TODO:
- Integrate this into exiftool, ffmpeg, magick commands
- Detect apng with ffmpeg (imagemagick treats it as a still png unless it's given a
    hint)
- Infallible conversion from Details to OutputFile - this might be
  impossible given that we don't currently store more than a mime type
  and a frame count to give any sort of indication of real type
- Try not to break API
 2023-07-11 23:11:25 -05:00
asonix
691bca286c Make object storage timeouts configurable 2023-07-11 13:01:58 -05:00
asonix
f5c39f9be5 Typo, error formatting 2023-07-10 17:15:43 -05:00
asonix
4be2eb67d1 Increase object storage signature expiry to 15 seconds 2023-07-10 16:28:51 -05:00
asonix
9004ecaadf Better classify process related errors 2023-07-10 15:29:41 -05:00
asonix
e10cb1401d Bump version 2023-07-09 15:16:04 -05:00
asonix
7dadbcf3ed Replace UnsupportedFormat error with more helpful errors 2023-07-09 15:07:49 -05:00
asonix
1694f49436 Use json for ffmpeg, imagemagick details parsing 2023-07-09 14:50:58 -05:00
asonix
686be1d93b Turn export_path into flag for sled 2023-07-08 19:56:26 -05:00
asonix
7768d4e58e Add database exports 
This should help admins running pict-rs on traditional filesystems to
keep valid backups of pict-rs' sled repo
 2023-07-08 17:36:01 -05:00
asonix
4680f9c3aa Instrument repo and store build methods 2023-07-08 10:42:03 -05:00
asonix
d51c348a3f Run health checks on old and new stores before migrating 2023-07-07 14:05:13 -05:00
asonix
3ccb8ecd8c Validate alias represents real file before setting not found 2023-07-07 13:33:27 -05:00
asonix
9e7376d411 Remove is_missing error, make Identifier and DeleteToken return Option on fetch 2023-07-07 13:17:26 -05:00
asonix
7274538b70 Try adding store healthcheck 2023-07-07 12:05:42 -05:00
asonix
c150903980 Expand store differentiation 2023-07-07 11:19:39 -05:00
asonix
75be876d3c Differentiate missing files in old vs new store for migration 2023-07-07 11:03:10 -05:00
asonix
508dd4340b Fix panic when migrating < 100 files, improve resumed migration messaging 2023-07-06 16:06:37 -05:00
asonix
3487cb0e30 Add ability to set 404 image 
Fix imagemagick blur arguments
 2023-07-05 19:07:03 -05:00
asonix
e7e4876908 Explicitly handle cases of invalid aliases 2023-07-05 16:46:44 -05:00
asonix
23e67b9697 Enhance migration handling of missing identifiers 
Queue a cleanup task for hashes that don't have original file identifiers
Be more specific about what field is missing when a field is missing
 2023-07-05 09:52:19 -05:00
asonix
2274dfecb4 Fix io-uring - read_to_end was polling Stream mapped through StreamReader after completion 2023-07-05 00:35:07 -05:00
asonix
c8e84b8914 Greatly improve error information during migration 2023-07-02 21:07:15 -05:00
asonix
e6f17cbac6 Log on successful migration 2023-06-29 11:45:06 -05:00
asonix
01d3610f46 Add basic progress indication for store migration 2023-06-29 11:39:47 -05:00
asonix
0d1df5ebc2 Avoid failed identifier cleanup on failed duplicate upload 2023-06-23 11:43:15 -05:00
asonix
7ba2fc9b26 Fix duplicate uploads potentially purging existing uploads on failure 2023-06-23 11:39:43 -05:00
asonix
58f0c328d9 Fix typo in error message 2023-06-23 11:20:55 -05:00
asonix
a1d48cbb42 Share client between object store and request extractor 
Add config to limit connection pool size
 2023-06-23 11:20:20 -05:00
asonix
071d788349 Include original error in object storage sendrequest error 2023-06-21 17:07:14 -05:00
asonix
a3ff072350 Remove unused cache duration parameter 2023-06-21 17:05:53 -05:00
asonix
6d2aef8cc0 Add jpegxl and avif support 2023-06-21 17:05:35 -05:00
asonix
554d852e68 Create unique errors for Repo and Store, separate from UploadError 
Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound
Allow skipping files that are not found during store migration
 2023-06-20 15:59:08 -05:00
asonix
1214b51ad7 Make use of FilesystemDefaults when constructing migrate commands 2023-06-19 15:04:36 -05:00
asonix
a3a986638d Improve gif handling. 
- Allow a fast-path exif cleaning if we'd re-encode a gif to a gif
- Use single-frame palettes to better map transparency from frame to frame

Unrelated:
- decrease ffmpeg logging when creating thumbnails
 2023-06-19 14:25:31 -05:00
asonix
22e526fd2a Allow type underscore (probably in instrument) 2023-06-01 17:33:43 -05:00
asonix
cc314c37a6 Use libopus for ffmpeg 2023-03-09 21:29:56 -06:00
asonix
dc27200a35 Add transparency to gifs, rustfmt 2023-02-25 14:31:57 -06:00
asonix
97ba4e806b Clippy 2023-02-25 13:01:55 -06:00
asonix
9a2f846cd1 Detect input transparency for gif/webm 2023-02-25 12:59:47 -06:00
asonix
281ac43dff Remove cache functionality 2023-02-25 11:34:48 -06:00
asonix
123488f617 clippy 2023-02-12 23:18:27 -06:00
asonix
788b87eceb Remove obsolute pre-0.3 migrate check 2023-02-12 23:13:47 -06:00
asonix
138499d47f Use transpose in a couple places 2023-02-05 20:50:59 -06:00
asonix
cc75d2d882 Clippy 2023-02-04 17:54:56 -06:00
asonix
4cc810d372 Add control over gif frame count 2023-02-04 17:52:23 -06:00
asonix
88ca9793e8 s/vrames/frames:v 2023-02-04 17:47:03 -06:00
asonix
40f57be0c7 Allow uploading small gifs 2023-02-04 17:32:36 -06:00
asonix
0aa3f574a5 Clippy 2023-01-29 11:57:59 -06:00
asonix
f6d6d54b88 Update deps 2023-01-29 11:47:28 -06:00
asonix
f0f40db8c3 Add healthcheck for db 2023-01-29 11:36:09 -06:00
asonix
5e5dd27a05 Retry migrate_file, remove variant before relating new one 2023-01-04 18:58:05 -06:00
asonix
28f7a139a0 Retry in-process, and rate-limit self 2023-01-04 17:26:05 -06:00
asonix
c55b1115d1 Increase client timeout, attempt to keep track of migration progress for resuming 2023-01-04 15:58:32 -06:00
asonix
dabcc24f98 Parse environment variables - main app 2023-01-04 15:05:34 -06:00
asonix
a489411c6c Parse environment variables 2023-01-04 14:42:04 -06:00
asonix
3c844d86c5 Allow configuring pict-rs through serializable types 2022-12-26 16:35:25 -06:00
asonix
2c1a37fb62 Extract pict-rs as library 2022-12-23 12:56:15 -06:00
asonix
74cd3392fa Borrow parent spans when creating children 2022-12-07 22:43:12 -06:00
asonix
3325b222a2 clippy 2022-11-26 11:18:34 -06:00
asonix
7037ee5791 Support auto-orienting photos 2022-10-15 11:13:24 -05:00
asonix
669b3fb86f Fix webp metadata stripping, more tracing cleanup 2022-10-01 22:47:52 -05:00
asonix
718f09c43a Clean tracing, simplify validation, rename InputFormat -> VideoFormat 2022-10-01 21:17:18 -05:00
asonix
e7cf21f862 Allow concurrency in small object uploads 2022-10-01 13:06:19 -05:00
asonix
fe2a499110 Fix sending upload parts (set size), optimize small file uploads 2022-10-01 13:00:07 -05:00
asonix
0307dc5a3b Introduce BytesStream to simplify aggregating, forwarding bytes 2022-10-01 12:06:33 -05:00
asonix
c767804430 Default to vp9 for uploaded video 2022-10-01 10:17:32 -05:00
asonix
85c5ce955d Clippy 2022-09-30 20:02:46 -05:00
asonix
d86d9a3228 Put proper file extension on videos 2022-09-30 20:00:14 -05:00
asonix
8eb2293808 Enable selecting video and audio codecs for uploaded media 2022-09-30 19:38:11 -05:00
asonix
f4542efcc1 Share ffprobe details code 2022-09-30 17:43:40 -05:00
asonix
3ceaaa391c Update clap 2022-09-28 18:23:41 -05:00
asonix
656b0225af Clippy 2022-09-27 23:23:06 -05:00