mirrors/pict-rs
1
0
Fork 0
mirror of https://git.asonix.dog/asonix/pict-rs.git synced 2024-11-10 19:51:02 +00:00
Code Issues Projects Releases Packages Wiki Activity
1135 commits 26 branches 313 tags 5.6 MiB
Commit graph

113 commits

Author SHA1 Message Date
asonix
88d4cc1a6a Exit after repo migration 2023-08-16 16:34:42 -05:00
asonix
f95e61e402 Expose repo migrations 2023-08-16 16:32:19 -05:00
asonix
ba27a1a223 Build out repo-repo migration 2023-08-16 16:09:40 -05:00
asonix
521cdd5b9d Enforce format on new details 2023-08-16 12:36:18 -05:00
asonix
4d909ba32a Make migration concurrent 2023-08-16 11:47:36 -05:00
asonix
1ee77b19d8 Make Repo object safe. Create migration 2023-08-15 19:19:03 -05:00
asonix
da876fd553 Make it compile 2023-08-14 14:25:19 -05:00
asonix
5d4486fdf4 Finish queue implementation update for sled 2023-08-13 19:47:20 -05:00
asonix
35d0c065e9 Start work on 0.5 db format - initial queuerepo changes 2023-08-13 14:12:38 -05:00
asonix
fee4ed1e3e Add process timeout 2023-08-05 12:41:06 -05:00
asonix
2a59a6492a Bring back hash_alias_key 2023-07-25 20:30:22 -05:00
asonix
bd3975f455 Simplify some repo functions, remove 0.3 migration 2023-07-25 20:08:18 -05:00
asonix
f221887599 Clippy 2023-07-23 16:04:17 -05:00
asonix
aac1bb7bc4 Finish media proxy implementation 2023-07-23 15:45:52 -05:00
asonix
4cca7d0f86 Prepare for proxying media 2023-07-23 12:10:36 -05:00
asonix
75bf425c6e Update alias query, process query to accept proxy url 2023-07-23 11:45:10 -05:00
asonix
7dc8ede8eb Gate cleanup behind read-only config 2023-07-22 20:26:28 -05:00
asonix
1f3298363f Attempt to mark unmarked variants as accessed on launch 2023-07-22 19:55:50 -05:00
asonix
fe1f7c869f Enable cleanup of variants 
Still TODO: doing a first pass to mark variants as accessed on launch
 2023-07-22 19:41:50 -05:00
asonix
47e13ec04e Mark variants as accessed on fetch 2023-07-22 18:50:04 -05:00
asonix
ce0df080f4 Add prometheus metrics 2023-07-22 16:47:59 -05:00
asonix
d661732fb4 Tidy imports 2023-07-22 12:47:32 -05:00
asonix
a390b21df1 Make running pict-rs fluent 2023-07-22 12:39:39 -05:00
asonix
3ca994ee31 Replace global config with passed-around config 
This will enable spinning up pict-rs multiple times in the same process with different configurations
 2023-07-22 12:31:01 -05:00
asonix
ac48003f45 Extract ProcessMap out of static 2023-07-22 11:16:19 -05:00
asonix
2f0a3618d8 Replace awc with reqwest 2023-07-21 16:58:31 -05:00
asonix
127dd6cdad Make max file count configurable 2023-07-18 20:32:17 -05:00
asonix
eeac900d7e Add quality value to process method, move -coalesce earlier in pipelines 2023-07-18 16:18:01 -05:00
asonix
c4c920191f Add read-only mode 2023-07-17 14:24:49 -05:00
asonix
dd1d509bb1 Enable configuring download client timeout 2023-07-17 13:44:31 -05:00
asonix
2961aae6e3 Port migration changes from 0.4.1 2023-07-16 22:07:42 -05:00
asonix
9871ad2ee0 Clippy 2023-07-14 23:16:57 -05:00
asonix
786f583d98 Add public_url to Store for optional redirects to public object store urls 
Add details generation to file migration to set the content-type for uploads
 2023-07-14 14:53:37 -05:00
asonix
2c22f2ee3a It compiles and runs, but doesn't work 2023-07-13 17:42:21 -05:00
asonix
33b83f97f2 It compiles 
Need to reinstate media limits
 2023-07-13 13:48:59 -05:00
asonix
ad1837f9dd VERY BROKEN: start replacing parts of pict-rs 2023-07-12 22:12:21 -05:00
asonix
58d9765594 WIP: Rework format detection and conversions 
- Add a bunch of types & methods that aren't used yet. The idea is
  creating the output from the known input parameters is pure, and works
  for any uploaded media
- Introduce a new kind of media "animation" which is neither an image
  nor a video. It is likely that animations can be processed properly at
  the /image/process.{ext} endpoint with a little massaging

TODO:
- Integrate this into exiftool, ffmpeg, magick commands
- Detect apng with ffmpeg (imagemagick treats it as a still png unless it's given a
    hint)
- Infallible conversion from Details to OutputFile - this might be
  impossible given that we don't currently store more than a mime type
  and a frame count to give any sort of indication of real type
- Try not to break API
 2023-07-11 23:11:25 -05:00
asonix
691bca286c Make object storage timeouts configurable 2023-07-11 13:01:58 -05:00
asonix
e10cb1401d Bump version 2023-07-09 15:16:04 -05:00
asonix
7dadbcf3ed Replace UnsupportedFormat error with more helpful errors 2023-07-09 15:07:49 -05:00
asonix
7768d4e58e Add database exports 
This should help admins running pict-rs on traditional filesystems to
keep valid backups of pict-rs' sled repo
 2023-07-08 17:36:01 -05:00
asonix
d51c348a3f Run health checks on old and new stores before migrating 2023-07-07 14:05:13 -05:00
asonix
3ccb8ecd8c Validate alias represents real file before setting not found 2023-07-07 13:33:27 -05:00
asonix
9e7376d411 Remove is_missing error, make Identifier and DeleteToken return Option on fetch 2023-07-07 13:17:26 -05:00
asonix
7274538b70 Try adding store healthcheck 2023-07-07 12:05:42 -05:00
asonix
c150903980 Expand store differentiation 2023-07-07 11:19:39 -05:00
asonix
75be876d3c Differentiate missing files in old vs new store for migration 2023-07-07 11:03:10 -05:00
asonix
508dd4340b Fix panic when migrating < 100 files, improve resumed migration messaging 2023-07-06 16:06:37 -05:00
asonix
3487cb0e30 Add ability to set 404 image 
Fix imagemagick blur arguments
 2023-07-05 19:07:03 -05:00
asonix
e7e4876908 Explicitly handle cases of invalid aliases 2023-07-05 16:46:44 -05:00
asonix
23e67b9697 Enhance migration handling of missing identifiers 
Queue a cleanup task for hashes that don't have original file identifiers
Be more specific about what field is missing when a field is missing
 2023-07-05 09:52:19 -05:00
asonix
c8e84b8914 Greatly improve error information during migration 2023-07-02 21:07:15 -05:00
asonix
e6f17cbac6 Log on successful migration 2023-06-29 11:45:06 -05:00
asonix
01d3610f46 Add basic progress indication for store migration 2023-06-29 11:39:47 -05:00
asonix
a1d48cbb42 Share client between object store and request extractor 
Add config to limit connection pool size
 2023-06-23 11:20:20 -05:00
asonix
554d852e68 Create unique errors for Repo and Store, separate from UploadError 
Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound
Allow skipping files that are not found during store migration
 2023-06-20 15:59:08 -05:00
asonix
281ac43dff Remove cache functionality 2023-02-25 11:34:48 -06:00
asonix
f0f40db8c3 Add healthcheck for db 2023-01-29 11:36:09 -06:00
asonix
5e5dd27a05 Retry migrate_file, remove variant before relating new one 2023-01-04 18:58:05 -06:00
asonix
28f7a139a0 Retry in-process, and rate-limit self 2023-01-04 17:26:05 -06:00
asonix
c55b1115d1 Increase client timeout, attempt to keep track of migration progress for resuming 2023-01-04 15:58:32 -06:00
asonix
3c844d86c5 Allow configuring pict-rs through serializable types 2022-12-26 16:35:25 -06:00
asonix
2c1a37fb62 Extract pict-rs as library 2022-12-23 12:56:15 -06:00