Commit graph

499 commits

Author SHA1 Message Date
asonix 735fcd1b49 Remove dependency on actix-web-codegen 2023-09-24 15:52:31 -05:00
asonix ecdbaa9c92 Use diesel-async re-export of Hook 2023-09-24 15:48:19 -05:00
asonix 745e990fa0 Remove direct dependency on actix-rt and actix-server 2023-09-24 15:45:10 -05:00
asonix 0926739d3c Remove once_cell 2023-09-24 15:32:00 -05:00
asonix e0968dcce3 Remove serde-cbor, num_cpus 2023-09-24 15:22:57 -05:00
asonix fd965bb1a5 Allow processing still images into animation formats 2023-09-24 13:20:33 -05:00
asonix f3ef2d870b Generate details during ingest 2023-09-24 12:58:16 -05:00
asonix 8f62b0c325 clippy 2023-09-24 12:42:23 -05:00
asonix 9ffdd0432f Infer video thumbnail format from image format 2023-09-24 12:37:43 -05:00
asonix 6c921817e1 Enable thumbnailing animations, use dynamic dispatch for a number of async readers 2023-09-24 11:54:16 -05:00
asonix cde4a72203 Add metrics to object storage body streams 2023-09-20 19:46:10 -05:00
asonix 49d3037358 Add metrics to object storage requests 2023-09-20 19:39:03 -05:00
asonix 7df6f3906e Add timeout, metrics to external validation request 2023-09-20 19:29:19 -05:00
asonix d0646dee25 Add config option for external validation timeout 2023-09-20 19:23:55 -05:00
asonix 4a5b9712aa Combine map_ok and map_err underlying impl 2023-09-10 23:51:07 -04:00
asonix 2b10c48619 Update io-uring to use streem for file bytes 2023-09-10 23:45:24 -04:00
asonix ea75ca24b5 Add track_caller to sync methods, update streem 2023-09-10 20:43:51 -04:00
asonix b2674f06d0 More streme 2023-09-10 20:08:01 -04:00
asonix 1b97ac1c5a Start replacing manual stream implementations with streem 2023-09-10 18:55:13 -04:00
asonix 858899b943 Add metrics & tracing to dropped uploads, turn pop into a single query 2023-09-07 19:20:41 -05:00
asonix 783df60f81 Fix on_conflict for postgres store migration path 2023-09-06 20:49:00 -05:00
asonix a761daca34 clippy 2023-09-05 20:55:11 -05:00
asonix e15a82c0c7 Add external validation check 2023-09-05 20:45:07 -05:00
asonix bc49f8ca37 Migrate with timestamp 2023-09-04 22:25:11 -05:00
asonix 9c50bbe23a match -> matches! 2023-09-04 22:06:28 -05:00
asonix 492b99922b 10 second worker delay after disconnect error 2023-09-04 22:04:15 -05:00
asonix 62ebc1c60a Move timeout after metrics - timeouts count as incomplete 2023-09-04 22:02:59 -05:00
asonix 3bd0f78e75 Use with_timeout for all timeouts 2023-09-04 21:58:57 -05:00
asonix a4b1ab7dfb Instrument postgres db calls 2023-09-04 21:51:27 -05:00
asonix 37448722de postgres: Rework job & upload notifications (more)
postgres: Add metrics to job push & pop, upload wait
sled: add upload wait metrics
2023-09-04 16:20:26 -05:00
asonix 00aa00c55c postgres: Enable 'previous page' when less than full limit fit on previous page 2023-09-03 21:58:31 -05:00
asonix a43de122f9 postgres: add already-claimed case, general: tracing paranoia 2023-09-03 21:30:47 -05:00
asonix 31caea438e Fix slow connection pool access 2023-09-03 20:05:29 -05:00
asonix 8a892ba622 it might work 2023-09-03 18:21:46 -05:00
asonix a2933dbebc Implement all the todos 2023-09-03 17:11:34 -05:00
asonix 94cb2a9ef3 Pass clippy 2023-09-03 16:59:41 -05:00
asonix 443d327edf Implement a couple more repo traits 2023-09-03 12:47:06 -05:00
asonix eac4cd54a4 Initial work for pg notifications 2023-09-02 20:13:32 -05:00
asonix e580e7701e Finish implementing HashRepo 2023-09-02 19:13:00 -05:00
asonix 8921f57a21 Remove Identifier trait, Replace with Arc<str> 2023-09-02 18:38:02 -05:00
asonix 8eb4cda256 Implement first query 2023-09-02 13:52:15 -05:00
asonix d475723087 Finish writing migrations & generate diesel schema 2023-09-02 13:35:30 -05:00
asonix 8c532c97e6 Initial postgres work 2023-09-02 11:52:55 -05:00
asonix ba3a23ed43 Add error codes 2023-09-01 20:51:01 -05:00
asonix fd74161c61 Rename some repo methods, generate UploadId in repo 2023-09-01 18:41:04 -05:00
asonix cbb66f1b75 Add internal endpoint for deleting alias without a delete token 2023-09-01 17:25:13 -05:00
asonix 8fb90a6f69 Give a meaningful distinction for format in details 2023-08-31 16:26:45 -05:00
asonix dce0827099 Increase default animation bounds, remove references to converting animations to videos 2023-08-30 21:55:53 -05:00
asonix 0b70ca49e4 clippy 2023-08-30 21:00:15 -05:00
asonix 721d9e1706 Don't try parsing an internal format for OldDetails 2023-08-30 20:55:38 -05:00
asonix 92ee60549f Enable copying input av1 stream from mp4 to webm 2023-08-30 20:40:08 -05:00
asonix b48a9233b2 Remove transcode from animation to video, make video transcoding 'optional'
Video transcoding still happens, but in many cases the video stream is able to be copied verbatim rather than being decoded & encoded
2023-08-30 20:37:54 -05:00
asonix 08fd96c2f7 clippy nit 2023-08-29 13:29:40 -05:00
asonix c0f6b1e988 Enable searching hashes by date 2023-08-29 13:27:18 -05:00
asonix 0146202236 Fully commit to HumanDate 2023-08-29 12:59:36 -05:00
asonix 64950bfe0e Shorten pagination slugs 2023-08-29 12:53:29 -05:00
asonix 1271ff2cc7 Clippy 2023-08-28 18:49:30 -05:00
asonix 2b6ea97773 Enable setting timestamp on hash creation, improve paging api 2023-08-28 18:43:24 -05:00
asonix 24812afeba Add paged hash access 2023-08-28 16:02:11 -05:00
asonix e302ab1f3d Migrate not-found alias 2023-08-28 13:40:19 -05:00
asonix b28cc38c38 Simplify Streamer 2023-08-23 19:12:59 -05:00
asonix 95637fdfe5 Remove direct dep on futures-util 2023-08-23 19:10:10 -05:00
asonix 8f50a15b25 Reduce reliance on futures-util 2023-08-23 11:59:42 -05:00
asonix 0352ada9d8 Don't doulbe-open repo 2023-08-16 16:39:41 -05:00
asonix 88d4cc1a6a Exit after repo migration 2023-08-16 16:34:42 -05:00
asonix f95e61e402 Expose repo migrations 2023-08-16 16:32:19 -05:00
asonix ba27a1a223 Build out repo-repo migration 2023-08-16 16:09:40 -05:00
asonix 1559d57f0a Don't overwrite existing variants 2023-08-16 15:12:16 -05:00
asonix 22cfbe979d Note how many failures have occurred for a given hash 2023-08-16 13:29:22 -05:00
asonix ee88823d41 clippy 2023-08-16 12:43:12 -05:00
asonix 521cdd5b9d Enforce format on new details 2023-08-16 12:36:18 -05:00
asonix f3c6239979 Allow skipping details generation if already generated 2023-08-16 11:57:09 -05:00
asonix 4d909ba32a Make migration concurrent 2023-08-16 11:47:36 -05:00
asonix ac996ff832 Remove unused error 2023-08-15 21:25:04 -05:00
asonix 1de257bb07 Ensure access values are unique 2023-08-15 21:18:25 -05:00
asonix bd3b668d81 Don't error on duplicate - happens on retried migration 2023-08-15 20:28:42 -05:00
asonix 9fe7410dd6 Add logging to migration 2023-08-15 19:33:19 -05:00
asonix 1ee77b19d8 Make Repo object safe. Create migration 2023-08-15 19:19:03 -05:00
asonix 09236d731d Fix poll after completion, misused compare_and_swap 2023-08-14 21:17:57 -05:00
asonix 26ca3a7195 Remove associated Bytes type 2023-08-14 20:00:00 -05:00
asonix da876fd553 Make it compile 2023-08-14 14:25:19 -05:00
asonix 3129f7844e BROKEN: start work on hash discriminant 2023-08-13 22:06:42 -05:00
asonix e4e93cddf9 Return JobId from push 2023-08-13 19:52:27 -05:00
asonix 5d4486fdf4 Finish queue implementation update for sled 2023-08-13 19:47:20 -05:00
asonix 35d0c065e9 Start work on 0.5 db format - initial queuerepo changes 2023-08-13 14:12:38 -05:00
asonix 0c4d632dda Consider timeouts a client error 2023-08-05 16:18:06 -05:00
asonix fee4ed1e3e Add process timeout 2023-08-05 12:41:06 -05:00
asonix 5f12be0c6a Clippy 2023-07-26 22:53:41 -05:00
asonix 2a59a6492a Bring back hash_alias_key 2023-07-25 20:30:22 -05:00
asonix bd3975f455 Simplify some repo functions, remove 0.3 migration 2023-07-25 20:08:18 -05:00
asonix f221887599 Clippy 2023-07-23 16:04:17 -05:00
asonix aac1bb7bc4 Finish media proxy implementation 2023-07-23 15:45:52 -05:00
asonix 4cca7d0f86 Prepare for proxying media 2023-07-23 12:10:36 -05:00
asonix 75bf425c6e Update alias query, process query to accept proxy url 2023-07-23 11:45:10 -05:00
asonix e59483c12c Add ProxyRepo 2023-07-23 10:23:46 -05:00
asonix d0c619bdf7 Clippy 2023-07-22 21:14:24 -05:00
asonix 77cdeab57e Improve metrics, add job metrics 2023-07-22 21:11:28 -05:00
asonix 7dc8ede8eb Gate cleanup behind read-only config 2023-07-22 20:26:28 -05:00
asonix 13d6e10886 Reduce log level on variants access 2023-07-22 20:20:58 -05:00
asonix 1f3298363f Attempt to mark unmarked variants as accessed on launch 2023-07-22 19:55:50 -05:00