505 commits

Author	SHA1	Message	Date
asonix	4bb9d81f51	Move internal middleware into own file	2023-09-30 17:33:01 -05:00
asonix	e3462f6664	payload middleware: switch to Rc, always inject if Payload isn't None	2023-09-30 17:24:48 -05:00
asonix	3267fb8301	Fail requests for malformed deadlines	2023-09-30 17:20:32 -05:00
asonix	65ea518542	Default log level to info	2023-09-30 16:28:04 -05:00
asonix	66e1711723	Enable proper draining of dropped request payloads ... Doing this as the outermost middleware ensures all endpoints are covered. Update request deadline to turn negative deadlines into immediate failures	2023-09-30 16:26:43 -05:00
asonix	6f44b05c0a	Remove references to Arbiter	2023-09-29 11:54:43 -05:00
asonix	735fcd1b49	Remove dependency on actix-web-codegen	2023-09-24 15:52:31 -05:00
asonix	ecdbaa9c92	Use diesel-async re-export of Hook	2023-09-24 15:48:19 -05:00
asonix	745e990fa0	Remove direct dependency on actix-rt and actix-server	2023-09-24 15:45:10 -05:00
asonix	0926739d3c	Remove once_cell	2023-09-24 15:32:00 -05:00
asonix	e0968dcce3	Remove serde-cbor, num_cpus	2023-09-24 15:22:57 -05:00
asonix	fd965bb1a5	Allow processing still images into animation formats	2023-09-24 13:20:33 -05:00
asonix	f3ef2d870b	Generate details during ingest	2023-09-24 12:58:16 -05:00
asonix	8f62b0c325	clippy	2023-09-24 12:42:23 -05:00
asonix	9ffdd0432f	Infer video thumbnail format from image format	2023-09-24 12:37:43 -05:00
asonix	6c921817e1	Enable thumbnailing animations, use dynamic dispatch for a number of async readers	2023-09-24 11:54:16 -05:00
asonix	cde4a72203	Add metrics to object storage body streams	2023-09-20 19:46:10 -05:00
asonix	49d3037358	Add metrics to object storage requests	2023-09-20 19:39:03 -05:00
asonix	7df6f3906e	Add timeout, metrics to external validation request	2023-09-20 19:29:19 -05:00
asonix	d0646dee25	Add config option for external validation timeout	2023-09-20 19:23:55 -05:00
asonix	4a5b9712aa	Combine map_ok and map_err underlying impl	2023-09-10 23:51:07 -04:00
asonix	2b10c48619	Update io-uring to use streem for file bytes	2023-09-10 23:45:24 -04:00
asonix	ea75ca24b5	Add track_caller to sync methods, update streem	2023-09-10 20:43:51 -04:00
asonix	b2674f06d0	More streme	2023-09-10 20:08:01 -04:00
asonix	1b97ac1c5a	Start replacing manual stream implementations with streem	2023-09-10 18:55:13 -04:00
asonix	858899b943	Add metrics & tracing to dropped uploads, turn pop into a single query	2023-09-07 19:20:41 -05:00
asonix	783df60f81	Fix on_conflict for postgres store migration path	2023-09-06 20:49:00 -05:00
asonix	a761daca34	clippy	2023-09-05 20:55:11 -05:00
asonix	e15a82c0c7	Add external validation check	2023-09-05 20:45:07 -05:00
asonix	bc49f8ca37	Migrate with timestamp	2023-09-04 22:25:11 -05:00
asonix	9c50bbe23a	match -> matches!	2023-09-04 22:06:28 -05:00
asonix	492b99922b	10 second worker delay after disconnect error	2023-09-04 22:04:15 -05:00
asonix	62ebc1c60a	Move timeout after metrics - timeouts count as incomplete	2023-09-04 22:02:59 -05:00
asonix	3bd0f78e75	Use with_timeout for all timeouts	2023-09-04 21:58:57 -05:00
asonix	a4b1ab7dfb	Instrument postgres db calls	2023-09-04 21:51:27 -05:00
asonix	37448722de	postgres: Rework job & upload notifications (more) ... postgres: Add metrics to job push & pop, upload wait sled: add upload wait metrics	2023-09-04 16:20:26 -05:00
asonix	00aa00c55c	postgres: Enable 'previous page' when less than full limit fit on previous page	2023-09-03 21:58:31 -05:00
asonix	a43de122f9	postgres: add already-claimed case, general: tracing paranoia	2023-09-03 21:30:47 -05:00
asonix	31caea438e	Fix slow connection pool access	2023-09-03 20:05:29 -05:00
asonix	8a892ba622	it might work	2023-09-03 18:21:46 -05:00
asonix	a2933dbebc	Implement all the todos	2023-09-03 17:11:34 -05:00
asonix	94cb2a9ef3	Pass clippy	2023-09-03 16:59:41 -05:00
asonix	443d327edf	Implement a couple more repo traits	2023-09-03 12:47:06 -05:00
asonix	eac4cd54a4	Initial work for pg notifications	2023-09-02 20:13:32 -05:00
asonix	e580e7701e	Finish implementing HashRepo	2023-09-02 19:13:00 -05:00
asonix	8921f57a21	Remove Identifier trait, Replace with Arc<str>	2023-09-02 18:38:02 -05:00
asonix	8eb4cda256	Implement first query	2023-09-02 13:52:15 -05:00
asonix	d475723087	Finish writing migrations & generate diesel schema	2023-09-02 13:35:30 -05:00
asonix	8c532c97e6	Initial postgres work	2023-09-02 11:52:55 -05:00
asonix	ba3a23ed43	Add error codes	2023-09-01 20:51:01 -05:00
asonix	fd74161c61	Rename some repo methods, generate UploadId in repo	2023-09-01 18:41:04 -05:00
asonix	cbb66f1b75	Add internal endpoint for deleting alias without a delete token	2023-09-01 17:25:13 -05:00
asonix	8fb90a6f69	Give a meaningful distinction for format in details	2023-08-31 16:26:45 -05:00
asonix	dce0827099	Increase default animation bounds, remove references to converting animations to videos	2023-08-30 21:55:53 -05:00
asonix	0b70ca49e4	clippy	2023-08-30 21:00:15 -05:00
asonix	721d9e1706	Don't try parsing an internal format for OldDetails	2023-08-30 20:55:38 -05:00
asonix	92ee60549f	Enable copying input av1 stream from mp4 to webm	2023-08-30 20:40:08 -05:00
asonix	b48a9233b2	Remove transcode from animation to video, make video transcoding 'optional' ... Video transcoding still happens, but in many cases the video stream is able to be copied verbatim rather than being decoded & encoded	2023-08-30 20:37:54 -05:00
asonix	08fd96c2f7	clippy nit	2023-08-29 13:29:40 -05:00
asonix	c0f6b1e988	Enable searching hashes by date	2023-08-29 13:27:18 -05:00
asonix	0146202236	Fully commit to HumanDate	2023-08-29 12:59:36 -05:00
asonix	64950bfe0e	Shorten pagination slugs	2023-08-29 12:53:29 -05:00
asonix	1271ff2cc7	Clippy	2023-08-28 18:49:30 -05:00
asonix	2b6ea97773	Enable setting timestamp on hash creation, improve paging api	2023-08-28 18:43:24 -05:00
asonix	24812afeba	Add paged hash access	2023-08-28 16:02:11 -05:00
asonix	e302ab1f3d	Migrate not-found alias	2023-08-28 13:40:19 -05:00
asonix	b28cc38c38	Simplify Streamer	2023-08-23 19:12:59 -05:00
asonix	95637fdfe5	Remove direct dep on futures-util	2023-08-23 19:10:10 -05:00
asonix	8f50a15b25	Reduce reliance on futures-util	2023-08-23 11:59:42 -05:00
asonix	0352ada9d8	Don't doulbe-open repo	2023-08-16 16:39:41 -05:00
asonix	88d4cc1a6a	Exit after repo migration	2023-08-16 16:34:42 -05:00
asonix	f95e61e402	Expose repo migrations	2023-08-16 16:32:19 -05:00
asonix	ba27a1a223	Build out repo-repo migration	2023-08-16 16:09:40 -05:00
asonix	1559d57f0a	Don't overwrite existing variants	2023-08-16 15:12:16 -05:00
asonix	22cfbe979d	Note how many failures have occurred for a given hash	2023-08-16 13:29:22 -05:00
asonix	ee88823d41	clippy	2023-08-16 12:43:12 -05:00
asonix	521cdd5b9d	Enforce format on new details	2023-08-16 12:36:18 -05:00
asonix	f3c6239979	Allow skipping details generation if already generated	2023-08-16 11:57:09 -05:00
asonix	4d909ba32a	Make migration concurrent	2023-08-16 11:47:36 -05:00
asonix	ac996ff832	Remove unused error	2023-08-15 21:25:04 -05:00
asonix	1de257bb07	Ensure access values are unique	2023-08-15 21:18:25 -05:00
asonix	bd3b668d81	Don't error on duplicate - happens on retried migration	2023-08-15 20:28:42 -05:00
asonix	9fe7410dd6	Add logging to migration	2023-08-15 19:33:19 -05:00
asonix	1ee77b19d8	Make Repo object safe. Create migration	2023-08-15 19:19:03 -05:00
asonix	09236d731d	Fix poll after completion, misused compare_and_swap	2023-08-14 21:17:57 -05:00
asonix	26ca3a7195	Remove associated Bytes type	2023-08-14 20:00:00 -05:00
asonix	da876fd553	Make it compile	2023-08-14 14:25:19 -05:00
asonix	3129f7844e	BROKEN: start work on hash discriminant	2023-08-13 22:06:42 -05:00
asonix	e4e93cddf9	Return JobId from push	2023-08-13 19:52:27 -05:00
asonix	5d4486fdf4	Finish queue implementation update for sled	2023-08-13 19:47:20 -05:00
asonix	35d0c065e9	Start work on 0.5 db format - initial queuerepo changes	2023-08-13 14:12:38 -05:00
asonix	0c4d632dda	Consider timeouts a client error	2023-08-05 16:18:06 -05:00
asonix	fee4ed1e3e	Add process timeout	2023-08-05 12:41:06 -05:00
asonix	5f12be0c6a	Clippy	2023-07-26 22:53:41 -05:00
asonix	2a59a6492a	Bring back hash_alias_key	2023-07-25 20:30:22 -05:00
asonix	bd3975f455	Simplify some repo functions, remove 0.3 migration	2023-07-25 20:08:18 -05:00
asonix	f221887599	Clippy	2023-07-23 16:04:17 -05:00
asonix	aac1bb7bc4	Finish media proxy implementation	2023-07-23 15:45:52 -05:00
asonix	4cca7d0f86	Prepare for proxying media	2023-07-23 12:10:36 -05:00
asonix	75bf425c6e	Update alias query, process query to accept proxy url	2023-07-23 11:45:10 -05:00