70 commits

Author	SHA1	Message	Date
asonix	55bc4b64c1	Add per-upload validations and per-upload preprocess steps	2024-03-27 19:00:54 -05:00
asonix	84a882392a	Start threading upload configuration into ingest	2024-03-27 16:57:22 -05:00
asonix	4976fcb2eb	Remove unneeded code	2024-03-09 22:53:46 -06:00
asonix	9fe586b9dd	Implement retries for jobs, start warning on long polls	2024-03-09 12:15:23 -06:00
asonix	25ef3861f1	Remove more unneeded 'static	2024-02-27 20:46:04 -06:00
asonix	04dcc9a0c8	Include file extension in identifiers	2024-02-27 20:41:25 -06:00
asonix	13fc0df31a	More piping	2024-02-24 19:07:48 -06:00
asonix	16bf18bda4	Remove some Unpin and 'static bounds	2024-02-22 18:05:04 -06:00
asonix	3a7d5b7bfb	Re-use try_from_stream where possible	2024-02-22 16:10:34 -06:00
asonix	c1e651c01a	Use BytesStream in more places	2024-02-22 16:03:04 -06:00
asonix	71d81681d5	It compiles	2024-02-04 15:45:47 -06:00
asonix	c176e4c686	clippy	2024-02-03 18:30:52 -06:00
asonix	7eba45f37e	cargo fix & cargo fmt	2024-02-03 18:19:14 -06:00
asonix	50e31f96da	It compiles again	2024-02-03 18:18:13 -06:00
asonix	cf7c753e65	Pass a State value around rather than a bunch of arguments	2024-02-03 13:31:54 -06:00
asonix	5805eb0aed	Enable configuring imagemagick security policy from pictrs	2024-01-31 21:32:01 -06:00
asonix	bfc2410552	Add trace-level logs to every loop (paranoid)	2023-12-28 11:58:38 -06:00
asonix	dc90e00324	Update metrics to 0.22 (plus mechanical changes)	2023-12-26 18:06:38 -06:00
asonix	5624671cbf	Rework ProcessRead to require a closure for using the internal AsyncRead	2023-12-22 20:52:58 -06:00
asonix	b94ba5fcfc	Inline process background future, clean tracing a bit	2023-12-22 13:12:19 -06:00
asonix	2523b6644b	Add AggregateTimeout, don't aggregate with permit	2023-12-04 15:39:34 -06:00
asonix	210af5d7d9	Add danger_dummy_mode	2023-11-11 14:22:12 -06:00
asonix	c795c1edfa	Replace most of actix-rt with tokio, give names to tasks	2023-10-20 19:08:11 -05:00
asonix	7b5a3020fa	Don't rely on global state for tmp_dir	2023-10-07 11:32:36 -05:00
asonix	fff4afe105	Start working on per-server temporary directory	2023-10-06 19:42:24 -05:00
asonix	d28c1ac628	Upgrade Concurrency: Update docs, configs, limit maxiumum details operations	2023-10-03 17:04:40 -05:00
asonix	0926739d3c	Remove once_cell	2023-09-24 15:32:00 -05:00
asonix	f3ef2d870b	Generate details during ingest	2023-09-24 12:58:16 -05:00
asonix	8f62b0c325	clippy	2023-09-24 12:42:23 -05:00
asonix	6c921817e1	Enable thumbnailing animations, use dynamic dispatch for a number of async readers	2023-09-24 11:54:16 -05:00
asonix	7df6f3906e	Add timeout, metrics to external validation request	2023-09-20 19:29:19 -05:00
asonix	1b97ac1c5a	Start replacing manual stream implementations with streem	2023-09-10 18:55:13 -04:00
asonix	e15a82c0c7	Add external validation check	2023-09-05 20:45:07 -05:00
asonix	a43de122f9	postgres: add already-claimed case, general: tracing paranoia	2023-09-03 21:30:47 -05:00
asonix	8921f57a21	Remove Identifier trait, Replace with Arc<str>	2023-09-02 18:38:02 -05:00
asonix	8f50a15b25	Reduce reliance on futures-util	2023-08-23 11:59:42 -05:00
asonix	ba27a1a223	Build out repo-repo migration	2023-08-16 16:09:40 -05:00
asonix	1ee77b19d8	Make Repo object safe. Create migration	2023-08-15 19:19:03 -05:00
asonix	da876fd553	Make it compile	2023-08-14 14:25:19 -05:00
asonix	fee4ed1e3e	Add process timeout	2023-08-05 12:41:06 -05:00
asonix	bd3975f455	Simplify some repo functions, remove 0.3 migration	2023-07-25 20:08:18 -05:00
asonix	77cdeab57e	Improve metrics, add job metrics	2023-07-22 21:11:28 -05:00
asonix	ce0df080f4	Add prometheus metrics	2023-07-22 16:47:59 -05:00
asonix	3ca994ee31	Replace global config with passed-around config ... This will enable spinning up pict-rs multiple times in the same process with different configurations	2023-07-22 12:31:01 -05:00
asonix	eeac900d7e	Add quality value to process method, move -coalesce earlier in pipelines	2023-07-18 16:18:01 -05:00
asonix	786f583d98	Add public_url to Store for optional redirects to public object store urls ... Add details generation to file migration to set the content-type for uploads	2023-07-14 14:53:37 -05:00
asonix	ec6c9aac1e	Better discovery validation, proper invoking of imagemagick	2023-07-13 19:21:57 -05:00
asonix	2c22f2ee3a	It compiles and runs, but doesn't work	2023-07-13 17:42:21 -05:00
asonix	ad1837f9dd	VERY BROKEN: start replacing parts of pict-rs	2023-07-12 22:12:21 -05:00
asonix	9e7376d411	Remove is_missing error, make Identifier and DeleteToken return Option on fetch	2023-07-07 13:17:26 -05:00