143 commits

Author	SHA1	Message	Date
asonix	6f95c72070	Add new feature for testing with errors, test & fix job retries	2024-03-10 22:02:27 -05:00
asonix	dff588aafd	Switch from deadpool to bb8	2024-03-10 15:59:08 -05:00
asonix	4976fcb2eb	Remove unneeded code	2024-03-09 22:53:46 -06:00
asonix	9fe586b9dd	Implement retries for jobs, start warning on long polls	2024-03-09 12:15:23 -06:00
asonix	4897c90ed7	Add blurhash cleanup for sled	2024-03-03 11:11:43 -06:00
asonix	3ecefcb64e	Save blurhashes in repo, improve some error responses, simplify extracting aliases	2024-03-02 13:27:58 -06:00
asonix	c1e651c01a	Use BytesStream in more places	2024-02-22 16:03:04 -06:00
asonix	71d81681d5	It compiles	2024-02-04 15:45:47 -06:00
asonix	d8c4a22ba9	Start describing metrics	2024-02-04 14:26:18 -06:00
asonix	7eba45f37e	cargo fix & cargo fmt	2024-02-03 18:19:14 -06:00
asonix	50e31f96da	It compiles again	2024-02-03 18:18:13 -06:00
asonix	5805eb0aed	Enable configuring imagemagick security policy from pictrs	2024-01-31 21:32:01 -06:00
asonix	e7357cbdaf	Count spawned tasks in cleanup, debug log queue length	2024-01-30 14:57:48 -06:00
asonix	34cadb86b8	Don't propogate pop span into blocking closure	2024-01-30 14:30:24 -06:00
asonix	dc12f6f79d	replace remaining web::block with spawn_blocking	2024-01-30 14:16:05 -06:00
asonix	819b83bab7	Use a far smaller pool to listen for db notifications	2024-01-25 16:50:36 -06:00
asonix	fe1132aec1	Enable at-most-once queueing for some cleanup jobs	2024-01-24 17:14:31 -06:00
asonix	fee2294901	clippy	2024-01-15 18:21:36 -05:00
asonix	19147e2035	postgres: allow connecting to TLS-enabled databases	2024-01-15 18:11:08 -05:00
asonix	688c797082	Remove LIMIT 10, single_value is already in use	2024-01-04 11:13:29 -06:00
asonix	6514487ea9	Index on queue_time for job queue, limit to 10 jobs in ID query	2024-01-04 11:01:25 -06:00
asonix	0fd6c459d0	Clippy	2024-01-04 10:42:34 -06:00
asonix	7d4f44a540	Set worker-id to None when marking a job as runnable after expired heartbeats	2024-01-04 10:29:37 -06:00
asonix	bfc2410552	Add trace-level logs to every loop (paranoid)	2023-12-28 11:58:38 -06:00
asonix	dc90e00324	Update metrics to 0.22 (plus mechanical changes)	2023-12-26 18:06:38 -06:00
asonix	b94ba5fcfc	Inline process background future, clean tracing a bit	2023-12-22 13:12:19 -06:00
asonix	c9155f7ce7	Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration	2023-12-17 23:15:27 -06:00
asonix	bd0b89b165	Clean spans a bit	2023-12-15 22:34:45 -06:00
asonix	cd80ab16c3	Downgrade retry warnings to debug	2023-12-11 13:08:10 -06:00
asonix	c795c1edfa	Replace most of actix-rt with tokio, give names to tasks	2023-10-20 19:08:11 -05:00
asonix	7b5a3020fa	Don't rely on global state for tmp_dir	2023-10-07 11:32:36 -05:00
asonix	914e21c043	Implement constant-time equality for delete tokens, inline alias cleanup	2023-10-04 12:11:29 -05:00
asonix	d28c1ac628	Upgrade Concurrency: Update docs, configs, limit maxiumum details operations	2023-10-03 17:04:40 -05:00
asonix	f223d477c7	Enable specifying concurrency for 0.4 to 0.5 migration	2023-10-03 16:27:19 -05:00
asonix	75551dc3a1	Increase deadpool wait timeout, helps with extreme concurrency	2023-10-03 16:15:49 -05:00
asonix	47751f3875	Downgrade some WARN logs to INFO, update docs for public methods	2023-09-30 17:52:58 -05:00
asonix	ecdbaa9c92	Use diesel-async re-export of Hook	2023-09-24 15:48:19 -05:00
asonix	745e990fa0	Remove direct dependency on actix-rt and actix-server	2023-09-24 15:45:10 -05:00
asonix	b2674f06d0	More streme	2023-09-10 20:08:01 -04:00
asonix	1b97ac1c5a	Start replacing manual stream implementations with streem	2023-09-10 18:55:13 -04:00
asonix	858899b943	Add metrics & tracing to dropped uploads, turn pop into a single query	2023-09-07 19:20:41 -05:00
asonix	783df60f81	Fix on_conflict for postgres store migration path	2023-09-06 20:49:00 -05:00
asonix	bc49f8ca37	Migrate with timestamp	2023-09-04 22:25:11 -05:00
asonix	9c50bbe23a	match -> matches!	2023-09-04 22:06:28 -05:00
asonix	62ebc1c60a	Move timeout after metrics - timeouts count as incomplete	2023-09-04 22:02:59 -05:00
asonix	3bd0f78e75	Use with_timeout for all timeouts	2023-09-04 21:58:57 -05:00
asonix	a4b1ab7dfb	Instrument postgres db calls	2023-09-04 21:51:27 -05:00
asonix	37448722de	postgres: Rework job & upload notifications (more) ... postgres: Add metrics to job push & pop, upload wait sled: add upload wait metrics	2023-09-04 16:20:26 -05:00
asonix	00aa00c55c	postgres: Enable 'previous page' when less than full limit fit on previous page	2023-09-03 21:58:31 -05:00
asonix	a43de122f9	postgres: add already-claimed case, general: tracing paranoia	2023-09-03 21:30:47 -05:00