379 commits

Author	SHA1	Message	Date
Alex Auvolat	f4346cc5f4	Update dependencies	2021-03-16 15:58:40 +01:00
Alex Auvolat	2a41b82384	Simpler Merkle & sync	2021-03-16 12:18:03 +01:00
Alex Auvolat	0aad2f2e06	some reordering	2021-03-16 11:47:39 +01:00
Alex Auvolat	515029d026	Refactor code	2021-03-16 11:43:58 +01:00
Alex Auvolat	1d9961e411	Simplify replication logic	2021-03-16 11:14:27 +01:00
Alex Auvolat	6a8439fd13	Some improvements in background worker but we terminate late	2021-03-15 23:14:12 +01:00
Alex Auvolat	0cd5b2ae19	WIP migrate to tokio 1	2021-03-15 22:36:41 +01:00
Alex Auvolat	4d4117f2b4	Refactor block resync loop; make workers infaillible	2021-03-15 20:09:44 +01:00
Alex Auvolat	667e4e72a8	Small fixes	2021-03-15 19:51:16 +01:00
Alex Auvolat	642bed601f	Make it case-insensitive	2021-03-15 19:16:42 +01:00
Alex Auvolat	5ee1d956b6	Allow manipulation of keys by their shorthand in the CLI	2021-03-15 19:14:26 +01:00
Alex Auvolat	537f652fec	Tiny things	2021-03-15 18:40:27 +01:00
Alex Auvolat	0290afe1f8	Make block rc code more understandable	2021-03-15 18:27:26 +01:00
Alex Auvolat	3bf2df622a	Time and metadata improvements	2021-03-15 16:21:41 +01:00
Alex Auvolat	097c339d98	Fix race condition	2021-03-15 15:26:29 +01:00
Alex Auvolat	bdcbdd1cd8	Fix list API bug	2021-03-15 14:46:37 +01:00
Alex Auvolat	9b118160a8	Optim & refactor	2021-03-12 22:06:56 +01:00
Alex Auvolat	831eb35763	cargo fmt	2021-03-12 21:52:19 +01:00
Alex Auvolat	c475471e7a	Implement table gc, currently for block_ref and version only	2021-03-12 19:57:37 +01:00
Alex Auvolat	f4aad8fe6e	cargo fmt	2021-03-12 18:16:03 +01:00
Alex Auvolat	5ab33fddac	Refactor CLI and prettify CLI outpu	2021-03-12 18:12:31 +01:00
Alex Auvolat	a1442f072a	Implement garage stats to get info on node contents	2021-03-12 15:40:54 +01:00
Alex Auvolat	cbe7e1a66a	Move table rpc client out of tableaux	2021-03-12 15:07:23 +01:00
Alex Auvolat	8860aa19b8	Make syncer have its own rpc client/server	2021-03-12 15:05:26 +01:00
Alex Auvolat	1fea257291	Don't sync at beginning	2021-03-12 14:51:17 +01:00
Alex Auvolat	7fdaf7aef0	Fix merkle updater not being notified; improved logging	2021-03-12 14:37:46 +01:00
Alex Auvolat	1ec49980ec	whoops	2021-03-11 19:30:24 +01:00
Alex Auvolat	3f7a496355	More security: don't delete stuff too easily	2021-03-11 19:06:27 +01:00
Alex Auvolat	f7c2cd1cd7	Add comment, and also whoops, this wasn't doing what we expected	2021-03-11 18:56:18 +01:00
Alex Auvolat	fae5104a2c	Add a nice warning	2021-03-11 18:50:32 +01:00
Alex Auvolat	db7a9d4948	Tiny changes	2021-03-11 18:45:26 +01:00
Alex Auvolat	046b649bcc	(not well tested) use merkle tree for sync	2021-03-11 18:28:27 +01:00
Alex Auvolat	94f3d28774	WIP big refactoring	2021-03-11 16:54:15 +01:00
Alex Auvolat	8d63738cb0	Checkpoint: add merkle tree in data table	2021-03-11 13:47:21 +01:00
Alex Auvolat	3214dd52dd	Very minor changes	2021-03-10 21:50:09 +01:00
Alex Auvolat	af7600f989	Correctly implement CompleteMultipartUpload with etag check of parts	2021-03-10 17:01:05 +01:00
Alex Auvolat	445912dc6a	Remove migration paths from 0.1 branch	2021-03-10 16:38:31 +01:00
Alex Auvolat	0fd7df8fa0	Switch to blake2 sum for identifying blocks by their data	2021-03-10 16:33:31 +01:00
Alex Auvolat	2afd2c81ba	Change hash function to blake2 for partition keys based on strings	2021-03-10 16:23:57 +01:00
Alex Auvolat	f319a7d374	Refactor model stuff, including cleaner CRDTs	2021-03-10 16:21:56 +01:00
Alex Auvolat	6a3dcf3974	Rename n_tokens into capacity	2021-03-10 14:52:03 +01:00
Alex Auvolat	7cda917b6b	update condition	2021-03-05 17:08:03 +01:00
Alex Auvolat	d7e005251d	Not fully tested: new multi-dc MagLev	2021-03-05 16:22:29 +01:00
Alex Auvolat	3882d5ba36	Remove epidemic propagation for fully replicated stuff: write directly to all nodes	2021-03-05 15:09:18 +01:00
Alex Auvolat	49c25a1509	Simulate stuff moving around	2021-02-25 10:53:33 +01:00
Alex Auvolat	5fe95ebae7	fix tracing	2021-02-24 12:18:01 +01:00
Alex Auvolat	13e2eda0c2	Arrange block manager	2021-02-24 11:58:03 +01:00
Alex Auvolat	09fd6ea7f0	I was tired yesterday	2021-02-24 11:05:59 +01:00
Alex Auvolat	a52ab69640	fix misuse of sled transactions	2021-02-23 22:45:36 +01:00
Alex Auvolat	20e6e9fa20	Update sled & try to debug deadlock (but its in sled...)	2021-02-23 21:27:28 +01:00
Alex Auvolat	bf25c95fe2	Make updated() be a sync function that doesn't fail	2021-02-23 20:25:15 +01:00
Alex Auvolat	28bc967c83	Handle correctly deletion dues to offloading	2021-02-23 19:59:43 +01:00
Alex Auvolat	55156cca9d	Several changes in table_sync: ... - separate path for case of offloading a partition we don't store - use sync::Mutex instead of tokio::Mutex, make less fn's async	2021-02-23 19:11:02 +01:00
Alex Auvolat	40763fd749	Cargo fmt	2021-02-23 18:46:25 +01:00
Alex Auvolat	6e6f7e8555	Replace some checksums where it makes sense	2021-02-23 18:14:37 +01:00
Alex Auvolat	e8e4418ca7	Add blake2 and xxhash hash functions	2021-02-23 17:52:28 +01:00
Alex Auvolat	b1b640ae8b	rename hash() to sha256sum(), we might want to change it at some places	2021-02-21 15:24:30 +01:00
Alex Auvolat	80892df8cc	Some refactoring	2021-02-21 13:11:10 +01:00
Alex Auvolat	3bcbbe1e31	More precise logging (warn only when returning a 500)	2021-02-20 00:30:39 +01:00
Alex Auvolat	10b983b8e7	Add verification of part numbers in CompleteMultipartUpload (WIP #30)	2021-02-20 00:13:07 +01:00
Alex Auvolat	1de96248e0	add application/xml header and missing xml escapes	2021-02-19 23:40:18 +01:00
Alex Auvolat	5d1fa591d9	Add compatibility list	2021-02-19 19:10:23 +01:00
Alex Auvolat	e64ecbdccd	S3 compatibility: return 404 instead of 400 on some multipart commands	2021-02-19 18:51:05 +01:00
Alex Auvolat	55a2a636ca	Implement ListObjectsV2	2021-02-19 16:44:06 +01:00
Alex Auvolat	02d512f3fd	Fix #28, extra headers being ignored (because of profound stupidity)	2021-02-19 12:38:22 +01:00
Alex Auvolat	76390085ef	Small improvements in the S3 put workflow	2021-02-19 12:11:02 +01:00
Alex Auvolat	36814be447	Fix S3 ListObjects result and replace println!s by debug!s	2021-01-16 16:05:54 +01:00
Alex Auvolat	e818f51073	Forgot a bump	2021-01-15 18:36:51 +01:00
Alex Auvolat	ceeb0732a2	Use 0.1.0b instead of 0.1.0 (for compatibility with new Error type)	2021-01-15 18:27:58 +01:00
Alex Auvolat	1d1d497e2b	Bump everything to 0.1.1	2021-01-15 17:54:48 +01:00
Alex Auvolat	851893a3f2	Do not accept domains such as [hello	2021-01-15 17:49:10 +01:00
Quentin	f8a40e8c4f	Explicitly set code path unreachable	2021-01-15 17:11:15 +01:00
Quentin	fad7bc405b	Behavior problem: do not panic anymore + add tests	2021-01-15 17:03:54 +01:00
Quentin	1e10c6a61c	Doc tests that do not compile/work must be tagged with ignore	2021-01-15 17:03:38 +01:00
Quentin	11a79a95dd	Simplify Error file	2021-01-15 16:25:44 +01:00
Quentin	c441a358cd	Remove unused dependencies	2021-01-15 16:16:32 +01:00
Quentin	f496e41ef4	Replace an already done check by unreachable!()	2021-01-15 15:44:44 +01:00
Quentin Dufour	2f4378a9c4	Fix formatting	2020-12-17 22:51:44 +01:00
Quentin Dufour	ccda9ab1ca	Merge branch 'master' into feature/website	2020-12-17 21:09:50 +01:00
Quentin Dufour	3132deca58	Web server access control	2020-12-17 20:43:14 +01:00
Quentin	011ff87b5f	Push update	2020-12-15 13:23:22 +01:00
Quentin	3bc4d57a0f	First implementation of the CLI	2020-12-15 12:48:24 +01:00
Quentin	a3566e49da	Start to implement Website CLI	2020-12-14 21:50:40 +01:00
Quentin	d0eb6a457f	Migrate RPC to new schema	2020-12-14 21:46:49 +01:00
Quentin	96388acf23	Implement migration	2020-12-12 21:35:29 +01:00
Alex Auvolat	8956db2a81	Make less things public	2020-12-12 17:58:19 +01:00
Alex Auvolat	5c6c067b0c	More documentation on CRDTs (we should probably extract this to a ... standalone crate!)	2020-12-12 17:06:40 +01:00
Quentin	e1ce2b228a	WIP table migration	2020-12-12 17:00:31 +01:00
Alex Auvolat	0b3084ca5f	Merge branch 'master' into doc/model	2020-12-12 16:05:28 +01:00
Quentin	e8c12072ce	Merge branch 'master' into feature/website	2020-12-10 20:12:56 +01:00
Quentin	51d0c14e44	CLI structure	2020-12-10 18:13:32 +01:00
Alex Auvolat	022b386a50	Improved compatibility on list API call	2020-12-06 15:39:03 +01:00
Quentin	986e15459a	Merge branch 'master' into feature/website	2020-12-06 15:21:09 +01:00
Alex Auvolat	4a5bbbb810	Propose ETag fix	2020-12-05 19:23:46 +01:00
Alex Auvolat	76b489f3d3	Reformulate patch	2020-12-05 19:20:07 +01:00
Quentin	bd7e3d1bd1	Fix Content-Length	2020-12-05 18:57:22 +01:00
Alex Auvolat	9f46fb699a	Content-range fix	2020-12-05 16:37:59 +01:00
Quentin	15f409d404	Merge branch 'master' into feature/website	2020-11-29 17:19:55 +01:00
Alex Auvolat	d54f15b2c6	Small optimisation	2020-11-29 17:07:14 +01:00
Alex Auvolat	fed97f37e1	ETag patch	2020-11-29 16:38:01 +01:00
Alex Auvolat	601ae25ad2	Small refactorings	2020-11-29 16:21:28 +01:00
Quentin	cbd10c1b0a	Add some doc on LWW	2020-11-23 18:17:48 +01:00
Quentin	8722e27600	CRDT doc	2020-11-23 17:49:21 +01:00
Quentin	aa320aa04a	Merge branch 'master' into feature/website	2020-11-22 19:54:47 +01:00
Quentin	17dc610f8a	Also check hash for < 3KB files	2020-11-22 11:14:46 +01:00
Quentin	435d5f9205	Fix base64/hex checksum comparison	2020-11-22 11:04:33 +01:00
Quentin	fb18f5e17a	Fix wrong http status code	2020-11-21 18:14:02 +01:00
Quentin	28efe341cb	Merge branch 'master' into feature/website	2020-11-21 18:01:50 +01:00
Quentin	b7a377308b	Handle HEAD	2020-11-21 17:58:14 +01:00
Quentin	a88fd49f71	Use handle_get	2020-11-21 17:50:19 +01:00
Quentin	0f33231ee6	We are able to serve a file	2020-11-21 15:15:25 +01:00
Quentin	d4c7f4e374	Fix host to key	2020-11-21 12:01:02 +01:00
Quentin	2f6eca4ef3	Merge remote-tracking branch 'origin/master' into feature/website	2020-11-21 10:52:27 +01:00
Alex Auvolat	a8b3c8fd58	data hexdump in warning	2020-11-20 23:53:54 +01:00
Alex Auvolat	f9be964c3f	Warning when cannot decode entry (data format incompatibilities)	2020-11-20 23:53:54 +01:00
Alex Auvolat	4f7f1d1cb3	less type bounds	2020-11-20 23:53:54 +01:00
Alex Auvolat	68be5072e5	simplify	2020-11-20 23:53:54 +01:00
Alex Auvolat	173f0dbac9	oops	2020-11-20 23:53:54 +01:00
Alex Auvolat	f8a04852a2	Convert bucket table to better CRDT representation	2020-11-20 23:53:54 +01:00
Alex Auvolat	e02e9e035e	Begin improve model to use better CRDTs	2020-11-20 23:53:54 +01:00
Quentin	5b363626f4	Support punnycode	2020-11-20 21:23:32 +01:00
Alex Auvolat	2dc9a48022	Fix reconciliation logic	2020-11-20 20:12:32 +01:00
Alex Auvolat	e9fd265ce6	Slight refactoring to make things clearer with DeletedFilter	2020-11-20 20:11:04 +01:00
Quentin	04f455ff7f	Make it compile again	2020-11-19 14:56:00 +01:00
Quentin	fc427b0b66	Merge branch 'master' into feature/website	2020-11-19 14:39:30 +01:00
Quentin	6076d869b1	Build error	2020-11-11 21:17:34 +01:00
Quentin	2765291796	Build path correctly	2020-11-11 19:48:01 +01:00
Alex Auvolat	7d7b9e95a9	Simplify and_then(Some) as map() and remove move	2020-11-11 16:36:48 +01:00
Quentin	5a5592c176	Replace with option syntaxic sugar	2020-11-11 16:12:42 +01:00
Quentin	d445c4ef9c	WIP fetch object	2020-11-11 15:24:25 +01:00
Quentin	3cb3994cd2	Add documentation to host_to_bucket	2020-11-10 17:05:10 +01:00
Quentin	cacf8ddf2d	Panic when it is a logical error	2020-11-10 15:52:20 +01:00
Quentin	d1b2fcc1e7	Rewrite for clarity	2020-11-10 15:48:40 +01:00
Quentin	ab62c59acb	Fix indent again	2020-11-10 15:40:33 +01:00
Quentin	8797eed0ab	Fixes due to integration tests	2020-11-10 15:32:04 +01:00
Quentin	1e52ee9f5b	Rewrite authority to host while staying on stack	2020-11-10 15:26:48 +01:00
Quentin	27795a390c	Fix formatting	2020-11-10 09:59:52 +01:00
Quentin	4093833ae8	Extract bucket	2020-11-10 09:57:07 +01:00
Quentin	09137fd6b5	Log host	2020-11-08 16:06:52 +01:00
Quentin	c78df603d7	Add some documentation	2020-11-08 16:02:16 +01:00
Quentin	71721f5bcf	Merge branch 'master' into feature/website	2020-11-08 15:53:33 +01:00
Quentin	0791e7164e	Parse host header	2020-11-08 15:47:25 +01:00
Alex Auvolat	a50fa70d45	Refactor error management in API part	2020-11-08 15:05:28 +01:00
Alex Auvolat	6a8b972f3a	Modif parse_bucket_key to fail when bucket is ""	2020-11-08 13:39:44 +01:00
Quentin	e1415f5976	Add a second test	2020-11-07 15:34:53 +01:00
Quentin	9a50ce12a8	Fix formatting	2020-11-07 13:59:30 +01:00
Quentin	8f4ada1965	Add a test for parse_bucket_key	2020-11-07 13:53:32 +01:00
Quentin	0d3bc169ee	It compiles!	2020-11-03 12:37:16 +01:00
Quentin	b3caa3628d	Fix description of the crate	2020-11-02 15:57:23 +01:00
Quentin	cea871d944	Skeleton to the new web API	2020-11-02 15:48:39 +01:00