Alex Auvolat
1bcd6fabbd
New buckets for 0.6.0: small changes
...
- Fix bucket delete
- fix merge of bucket creation date
- Replace deletable with option in aliases
Rationale: if two aliases point to conflicting bucket, resolving
by making an arbitrary choice risks making data accessible when it
shouldn't be. We'd rather resolve to deleting the alias until
someone puts it back.
2022-01-04 12:52:47 +01:00
Alex Auvolat
beeef4758e
Some movement of helper code and refactoring of error handling
2022-01-04 12:52:46 +01:00
Alex Auvolat
d8ab5bdc3e
New buckets for 0.6.0: fix model and migration
2022-01-04 12:47:28 +01:00
Alex Auvolat
c7d5c73244
Add must_use to some CRDT functions
2022-01-04 12:47:28 +01:00
Alex Auvolat
87121dce9d
New buckets for 0.6.0: documentation and build files
2022-01-04 12:47:06 +01:00
Alex Auvolat
b1cfd16913
New buckets for 0.6.0: small fixes, including:
...
- ensure bucket names are correct aws s3 names
- when making aliases, ensure timestamps of links in both ways are the
same
- fix small remarks by trinity
- don't have a separate website_access field
2022-01-04 12:46:41 +01:00
Alex Auvolat
0bbb6673e7
Model changes
2022-01-04 12:45:52 +01:00
Alex Auvolat
5b1117e582
New model for buckets
2022-01-04 12:45:46 +01:00
trinity-1686a
1eb972b1ac
Add compression using zstd ( #173 )
...
fix #27
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/173
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
2021-12-15 11:26:43 +01:00
Alex Auvolat
c94406f428
Improve how node roles are assigned in Garage
...
- change the terminology: the network configuration becomes the role
table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
changes are staged in a CRDT data structure. Then, once the user is
happy with the changes, they can commit them all at once (or revert
them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm
This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
2021-11-16 16:05:53 +01:00
Trinity Pointard
9c58ec28d3
add support for vhost-style s3 bucket
2021-11-16 15:41:41 +01:00
Trinity Pointard
9d7535c3f5
allow missing bootstrap_peers in garage.toml
2021-11-05 16:36:25 +01:00
Alex Auvolat
2090a6187f
Add tranquilizer mechanism to improve on token bucket mechanism
2021-11-04 13:26:59 +01:00
Alex Auvolat
6f13d083ab
Add semaphore to limit RAM used by buffered outgoing requests
2021-11-03 18:02:57 +01:00
Alex Auvolat
6b47c294f5
Refactoring on repair commands
2021-10-27 11:14:55 +02:00
Alex Auvolat
43e13a501d
Use published netapp crate instead of git repo
2021-10-26 10:36:57 +02:00
Alex Auvolat
de4276202a
Improve CLI, adapt tests, update documentation
2021-10-25 14:21:48 +02:00
Alex Auvolat
1b450c4b49
Improvements to CLI and various fixes for netapp version
...
Discovery via consul, persist peer list to file
2021-10-22 16:55:24 +02:00
Alex Auvolat
4067797d01
First port of Garage to Netapp
2021-10-22 15:55:18 +02:00
Alex Auvolat
b9127dd6f8
Prepare for v0.3.0 and add migration path from v0.2.1.x
2021-05-28 15:29:58 +02:00
Alex Auvolat
b490ebc7f6
Many improvements on ring/replication and its configuration:
...
- Explicit "replication_mode" configuration parameters that takes
either "none", "2" or "3" as values, instead of letting user configure
replication factor themselves. These are presets whose corresponding
replication/quorum values can be found in replication/mode.rs
- Explicit support for single-node and two-node deployments
(number of nodes must be at least "replication_mode", with "none"
we can have only one node)
- Ring is now stored much more compactly with 256*8 + n*32 bytes,
instead of 256*32 bytes
- Support for gateway-only nodes that do not store data
(these nodes still need a metadata_directory to store the list
of bucket and keys since those are stored on all nodes; it also
technically needs a data_directory to start but it will stay
empty unless we have bugs)
2021-05-28 14:07:36 +02:00
Alex Auvolat
6ccffc3162
Improved XML serialization
...
- Use quick_xml and serde for all XML response returned by the S3 API.
- Include tests for all structs used to generate XML
- Remove old manual XML escaping function which was unsafe
2021-05-06 22:37:15 +02:00
Trinity Pointard
e4b9e4e24d
rename types to CamelCase
2021-05-03 22:15:09 +02:00
Trinity Pointard
f05bb111c2
fix clippy warnings on util and rpc
2021-05-03 22:11:41 +02:00
Alex Auvolat
575726358c
Tune Sled configuration
...
- Make sled cache size and flush interval configurable
- Set less agressive default values:
- cache size 128MB instead of 1GB
- Flush interval 2 seconds instead of .5 seconds
2021-05-03 17:27:43 +02:00
Trinity Pointard
74373aebcf
make most requested changes
2021-04-27 16:47:08 +02:00
Trinity Pointard
f871689571
run cargo fmt on util and make missing doc warning
2021-04-27 16:37:10 +02:00
Trinity Pointard
f9bd2d8fb7
document util crate
2021-04-27 16:37:10 +02:00
Alex Auvolat
64b91c2645
Keep old data
2021-04-27 16:37:09 +02:00
Alex Auvolat
9ced9f78dc
Improve bootstraping: do it regularly; persist peer list
2021-04-27 16:37:08 +02:00
Alex Auvolat
f859d15062
update to v0.2.1
2021-03-19 13:39:18 +01:00
Alex Auvolat
fd8f4caa81
Support old CPUs
2021-03-19 12:19:40 +01:00
Alex Auvolat
4c26a0b9c1
Update Cargo.toml files with AGPL license info
2021-03-18 21:59:17 +01:00
Trinity Pointard
f17cb6c969
resolve domain to multiple addresses
...
And warn instead of failling when a domain can't be resolved
2021-03-18 21:04:30 +01:00
Trinity Pointard
c8a7ce5cdf
remove domain resolution for *_bind_addr
2021-03-18 19:47:51 +01:00
Trinity Pointard
81e9db783f
simplify addresse deserialialiser and limit allocations
2021-03-18 19:47:51 +01:00
Trinity Pointard
ae3b7029a9
add support for using domain name in configuration
2021-03-18 19:47:51 +01:00
Alex Auvolat
dead945c8f
Prepare for release 0.2
2021-03-18 19:33:15 +01:00
Alex Auvolat
f4346cc5f4
Update dependencies
2021-03-16 15:58:40 +01:00
Alex Auvolat
6a8439fd13
Some improvements in background worker but we terminate late
2021-03-15 23:14:12 +01:00
Alex Auvolat
0cd5b2ae19
WIP migrate to tokio 1
2021-03-15 22:36:41 +01:00
Alex Auvolat
4d4117f2b4
Refactor block resync loop; make workers infaillible
2021-03-15 20:09:44 +01:00
Alex Auvolat
3bf2df622a
Time and metadata improvements
2021-03-15 16:21:41 +01:00
Alex Auvolat
c475471e7a
Implement table gc, currently for block_ref and version only
2021-03-12 19:57:37 +01:00
Alex Auvolat
8d63738cb0
Checkpoint: add merkle tree in data table
2021-03-11 13:47:21 +01:00
Alex Auvolat
3882d5ba36
Remove epidemic propagation for fully replicated stuff: write directly to all nodes
2021-03-05 15:09:18 +01:00
Alex Auvolat
20e6e9fa20
Update sled & try to debug deadlock (but its in sled...)
2021-02-23 21:27:28 +01:00
Alex Auvolat
40763fd749
Cargo fmt
2021-02-23 18:46:25 +01:00
Alex Auvolat
e8e4418ca7
Add blake2 and xxhash hash functions
2021-02-23 17:52:28 +01:00
Alex Auvolat
b1b640ae8b
rename hash() to sha256sum(), we might want to change it at some places
2021-02-21 15:24:30 +01:00
Alex Auvolat
1d1d497e2b
Bump everything to 0.1.1
2021-01-15 17:54:48 +01:00
Quentin
fc427b0b66
Merge branch 'master' into feature/website
2020-11-19 14:39:30 +01:00
Quentin
2765291796
Build path correctly
2020-11-11 19:48:01 +01:00
Quentin
4093833ae8
Extract bucket
2020-11-10 09:57:07 +01:00
Alex Auvolat
a50fa70d45
Refactor error management in API part
2020-11-08 15:05:28 +01:00
Quentin
cea871d944
Skeleton to the new web API
2020-11-02 15:48:39 +01:00
Quentin
104e2ce0a2
Add "web" configuration entry
2020-10-31 17:28:56 +01:00
Alex Auvolat
cc65cdc0fe
Add license, description and repository to .toml files
2020-07-07 14:14:58 +02:00
Alex Auvolat
fbe8fe81f2
Add automatic peer discovery from Consul
2020-06-30 18:33:14 +02:00
Alex Auvolat
db6f1f35a8
Rename epidemic_factor to epidemic_fanout (that's what it is); complete conf example in readme
2020-06-30 15:03:34 +02:00
Alex Auvolat
b46a7788d1
Implement HTTP ranges in get
2020-05-04 13:09:23 +00:00
Alex Auvolat
d867bbcfb5
Implement DeleteObjects
2020-05-01 15:52:35 +00:00
Alex Auvolat
f2e05986c4
Starting to be S3 compatible
2020-04-24 17:46:52 +00:00
Alex Auvolat
d8f5e643bc
Split code for modular compilation
2020-04-24 10:10:01 +00:00
