mirrors/garage
1
0
Fork 0
mirror of https://git.deuxfleurs.fr/Deuxfleurs/garage.git synced 2024-11-21 23:51:00 +00:00
Code Issues Projects Releases Packages Wiki Activity
garage/Cargo.toml

155 lines
4.3 KiB
TOML
Raw Normal View History

Split code for modular compilation
2020-04-24 10:10:01 +00:00
[workspace]
Use the new cargo feature resolver "2" Garage currently uses the legacy resolver "1". The new one is used by default if the root package specifies 'edition = 2021', which Garage does not (yet). The problem with the legacy resolver is, among others, that features enabled by dev-dependencies are propagated to normal dependencies. This affects e.g. hyper - one of the dev-dependencies enables "http2" feature that adds many extra dependencies. If we build garage without opentelemetry-otlp (this is enabled in the following commit), there's no normal dependency enabling "http2" feature. See https://doc.rust-lang.org/cargo/reference/resolver.html#feature-resolver-version-2
2022-09-04 13:26:19 +00:00
resolver = "2"
Split code for modular compilation
2020-04-24 10:10:01 +00:00
members = [
Abstract database behind generic interface and implement alternative drivers (#322) - [x] Design interface - [x] Implement Sled backend - [x] Re-implement the SledCountedTree hack ~~on Sled backend~~ on all backends (i.e. over the abstraction) - [x] Convert Garage code to use generic interface - [x] Proof-read converted Garage code - [ ] Test everything well - [x] Implement sqlite backend - [x] Implement LMDB backend - [ ] (Implement Persy backend?) - [ ] (Implement other backends? (like RocksDB, ...)) - [x] Implement backend choice in config file and garage server module - [x] Add CLI for converting between DB formats - Exploit the new interface to put more things in transactions - [x] `.updated()` trigger on Garage tables Fix #284 **Bugs** - [x] When exporting sqlite, trees iterate empty?? - [x] LMDB doesn't work **Known issues for various back-ends** - Sled: - Eats all my RAM and also all my disk space - `.len()` has to traverse the whole table - Is actually quite slow on some operations - And is actually pretty bad code... - Sqlite: - Requires a lock to be taken on all operations. The lock is also taken when iterating on a table with `.iter()`, and the lock isn't released until the iterator is dropped. This means that we must be VERY carefull to not do anything else inside a `.iter()` loop or else we will have a deadlock! Most such cases have been eliminated from the Garage codebase, but there might still be some that remain. If your Garage-over-Sqlite seems to hang/freeze, this is the reason. - (adapter uses a bunch of unsafe code) - Heed (LMDB): - Not suited for 32-bit machines as it has to map the whole DB in memory. - (adpater uses a tiny bit of unsafe code) **My recommendation:** avoid 32-bit machines and use LMDB as much as possible. **Converting databases** is actually quite easy. For example from Sled to LMDB: ```bash cd src/db cargo run --features cli --bin convert -- -i path/to/garage/meta/db -a sled -o path/to/garage/meta/db.lmdb -b lmdb ``` Then, just add this to your `config.toml`: ```toml db_engine = "lmdb" ``` Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/322 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>
2022-06-08 08:01:44 +00:00
"src/db",
Split code for modular compilation
2020-04-24 10:10:01 +00:00
"src/util",
[import-netapp] import Netapp code into Garage codebase
2024-02-13 11:55:41 +00:00
"src/net",
Split code for modular compilation
2020-04-24 10:10:01 +00:00
"src/rpc",
"src/table",
Add missing src/block to toplevel cargo.toml
2022-03-23 09:26:10 +00:00
"src/block",
Rename garage_core to garage_model
2020-07-07 11:59:22 +00:00
"src/model",
Split code for modular compilation
2020-04-24 10:10:01 +00:00
"src/api",
Skeleton to the new web API
2020-11-02 14:48:39 +00:00
"src/web",
Add a K2V client library and CLI (#303) lib.rs could use getting split in modules, but I'm not sure how exactly Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/303 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
2022-05-18 20:24:09 +00:00
"src/garage",
"src/k2v-client",
Split format_table into separate crate and reduce k2v-client dependencies
2023-05-17 11:01:37 +00:00
"src/format-table",
Split code for modular compilation
2020-04-24 10:10:01 +00:00
]
Pretty logging
2020-04-21 12:54:55 +00:00
Add a K2V client library and CLI (#303) lib.rs could use getting split in modules, but I'm not sure how exactly Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr> Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/303 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
2022-05-18 20:24:09 +00:00
default-members = ["src/garage"]
Declare garage crates using workspace.dependencies This will allow to really disable "sled" feature without declaring `default-features = false` in every Cargo.toml where garage_db and garage_model is used. See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table
2023-05-07 16:12:01 +00:00
[workspace.dependencies]
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
# Internal Garage crates
fix clippy lint in format-table crate
2023-05-17 11:06:37 +00:00
format_table = { version = "0.1.1", path = "src/format-table" }
[next-0.10] bump version number to 1.0
2024-03-28 14:19:44 +00:00
garage_api = { version = "1.0.0", path = "src/api" }
garage_block = { version = "1.0.0", path = "src/block" }
garage_db = { version = "1.0.0", path = "src/db", default-features = false }
garage_model = { version = "1.0.0", path = "src/model", default-features = false }
garage_net = { version = "1.0.0", path = "src/net" }
garage_rpc = { version = "1.0.0", path = "src/rpc" }
garage_table = { version = "1.0.0", path = "src/table" }
garage_util = { version = "1.0.0", path = "src/util" }
garage_web = { version = "1.0.0", path = "src/web" }
bump k2v-client to v0.0.4
2023-05-22 08:47:15 +00:00
k2v-client = { version = "0.0.4", path = "src/k2v-client" }
Declare garage crates using workspace.dependencies This will allow to really disable "sled" feature without declaring `default-features = false` in every Cargo.toml where garage_db and garage_model is used. See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table
2023-05-07 16:12:01 +00:00
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
# External crates from crates.io
arc-swap = "1.0"
[fix-auth-ct-eq] use argon2 hashing and verification for admin/metrics token checking
2024-02-29 09:57:07 +00:00
argon2 = "0.5"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
async-trait = "0.1.7"
backtrace = "0.3"
base64 = "0.21"
blake2 = "0.10"
bytes = "1.0"
bytesize = "1.1"
[import-netapp] import Netapp code into Garage codebase
2024-02-13 11:55:41 +00:00
cfg-if = "1.0"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
chrono = "0.4"
[s3-checksum] implement x-amz-checksum-* headers
2024-03-21 13:06:59 +00:00
crc32fast = "1.4"
crc32c = "0.6"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
crypto-common = "0.1"
digest = "0.10"
err-derive = "0.3"
gethostname = "0.4"
git-version = "0.3.4"
hex = "0.4"
hexdump = "0.1"
hmac = "0.12"
[dep-upgrade-202402] upgrade easy dependencies
2024-02-05 12:10:56 +00:00
idna = "0.5"
itertools = "0.12"
add rpc_public_addr_subnet config option In case `rpc_public_addr` is not set, but autodiscovery is used, this allows filtering the list of automatically discovered IPs to a specific subnet. For example, if nodes should pick *their* IP inside a specific subnet, but you don't want to explicitly write the IP down (as it's dynamic, or you want to share configs across nodes), you can use this option.
2024-04-23 08:57:43 +00:00
ipnet = "2.9.0"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
lazy_static = "1.4"
md-5 = "0.10"
mktemp = "0.5"
nix = { version = "0.27", default-features = false, features = ["fs"] }
nom = "7.1"
parse_duration = "2.1"
pin-project = "1.0.12"
[dep-upgrade-202402] upgrade easy dependencies
2024-02-05 12:10:56 +00:00
pnet_datalink = "0.34"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
rand = "0.8"
[s3-checksum] implement x-amz-checksum-* headers
2024-03-21 13:06:59 +00:00
sha1 = "0.10"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
sha2 = "0.10"
timeago = { version = "0.4", default-features = false }
xxhash-rust = { version = "0.8", default-features = false, features = ["xxh3"] }
[sse-c] Implement SSE-C encryption
2024-02-23 15:49:50 +00:00
aes-gcm = { version = "0.10", features = ["aes", "stream"] }
[import-netapp] import Netapp code into Garage codebase
2024-02-13 11:55:41 +00:00
sodiumoxide = { version = "0.2.5-0", package = "kuska-sodiumoxide" }
kuska-handshake = { version = "0.2.0", features = ["default", "async_std"] }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
clap = { version = "4.1", features = ["derive", "env"] }
pretty_env_logger = "0.5"
structopt = { version = "0.3", default-features = false }
[syslog] Add support to logging to syslog Original patch by Jakub Jirutka for Alpine Linux port.
2024-03-20 13:21:08 +00:00
syslog-tracing = "0.3"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
heed = { version = "0.11", default-features = false, features = ["lmdb"] }
[sqlite-r2d2] implement connection pooling in sqlite backend
2024-03-18 16:08:54 +00:00
rusqlite = "0.31.0"
r2d2 = "0.8"
r2d2_sqlite = "0.24"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
async-compression = { version = "0.4", features = ["tokio", "zstd"] }
[dep-upgrade-202402] upgrade easy dependencies
2024-02-05 12:10:56 +00:00
zstd = { version = "0.13", default-features = false }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
quick-xml = { version = "0.26", features = [ "serialize" ] }
rmp-serde = "1.1.2"
serde = { version = "1.0", default-features = false, features = ["derive", "rc"] }
serde_bytes = "0.11"
serde_json = "1.0"
[fix-cargo-toml] fix cargo warnings in Cargo.toml files
2024-02-15 09:39:06 +00:00
toml = { version = "0.8", default-features = false, features = ["parse"] }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
# newer version requires rust edition 2021
[dep-upgrade-202402] update toml, kube and k8s-openapi
2024-02-08 17:49:38 +00:00
k8s-openapi = { version = "0.21", features = ["v1_24"] }
kube = { version = "0.88", default-features = false, features = ["runtime", "derive", "client", "rustls-tls"] }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
schemars = "0.8"
reqwest = { version = "0.11", default-features = false, features = ["rustls-tls-manual-roots", "json"] }
form_urlencoded = "1.0.0"
[dep-upgrade-202402] wip: port to http/hyper crates v1
2024-02-05 17:49:54 +00:00
http = "1.0"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
httpdate = "1.0"
http-range = "0.1"
[dep-upgrade-202402] wip: port to http/hyper crates v1
2024-02-05 17:49:54 +00:00
http-body-util = "0.1"
[fix-cargo-toml] fix cargo warnings in Cargo.toml files
2024-02-15 09:39:06 +00:00
hyper = { version = "1.0", default-features = false }
[dep-upgrade-202402] rename BytesBody into ErrorBody for clarity
2024-02-07 13:45:52 +00:00
hyper-util = { version = "0.1", features = [ "full" ] }
[dep-upgrade-202402] upgrade easy dependencies
2024-02-05 12:10:56 +00:00
multer = "3.0"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
percent-encoding = "2.2"
[dep-upgrade-202402] upgrade easy dependencies
2024-02-05 12:10:56 +00:00
roxmltree = "0.19"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
url = "2.3"
futures = "0.3"
futures-util = "0.3"
[import-netapp] import Netapp code into Garage codebase
2024-02-13 11:55:41 +00:00
tokio = { version = "1.0", default-features = false, features = ["net", "rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }
tokio-util = { version = "0.7", features = ["compat", "io"] }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
tokio-stream = { version = "0.1", features = ["net"] }
opentelemetry = { version = "0.17", features = [ "rt-tokio", "metrics", "trace" ] }
opentelemetry-prometheus = "0.10"
opentelemetry-otlp = "0.10"
[import-netapp] import Netapp code into Garage codebase
2024-02-13 11:55:41 +00:00
opentelemetry-contrib = "0.9"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
prometheus = "0.13"
# used by the k2v-client crate only
[fix-cargo-toml] fix cargo warnings in Cargo.toml files
2024-02-15 09:39:06 +00:00
aws-sigv4 = { version = "1.1" }
[dep-upgrade-202402] upgrade to http/hyper 1.x for tests
2024-02-05 18:57:35 +00:00
hyper-rustls = { version = "0.26", features = ["http2"] }
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
log = "0.4"
thiserror = "1.0"
# ---- used only as build / dev dependencies ----
assert-json-diff = "2.0"
rustc_version = "0.4.0"
static_init = "1.0"
update Cargo dependencies Some aws-* crates have been pinned because newer versions don't compile with rustc 1.73. These crates are only used for integration testing, no crates which are part of the final Garage version are concerned.
2024-08-25 09:19:38 +00:00
aws-config = "=1.1.4"
aws-credential-types = "=1.1.4"
aws-types = "=1.1.4"
aws-sdk-config = "=1.13.0"
aws-sdk-s3 = "=1.14.0"
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 12:02:02 +00:00
(Try to) disable LTO ?
2020-04-10 20:55:01 +00:00
[profile.dev]
[lto-nix] enable LTO for release builds using Nix
2024-02-07 18:41:23 +00:00
#lto = "thin" # disabled for now, adds 2-4 min to each CI build
(Try to) disable LTO ?
2020-04-10 20:55:01 +00:00
lto = "off"
Enable debug info in release profile
2020-04-16 18:46:43 +00:00
[profile.release]
Cargo.toml: Enable full LTO in release builds and thin in dev builds.
2024-02-07 16:11:27 +00:00
lto = true
codegen-units = 1
opt-level = "s"
strip = true
Reference in a new issue Copy permalink