mirror of
https://git.deuxfleurs.fr/Deuxfleurs/garage.git
synced 2024-11-13 03:31:30 +00:00
Finish writing about Garage features, and fix from-source instructions
This commit is contained in:
Alex Auvolat
parent
f6aebefcc9
commit
1d0a610690
3 changed files with 110 additions and 49 deletions
|
|
@ -5,12 +5,14 @@ weight = 25
|
|||
|
||||
## Configuring a bucket for website access
|
||||
|
||||
There are two methods to expose buckets as website:
|
||||
There are three methods to expose buckets as website:
|
||||
|
||||
1. using the PutBucketWebsite S3 API call, which is allowed for access keys that have the owner permission bit set
|
||||
|
||||
2. from the Garage CLI, by an adminstrator of the cluster
|
||||
|
||||
3. using the Garage administration API
|
||||
|
||||
The `PutBucketWebsite` API endpoint [is documented](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) in the official AWS docs.
|
||||
This endpoint can also be called [using `aws s3api`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-website.html) on the command line.
|
||||
The website configuration supported by Garage is only a subset of the possibilities on Amazon S3: redirections are not supported, only the index document and error document can be specified.
|
||||
|
|
|
|||
|
|
@ -20,57 +20,76 @@ sudo apt-get update
|
|||
sudo apt-get install build-essential
|
||||
```
|
||||
|
||||
## Using source from the Gitea repository (recommended)
|
||||
## Building from source from the Gitea repository
|
||||
|
||||
The primary location for Garage's source code is the
|
||||
[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage).
|
||||
[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage),
|
||||
which contains all of the released versions as well as the code
|
||||
for the developpement of the next version.
|
||||
|
||||
Clone the repository and build Garage with the following commands:
|
||||
Clone the repository and enter it as follows:
|
||||
|
||||
```bash
|
||||
git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git
|
||||
cd garage
|
||||
cargo build
|
||||
```
|
||||
|
||||
Be careful, as this will make a debug build of Garage, which will be extremely slow!
|
||||
To make a release build, invoke `cargo build --release` (this takes much longer).
|
||||
|
||||
The binaries built this way are found in `target/{debug,release}/garage`.
|
||||
|
||||
## Using source from `crates.io`
|
||||
|
||||
Garage's source code is published on `crates.io`, Rust's official package repository.
|
||||
This means you can simply ask `cargo` to download and build this source code for you:
|
||||
If you wish to build a specific version of Garage, check out the corresponding tag. For instance:
|
||||
|
||||
```bash
|
||||
cargo install garage
|
||||
git tag # List available tags
|
||||
git checkout v0.8.0 # Change v0.8.0 with the version you wish to build
|
||||
```
|
||||
|
||||
That's all, `garage` should be in `$HOME/.cargo/bin`.
|
||||
Otherwise you will be building a developpement build from the `main` branch
|
||||
that includes all of the changes to be released in the next version.
|
||||
Be careful that such a build might be unstable or contain bugs,
|
||||
and could be incompatible with nodes that run stable versions of Garage.
|
||||
|
||||
You can add this folder to your `$PATH` or copy the binary somewhere else on your system.
|
||||
For instance:
|
||||
Finally, build Garage with the following command:
|
||||
|
||||
```bash
|
||||
sudo cp $HOME/.cargo/bin/garage /usr/local/bin/garage
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
The binary built this way can now be found in `target/release/garage`.
|
||||
You may simply copy this binary to somewhere in your `$PATH` in order to
|
||||
have the `garage` command available in your shell, for instance:
|
||||
|
||||
## Selecting features to activate in your build
|
||||
```bash
|
||||
sudo cp target/release/garage /usr/local/bin/garage
|
||||
```
|
||||
|
||||
Garage supports a number of compilation options in the form of Cargo features,
|
||||
If you are planning to develop Garage,
|
||||
you might be interested in producing debug builds, which compile faster but run slower:
|
||||
this can be done by removing the `--release` flag, and the resulting build can then
|
||||
be found in `target/debug/garage`.
|
||||
|
||||
## List of available Cargo feature flags
|
||||
|
||||
Garage supports a number of compilation options in the form of Cargo feature flags,
|
||||
which can be used to provide builds adapted to your system and your use case.
|
||||
The following features are available:
|
||||
To produce a build with a given set of features, invoke the `cargo build` command
|
||||
as follows:
|
||||
|
||||
| Feature | Enabled | Description |
|
||||
| ------- | ------- | ----------- |
|
||||
| `bundled-libs` | BY DEFAULT | Use bundled version of sqlite3, zstd, lmdb and libsodium |
|
||||
| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium if available (exclusive with `bundled-libs`, build using `cargo build --no-default-features --features system-libs`) |
|
||||
| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your Garage cluster must have it enabled as well) |
|
||||
| `kubernetes-discovery` | optional | Enable automatic registration and discovery of cluster nodes through the Kubernetes API |
|
||||
| `metrics` | BY DEFAULT | Enable collection of metrics in Prometheus format on the admin API |
|
||||
```bash
|
||||
# This will build the default feature set plus feature1, feature2 and feature3
|
||||
cargo build --release --features feature1,feature2,feature3
|
||||
# This will build ONLY feature1, feature2 and feature3
|
||||
cargo build --release --no-default-features \
|
||||
--features feature1,feature2,feature3
|
||||
```
|
||||
|
||||
The following feature flags are available in v0.8.0:
|
||||
|
||||
| Feature flag | Enabled | Description |
|
||||
| ------------ | ------- | ----------- |
|
||||
| `bundled-libs` | *by default* | Use bundled version of sqlite3, zstd, lmdb and libsodium |
|
||||
| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium<br>if available (exclusive with `bundled-libs`, build using<br>`cargo build --no-default-features --features system-libs`) |
|
||||
| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your<br>Garage cluster must have it enabled as well) |
|
||||
| `kubernetes-discovery` | optional | Enable automatic registration and discovery<br>of cluster nodes through the Kubernetes API |
|
||||
| `metrics` | *by default* | Enable collection of metrics in Prometheus format on the admin API |
|
||||
| `telemetry-otlp` | optional | Enable collection of execution traces using OpenTelemetry |
|
||||
| `sled` | BY DEFAULT | Enable using Sled to store Garage's metadata |
|
||||
| `sled` | *by default* | Enable using Sled to store Garage's metadata |
|
||||
| `lmdb` | optional | Enable using LMDB to store Garage's metadata |
|
||||
| `sqlite` | optional | Enable using Sqlite3 to store Garage's metadata |
|
||||
|
|
|
|||
|
|
@ -26,6 +26,11 @@ a storage plan that best exploits the available storage capacity while satisfyin
|
|||
To learn more about geo-distributed Garage clusters,
|
||||
read our documentation on [setting up a real-world deployment](@/documentation/cookbook/real-world.md).
|
||||
|
||||
### Standalone/self-contained
|
||||
|
||||
Garage is extremely simple to deploy, and does not depend on any external service to run.
|
||||
This makes setting up and administering storage clusters, we hope, as easy as it could be.
|
||||
|
||||
### Flexible topology
|
||||
|
||||
A Garage cluster can very easily evolve over time, as storage nodes are added or removed.
|
||||
|
|
@ -42,11 +47,11 @@ of going through a central bottleneck (the leader node).
|
|||
As a consequence, requests can be handled much faster, even in cases where latency
|
||||
between cluster nodes is important (see our [benchmarks](@/documentation/design/benchmarks/index.md) for data on this).
|
||||
This is particularly usefull when nodes are far from one another and talk to one other through standard Internet connections.
|
||||
|
||||
|
||||
### Several replication modes
|
||||
|
||||
Garage supports a variety of replication modes, with 1 copy, 2 copies or 3 copies of your data,
|
||||
and with various levels of consistency.
|
||||
and with various levels of consistency, in order to adapt to a variety of usage scenarios.
|
||||
Read our reference page on [supported replication modes](@/documentation/reference-manual/configuration.md#replication-mode)
|
||||
to select the replication mode best suited to your use case (hint: in most cases, `replication_mode = "3"` is what you want).
|
||||
|
||||
|
|
@ -54,32 +59,67 @@ to select the replication mode best suited to your use case (hint: in most cases
|
|||
|
||||
A storage bucket can easily be configured to be served directly by Garage as a static web site.
|
||||
Domain names for multiple websites directly map to bucket names, making it easy to build
|
||||
a platform for your user's to autonomously build and host their websites over Garage.
|
||||
a platform for your users to autonomously build and host their websites over Garage.
|
||||
Surprisingly, none of the other alternative S3 implementations we surveyed (such as Minio
|
||||
or CEPH) support publishing static websites from S3 buckets, a feature that is however
|
||||
directly inherited from S3 on AWS.
|
||||
Read more on our [dedicated documentation page](@/documentation/cookbook/exposing-websites.md).
|
||||
|
||||
### Bucket names as aliases
|
||||
|
||||
- the same bucket may have multiple names (useful when exposing websites for example)
|
||||
In Garage, a bucket may have several names, known as aliases.
|
||||
Aliases can easily be added and removed on demand:
|
||||
this allows to easily rename buckets if needed
|
||||
without having to copy all of their content, something that cannot be done on AWS.
|
||||
For buckets served as static websites, having multiple aliases for a bucket can allow
|
||||
exposing the same content under different domain names.
|
||||
|
||||
- bucket renaming is possible
|
||||
Garage also supports bucket aliases which are local to a single user:
|
||||
this allows different users to have different buckets with the same name, thus avoiding naming collisions.
|
||||
This can be helpfull for instance if you want to write an application that creates per-user buckets with always the same name.
|
||||
|
||||
- Scoped buckets: 2 users can have a different bucket with the same name -> avoid collision. Helpful if you want to write an application that creates per-user bucket always with the same name.
|
||||
|
||||
### Standalone/self contained
|
||||
|
||||
|
||||
### Integration with Kubernetes and Nomad
|
||||
|
||||
Many node discovery methods: Kubernetes integration, Nomad integration through Consul
|
||||
|
||||
### Support for changing IP addresses
|
||||
|
||||
(as long as all nodes don't change their IP at the same time)
|
||||
This feature is totally invisible to S3 clients and does not break compatibility with AWS.
|
||||
|
||||
### Cluster administration API
|
||||
|
||||
Garage provides a fully-fledged REST API to administer your cluster programatically.
|
||||
Functionnality included in the admin API include: setting up and monitoring
|
||||
cluster nodes, managing access credentials, and managing storage buckets and bucket aliases.
|
||||
A full reference of the administration API is available [here](@/documentation/reference-manual/admin-api.md).
|
||||
|
||||
### Metrics and traces
|
||||
|
||||
### (experimental) K2V API
|
||||
|
||||
Garage makes some internal metrics available in the Prometheus data format,
|
||||
which allows you to build interactive dashboards to visualize the load and internal state of your storage cluster.
|
||||
|
||||
For developpers and performance-savvy administrators,
|
||||
Garage also supports exporting traces of what it does internally in OpenTelemetry format.
|
||||
This allows to monitor the time spent at various steps of the processing of requests,
|
||||
in order to detect potential performance bottlenecks.
|
||||
|
||||
### Kubernetes and Nomad integrations
|
||||
|
||||
Garage can automatically discover other nodes in the cluster thanks to integration
|
||||
with orchestrators such as Kubernetes and Nomad (when used with Consul).
|
||||
This eases the configuration of your cluster as it removes one step where nodes need
|
||||
to be manually connected to one another.
|
||||
|
||||
### Support for changing IP addresses
|
||||
|
||||
As long as all of your nodes don't thange their IP address at the same time,
|
||||
Garage should be able to tolerate nodes with changing/dynamic IP addresses,
|
||||
as nodes will regularly exchange the IP addresses of their peers and try to
|
||||
reconnect using newer addresses when existing connections are broken.
|
||||
|
||||
### K2V API (experimental)
|
||||
|
||||
As part of an ongoing research project, Garage can expose an experimental key/value storage API called K2V.
|
||||
K2V is made for the storage and retrieval of many small key/value pairs that need to be processed in bulk.
|
||||
This completes the S3 API with an alternative that can be used to easily store and access metadata
|
||||
related to objects stored in an S3 bucket.
|
||||
|
||||
In the context of our research project, [Aérogramme](https://aerogramme.deuxfleurs.fr),
|
||||
K2V is used to provide metadata and log storage for operations on encrypted e-mail storage.
|
||||
|
||||
Learn more on the specification of K2V [here](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)
|
||||
and on how to enable it in Garage [here](@/documentation/reference-manual/k2v.md).
|
||||
|
|
|
|||
Loading…
Reference in a new issue