Add skeleton for backups, fuse and code sections

This commit is contained in:
Quentin Dufour 2021-11-10 18:05:07 +01:00
parent 1928f59d54
commit 76d21be1b9
No known key found for this signature in database
GPG key ID: A98E9B769E4FF428
5 changed files with 154 additions and 1 deletions

View file

@ -18,7 +18,9 @@
- [Websites (Hugo, Jekyll, Publii...)](./connect/websites.md) - [Websites (Hugo, Jekyll, Publii...)](./connect/websites.md)
- [Repositories (Docker, Nix, Git...)](./connect/repositories.md) - [Repositories (Docker, Nix, Git...)](./connect/repositories.md)
- [CLI tools (rclone, awscli, mc...)](./connect/cli.md) - [CLI tools (rclone, awscli, mc...)](./connect/cli.md)
- [Backups (restic, duplicity...)](./connect/backup.md)
- [Your code (PHP, JS, Go...)](./connect/code.md) - [Your code (PHP, JS, Go...)](./connect/code.md)
- [FUSE (s3fs, goofys, s3backer...)](./connect/fs.md)
- [Reference Manual](./reference_manual/index.md) - [Reference Manual](./reference_manual/index.md)

View file

@ -0,0 +1,33 @@
# Backups (restic, duplicity...)
Backups are essential for disaster recovery but they are not trivial to manage.
Using Garage as your backup target will enable you to scale your storage as needed while ensuring high availability.
## Borg Backup
Borg Backup is very popular among the backup tools but it is not yet compatible with the S3 API.
We recommend using any other tool listed in this guide because they are all compatible with the S3 API.
If you still want to use Borg, you can use it with `rclone mount`.
## Restic
*External links:* [Restic Documentation > Amazon S3](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3)
## Duplicity
*External links:* [Duplicity > man](https://duplicity.gitlab.io/duplicity-web/vers8/duplicity.1.html) (scroll to "URL Format" and "A note on Amazon S3")
## Duplicati
*External links:* [Duplicati Documentation > Storage Providers](https://github.com/kees-z/DuplicatiDocs/blob/master/docs/05-storage-providers.md#s3-compatible)
## knoxite
*External links:* [Knoxite Documentation > Storage Backends](https://knoxite.com/docs/storage-backends/#amazon-s3)
## kopia
*External links:* [Kopia Documentation > Repositories](https://kopia.io/docs/repositories/#amazon-s3)

View file

@ -1 +1,79 @@
# Your code (PHP, JS, Go...) # Your code (PHP, JS, Go...)
If you are developping a new application, you may want to use Garage to store your user's media.
The S3 API that Garage uses is a standard REST API, so as long as you can make HTTP requests,
you can query it. You can check the [S3 REST API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html) from Amazon to learn more.
Developping your own wrapper around the REST API is time consuming and complicated.
Instead, there are some libraries already avalaible.
Some of them are maintained by Amazon, some by Minio, others by the community.
## PHP
- Amazon aws-sdk-php
- [Installation](https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/getting-started_installation.html)
- [Reference](https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html)
- [Example](https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/s3-examples-creating-buckets.html)
## Javascript
- Minio SDK
- [Reference](https://docs.min.io/docs/javascript-client-api-reference.html)
- Amazon aws-sdk-js
- [Installation](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/getting-started.html)
- [Reference](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html)
- [Example](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/s3-example-creating-buckets.html)
## Golang
- Minio minio-go-sdk
- [Reference](https://docs.min.io/docs/golang-client-api-reference.html)
- Amazon aws-sdk-go-v2
- [Installation](https://aws.github.io/aws-sdk-go-v2/docs/getting-started/)
- [Reference](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/s3)
- [Example](https://aws.github.io/aws-sdk-go-v2/docs/code-examples/s3/putobject/)
## Python
- Minio SDK
- [Reference](https://docs.min.io/docs/python-client-api-reference.html)
- Amazon boto3
- [Installation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html)
- [Reference](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html)
- [Example](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html)
## Java
- Minio SDK
- [Reference](https://docs.min.io/docs/java-client-api-reference.html)
- Amazon aws-sdk-java
- [Installation](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html)
- [Reference](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Client.html)
- [Example](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/examples-s3-objects.html)
## Rust
- Amazon aws-rust-sdk
- [Github](https://github.com/awslabs/aws-sdk-rust)
## .NET
- Minio SDK
- [Reference](https://docs.min.io/docs/dotnet-client-api-reference.html)
- Amazon aws-dotnet-sdk
## C++
- Amazon aws-cpp-sdk
## Haskell
- Minio SDK
- [Reference](https://docs.min.io/docs/haskell-client-api-reference.html)

View file

@ -0,0 +1,30 @@
# FUSE (s3fs, goofys, s3backer...)
**WARNING! Garage is not POSIX compatible.
Mounting S3 buckets as filesystems will not provide POSIX compatibility.
If you are not careful, you will lose or corrupt your data.**
Do not use these FUSE filesystems to store any database files (eg. MySQL, Postgresql, Mongo or sqlite),
any daemon cache (dovecot, openldap, gitea, etc.),
and more generally any software that use locking, advanced filesystems features or make any synchronisation assumption.
Ideally, avoid these solutions at all for any serious or production use.
## rclone mount
*External link:* [rclone documentation > rclone mount](https://rclone.org/commands/rclone_mount/)
## s3fs
*External link:* [s3fs github > README.md](https://github.com/s3fs-fuse/s3fs-fuse#examples)
## goofys
*External link:* [goofys github > README.md](https://github.com/kahing/goofys#usage)
## s3backer
*External link:* [s3backer github > manpage](https://github.com/archiecobbs/s3backer/wiki/ManPage)
## csi-s3
*External link:* [csi-s3 Github > README.md](https://github.com/ctrox/csi-s3)

View file

@ -1,5 +1,7 @@
# Repositories (Docker, Nix, Git...) # Repositories (Docker, Nix, Git...)
Whether you need to store and serve binary packages or source code, you may want to deploy a tool referred as a repository or registry.
Garage can also help you serve this content.
## Gitea ## Gitea
@ -45,7 +47,6 @@ If it worked, you should see some content in your gitea bucket (you must configu
``` ```
$ aws s3 ls s3://gitea/avatars/ $ aws s3 ls s3://gitea/avatars/
2021-11-10 12:35:47 190034 616ba79ae2b84f565c33d72c2ec50861 2021-11-10 12:35:47 190034 616ba79ae2b84f565c33d72c2ec50861
``` ```
@ -53,10 +54,19 @@ $ aws s3 ls s3://gitea/avatars/
## Gitlab ## Gitlab
*External link:* [Gitlab Documentation > Object storage](https://docs.gitlab.com/ee/administration/object_storage.html)
## Private NPM Registry (Verdacio)
*External link:* [Verdaccio Github Repository > aws-storage plugin](https://github.com/verdaccio/verdaccio/tree/master/packages/plugins/aws-storage)
## Docker ## Docker
Not yet compatible, follow [#103](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/103). Not yet compatible, follow [#103](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/103).
*External link:* [Docker Documentation > Registry storage drivers > S3 storage driver](https://docs.docker.com/registry/storage-drivers/s3/)
## Nix ## Nix
Nix has no repository in its terminology: instead, it breaks down this concept in 2 parts: binary cache and channel. Nix has no repository in its terminology: instead, it breaks down this concept in 2 parts: binary cache and channel.