mirrors/garage
1
0
Fork 0
mirror of https://git.deuxfleurs.fr/Deuxfleurs/garage.git synced 2024-11-14 12:11:17 +00:00
Code Issues Projects Releases Packages Wiki Activity
1286 commits 27 branches 74 tags 44 MiB
Commit graph

241 commits

Author SHA1 Message Date
Alex Auvolat
26849ed066
Add step to 0.6.0 migration guide 2022-01-27 14:31:25 +01:00
Alex Auvolat
ea7fb901eb
Implement {Put,Get,Delete}BucketCors and CORS in general 
- OPTIONS request against API endpoint
- Returning corresponding CORS headers on API calls
- Returning corresponding CORS headers on website GET's
 2022-01-24 11:58:00 +01:00
Quentin Dufour
94f0e7c135 Test ListParts endpoint with awscli 2022-01-21 10:42:35 +01:00
Alex Auvolat
60c0033c8b
Update documentation 2022-01-13 14:25:22 +01:00
Alex Auvolat
6617a72220
Implement UploadPartCopy 2022-01-13 13:58:47 +01:00
Quentin
b4592a00fe Implement ListMultipartUploads (#171) 
Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2.

It took me some times as I wanted to propose the following things:
  - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476).
  - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator)
  - An IO closure to make unit tests possibles.
  - Unit tests, to track regressions and document how to interact with the code
  - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK.

Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications.

Some points that we might want to discuss:
  - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation.
  - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token.
  - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now).

Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/171
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
 2022-01-12 19:04:55 +01:00
Alex Auvolat
9cb2e9e57c
Add documentation to migrate to 0.6 2022-01-10 14:42:58 +01:00
Alex Auvolat
8f39360f22
Update documentation 2022-01-07 17:14:37 +01:00
trinity-1686a
945b75dbf1 update s3 compatibility list (#177) 
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/177
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 15:05:54 +01:00
trinity-1686a
1eb972b1ac Add compression using zstd (#173) 
fix #27

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/173
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 11:26:43 +01:00
Quentin Dufour
3b3a1f275f
Add a second plot 2021-12-13 11:58:03 +01:00
Quentin Dufour
dba9af2968
Update benchmark 2021-12-09 18:42:45 +01:00
Quentin Dufour
e9358054ac
Typos and dead links 2021-12-08 14:40:14 +01:00
Quentin Dufour
f9e5520ffb
Add a benchmark page with a first benchmark 2021-12-08 11:30:07 +01:00
Alex Auvolat
4b369347c0
S3 compatibility target 2021-12-06 17:18:45 +01:00
Alex Auvolat
224c89ad6e
Reorganize and improve documentation 2021-12-06 16:33:01 +01:00
Quentin Dufour
7c2037ba87
WIP front page garage 2021-12-06 15:33:43 +01:00
Alex Auvolat
c94406f428
Improve how node roles are assigned in Garage 
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
 2021-11-16 16:05:53 +01:00
Trinity Pointard
53888995bd update doc and comments 2021-11-16 15:41:41 +01:00
Trinity Pointard
57df9c6e2d add s3_api.root_domain to doc book 2021-11-16 15:41:41 +01:00
adrien
cdeb5b4dbb added link to RFID Garage talk (#155) 
Co-authored-by: ADRN <adrien@luxeylab.net>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/155
Co-authored-by: adrien <adrien@luxeylab.net>
Co-committed-by: adrien <adrien@luxeylab.net>
 2021-11-16 15:39:08 +01:00
Quentin Dufour
100aad8bf4
Add rclone mount doc 2021-11-10 18:38:17 +01:00
Quentin Dufour
80a87929b0
Improve CLI documentation 2021-11-10 18:18:34 +01:00
Quentin Dufour
76d21be1b9
Add skeleton for backups, fuse and code sections 2021-11-10 18:05:07 +01:00
Quentin Dufour
1928f59d54
Add documentation for Gitea. 2021-11-10 12:41:09 +01:00
Quentin Dufour
323514be15
Documentation for Nix binary cache 2021-11-10 10:02:22 +01:00
Quentin Dufour
ad8d5139cf
hugo deploy does not build website, fix doc 2021-11-10 10:02:22 +01:00
Alex Auvolat
08b1e8a7ea
Move design draft to separate file; write about GC in internals 2021-11-09 12:25:33 +01:00
Quentin Dufour
8e25a37f0e
Add documentation for nginx 2021-11-08 12:20:40 +01:00
Quentin Dufour
e342db19aa
Add documentation about Gateways 2021-11-08 12:20:40 +01:00
Quentin Dufour
f3405b6378
Doc about exposing your website 2021-11-08 12:20:40 +01:00
Quentin Dufour
860ccf2811
Harden Garage's systemd service 2021-11-08 12:20:40 +01:00
Quentin Dufour
9df7559446
Documentation for hugo, jekyll and publii 2021-11-08 12:20:40 +01:00
Quentin Dufour
a97467075d
Add documentation for synapse-s3-storage-provider 2021-11-08 12:20:40 +01:00
Trinity Pointard
da6efb4b23 fix missing bootstrap_peers in doc 2021-11-05 11:21:50 +01:00
Quentin Dufour
e93d7fb228
Add Peertube + improve CLI instructions 2021-11-03 14:39:14 +01:00
Quentin Dufour
eaf54efb25
Add doc for Nextcloud 2021-11-03 14:07:55 +01:00
Quentin Dufour
93f8d59e4c
Extract toolchain build from the CI 2021-10-29 11:34:01 +02:00
ADRN
4e8af1d956 Modified the 'Funding' sentence to remove 'promise' since we actually got the first instalment 2021-10-26 13:34:28 +02:00
Alex Auvolat
3e7f766d95
CLI: default rpc_host 2021-10-26 11:36:30 +02:00
Alex Auvolat
b2c51844a1
Add download link on homepage 2021-10-25 15:55:30 +02:00
Alex Auvolat
de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00
Quentin Dufour
dc017a0cab
Build Garage with Nix 2021-10-19 16:56:07 +02:00
Alex Auvolat
1aed317818
Small changes on NGI kickoff talk 2021-10-07 11:12:34 +02:00
Quentin
c5574c8409 Add links and put logos in a flexbox 2021-09-28 10:21:10 +02:00
mricher
de0228ca2a
Doc: add funding disclaimer for NGI/EU grant 
PNG logs optimized, render to be checked by @quentin. Fix #106.
 2021-09-25 17:21:07 +02:00
Alex Auvolat
df345e37db
Add sticker and NGI kickoff talk 2021-09-12 13:37:33 +02:00
Alex Auvolat
42f692b1e0
Documentation fixes (typo and small reorganization) 2021-05-31 23:55:51 +02:00
Alex Auvolat
14fd3df654
Write about S3 client configuration 2021-05-31 17:41:21 +02:00
Alex Auvolat
56ac9fd460
Updates to documentation 2021-05-31 17:23:35 +02:00
Alex Auvolat
d76a8576f4
Reorganize documentation 2021-05-31 17:13:36 +02:00
Trinity Pointard
289521886b make most changes suggested during install-party 2021-05-29 21:37:49 +02:00
Alex Auvolat
ebd21b325e
Write documentation on configuration file and other improvements 2021-05-28 18:00:59 +02:00
Alex Auvolat
339c611789
Add links to git/matrix/drone on top of documentation 2021-05-03 10:45:45 +02:00
Alex Auvolat
a981244f11
Replace talk links with permalinks 2021-04-30 15:51:32 +02:00
LUXEY Adrien
be3b1d8f91 created doc/talks subfolder and added my talk and the previous one done at the wide team 2021-04-30 15:37:43 +02:00
Alex Auvolat
bf36f1f16a
Update documentation 2021-04-27 16:37:10 +02:00
Alex Auvolat
9589d10165
fix command for adding node 2021-04-27 16:37:08 +02:00
Alex Auvolat
f9d77b6cd9
Section on recovering from failures 2021-04-27 16:37:08 +02:00
LUXEY Adrien
ee00ac59b7
[doc] Added mention that GarageHQ is hosted with Garage 2021-04-27 16:37:08 +02:00
Alex Auvolat
c9943aab1f
fix typos 2021-04-27 16:37:07 +02:00
LUXEY Adrien
d0692b75b0
intro.md: fix some typos, errors & dead links, plus some stylistic stuff 
modifié :         doc/book/src/intro.md
 2021-04-27 16:37:07 +02:00
Quentin Dufour
4a6ed223dc
Fix garage_util description 2021-04-27 16:37:07 +02:00
Quentin Dufour
c409ee89f6
Fix some typos 2021-04-27 16:37:07 +02:00
Quentin Dufour
c1d64333c5
Fix a table in the doc 2021-04-27 16:37:04 +02:00
Alex Auvolat
c5e24de159 Center logo in book, add book CI 2021-03-19 14:25:57 +01:00
Alex Auvolat
4348bde180 Merge branch 'dev-0.2' 2021-03-18 19:27:02 +01:00
Quentin Dufour
ea21c54434 Add handle files section to the doc 2021-03-17 22:44:35 +01:00
Quentin Dufour
1a5af9d1fc WIP getting started 2021-03-17 22:06:37 +01:00
Quentin Dufour
b82a61fba2 Simplify our README 2021-03-17 20:58:30 +01:00
Quentin Dufour
44d0815ff9 Wrote daemon 2021-03-17 20:04:27 +01:00
Quentin Dufour
468e45ed7f WIP doc 2021-03-17 18:01:06 +01:00
Quentin Dufour
60f994a118 Working on the getting started guide 2021-03-17 17:24:11 +01:00
Quentin Dufour
002538f92c Refactor file organization 2021-03-17 16:15:18 +01:00
Quentin Dufour
c50113acf3 Work on structure + Getting started is reworked 2021-03-17 15:44:29 +01:00
Quentin Dufour
0afc701a69 Doc skeleton + intro 2021-03-17 14:44:14 +01:00
Alex Auvolat
797cda1c33 Add logo in repo 2021-03-17 10:26:50 +01:00
Alex Auvolat
46c7226fe4 New logo!! 2021-03-17 08:41:07 +01:00
Alex Auvolat
d7e148d302 Description of MultiDC MagLev 2021-02-25 11:37:42 +01:00
Alex Auvolat
0522983aec precisions 2021-02-25 11:27:13 +01:00
Alex Auvolat
fdf908e845 Add precisions 2021-02-25 11:23:22 +01:00
Alex Auvolat
2b4b69938f Add write-up about load-balancing 2021-02-25 11:18:44 +01:00
Alex Auvolat
0ddfee92c5 add precision 2021-02-19 19:11:55 +01:00
Alex Auvolat
5d1fa591d9 Add compatibility list 2021-02-19 19:10:23 +01:00
Alex Auvolat
f26c795c99 Add warning 2021-01-23 19:15:57 +01:00
Alex Auvolat
6f6bf23bec Move doc files here from Deuxfleur's site repo 2021-01-23 19:12:10 +01:00
Alex Auvolat
f844d4ee9b Add slide on consistency 2020-12-01 17:42:13 +01:00
Alex Auvolat
7642229d54 Two new slides 2020-12-01 14:31:13 +01:00
Alex Auvolat
ad432eb154 Add some first technical slides 2020-12-01 13:46:23 +01:00
Quentin
9a57a0319a Talk 2020-11-30 17:32:37 +01:00
Alex Auvolat
9b3aabfcbf Add talk template 2020-11-30 15:54:31 +01:00