pict-rs 0.4.5

Overview

pict-rs 0.4.5 contains a new commandline option for improving throughput when migrating storage, as well as a change in behavior for the delete endpoint. Additionally there are a few dependency upgrades.

Features

Migration Concurrency

Changes

Inline Delete

Upgrade Notes

There's no significant changes from 0.4.4, so upgrading should be as simple as pulling a new version of pict-rs.

Descriptions

Migration Concurrency

When migrating from filesystem storage to object storage, pict-rs initially migrated every file sequentially. This was extremely slow, so in 0.4.1 this was made concurrent. Since then, the object storage migration has moved 32 files at a time. This still might be too slow for some large deployments, so pict-rs 0.4.5 is making this value configurable.

Thew new --concurrency flag can be passed to the migrate-store subcommand with any integer value. By default, pict-rs sets this value to 32, but for large deployments, setting it to 128, 256, or even 512 might be useful. Note that increasing the concurrency will increase the number of connections to the object storage provider.

Example:

$ pict-rs \
    migrate-store \
        --concurrency 32 \
    filesystem \
        -p /path/to/files \
    object-storage \
        -e https://object-storage-endpoint \
        -b bucket-name \
        -r region \
        -a access-key \
        -s secret-key \
    sled \
        -p /path/to/sled-repo

Inline Delete

Previously, the endpoint for deleting images would always return an HTTP 204 status after queueing a job to delete the image. This meant that if an invalid delete token was provided, or if the deletion failed for some other reason, the client making the request would not be informed of this. pict-rs 0.4.5 now performs this deletion inline to ensure the client is made aware of problems.

1.9 KiB Raw Permalink Blame History