pict-rs/releases/0.4.1.md

 # pict-rs 0.4.1

## Overview

Today I am releasing pict-rs 0.4.1. This includes a couple bugfixes and a few requested features.
Most notably is the object-storage migration now uploads files concurrently. It should be much
faster than before.

### Features

- [Add Concurrency to Migration](#add-concurrency-to-migration)
- [Improved Error Messages](#improved-error-messages)
- [Configurable Object Storage Timeouts](#configurable-object-storage-timeouts)
- [Set Content-Type in Object Storage](#set-content-type-in-object-storage)


### Bugfix

- [Alias Cleanup](#alias-cleanup)
- [ImageMagick Security Policy](#imagemagick-security-policy)
- [Broken Pipe on Upload Error](#broken-pipe-on-upload-error)


## Upgrade Notes

There's no significant changes from 0.4, so upgrading should be as simple as pulling a newer version
of pict-rs.


## Descriptions

### Add Concurrency to Migration

Adding concurrency means changing how pict-rs kept track of migration progress. Previously it was
setting a single flag with the "newest" hash that it had migrated, so a resuming migration would
iterate until it found that hash before continuing linearly. Now, pict-rs keeps track of each
individual file. That way it does not matter the order in which files are migrated.

The migration is hardcoded to try moving 32 files at a time, so ideally this would mean a 32x
speedup over the previous linear version. Whether it actually ends up being so fast is yet to be
seen.


### Improved Error Messages

Previously errors encountered when failing to run a process were pretty opaque. pict-rs relies on
running the `ffmpeg`, `ffprobe`, `magick`, and `exiftool` commands, and if it could not run them it
would print something vague like "File not found" or "Permission Denied", without much context for
what wasn't found or what was denied.

Now, pict-rs properly informs the admin that "Required command magick not found" or "Cannot run
command ffmpeg due to invalid permissions on binary".

Additionally, the error messages returned in the API were not very useful either. "Error in ffmpeg"
or "Error in imagemagick" are not super useful. Now pict-rs will return the "root cause" of errors,
which is generally more helpful. There's still more work to be done here, so this topic will be
mentioned in future releases.


### Configurable Object Storage Timeouts

pict-rs now provides the ability to increase the HTTP timeout and the Signature Expiration for
requests to object storage. This may not be super helpful, but if your object storage is
particularly slow you can now increase the values.

The relevant variables are `PICTRS__SToRE__SIGNATURE_EXPIRATION` and
`PICTRS__STORE__CLIENT_TIMEOUT`.

In toml, that would be
```toml
[store]
signature_expiration = 15
client_timeout = 30
```


### Set Content-Type in Object Storage

This is not immediately useful, but it means that the object storage will begin knowing about the
content-type of uploaded files, so in the future when pict-rs supports serving directly from
object-storage it will provide a meaningful content-type header to browsers.


### Alias Cleanup

There was a bug introduced in 0.4 that made using the "delete" endpoint not actually remove the file
from storage. It would un-link the alias so it couldn't be accessed, but full deletion never
happened. This was a result of me reordering a couple lines in the alias cleanup. 0.4.1 fixes this
issue by un-reordering them.


### ImageMagick Security Policy

The security policy used in the pict-rs docker container has been updated to allow for more file
types. This makes the new JSON-based imagemagick and ffmpeg output parsers work, and might make one
or two rejected files now no longer be rejected. I don't know the full extent to which this was a
real problem in 0.4, but regardless it's better now.


### Broken Pipe on Upload Error

Previously if a file was uploaded that was too large, it would be rejected immediately with a file
size error. This sometimes resulted in clients receiving a Broken Pipe error on the connection
instead of properly receiving an HTTP Response. This has been fixed in the `actix-form-data` library
(that I also maintain) by ensuring that we read the entire upload (dropping the bytes immediately)
before replying.