Commit graph

19 commits

Author SHA1 Message Date
Arun Raghavan
8f96509f03 aws: s3: Enable tests again
We lost the environment variable checks during the addition of the
putobjectsink tests, which caused failures on MR branches.

It would be nicer to use some other mechanism to validate the tests can
run, so we don't count on only the environmnent, but for now this will
have to do.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1629>
2024-06-18 11:58:43 -04:00
Sebastian Dröge
5aedcab32f Revert "aws: s3: Re-enable tests"
This reverts commit b4b56eb282.
The tests are still failing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1624>
2024-06-18 08:50:07 +03:00
Arun Raghavan
b4b56eb282 aws: s3: Re-enable tests
These seem to have stopped working due to bad/rotated creds. Should work
fine now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1623>
2024-06-17 06:08:18 -04:00
François Laignel
9f27bde36a aws: use fixed BehaviorVersion
Quoting [`BehaviorVersion` documentation]:

> Over time, new best-practice behaviors are introduced. However, these
> behaviors might not be backwards compatible. For example, a change which
> introduces new default timeouts or a new retry-mode for all operations might
> be the ideal behavior but could break existing applications.

This commit uses `BehaviorVersion::v2023_11_09()`, which is the latest
major version at the moment. When a new major version is released, the method
will be deprecated, which will warn us of the new version and let us decide
when to upgrade, after any changes if required. This is safer that using
`latest()` which would silently use a different major version, possibly
breaking existing code.

[`BehaviorVersion` documentation]: https://docs.rs/aws-config/1.1.8/aws_config/struct.BehaviorVersion.html

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1520>
2024-03-26 17:44:16 +01:00
Sebastian Dröge
73a53e38c4 aws: s3: Disable remaining tests too for now
They fail state changes, which cases `GstHarness` to abort.
2024-01-16 09:13:41 +02:00
Arun Raghavan
fd3675aac0 aws: s3: Temporarily disable putobject tests
Disabling while we figure out why it's failing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1429>
2024-01-15 21:43:25 -05:00
Arun Raghavan
8b18ca15b5 Revert "aws: Disable putobjectsink tests for now"
This reverts commit b128d127c2.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/472
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1416>
2024-01-11 15:38:36 -05:00
Sebastian Dröge
b128d127c2 aws: Disable putobjectsink tests for now
See https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/472

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1413>
2023-12-22 13:25:12 +02:00
Arun Raghavan
410d104ad6 aws: s3putobjectsink: Add a flush-on-error property
Makes sure we can send out data even if the pipeline shutdown in error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1337>
2023-12-18 16:13:48 -05:00
Arun Raghavan
12dbf50ddc aws: s3putobjectsink: Add some thresholds for flushing
Lets us connect when we perform a flush

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1337>
2023-12-18 16:13:48 -05:00
Arun Raghavan
a54b2dd39e aws: s3: Add a new awss3putobjectsink
When streaming small amounts of data, using awss3sink might not be a
good idea, as we need to accumulate at least 5 MB of data for a
multipart upload (or we flush on EOS).

The alternative, while inefficient, is to do a complete PutObject of
_all_ the data periodically so as to not lose data in case of a pipeline
failure. This element makes a start on this idea by doing a PutObject
for every buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1337>
2023-12-18 10:39:23 -05:00
Sebastian Dröge
66c62d69b9 aws: Stop using deprecated aws_config function in the test 2023-11-18 10:16:24 +02:00
Arun Raghavan
771741c10c Revert "s3: tests: Remove emoji-based tests for now"
This reverts commit a49a5dcb11.

Now that hotdoc should work with emoji, let's bring the tests back.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1386>
2023-11-09 11:50:53 -05:00
Arun Raghavan
a49a5dcb11 s3: tests: Remove emoji-based tests for now
These break hotdoc, which we need to fix first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1333>
2023-10-24 12:52:12 -04:00
Arun Raghavan
51129febeb aws: s3sink: Fix handling of special characters in key
Properly URL-encode the string if needed, and add some tests for a
couple of cases.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/431
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1333>
2023-10-24 12:52:12 -04:00
Sebastian Dröge
6fe806c2b5 aws: Update to AWS SDK 0.55/0.25
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1152>
2023-03-31 09:12:26 +00:00
Sebastian Dröge
3b4c48d9f5 Fix various new clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1062>
2023-01-25 10:31:19 +02:00
Arun Raghavan
54c84a7211 aws: Skip s3 test on Windows until we figure out why it times out 2022-11-02 13:14:08 -04:00
Arun Raghavan
e66378d254 aws: Add a test for s3src/s3sink
This does rely on AWS credentials being provided in the environment, but
the test will be ignored if those are missing.
2022-10-18 09:51:34 +00:00