From 0d93910a394b093b5c310346adf52fef0a060bb1 Mon Sep 17 00:00:00 2001 From: Wonchul Lee Date: Tue, 22 Dec 2020 16:58:22 +0900 Subject: [PATCH] tutorial: Sync doc link with gstreamer crates --- tutorial/tutorial-1.md | 11 ++++++----- tutorial/tutorial-2.md | 4 ++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/tutorial/tutorial-1.md b/tutorial/tutorial-1.md index 80eb2d87..1dea773d 100644 --- a/tutorial/tutorial-1.md +++ b/tutorial/tutorial-1.md @@ -222,7 +222,8 @@ pub fn register(plugin: &gst::Plugin) -> Result<(), glib::BoolError> { This defines a Rust wrapper type for our element subclass. This is similar to `gst::Element` and others and provides the public interface to our element. -In addition, we also define a `register` function (the one that is already called from our `plugin_init` function). When `register` function is called it registers the element factory with GStreamer based on the type ID, to be able to create new instances of it with the name “rsrgb2gray” (e.g. when using [`gst::ElementFactory::make`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/struct.ElementFactory.html#method.make)). The `static_type` function will register the type with the GObject type system on the first call and the next time it's called (or on all the following calls) it will return the type ID. +In addition, we also define a `register` function (the one that is already called from our `plugin_init` function). When `register` function is called it registers the element factory with GStreamer based on the type ID, to be able to create new instances of it with the name “rsrgb2gray” (e.g. when using [`gst::ElementFactory::make`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/struct.ElementFactory.html#method.make)). The `static_type` function will register the type with the GObject type system on the first call and the next time it's called (or on all the following calls) it will return the type ID. + ## Type Class & Instance Initialization @@ -262,7 +263,7 @@ impl ObjectSubclass for Rgb2Gray { In the `new` function we return our empty struct. -In the `class_init` function we, again, set up some metadata for our new element. In this case these are a description, a classification of our element, a longer description and the author. The metadata can later be retrieved and made use of via the [`Registry`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/struct.Registry.html) and [`PluginFeature`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/struct.PluginFeature.html)/[`ElementFactory`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/struct.ElementFactory.html) API. We also configure the `BaseTransform` class and define that we will never operate in-place (producing our output in the input buffer), and that we don’t want to work in passthrough mode if the input/output formats are the same. +In the `class_init` function we, again, set up some metadata for our new element. In this case these are a description, a classification of our element, a longer description and the author. The metadata can later be retrieved and made use of via the [`Registry`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/struct.Registry.html) and [`PluginFeature`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/struct.PluginFeature.html)/[`ElementFactory`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/struct.ElementFactory.html) API. We also configure the `BaseTransform` class and define that we will never operate in-place (producing our output in the input buffer), and that we don’t want to work in passthrough mode if the input/output formats are the same. Additionally we need to implement various traits on the Rgb2Gray struct, which will later be used to override virtual methods of the various parent classes of our element. For now we can keep the trait implementations empty. There is one trait implementation required per parent class. @@ -439,7 +440,7 @@ impl ObjectSubclass for Rgb2Gray { } ``` -We define a new struct State that contains the input and output caps, stored in a [`VideoInfo`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer_video/struct.VideoInfo.html). `VideoInfo` is a struct that contains various fields like width/height, framerate and the video format and allows to conveniently with the properties of (raw) video formats. We have to store it inside a [`Mutex`](https://doc.rust-lang.org/std/sync/struct.Mutex.html) in our `Rgb2Gray` struct as this can (in theory) be accessed from multiple threads at the same time. +We define a new struct State that contains the input and output caps, stored in a [`VideoInfo`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer_video/struct.VideoInfo.html). `VideoInfo` is a struct that contains various fields like width/height, framerate and the video format and allows to conveniently with the properties of (raw) video formats. We have to store it inside a [`Mutex`](https://doc.rust-lang.org/std/sync/struct.Mutex.html) in our `Rgb2Gray` struct as this can (in theory) be accessed from multiple threads at the same time. Whenever input/output caps are configured on our element, the `set_caps` virtual method of `BaseTransform` is called with both caps (i.e. in the very beginning before the data flow and whenever it changes), and all following video frames that pass through our element should be according to those caps. Once the element is shut down, the `stop` virtual method is called and it would make sense to release the `State` as it only contains stream-specific information. We’re doing this by adding the following to the `BaseTransformImpl` trait implementation @@ -697,9 +698,9 @@ impl BaseTransformImpl for Rgb2Gray { } ``` -What happens here is that we first of all lock our state (the input/output `VideoInfo`) and error out if we don’t have any yet (which can’t really happen unless other elements have a bug, but better safe than sorry). After that we map the input buffer readable and the output buffer writable with the [`VideoFrameRef`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer_video/video_frame/struct.VideoFrameRef.html) API. By mapping the buffers we get access to the underlying bytes of them, and the mapping operation could for example make GPU memory available or just do nothing and give us access to a normally allocated memory area. We have access to the bytes of the buffer until the `VideoFrameRef` goes out of scope. +What happens here is that we first of all lock our state (the input/output `VideoInfo`) and error out if we don’t have any yet (which can’t really happen unless other elements have a bug, but better safe than sorry). After that we map the input buffer readable and the output buffer writable with the [`VideoFrameRef`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer_video/video_frame/struct.VideoFrameRef.html) API. By mapping the buffers we get access to the underlying bytes of them, and the mapping operation could for example make GPU memory available or just do nothing and give us access to a normally allocated memory area. We have access to the bytes of the buffer until the `VideoFrameRef` goes out of scope. -Instead of `VideoFrameRef` we could’ve also used the [`gst::Buffer::map_readable()`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/buffer/struct.BufferRef.html#method.map_readable) and [`gst::Buffer::map_writable()`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/buffer/struct.BufferRef.html#method.map_writable) API, but different to those the `VideoFrameRef` API also extracts various metadata from the raw video buffers and makes them available. For example we can directly access the different planes as slices without having to calculate the offsets ourselves, or we get directly access to the width and height of the video frame. +Instead of `VideoFrameRef` we could’ve also used the [`gst::Buffer::map_readable()`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/buffer/struct.BufferRef.html#method.map_readable) and [`gst::Buffer::map_writable()`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/buffer/struct.BufferRef.html#method.map_writable) API, but different to those the `VideoFrameRef` API also extracts various metadata from the raw video buffers and makes them available. For example we can directly access the different planes as slices without having to calculate the offsets ourselves, or we get directly access to the width and height of the video frame. After mapping the buffers, we store various information we’re going to need later in local variables to save some typing later. This is the width (same for input and output as we never changed the width in `transform_caps`), the input and out (row-) stride (the number of bytes per row/line, which possibly includes some padding at the end of each line for alignment reasons), the output format (which can be BGRx or GRAY8 because of how we implemented `transform_caps`) and the pointers to the first plane of the input and output (which in this case also is the only plane, BGRx and GRAY8 both have only a single plane containing all the RGB/gray components). diff --git a/tutorial/tutorial-2.md b/tutorial/tutorial-2.md index cdca62dc..ce70695f 100644 --- a/tutorial/tutorial-2.md +++ b/tutorial/tutorial-2.md @@ -476,7 +476,7 @@ First of all, we need to get notified whenever the caps that our source is confi } ``` -In here we parse the caps into a [`AudioInfo`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer_audio/struct.AudioInfo.html) and then store that in our internal state, while updating various fields. We tell the base class about the number of bytes each buffer is usually going to hold, and update our current sample position, the stop sample position (when a seek with stop position happens, we need to know when to stop) and our accumulator. This happens by scaling both positions by the old and new sample rate. If we don't have an old sample rate, we assume nanoseconds (this will make more sense once seeking is implemented). The scaling is done with the help of the [`muldiv`](https://crates.io/crates/muldiv) crate, which implements scaling of integer types by a fraction with protection against overflows by doing up to 128 bit integer arithmetic for intermediate values. +In here we parse the caps into a [`AudioInfo`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer_audio/struct.AudioInfo.html) and then store that in our internal state, while updating various fields. We tell the base class about the number of bytes each buffer is usually going to hold, and update our current sample position, the stop sample position (when a seek with stop position happens, we need to know when to stop) and our accumulator. This happens by scaling both positions by the old and new sample rate. If we don't have an old sample rate, we assume nanoseconds (this will make more sense once seeking is implemented). The scaling is done with the help of the [`muldiv`](https://crates.io/crates/muldiv) crate, which implements scaling of integer types by a fraction with protection against overflows by doing up to 128 bit integer arithmetic for intermediate values. The accumulator is the updated based on the current phase of the sine wave at the current sample position. @@ -507,7 +507,7 @@ As a last step we post a new `LATENCY` message on the bus whenever the sample } ``` -Here we take the caps that are passed in, truncate them (i.e. remove all but the very first [`Structure`](https://slomo.pages.freedesktop.org/rustdocs/gstreamer/gstreamer/structure/struct.Structure.html)) and then manually fixate the sample rate to the closest value to 48kHz. By default, caps fixation would result in the lowest possible sample rate but this is usually not desired. +Here we take the caps that are passed in, truncate them (i.e. remove all but the very first [`Structure`](https://gstreamer.pages.freedesktop.org/gstreamer-rs/gstreamer/structure/struct.Structure.html)) and then manually fixate the sample rate to the closest value to 48kHz. By default, caps fixation would result in the lowest possible sample rate but this is usually not desired. For good measure, we also fixate the number of channels to the closest value to 1, but this would already be the default behaviour anyway. And then chain up to the parent class' implementation of `fixate`, which for now basically does the same as `caps.fixate()`. After this, the caps are fixated, i.e. there is only a single `Structure` left and all fields have concrete values (no ranges or sets).