mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-07-02 20:55:55 +00:00
215 lines
8.8 KiB
XML
215 lines
8.8 KiB
XML
<chapter id="chapter-allocation" xreflabel="Memory allocation">
|
|
<title>Memory allocation</title>
|
|
<para>
|
|
Memory allocation and management is a very important topic in
|
|
multimedia. High definition video uses many magabytes to store
|
|
one single frame of video. It is important to reuse the memory
|
|
when possible instead of constantly allocating and freeing
|
|
the memory.
|
|
</para>
|
|
<para>
|
|
Multimedia systems usually use special purpose chips, such as
|
|
DSPs or GPUs to perform the heavy lifting (especially for video).
|
|
These special purpose chips have usually strict requirements
|
|
for the memory that they can operate on and how the memory
|
|
is accessed.
|
|
</para>
|
|
<para>
|
|
This chapter talks about the memory management features that
|
|
&GStreamer; plugins can use. We will first talk about the
|
|
lowlevel <classname>GstMemory</classname> object that manages
|
|
access to a piece of memory. We then continue with
|
|
<classname>GstBuffer</classname> that is used to exchange data
|
|
between plugins (and the application) and that uses
|
|
<classname>GstMemory</classname>. We talk about
|
|
<classname>GstMeta</classname> that can be placed on buffers to
|
|
give extra info about the buffer and its memory.
|
|
For efficiently managing buffers of the same size, we take a
|
|
look at <classname>GstBufferPool</classname>. To conclude this
|
|
chapter we take a look at the GST_QUERY_ALLOCATION query that
|
|
is used to negotiate memory management options between elements.
|
|
</para>
|
|
|
|
<sect1 id="section-allocation-memory" xreflabel="GstMemory">
|
|
<title>GstMemory</title>
|
|
<para>
|
|
<classname>GstMemory</classname> is an object that manages a region
|
|
of memory. The memory object points to a region of memory of
|
|
<quote>maxsize</quote>. The area in this memory starting at
|
|
<quote>offset</quote> and for <quote>size</quote> bytes is the
|
|
accessible region in the memory. the maxsize of the memory can
|
|
never be changed after the object is created, however, the offset
|
|
and size can be changed.
|
|
</para>
|
|
<para>
|
|
Data access to the memory wrapped by the <classname>GstMemory</classname>
|
|
object is always protected with a <function>gst_memory_map()</function>
|
|
and <function>gst_memory_unmap()</function> pair. An access mode
|
|
(read/write) must be given when mapping memory. The map
|
|
function returns a pointer to the valid memory region that can
|
|
then be accessed according to the requested access mode.
|
|
</para>
|
|
<para>
|
|
<classname>GstMemory</classname> objects are created by a
|
|
<classname>GstAllocator</classname> object. To implement support
|
|
for a new kind of memory type, you must implement a new allocator
|
|
object.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-allocation-buffer" xreflabel="GstBuffer">
|
|
<title>GstBuffer</title>
|
|
<para>
|
|
A <classname>GstBuffer</classname> is an lightweight object that
|
|
is passed from an upstream to a downstream element and contains
|
|
memory and metadata. It represents the multimedia content that
|
|
is pushed or pull downstream by elements.
|
|
</para>
|
|
<para>
|
|
The buffer contains one or more <classname>GstMemory</classname>
|
|
objects thet represent the data in the buffer.
|
|
</para>
|
|
<para>
|
|
Metadata in the buffer consists of:
|
|
</para>
|
|
<itemizedlist mark="opencircle">
|
|
<listitem>
|
|
<para>
|
|
DTS and PTS timestamps. These represent the decoding and
|
|
presentation timestamps of the buffer content and is used by
|
|
synchronizing elements to schedule buffers. Both these timestamps
|
|
can be GST_CLOCK_TIME_NONE when unknown/undefined.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
The duration of the buffer contents. This duration can be
|
|
GST_CLOCK_TIME_NONE when unknown/undefined.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Media specific offsets and offset_end. For video this is the
|
|
frame number in the stream and for audio the sample number. Other
|
|
definitions for other media exist.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Arbitrary structures via <classname>GstMeta</classname>, see below.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
A buffer is writable when the refcount of the object is exactly 1, meaning
|
|
that only one object is holding a ref to the buffer. You can only
|
|
modify anything in the buffer when the buffer is writable. This means
|
|
that you need to call <function>gst_buffer_make_writable()</function>
|
|
before changing the timestamps, offsets, metadata or adding and
|
|
removing memory blocks.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-allocation-meta" xreflabel="GstMeta">
|
|
<title>GstMeta</title>
|
|
<para>
|
|
With the <classname>GstMeta</classname> system you can add arbitrary
|
|
structures of on buffers. These structures describe extra properties
|
|
of the buffer such as cropping, stride, region of interest etc.
|
|
</para>
|
|
<para>
|
|
Metadata is also used to store, for example, the X image that is
|
|
backing up the memory of the buffer. This makes it easier for elements
|
|
to locate the X image from the buffer.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-allocation-bufferpool" xreflabel="GstBufferPool">
|
|
<title>GstBufferPool</title>
|
|
<para>
|
|
The <classname>GstBufferPool</classname> object provides a convenient
|
|
base class for managing lists of reusable buffers. Essential for this
|
|
object is that all the buffers have the same properties such as size,
|
|
padding, metadata and alignment.
|
|
</para>
|
|
<para>
|
|
A bufferpool object can be configured to manage a minimum and maximum
|
|
amount of buffers of a specific size. A bufferpool can also be
|
|
configured to use a specific <classname>GstAllocator</classname> for
|
|
the memory of the buffers. There is support in the bufferpool to enable
|
|
bufferpool specific options, such as adding <classname>GstMeta</classname>
|
|
to the buffers in the pool or such as enabling specific padding on
|
|
the memory in the buffers.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-allocation-query" xreflabel="GST_QUERY_ALLOCATION">
|
|
<title>GST_QUERY_ALLOCATION</title>
|
|
<para>
|
|
The ALLOCATION query is used to negotiate
|
|
<classname>GstMeta</classname>, <classname>GstBufferPool</classname>
|
|
and <classname>GstAllocator</classname> between elements. Negotiation
|
|
of the allocation strategy is always initiated and decided by a srcpad
|
|
after it has negotiated a format and before it decides to push buffers.
|
|
A sinkpad can suggest an allocation strategy but it is ultimately the
|
|
source pad that will decide based on the suggestions of the downstream
|
|
sink pad.
|
|
</para>
|
|
<para>
|
|
The source pad will do a GST_QUERY_ALLOCATION with the negotiated caps
|
|
as a parameter. This is needed so that the downstream element knows
|
|
what media type is being handled. A downstream sink pad can answer the
|
|
allocation query with the following results:
|
|
</para>
|
|
<itemizedlist mark="opencircle">
|
|
<listitem>
|
|
<para>
|
|
An array of possible <classname>GstBufferPool</classname> suggestions
|
|
with suggested size, minimum and maximum amount of buffers.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
An array of GstAllocator objects along with suggested allocation
|
|
parameters such as flags, prefix, alignment and padding. These
|
|
allocators can also be configured in a bufferpool when this is
|
|
supported by the bufferpool.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
An array of supported <classname>GstMeta</classname> implementations
|
|
along with metadata specific parameters.
|
|
It is important that the upstream element knows what kind of
|
|
metadata is supported downstream before it places that metadata
|
|
on buffers.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
When the GST_QUERY_ALLOCATION returns, the source pad will select
|
|
from the available bufferpools, allocators and metadata how it will
|
|
allocate buffers.
|
|
</para>
|
|
<para>
|
|
In many baseclasses you will see the following virtual methods for
|
|
influencing the allocation strategy:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
<function>propose_allocation ()</function> should suggest
|
|
allocation parameters for the upstream element.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<function>decide_allocation ()</function> should decide the
|
|
allocation parameters from the suggestions received from
|
|
downstream.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect1>
|
|
</chapter>
|