mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-27 12:11:13 +00:00
160 lines
5.6 KiB
Text
160 lines
5.6 KiB
Text
GstBuffer
|
|
---------
|
|
|
|
This document describes the design for buffers.
|
|
|
|
A GstBuffer is the object that is passed from an upstream element to a
|
|
downstream element and contains memory and metadata information.
|
|
|
|
Requirements
|
|
~~~~~~~~~~~~
|
|
|
|
- It must be fast
|
|
* allocation, free, low fragmentation
|
|
- Must be able to attach multiple memory blocks to the buffer
|
|
- Must be able to attach artibtrary metadata to buffers
|
|
- efficient handling of subbuffer, copy, span, trim
|
|
|
|
Lifecycle
|
|
~~~~~~~~~
|
|
|
|
GstMemory extends from GstMiniObject and therefore uses its lifecycle
|
|
management (See part-miniobject.txt).
|
|
|
|
Writability
|
|
~~~~~~~~~~~
|
|
|
|
When a Buffers is writable as returned from gst_buffer_is_writable():
|
|
|
|
- metadata can be added/removed and the metadata can be changed
|
|
- GstMemory blocks can be added/removed
|
|
|
|
The individual memory blocks have their own refcounting and READONLY flags
|
|
that might influence their writability.
|
|
|
|
Buffers can be made writable with gst_buffer_make_writable(). This will copy the
|
|
buffer with the metadata and will ref the memory in the buffer. This means that
|
|
the memory is not automatically copied when copying buffers.
|
|
|
|
|
|
Managing GstMemory
|
|
------------------
|
|
|
|
A GstBuffer contains an array of pointers to GstMemory objects.
|
|
|
|
When the buffer is writable, gst_buffer_insert_memory() can be used to add a
|
|
new GstMemory object to the buffer. When the array of memory is full, memory
|
|
will be merged to make room for the new memory object.
|
|
|
|
gst_buffer_n_memory() is used to get the amount of memory blocks on the
|
|
GstBuffer.
|
|
|
|
With gst_buffer_peek_memory(), memory can be retrieved from the memory array.
|
|
The desired access pattern for the memory block should be specified so that
|
|
appropriate checks can be made and, in case of GST_MAP_WRITE, a writable copy
|
|
can be constructed when needed.
|
|
|
|
gst_buffer_remove_memory_range() and gst_buffer_remove_memory() can be used to
|
|
remove memory from the GstBuffer.
|
|
|
|
|
|
Subbuffers
|
|
----------
|
|
|
|
Subbuffers are made by copying only a region of the memory blocks and copying
|
|
all of the metadata.
|
|
|
|
|
|
Span
|
|
----
|
|
|
|
Spanning will merge together the data of 2 buffers into a new buffer
|
|
|
|
|
|
Data access
|
|
-----------
|
|
|
|
Accessing the data of the buffer can happen by retrieving the individual
|
|
GstMemory objects in the GstBuffer or my using the gst_buffer_map() and
|
|
gst_buffer_unmap() function.
|
|
|
|
The _map and _unmap function will always return the memory of all blocks as one
|
|
large contiguous region of memory. Using the _map and _unmap function might be
|
|
more convenient than accessing the individual memory blocks at the expense of
|
|
being more expensive because it might perform memcpy operations.
|
|
|
|
For buffers with only one GstMemory object (the most common case), _map and
|
|
_unmap have no performance penalty at all.
|
|
|
|
|
|
* Read access with 1 memory block
|
|
|
|
The memory block is accessed and mapped for read access.
|
|
The memory block is unmapped after usage
|
|
|
|
* write access with 1 memory block
|
|
|
|
The buffer should be writable or this operation will fail..
|
|
The memory block is accessed. If the memory block is readonly, a copy is made
|
|
and the original memory block is replaced with this copy. then the memory
|
|
block is mapped in write mode.
|
|
The memory block is unmapped after usage.
|
|
|
|
* Read access with multiple memory blocks
|
|
|
|
The memory blocks are combined into one large memory block. If the buffer is
|
|
writable, The memory blocks are replace with this new memory block. If the
|
|
buffer is not writable, the memory is returned as is.
|
|
The memory block is then mapped in read mode.
|
|
|
|
When the memory is unmapped after usage and the buffer has multiple memory
|
|
blocks, this means that the map operation was not able to store the combined
|
|
buffer and it thus returned memory that should be freed. Otherwise, the memory
|
|
is unmapped.
|
|
|
|
* Write access with multiple memory blocks
|
|
|
|
The buffer should be writable or the operation fails. The memory blocks are
|
|
combined into one large memory block and the existing blocks are replaced with
|
|
this new block. The memory is then mapped in write mode.
|
|
The memory is unmapped after usage.
|
|
|
|
|
|
Use cases
|
|
---------
|
|
|
|
Generating RTP packets from h264 video
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
We receive as input a GstBuffer with an encoded h264 image and we need to
|
|
create RTP packets containing this h264 data as the payload. We typically need
|
|
to fragment the h264 data into multiple packets, each with their own RTP and
|
|
payload specific header.
|
|
|
|
+-------+-------+---------------------------+--------+
|
|
input H264 buffer: | NALU1 | NALU2 | ..... | NALUx |
|
|
+-------+-------+---------------------------+--------+
|
|
|
|
|
V
|
|
array of +-+ +-------+ +-+ +-------+ +-+ +-------+
|
|
output buffers: | | | NALU1 | | | | NALU2 | .... | | | NALUx |
|
|
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
|
: : : :
|
|
\-----------/ \-----------/
|
|
buffer 1 buffer 2
|
|
|
|
The output buffer array consists of x buffers consisting of an RTP payload header
|
|
and a subbuffer of the original input H264 buffer. Since the rtp headers and
|
|
the h264 data don't need to be contiguous in memory, they are added to the buffer
|
|
as separate GstMemory blocks and we can avoid to memcpy the h264 data into
|
|
contiguous memory.
|
|
|
|
A typical udpsink will then use something like sendmsg to send the memory regions
|
|
on the network inside one UDP packet. This will further avoid having to memcpy
|
|
data into contiguous memory.
|
|
|
|
Using bufferlists, the complete array of output buffers can be pushed in one
|
|
operation to the peer element.
|
|
|
|
|
|
|