mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-19 00:01:23 +00:00
38b96e6c8c
gst_buffer_take_memory -> gst_buffer_insert_memory because insert is what the method does. Make all methods deal with ranges so that we can replace, merge, remove and map a certain subset of the memory in a buffer. With the new methods we can make some code nicer and reuse more code. Being able to deal with a subset of the buffer memory allows us to optimize more cases later (most notably RTP headers and payload that could be in different memory objects). Make some more convenient macros that call the more generic range methods.
154 lines
5.5 KiB
Text
154 lines
5.5 KiB
Text
GstBuffer
|
|
---------
|
|
|
|
This document describes the design for buffers.
|
|
|
|
A GstBuffer is the object that is passed from an upstream element to a
|
|
downstream element and contains memory and metadata information.
|
|
|
|
Requirements
|
|
~~~~~~~~~~~~
|
|
|
|
- It must be fast
|
|
* allocation, free, low fragmentation
|
|
- Must be able to attach multiple memory blocks to the buffer
|
|
- Must be able to attach artibtrary metadata to buffers
|
|
- efficient handling of subbuffer, copy, span, trim
|
|
|
|
Writability
|
|
-----------
|
|
|
|
The Buffers is writable when the refcount is 1. This means that:
|
|
|
|
- metadata can be added/removed and the metadata can be changed
|
|
- GstMemory blocks can be added/removed
|
|
|
|
The individual memory blocks have their own refcounting and READONLY flags
|
|
that might influence their writability.
|
|
|
|
Buffers can be made writable with gst_buffer_make_writable(). This will copy the
|
|
buffer with the metadata and will ref the memory in the buffer. This means that
|
|
the memory is not automatically copied when copying buffers.
|
|
|
|
|
|
Managing GstMemory
|
|
------------------
|
|
|
|
A GstBuffer contains an array of pointers to GstMemory objects.
|
|
|
|
When the buffer is writable, gst_buffer_insert_memory() can be used to add a
|
|
new GstMemory object to the buffer. When the array of memory is full, memory
|
|
will be merged to make room for the new memory object.
|
|
|
|
gst_buffer_n_memory() is used to get the amount of memory blocks on the
|
|
GstBuffer.
|
|
|
|
With gst_buffer_peek_memory(), memory can be retrieved from the memory array.
|
|
The desired access pattern for the memory block should be specified so that
|
|
appropriate checks can be made and, in case of GST_MAP_WRITE, a writable copy
|
|
can be constructed when needed.
|
|
|
|
gst_buffer_remove_memory_range() and gst_buffer_remove_memory() can be used to
|
|
remove memory from the GstBuffer.
|
|
|
|
|
|
Subbuffers
|
|
----------
|
|
|
|
Subbuffers are made by copying only a region of the memory blocks and copying
|
|
all of the metadata.
|
|
|
|
|
|
Span
|
|
----
|
|
|
|
Spanning will merge together the data of 2 buffers into a new buffer
|
|
|
|
|
|
Data access
|
|
-----------
|
|
|
|
Accessing the data of the buffer can happen by retrieving the individual
|
|
GstMemory objects in the GstBuffer or my using the gst_buffer_map() and
|
|
gst_buffer_unmap() function.
|
|
|
|
The _map and _unmap function will always return the memory of all blocks as one
|
|
large contiguous region of memory. Using the _map and _unmap function might be
|
|
more convenient than accessing the individual memory blocks at the expense of
|
|
being more expensive because it might perform memcpy operations.
|
|
|
|
For buffers with only one GstMemory object (the most common case), _map and
|
|
_unmap have no performance penalty at all.
|
|
|
|
|
|
* Read access with 1 memory block
|
|
|
|
The memory block is accessed and mapped for read access.
|
|
The memory block is unmapped after usage
|
|
|
|
* write access with 1 memory block
|
|
|
|
The buffer should be writable or this operation will fail..
|
|
The memory block is accessed. If the memory block is readonly, a copy is made
|
|
and the original memory block is replaced with this copy. then the memory
|
|
block is mapped in write mode.
|
|
The memory block is unmapped after usage.
|
|
|
|
* Read access with multiple memory blocks
|
|
|
|
The memory blocks are combined into one large memory block. If the buffer is
|
|
writable, The memory blocks are replace with this new memory block. If the
|
|
buffer is not writable, the memory is returned as is.
|
|
The memory block is then mapped in read mode.
|
|
|
|
When the memory is unmapped after usage and the buffer has multiple memory
|
|
blocks, this means that the map operation was not able to store the combined
|
|
buffer and it thus returned memory that should be freed. Otherwise, the memory
|
|
is unmapped.
|
|
|
|
* Write access with multiple memory blocks
|
|
|
|
The buffer should be writable or the operation fails. The memory blocks are
|
|
combined into one large memory block and the existing blocks are replaced with
|
|
this new block. The memory is then mapped in write mode.
|
|
The memory is unmapped after usage.
|
|
|
|
|
|
Use cases
|
|
---------
|
|
|
|
Generating RTP packets from h264 video
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
We receive as input a GstBuffer with an encoded h264 image and we need to
|
|
create RTP packets containing this h264 data as the payload. We typically need
|
|
to fragment the h264 data into multiple packets, each with their own RTP and
|
|
payload specific header.
|
|
|
|
+-------+-------+---------------------------+--------+
|
|
input H264 buffer: | NALU1 | NALU2 | ..... | NALUx |
|
|
+-------+-------+---------------------------+--------+
|
|
|
|
|
V
|
|
array of +-+ +-------+ +-+ +-------+ +-+ +-------+
|
|
output buffers: | | | NALU1 | | | | NALU2 | .... | | | NALUx |
|
|
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
|
: : : :
|
|
\-----------/ \-----------/
|
|
buffer 1 buffer 2
|
|
|
|
The output buffer array consists of x buffers consisting of an RTP payload header
|
|
and a subbuffer of the original input H264 buffer. Since the rtp headers and
|
|
the h264 data don't need to be contiguous in memory, they are added to the buffer
|
|
as separate GstMemory blocks and we can avoid to memcpy the h264 data into
|
|
contiguous memory.
|
|
|
|
A typical udpsink will then use something like sendmsg to send the memory regions
|
|
on the network inside one UDP packet. This will further avoid having to memcpy
|
|
data into contiguous memory.
|
|
|
|
Using bufferlists, the complete array of output buffers can be pushed in one
|
|
operation to the peer element.
|
|
|
|
|
|
|