Memory allocation and management is a very important topic in multimedia. High definition video uses many magabytes to store one single frame of video. It is important to reuse the memory when possible instead of constantly allocating and freeing the memory. Multimedia systems usually use special purpose chips, such as DSPs or GPUs to perform the heavy lifting (especially for video). These special purpose chips have usually strict requirements for the memory that they can operate on and how the memory is accessed. This chapter talks about the memory management features that &GStreamer; plugins can use. We will first talk about the lowlevel GstMemory object that manages access to a piece of memory. We then continue with GstBuffer that is used to exchange data between plugins (and the application) and that uses GstMemory. We talk about GstMeta that can be placed on buffers to give extra info about the buffer and its memory. For efficiently managing buffers of the same size, we take a look at GstBufferPool. To conclude this chapter we take a look at the GST_QUERY_ALLOCATION query that is used to negotiate memory management options between elements. GstMemory is an object that manages a region of memory. The memory object points to a region of memory of maxsize. The area in this memory starting at offset and for size bytes is the accessible region in the memory. the maxsize of the memory can never be changed after the object is created, however, the offset and size can be changed. Data access to the memory wrapped by the GstMemory object is always protected with a gst_memory_map() and gst_memory_unmap() pair. An access mode (read/write) must be given when mapping memory. The map function returns a pointer to the valid memory region that can then be accessed according to the requested access mode. GstMemory objects are created by a GstAllocator object. To implement support for a new kind of memory type, you must implement a new allocator object. A GstBuffer is an lightweight object that is passed from an upstream to a downstream element and contains memory and metadata. It represents the multimedia content that is pushed or pull downstream by elements. The buffer contains one or more GstMemory objects thet represent the data in the buffer. Metadata in the buffer consists of: DTS and PTS timestamps. These represent the decoding and presentation timestamps of the buffer content and is used by synchronizing elements to schedule buffers. Both these timestamps can be GST_CLOCK_TIME_NONE when unknown/undefined. The duration of the buffer contents. This duration can be GST_CLOCK_TIME_NONE when unknown/undefined. Media specific offsets and offset_end. For video this is the frame number in the stream and for audio the sample number. Other definitions for other media exist. Arbitrary structures via GstMeta, see below. A buffer is writable when the refcount of the object is exactly 1, meaning that only one object is holding a ref to the buffer. You can only modify anything in the buffer when the buffer is writable. This means that you need to call gst_buffer_make_writable() before changing the timestamps, offsets, metadata or adding and removing memory blocks. With the GstMeta system you can add arbitrary structures of on buffers. These structures describe extra properties of the buffer such as cropping, stride, region of interest etc. Metadata is also used to store, for example, the X image that is backing up the memory of the buffer. This makes it easier for elements to locate the X image from the buffer. The GstBufferPool object provides a convenient base class for managing lists of reusable buffers. Essential for this object is that all the buffers have the same properties such as size, padding, metadata and alignment. A bufferpool object can be configured to manage a minimum and maximum amount of buffers of a specific size. A bufferpool can also be configured to use a specific GstAllocator for the memory of the buffers. There is support in the bufferpool to enable bufferpool specific options, such as adding GstMeta to the buffers in the pool or such as enabling specific padding on the memory in the buffers. The ALLOCATION query is used to negotiate GstMeta, GstBufferPool and GstAllocator between elements. Negotiation of the allocation strategy is always initiated and decided by a srcpad after it has negotiated a format and before it decides to push buffers. A sinkpad can suggest an allocation strategy but it is ultimately the source pad that will decide based on the suggestions of the downstream sink pad. The source pad will do a GST_QUERY_ALLOCATION with the negotiated caps as a parameter. This is needed so that the downstream element knows what media type is being handled. A downstream sink pad can answer the allocation query with the following results: An array of possible GstBufferPool suggestions with suggested size, minimum and maximum amount of buffers. An array of GstAllocator objects along with suggested allocation parameters such as flags, prefix, alignment and padding. These allocators can also be configured in a bufferpool when this is supported by the bufferpool. An array of supported GstMeta implementations along with metadata specific parameters. It is important that the upstream element knows what kind of metadata is supported downstream before it places that metadata on buffers. When the GST_QUERY_ALLOCATION returns, the source pad will select from the available bufferpools, allocators and metadata how it will allocate buffers. In many baseclasses you will see the following virtual methods for influencing the allocation strategy: propose_allocation () should suggest allocation parameters for the upstream element. decide_allocation () should decide the allocation parameters from the suggestions received from downstream.