gstreamer/docs/design/part-memory.txt

GstMemory
---------

This document describes the design of the memory objects.

GstMemory objects are usually added to GstBuffer objects and contain the
multimedia data passed around in the pipeline.

Requirements
~~~~~~~~~~~~

 - It must be possible to have different memory allocators
 - It must be possible to efficiently share memory objects, copy, span
   and trim.


Allocators
~~~~~~~~~~

 GstMemory objects are created by allocators. Allocators are created from
 a GstMemoryInfo structure.

  struct _GstMemoryInfo {
    const gchar              *mem_type;

    GstAllocatorAllocFunction alloc;

    GstMemoryMapFunction      mem_map;
    GstMemoryUnmapFunction    mem_unmap;
    GstMemoryFreeFunction     mem_free;

    GstMemoryCopyFunction     mem_copy;
    GstMemoryShareFunction    mem_share;
    GstMemoryIsSpanFunction   mem_is_span;
  };

 Allocators are refcounted. It is also possible to register the allocator to the
 GStreamer system. This way, the allocator can be retrieved by name.

 After an allocator is created, new GstMemory can be created with 

   GstMemory * gst_allocator_alloc (const GstAllocator * allocator,
                                    gsize maxsize, gsize align);

 The GstMemory object is a refcounted object that must be freed with
 gst_memory_unref ().

 It is also possible to create a new GstMemory object that wraps existing
 memory with:
  
   GstMemory * gst_memory_new_wrapped  (GstMemoryFlags flags,
                                        gpointer data, gsize maxsize,
                                        gsize offset, gsize size,
                                        gpointer user_data,
                                        GDestroyNotify notify);

Lifecycle
~~~~~~~~~

GstMemory objects are refcounted. When the GstMemory object is first created, it
has a refcount of 1. 

Each variable holding a reference to the GstMemory object is responsible for
updating the refcount.

When the refcount reaches 0, and thus no objects hold a reference anymore, we
can free the memory. The GstMemoryFreeFunction of the allocator will be called
to cleanup the memory.


Sharing
-------

GstMemory objects can be shared between multiple GstBuffer objects. It is
important that when a thread writes to the shared memory that the other
buffer don't not see the changes.

We add a separate shared counter that counts the amount of objects that share
this GstMemory object. The counter is initially 0, meaning that the object is
not shared with any object. When a GstBuffer (or other object) adds a ref to
the GstMemorty, it will also increase the shared count.

When the GstMemory is removed from the buffer, the ref count and the shared
counter will be decreased.

We don't want to use the refcount for this purpose because language bindings
might keep arbitrary references to the object.

A GstMemory object with a shared counter > 1 is not writable. Any attempt to
map with WRITE access or resize will fail. _make_mapped() with WRITE access
will make a copy.




Memory layout
~~~~~~~~~~~~~

 GstMemory manages a memory region. The accesible part of the managed region is
 defined by an offset relative to the start of the region and a size. This
 means that the managed region can be larger than what is visible to the user of
 GstMemory API.

 Schematically, GstMemory has a pointer to a memory region of _maxsize_. The area
 starting from _offset_ and _size_ is accessible.

               memory
    GstMemory  ->*----------------------------------------------------*
                 ^----------------------------------------------------^ 
                                   maxsize
                      ^--------------------------------------^         
                     offset            size

 The current properties of the accessible memory can be retrieved with:

   gsize       gst_memory_get_sizes  (GstMemory *mem, gsize *offset, gsize *maxsize);

 The offset and size can be changed with:

   void        gst_memory_resize     (GstMemory *mem, gssize offset, gsize size);


Data Access
~~~~~~~~~~~

 Access to the memory region is always controlled with a map and unmap method
 call. This allows the implementation to monitor the access patterns or set up
 the required memory mappings when needed.

 Mapping a memory region requires the caller to specify the access method: READ
 and/or WRITE.

 After the data has been accessed in the object, the unmap call must be
 performed.

 It is allowed to map multiple times with different access modes. for each of
 the map calls, an corresponding unmap call needs to be made. WRITE-only memory
 cannot be mapped in READ mode and READ-only memory cannot be mapped in WRITE
 mode.
 
 The memory pointer returned from the map call is guaranteed to remain valid in
 the requested mapping mode until the corresponding unmap call is performed on
 the pointer.
 
 When multiple map operations are nested and return the same pointer, the pointer
 is valid until the last unmap call is done.

 When the final reference on a memory object is dropped, all outstanding
 mappings should have been unmapped.

 Resizing a GstMemory does not influence any current mappings an any way.

Copy
~~~~

 A GstMemory copy can be made with the gst_memory_copy() call. Normally,
 allocators will implement a custom version of this function to make a copy of
 the same kind of memory as the original one.

 This is what the fallback version of the copy function will do, albeit slower
 than what as custom implementation could do.

 The copy operation is only required to copy the visible range of the memory
 block.


Share
~~~~~

 A memory region can be shared between GstMemory object with the
 gst_memory_share() operation.