Decoder API Tutorial

This tutorial provides an introduction to writing a h265 video decoding using libde265 using the C language API.

A look around the source tree

When checking out the source repository, the source is organized into these subdirectories:

• libde265 - main library source code. Everything below libde265/encoder is required only for h265 encoding and can be omitted for a decoder-only use.
• dec265 - example h265 video decoder
• enc265 - example h265 video encoder
• sherlock265 - h265 video analysis tool (shows motion-vectors, CTB subdivisions, coding modes)
• tools - miscellaneous tools mainly for encoder development

The code inside 'libde265' is usually compiled into a link-library, while the other directories contain executable programs. We include build scripts based on autoconf and/or CMake.

libde265 is written in C++11, but provides a plain C interface for easy integration. All decoder functions are defined in the header de265.h. This file also is the reference documentation. The header en265.h includes the encoder API and is not needed for a decoder-only software.

This documentation refers to the new API of libde265, which is currently developed in branch frame-parallel. The old API for branch master is very similar, though, and most of what is written here also applies.

Writing a simple video decoder

First, we need to create a decoder context instance using

de265_decoder_context* de265_new_decoder(void)

The decoder uses background threads to do the actual decoding. Before using the decoder, we have to start these background threads with

de265_error de265_start_worker_threads(de265_decoder_context*, int number_of_threads)

Decoding without background threads is currently not supported. Hence, this function must be called. When the decoder is not needed any more, the decoder context must be freed again after use by

de265_error de265_free_decoder(de265_decoder_context*)

Decoding loop

Decoding the video consists of asynchronously pushing compressed video data into the decoder context and pulling decoded frames out of the decoder. You can push in as much data as you want and pull out frames whenever you want. There exist several functions for pushing data into the decoder. The most frequently used might be

de265_error de265_push_data(de265_decoder_context*, const void* data, int length,
                            de265_PTS pts, void* user_data);

This pushes length bytes of data into the decoder. The data will be copied into the decoder context. You can also provide a pts timestamp and user_data. These are not actually used by the decoder, but returned again in the decoded image decoded from this data for your use. The data provided to this function should be a h265 only elementary bitstream with startcode markers. At the end of the file, you should call

de265_error de265_push_end_of_stream(de265_decoder_context*);

to tell the decoder that no more data will follow this call. It will then flush all its output queues.

If your data consists of packetized data without startcodes, you can also push complete NAL units into the decoder context using

de265_error de265_push_NAL(de265_decoder_context*, const void* data, int length,
                           de265_PTS pts, void* user_data);

Whenever there is enough data available, the decoder will do some work and add decoded images to its output queue. To find out whether the decoder needs more data and whether there are images waiting in the output queue, you can call

int de265_get_action(de265_decoder_context*, int blocking);

This will return a bit-set telling the current decoder state:

#define de265_action_push_more_input     1
#define de265_action_get_image           2
#define de265_action_end_of_stream       4

if de265_action_push_more_input is set, then the decoder is currently stalled because it requires more input data. If de265_action_get_image is set, you decoded images are waiting in the output queue. de265_action_end_of_stream indicates that the decoder has completely decoded its input and stopped. However, there may still be images waiting in the output queue.

Usually, this function returns immediately, even if the returned bit-set is empty. However, you can also call this with the blocking parameter set to 1, which makes the function block until there is at least one action required.

Decoded images can be extracted from the decoder context using

const struct de265_image* de265_get_next_picture(de265_decoder_context*);

If there are no images available, NULL is returned. After use, you have to free the images with

void de265_release_picture(const de265_image*);

You can keep hold of several images, if you like, but at some time, you must release them.

Decoding loop example

This is an example for a decoding loop. For a more complete example, see dec265/dec265.cc.

for (;;) {
  int action = de265_get_action(context, 1);

  if (action & de265_action_push_more_data) {
    // ... load more video data into 'buf'

    de265_error err = de265_push_data(context, buf, n, 0, NULL);
  }

  if (action & de265_action_get_image) {
    // show available images

    const de265_image* img = de265_get_next_picture(ctx);
    if (img) {
      // ... display image

      de265_release_picture(img);
    }
  }

  if (action & de265_action_end_of_stream) {
    break;
  }
}

Accessing image data

The de265_image* is an opaque pointer. To access the image data, you have to use the following functions:

int de265_get_image_width(const struct de265_image*,int channel);
int de265_get_image_height(const struct de265_image*,int channel);
const uint8_t* de265_get_image_plane(const struct de265_image*, int channel, int* out_stride);

enum de265_chroma de265_get_chroma_format(const struct de265_image*);
int de265_get_bits_per_pixel(const struct de265_image*,int channel);
de265_PTS de265_get_image_PTS(const struct de265_image*);
void* de265_get_image_user_data(const struct de265_image*);
void* de265_get_image_plane_user_data(const struct de265_image*, int channel);

Channels are 0,1,2 for Y,Cb,Cr, respectively.

Advanced memory management

For high-performance decoding, you might want the decoder to decode directly into user-supplied image buffers to avoid copying of images. This is possible by writing your own stub functions for allocation and deallocation of image buffers and passing them to the decoder context.

Specifically, you have to fill in this data structure:

struct de265_image_allocation
{
  int  (*get_buffer)(struct de265_image_intern* img,
                     const struct de265_image_spec* spec,
                     void* alloc_userdata);
  void (*release_buffer)(struct de265_image_intern* img,
                         void* alloc_userdata);

  void* alloc_userdata;
};

The de265_image_spec defines the properties of the image to be allocated (width, height, chroma format, alignment). The alloc_userdata variable is passed to each call of get_buffer() and release_buffer() but not used otherwise by the decoder.

The get_buffer() allocation function also received a de265_image_intern for which the image buffers should be set. Call de265_set_image_plane_intern() for each of the planes (Y,Cb,Cr) to set the memory buffers:

void de265_set_image_plane_intern(struct de265_image_intern* img,
                       int cIdx,
                           void* mem, int stride,
                           void *userdata);

The element userdata is again optional, copied to the respective image plane of de265_image_intern and not used by the decoder itself. You can use it, for example, when releasing the image.

Register your custom allocation function with

void de265_set_image_allocation_functions(de265_decoder_context*,
                                          struct de265_image_allocation*);

For a complete reference to the decoder API, check the file de265.h.