Decoder API Tutorial

This tutorial provides an introduction to writing a h265 video decoding using libde265 using the C language API.

A look around the source tree

When checking out the source repository, the source is organized into these subdirectories:

• libde265 - main library source code. Everything below libde265/encoder is required only for h265 encoding and can be omitted for a decoder-only use.
• dec265 - example h265 video decoder
• enc265 - example h265 video encoder
• sherlock265 - h265 video analysis tool (shows motion-vectors, CTB subdivisions, coding modes)
• tools - miscellaneous tools mainly for encoder development

The code inside 'libde265' is usually compiled into a link-library, while the other directories contain executable programs. We include build scripts based on autoconf and/or CMake.

Writing a simple video decoder

First, we need to create a decoder context instance using de265_decoder_context* de265_new_decoder(void)

when using multi-threaded decoding, we should tell it the number of threads to use for decoding de265_error de265_start_worker_threads(de265_decoder_context*, int number_of_threads)

the decoder context must be freed again after use by de265_error de265_free_decoder(de265_decoder_context*)

Decoding loop

Decoding the video consists of pushing compressed video data into the decoder context, asking the decoder context to do some decoding work, and pulling decoded frames out of the decoder. There exist several functions for pushing data into the decoder. The most frequently used might be de265_error de265_push_data(de265_decoder_context*, const void* data, int length, de265_PTS pts, void* user_data);

This pushes length bytes of data into the decoder. The data will be copied into the decoder context. You can also provide a pts timestamp and user_data these are not actually used by the decoder, but returned again in the decoded image decoded from this data for your use. The data should be a h265 only elementary bitstream with startcode markers. At the end of the file, you should call de265_error de265_flush_data(de265_decoder_context*); to tell the decoder that there will not follow any more data.

If your data consists of packetized data without startcodes, you can also push complete NAL units into the decoder context using de265_error de265_push_NAL(de265_decoder_context*, const void* data, int length, de265_PTS pts, void* user_data);

Regularly, e.g. after pushing new data into the decoder, you should call de265_error de265_decode(de265_decoder_context*, int* more); to do some decoding work. The amount of video data decoded in each call is undefined. I.e., you cannot assume that each call to de265_decode() will decode exactly one frame. It may be less (no frame at all), or several frames might become available at once (e.g. because of the video frame reordering). The more parameter is a boolean output parameter that indicates whether there is more decoding work to do. If yes, you should call this function again.

This function might also return the errors

• DE265_ERROR_IMAGE_BUFFER_FULL - all output image buffers are full, some images must be collected from the decoder context to continue decoding
• DE265_ERROR_WAITING_FOR_INPUT_DATA - there is not enough input data available to continue decoding, you have to insert more data

Regularly, you should collect decoded images from the decoder context using const struct de265_image* de265_get_next_picture(de265_decoder_context*); If there are no images available, NULL is returned. After use, you have to free the images with void de265_release_picture(const de265_image*); You can keep hold of several images, if you like, but at some time, you must release them.

Decoding loop example

This is an example for a decoding loop. For a more complete example, see dec265/dec265.cc.

for (;;) {
  // ... load more video data into 'buf'

  de265_error err = de265_push_data(context, buf, n, 0, NULL);

  int more=1;
  while (more) {
    more = 0;

    err = de265_decode(context, &more);
    if (err != DE265_OK) {
      break;
    }

    // show available images                                                               

    const de265_image* img = de265_get_next_picture(ctx);
    if (img) {
      // ... display image

      de265_release_picture(img);
    }
  }
}

Accessing image data

The de265_image* is an opaque pointer. To access the image data, you have to use the following functions: int de265_get_image_width(const struct de265_image*,int channel); int de265_get_image_height(const struct de265_image*,int channel); const uint8_t* de265_get_image_plane(const struct de265_image*, int channel, int* out_stride);

enum de265_chroma de265_get_chroma_format(const struct de265_image*);
int de265_get_bits_per_pixel(const struct de265_image*,int channel);
de265_PTS de265_get_image_PTS(const struct de265_image*);
void* de265_get_image_user_data(const struct de265_image*);
void* de265_get_image_plane_user_data(const struct de265_image*, int channel);

Channels are 0,1,2 for Y,Cb,Cr, respectively.

Advanced memory management

For high-performance decoding, you might want the decoder to decode directly into user-supplied image buffers to avoid copying of images. This is possible by writing your own stub functions for allocation and deallocation of image buffers and passing them to the decoder context.

Specifically, you have to fill in this data structure: struct de265_image_allocation { int (get_buffer)(struct de265_image img, const struct de265_image_spec* spec, void* userdata); void (release_buffer)(struct de265_image img, void* userdata);

  void* userdata;
};

The de265_image_spec defines the properties of the image to be allocated (width, height, chroma format, alignment). The userdata variable is passed to each call of get_buffer() and release_buffer() but not used otherwise by the decoder.

The get_buffer() allocation function also received a de265_image for which the image buffers should be set. Use de265_set_image_plane() to set the image buffers for all planes (Y,Cb,Cr): void de265_set_image_plane(struct de265_image* img, int cIdx, void* mem, int stride, void *userdata);

The element userdata is again optional, returned in the decoded respective image plane of de265_image and not used by the decoder itself.

Register your custom allocation function with void de265_set_image_allocation_functions(de265_decoder_context*, struct de265_image_allocation*);