Zencoder: CAE Guide

Product(s)
Zencoder
Role(s)
API Developer
API(s)
Zencoder API

This topic provides a guide to using Context Aware Encoding (CAE).

Overview

Context Aware Encoding (CAE) analyzes each source video and intelligently builds a custom bitrate ladder (set of renditions) for each piece of content. In addition, Context Aware Encoding takes into account constraints associated with the delivery network and device being used to view the content. It decides how many renditions are needed and what resolutions and bitrates to use for each, while still maintaining a consistent level of quality across all titles. This results in massive savings on storage and bandwidth costs, while improving the playback experience for users. For additional information, please see the Overview of Context Aware Encoding.

Process

CAE is enabled on a Zencoder Job by setting generate_dynamic_profile to true and adding dynamic_profile_rendition values to Outputs of the Job. In addition, you can add dynamic_profile_options, but this is not required. After downloading and inspecting the Input source file for the Job, Zencoder runs the CAE dynamic profile generator on the source file. The resulting dynamic profile ladder settings are added to each corresponding Output before they are encoded. The following settings are automatically set for Outputs with a dynamic_profile_rendition value:

  • width
  • height
  • frame_rate
  • video_codec
  • video_codec_profile
  • video_codec_tier
  • video_codec_level
  • video_reference_frames
  • video_bframes
  • video_bitrate
  • decoder_bitrate_cap
  • decoder_buffer_size
  • keyframe_rate
  • fixed_keyframe_interval

Other Outputs without a dynamic_profile_rendition will begin encoding as soon as the Input source is inspected, concurrently while the dynamic profile generator is running.

Guidelines for configuring dynamic_profile_options

Number of Renditions

The minimum and maximum number of renditions to create is controlled with the min_renditions and max_renditions settings. For best performance, it is recommended to leave a certain gap between these limits (e.g. set min_renditions to 2 and max_renditions to 10), allowing CAE profile generator to select number most suitable for each content. For easy to encode content, it may produce fewer renditions, while for more complex content it may produce more. It is also recommended to set min_renditions to be as small as possible, to allow for saving bandwidth when more renditions are unnecessary.

Resolutions

By default, CAE uses a ladder of about 30 standard resolutions, ranging from 192x108 to 7680x4320. It can be limited on either the low or high end by using the min_resolution and max_resolution settings. Alternately, a custom list of resolutions and associated codec settings can be provided in the video_configurations setting.

GOP length

The keyframe_rate setting controls the maximum GOP length of the encoded content. Set this to correspond with segment_seconds for HLS/DASH encoding. Otherwise a value of 0.5 is recommended, corresponding to a 2-second GOP length.

Bitrate coverage

The limits of the bitrates to be used can controlled by the min_bitrate and max_bitrate settings. This will instruct CAE to allocate at least min_bitrate Kbps for the first rendition and constrain the last rendition to at most max_bitrate Kbps.

To ensure the rendition bitrates are spaced appropriately, use the min_granularity and max_granularity settings. For example, these can be used to make sure the step from one bitrate up to the next is not less than a 50% increase in bitrate or more than a 100% increase in bitrate. Please note that most streaming authoring guidelines recommend granularity doesn’t exceed 100%.

Startup latency, reliability at poor network conditions

To ensure the bitrate ladder always starts with a low enough bitrate, use the max_first_rendition_bitrate setting. The lower the first rendition bitrate is set, the less likely a video player will need to buffer due to low connection speeds. Additionally, if the streaming manifest indicates to load the lowest bitrate first, this will also directly affect startup latency.

Support for legacy devices

To support older devices that are only capable of Baseline profile H.264 playback, use the select_baseline_profile_configuration setting. When this is set, at least one rendition will be encoded with Baseline profile. Additional constraints (codec profile, level, number of reference frames, b-frames, etc.) can be controlled with a custom video_configurations setting.

VBR variability control

The maximum peak bitrates can be controlled with the bitrate_cap_to_bitrate_ratio and bitrate_cap_offset settings. If a bitrate “R” is selected for a rendition, then the decoder_bitrate_cap is calculated as:

decoder_bitrate_cap = R * bitrate_cap_to_bitrate_ratio + bitrate_cap_offset

Similarly, the decoder buffer limits can be controlled with the buffer_size_to_bitrate_ratio and buffer_size_offset settings. The decoder_buffer_size is calculated as:

decoder_buffer_size = R * buffer_size_to_bitrate_ratio + buffer_size_offset

Note that peak bitrate and buffer limits may also be constrained by the codec profile and/or level settings. Some general device compatibility issues can be avoided by limiting bitrate_cap_to_bitrate_ratio to 2.0 or below, and buffer_size_to_bitrate_ratio to 2.0 * GOP length (in seconds).

CAE and segmented outputs

CAE is compatible with segmented outputs in Zencoder. On the Output rendition settings, set the type as segmented and the appropriate segment_seconds value. Also, keyframe_rate and fixed_keyframe_interval settings must be set in dynamic_profile_options.

CAE - HLS example

{
 "input": "s3://test_bucket/test_media.mp4",
 "generate_dynamic_profile": true,
 "dynamic_profile_options": {
   "max_renditions": 6,
   "min_renditions": 2,
   "select_baseline_profile_configuration": true,
   "keyframe_rate": "0.5",
   "fixed_keyframe_interval": "true"
 },
 "outputs": [
   {
     "label": "CAE1",
     "dynamic_profile_rendition": 1,
     "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE1",
     "filename": "CAE1.m3u8",
     "type": "segmented",
     "segment_seconds": 2,
     "byte_range_segmenting": true,
     "generate_keyframe_manifest": "true",
     "hls_protocol_version": 7,
     "skip_audio": true
   },
   {
     "label": "CAE2",
     "dynamic_profile_rendition": 2,
     "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE2",
     "filename": "CAE2.m3u8",
     "type": "segmented",
     "segment_seconds": 2,
     "byte_range_segmenting": true,
     "generate_keyframe_manifest": "true",
     "hls_protocol_version": 7,
     "skip_audio": true
   },
   {
     "label": "CAE3",
     "dynamic_profile_rendition": 3,
     "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE3",
     "filename": "CAE3.m3u8",
     "type": "segmented",
     "segment_seconds": 2,
     "byte_range_segmenting": true,
     "generate_keyframe_manifest": "true",
     "hls_protocol_version": 7,
     "skip_audio": true
   },
   {
     "label": "CAE4",
     "dynamic_profile_rendition": 4,
     "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE4",
     "filename": "CAE4.m3u8",
     "type": "segmented",
     "segment_seconds": 2,
     "byte_range_segmenting": true,
     "generate_keyframe_manifest": "true",
     "hls_protocol_version": 7,
     "skip_audio": true
   },
   {
     "label": "CAE5",
     "dynamic_profile_rendition": 5,
     "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE5",
     "filename": "CAE5.m3u8",
       "type": "segmented",
       "segment_seconds": 2,
       "byte_range_segmenting": true,
       "generate_keyframe_manifest": "true",
       "hls_protocol_version": 7,
       "skip_audio": true
     },
     {
       "label": "CAE6",
       "dynamic_profile_rendition": 6,
       "base_url": "s3://test_bucket/outputs/HLS_CAE/CAE6",
       "filename": "CAE6.m3u8",
       "type": "segmented",
       "segment_seconds": 2,
       "byte_range_segmenting": true,
       "generate_keyframe_manifest": "true",
       "hls_protocol_version": 7,
       "skip_audio": true
     }
     {
       "label": "Audio",
       "audio_bitrate": "128",
       "type": "segmented",
       "segment_seconds": 2,
       "byte_range_segmenting": true,
       "skip_video": true,
       "base_url": "s3://test_bucket/outputs/HLS_CAE/Audio/128k/",
       "filename": "audio_128k.ac3"
     },
     {
       "base_url": "s3://test_bucket/outputs/HLS_CAE/",
       "filename": "master_playlist.m3u8",
       "hls_protocol_version": 7,
       "type": "playlist",
       "allow_skipped_sources": true,
       "alternate_audio": {
          "128k_audio": {
           "path": "Audio/128k/audio_128k.m3u8",
           "language": "en",
           "source": "Audio"
         }
       }
       ],
       "streams": [
         {
           "path": "CAE1/CAE1.m3u8",
           "source": "CAE1",
           "audio": "128k_audio"
         },
         {
           "path": "CAE2/CAE2.m3u8",
           "source": "CAE2",
           "audio": "128k_audio"
         },
         {
           "path": "CAE3/CAE3.m3u8",
           "source": "CAE3",
           "audio": "128k_audio"
         },
         {
           "path": "CAE4/CAE4.m3u8",
           "source": "CAE4",
           "audio": "128k_audio"
         },
         {
           "path": "CAE5/CAE5.m3u8",
           "source": "CAE5",
           "audio": "128k_audio"
         },
         {
           "path": "CAE6/CAE6.m3u8",
           "source": "CAE6",
           "audio": "128k_audio"
         }
       ]
     }
   ]
  }

CAE - DASH example


{
 "input": "s3://test_bucket/test.mov",
 "generate_dynamic_profile": true,
 "dynamic_profile_options": {
   "max_renditions": 6,
   "min_renditions": 2,
   "select_baseline_profile_configuration": true,
   "keyframe_rate": "0.5",
   "fixed_keyframe_interval": "true"
 },
 "outputs": [
   {
     "label": "CAE1",
     "dynamic_profile_rendition": 1,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE1",
     "filename": "CAE1.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "CAE2",
     "dynamic_profile_rendition": 2,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE2",
     "filename": "CAE2.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "CAE3",
     "dynamic_profile_rendition": 3,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE3",
     "filename": "CAE3.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "CAE4",
     "dynamic_profile_rendition": 4,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE4",
     "filename": "CAE4.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "CAE5",
     "dynamic_profile_rendition": 5,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE5",
     "filename": "CAE5.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "CAE6",
     "dynamic_profile_rendition": 6,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/CAE6",
     "filename": "CAE6.mpd",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_audio": true,
     "public": true
   },
   {
     "label": "Audio",
     "audio_bitrate": "128",
     "type": "segmented",
     "segment_seconds": 10,
     "streaming_delivery_format": "dash",
     "skip_video": true,
     "base_url": "s3://test_bucket/outputs/DASH_CAE/Audio/128k/",
     "filename": "audio_128k.mpd",
     "public": true
   },
   {
     "base_url": "s3://test_bucket/outputs/DASH_CAE/",
     "filename": "master_playlist.mpd",
     "streaming_delivery_format": "dash",
     "type": "playlist",
     "allow_skipped_sources": true,
     "public": true,
     "streams": [
       {
         "path": "CAE1/CAE1.mpd",
         "source": "CAE1"
       },
       {
         "path": "CAE2/CAE2.mpd",
         "source": "CAE2"
       },
       {
         "path": "CAE3/CAE3.mpd",
         "source": "CAE3"
       },
       {
         "path": "CAE4/CAE4.mpd",
         "source": "CAE4"
       },
       {
         "path": "CAE5/CAE5.mpd",
         "source": "CAE5"
       },
       {
         "path": "CAE6/CAE6.mpd",
         "source": "CAE6"
       },
       {
         "path": "Audio/128k/audio_128k.mpd",
         "source": "Audio"
       }
     ]
   }
 ]
}