Using Dynamic Ingest with S3

Product(s)
Video Cloud
Role(s)
API Developer
Task(s)
Add Videos/Assets
API(s)
Dynamic Ingest API

In this topic, you will learn how to ingest videos from your Amazon S3 account using the Dynamic Ingest API.

Overview

Dynamic Ingest can download videos from your Amazon S3 bucket. If your S3 content is accessible to Everyone, there is nothing you need to do. If your content is protected, however, you will need to add a bucket policy allowing Dynamic Ingest to access your videos. This document explains how to do that.

Setting the S3 Policy

To set up your policy to allow Dynamic Ingest to access your videos, go to the AWS Console. Select your bucket, then select the "Permissions" tab, then click the "Bucket Policy" button. On the last row click Add bucket policy. Below is our recommended policy:

  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "AddPerm",
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::395540211253:root"
        },
        "Action": [
          "s3:GetObject",
          "s3:PutObject",
          "s3:GetObjectAcl",
          "s3:PutObjectAcl",
          "s3:ListMultipartUploadParts"
        ],
        "Resource": [
          "arn:aws:s3:::MY-BUCKETNAME/*"
        ]
      },
      {
        "Sid": "Stmt1295042087538",
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::749779118921:user/videocloud-ingestion"
        },
        "Action": [
        "s3:ListBucketMultipartUploads",
        "s3:GetBucketLocation"
        ],
        "Resource": "arn:aws:s3:::MY-BUCKETNAME/*"
      }
    ]
  }

There is only one change you must make to this bucket policy before it's ready to go:

  1. Replace MY-BUCKETNAME with the name of your bucket. Leave the /* at the end of the Resource line, as that will apply the policy to every file within the bucket.

  2. Optionally, replace the Action array with the actions you want to allow. Valid actions that Video Cloud may need are:

    Object Permissions - first action array in the example above

    • s3:GetObject - allow Video Cloud to download files from the bucket
    • s3:PutObject - allow Video Cloud to upload files to the bucket
    • s3:GetObjectAcl - allow Video Cloud to get ACL permissions for files
    • s3:PutObjectAcl - allow Video Cloud to set ACL permissions for other users on files
    • s3:ListMultipartUploadParts - allow Video Cloud to list which parts of a multipart file have been uploaded to the bucket

    Bucket Permissions - second action array in the example above

    • s3:GetBucketLocation - allow Video Cloud to get the location of the bucket (US, EU, Asia, etc.)
    • s3:ListBucketMultipartUploads - allow Video Cloud to view any multipart uploads currently happening on the bucket

    Video Cloud does not need any write permissions.

Note that there are two statements in the policy. The first statement affects keys within the bucket, while the second part affects the bucket itself.

The rest of the policy can be left alone and it will work fine. Amazon's documentation contains more information about the elements of a Bucket Policy.