MP4 をストリーミング用動画フォーマット HLS に自動変換する AWS Step Functions ジョブをつくる

こんにちは、広野です。

以前、以下の記事で紹介していた AWS Step Functions のジョブを最新化したので紹介します。今回のジョブは社内教育用の動画作成で活用しています。

※上記記事は古いので、お勧めしません。

動画は Amazon CloudFront 経由、Amazon Cognito 認証付きで配信しています。参考記事はこちら。

 

アーキテクチャ

基本的には以下の記事と同じですので、割愛します。

 

仕様

  • MP4 ファイルを Amazon S3 バケットの input フォルダに配置したら、自動で AWS Step Functions ステートマシンが呼び出される。
  • ステートマシンは、AWS Elemental MediaConvert を呼び出し、S3 上の MP4 ファイルを GIF に変換し S3 バケットの output フォルダに保存する。
  • 最後に完了したことを Amazon SNS で通知する。
  • コーデックは H.264 を採用。互換性重視のため。H.265 ではブラウザによっては再生できない。
  • 画質は 360p と 720p の 2段階。アダプティブビットレート (ABR) 対応で視聴者のネットワーク状況に応じてプレーヤーが自動で画質を切り替えられるようにした。

インプット

以下のように、MP4 ファイルを S3 バケットに配置します。

アウトプット

以下のように、m3u8 ファイル群が生成されます。プレーヤーに、この例では sample-cat.m3u8 を指定することになります。

 

AWS CloudFormation テンプレート

実際にデプロイしたときに使用した AWS CloudFormation テンプレートを貼り付けます。詳細な設定はこちらをご覧ください。

AWSTemplateFormatVersion: 2010-09-09
Description: The CloudFormation template that creates a Step Functions state machine. It provides converting MP4 to HLS (H.264, AAC, ABR 360p/720p). The IAM role MediaConvert_Default_Role must be created in your AWS account before creating a job. Please refer https://docs.aws.amazon.com/mediaconvert/latest/ug/creating-the-iam-role-in-mediaconvert-configured.html for details.

# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------#
Parameters:
  SystemName:
    Type: String
    Description: System name. use lower case only. (e.g. example)
    Default: example
    MaxLength: 10
    MinLength: 1

  SubName:
    Type: String
    Description: System sub name. use lower case only. (e.g. prod or dev)
    Default: dev
    MaxLength: 10
    MinLength: 1

Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: "General Configuration"
        Parameters:
          - SystemName
          - SubName

Resources:
# ------------------------------------------------------------#
# S3
# ------------------------------------------------------------#
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
      LifecycleConfiguration:
        Rules:
          - Id: AutoDelete
            Status: Enabled
            ExpirationInDays: 14
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      NotificationConfiguration:
        EventBridgeConfiguration:
          EventBridgeEnabled: true
      Tags:
        - Key: Cost
          Value: !Sub ${SystemName}-${SubName}

# ------------------------------------------------------------#
# Elemental MediaConvert
# ------------------------------------------------------------#
  MediaConvertQueue:
    Type: AWS::MediaConvert::Queue
    Properties:
      Description: !Sub For ${SystemName}-${SubName}
      Name: !Sub ${SystemName}-${SubName}-mp4-hls-conv
      PricingPlan: ON_DEMAND
      Status: ACTIVE
      Tags:
        Cost: !Sub ${SystemName}-${SubName}

  MediaConvertJobTemplate:
    Type: AWS::MediaConvert::JobTemplate
    Properties:
      Name: !Sub ${SystemName}-${SubName}-mp4-hls-abr-360-720
      Description: !Sub The transcoding configuration for ${SystemName}-${SubName} (MP4 to HLS ABR 360p/720p)
      Category: !Ref SystemName
      AccelerationSettings:
        Mode: DISABLED
      Priority: 0
      Queue: !GetAtt MediaConvertQueue.Arn
      SettingsJson:
        Inputs:
          - VideoSelector:
              ColorSpace: FOLLOW
              SampleRange: FOLLOW
              Rotate: DEGREE_0
            AudioSelectors:
              Audio Selector 1:
                DefaultSelection: DEFAULT
            FilterEnable: AUTO
            PsiControl: IGNORE_PSI
            FilterStrength: 0
            DeblockFilter: DISABLED
            DenoiseFilter: DISABLED
            InputScanType: AUTO
            TimecodeSource: ZEROBASED
        OutputGroups:
          - CustomName: HLS_output
            Name: Apple HLS
            Outputs:
              # 360p video
              - ContainerSettings:
                  Container: M3U8
                NameModifier: _360p
                VideoDescription:
                  Width: 640
                  Height: 360
                  ScalingBehavior: DEFAULT
                  Sharpness: 50
                  CodecSettings:
                    Codec: H_264
                    H264Settings:
                      InterlaceMode: PROGRESSIVE
                      RateControlMode: QVBR
                      MaxBitrate: 800000
                      CodecProfile: MAIN
                      CodecLevel: AUTO
                      FramerateControl: INITIALIZE_FROM_SOURCE
                      GopSize: 2
                      GopSizeUnits: SECONDS
                      GopClosedCadence: 1
                      GopBReference: DISABLED
                      EntropyEncoding: CABAC
                      AdaptiveQuantization: HIGH
                      SpatialAdaptiveQuantization: ENABLED
                      TemporalAdaptiveQuantization: ENABLED
                      FlickerAdaptiveQuantization: DISABLED
                      SceneChangeDetect: TRANSITION_DETECTION
                      QualityTuningLevel: SINGLE_PASS_HQ
                      ParControl: INITIALIZE_FROM_SOURCE
              # 720p video
              - ContainerSettings:
                  Container: M3U8
                NameModifier: _720p
                VideoDescription:
                  Width: 1280
                  Height: 720
                  ScalingBehavior: DEFAULT
                  Sharpness: 50
                  CodecSettings:
                    Codec: H_264
                    H264Settings:
                      InterlaceMode: PROGRESSIVE
                      RateControlMode: QVBR
                      MaxBitrate: 2500000
                      CodecProfile: MAIN
                      CodecLevel: AUTO
                      FramerateControl: INITIALIZE_FROM_SOURCE
                      GopSize: 2
                      GopSizeUnits: SECONDS
                      GopClosedCadence: 1
                      GopBReference: DISABLED
                      EntropyEncoding: CABAC
                      AdaptiveQuantization: HIGH
                      SpatialAdaptiveQuantization: ENABLED
                      TemporalAdaptiveQuantization: ENABLED
                      FlickerAdaptiveQuantization: DISABLED
                      SceneChangeDetect: TRANSITION_DETECTION
                      QualityTuningLevel: SINGLE_PASS_HQ
                      ParControl: INITIALIZE_FROM_SOURCE
              # Audio
              - ContainerSettings:
                  Container: M3U8
                NameModifier: _audio
                AudioDescriptions:
                  - AudioTypeControl: FOLLOW_INPUT
                    AudioSourceName: Audio Selector 1
                    CodecSettings:
                      Codec: AAC
                      AacSettings:
                        Bitrate: 96000
                        RateControlMode: CBR
                        CodecProfile: LC
                        CodingMode: CODING_MODE_2_0
                        SampleRate: 48000
                        Specification: MPEG4
                    LanguageCodeControl: FOLLOW_INPUT
            OutputGroupSettings:
              Type: HLS_GROUP_SETTINGS
              HlsGroupSettings:
                SegmentLength: 10
                MinSegmentLength: 0
                Destination: !Sub s3://${S3Bucket}/output/
                DestinationSettings:
                  S3Settings:
                    StorageClass: STANDARD
                SegmentControl: SEGMENTED_FILES
                ManifestCompression: NONE
                DirectoryStructure: SINGLE_DIRECTORY
        TimecodeConfig:
          Source: ZEROBASED
      StatusUpdateInterval: SECONDS_60
      Tags:
        Cost: !Sub ${SystemName}-${SubName}
    DependsOn:
      - MediaConvertQueue

# ------------------------------------------------------------#
# Step Functions State Machine
# ------------------------------------------------------------#
  StateMachineMp4HlsConv:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      StateMachineName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
      StateMachineType: STANDARD
      DefinitionSubstitutions:
        DsSystemName: !Ref SystemName
        DsSubName: !Ref SubName
        DSRegion: !Ref AWS::Region
        DsAwsAccountId: !Ref AWS::AccountId
        DsMediaConvertJobTemplateArn: !GetAtt MediaConvertJobTemplate.Arn
        DsMediaConvertQueueArn: !GetAtt MediaConvertQueue.Arn
        DsSnsTopicArn: !GetAtt SNSTopic.TopicArn
      DefinitionString: |-
        {
          "StartAt": "CreateJob",
          "States": {
            "CreateJob": {
              "Type": "Task",
              "Resource": "arn:aws:states:::mediaconvert:createJob.sync",
              "Arguments": {
                "JobTemplate": "${DsMediaConvertJobTemplateArn}",
                "Queue": "${DsMediaConvertQueueArn}",
                "Role": "arn:aws:iam::${DsAwsAccountId}:role/service-role/MediaConvert_Default_Role",
                "Settings": {
                  "Inputs": [
                    {
                      "FileInput": "{% 's3://' & $states.input.detail.bucket.name & '/' & $states.input.detail.object.key %}"
                    }
                  ]
                },
                "Tags": {
                  "Cost": "${DsSystemName}-${DsSubName}"
                }
              },
              "Next": "SnsPublish",
              "QueryLanguage": "JSONata",
              "TimeoutSeconds": 600
            },
            "SnsPublish": {
              "Type": "Task",
              "Resource": "arn:aws:states:::sns:publish",
              "QueryLanguage": "JSONata",
              "Arguments": {
                "TopicArn": "${DsSnsTopicArn}",
                "Message": {
                  "Input": "{% $states.input.Job.Settings.Inputs[0].FileInput %}",
                  "Status": "{% $states.input.Job.Status %}",
                  "Messages": "{% $states.input.Job.Messages %}"
                }
              },
              "TimeoutSeconds": 30,
              "End": true
            }
          },
          "TimeoutSeconds": 660,
          "Comment": "For converting a MP4 media to HLS (ABR 360p/720p)"
        }
      LoggingConfiguration:
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt LogGroupStateMachineMp4HlsConv.Arn
        IncludeExecutionData: true
        Level: ERROR
      RoleArn: !GetAtt StateMachineMp4HlsConvRole.Arn
      TracingConfiguration:
        Enabled: false
      Tags:
        - Key: Cost
          Value: !Sub ${SystemName}-${SubName}
    DependsOn:
      - LogGroupStateMachineMp4HlsConv
      - StateMachineMp4HlsConvRole
      - MediaConvertJobTemplate

# ------------------------------------------------------------#
# Step Functions State Machine LogGroup (CloudWatch Logs)
# ------------------------------------------------------------#
  LogGroupStateMachineMp4HlsConv:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub /aws/vendedlogs/states/${SystemName}-${SubName}-mp4-hls-conv
      RetentionInDays: 365
      Tags:
        - Key: Cost
          Value: !Sub ${SystemName}-${SubName}

# ------------------------------------------------------------#
# Step Functions State Machine Execution Role (IAM)
# ------------------------------------------------------------#
  StateMachineMp4HlsConvRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${SystemName}-${SubName}-StateMachineMp4HlsConvRole
      Description: This role allows State Machines to invoke specified AWS resources.
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                states.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /service-role/
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
        - arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess
        - arn:aws:iam::aws:policy/AWSElementalMediaConvertFullAccess
      Policies:
        - PolicyName: !Sub ${SystemName}-${SubName}-StateMachineMp4HlsConvPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - "s3:GetObject"
                Resource:
                  - !Sub "${S3Bucket.Arn}/input/*"
              - Effect: Allow
                Action:
                  - "s3:PutObject"
                Resource:
                  - !Sub "${S3Bucket.Arn}/output/*"
              - Effect: Allow
                Action:
                  - "events:PutTargets"
                  - "events:PutRule"
                  - "events:DescribeRule"
                Resource: 
                  - !Sub "arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/StepFunctionsGetEventsForMediaConvertJobRule"
              - Effect: Allow
                Action:
                  - "sns:Publish"
                Resource: 
                  - !GetAtt SNSTopic.TopicArn
    DependsOn:
      - S3Bucket
      - SNSTopic

# ------------------------------------------------------------#
# EventBridge Rule for starting State Machine
# ------------------------------------------------------------#
  EventBridgeRuleStartSfn:
    Type: AWS::Events::Rule
    Properties:
      Name: !Sub ${SystemName}-${SubName}-mp4-hls-conv-start-sfn
      Description: !Sub This rule starts mp4 hls converter Sfn for ${SystemName}-${SubName}. The trigger is the S3 event notifications.
      EventBusName: !Sub "arn:aws:events:${AWS::Region}:${AWS::AccountId}:event-bus/default"
      EventPattern:
        source:
          - "aws.s3"
        detail-type:
          - "Object Created"
        detail:
          bucket:
            name:
              - !Ref S3Bucket
          object:
            key:
              - wildcard: "input/*.mp4"
      State: ENABLED
      Targets:
        - Arn: !GetAtt StateMachineMp4HlsConv.Arn
          Id: !Sub ${SystemName}-${SubName}-mp4-hls-conv-start-sfn
          RoleArn: !GetAtt EventBridgeRuleSfnRole.Arn
    DependsOn:
      - EventBridgeRuleSfnRole

# ------------------------------------------------------------#
# EventBridge Rule Invoke Step Functions State Machine Role (IAM)
# ------------------------------------------------------------#
  EventBridgeRuleSfnRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub ${SystemName}-${SubName}-EventBridgeHlsSfnRole
      Description: !Sub This role allows EventBridge to invoke mp4 hls converter Sfn for ${SystemName}-${SubName}.
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
              - events.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: !Sub ${SystemName}-${SubName}-EventBridgeHlsSfnPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - "states:StartExecution"
                Resource:
                  - !GetAtt StateMachineMp4HlsConv.Arn
    DependsOn:
      - StateMachineMp4HlsConv

# ------------------------------------------------------------#
# SNS Topic
# ------------------------------------------------------------#
  SNSTopic:
    Type: AWS::SNS::Topic
    Properties:
      TracingConfig: PassThrough
      DisplayName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
      FifoTopic: false
      Tags:
        - Key: Cost
          Value: !Sub ${SystemName}-${SubName}

 

AWS Elemental MediaConvert は、前提として MediaConvert に割り当てる IAM ロールが必要になります。以下のドキュメント参考に作成が必要ですが、今回は雑に広い権限を作成をしています。以前は権限決め打ちで、かつ AWS アカウント共通で持たないといけなかった記憶がありますが、今は細かく定義できるようになっています。

AWSTemplateFormatVersion: 2010-09-09
Description: The CloudFormation template that creates a MediaConvert_Default_Role in your AWS account. It is needed when you create MediaConvert jobs.

Resources:
# ------------------------------------------------------------#
# Elemental MediaConvert Default Role (IAM)
# ------------------------------------------------------------#
  MediaConvertDefaultRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: MediaConvert_Default_Role
      Description: This role allows MediaConvert to execute jobs.
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
              - mediaconvert.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /service-role/
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonS3FullAccess
        - arn:aws:iam::aws:policy/AmazonAPIGatewayInvokeFullAccess

 

まとめ

いかがでしたでしょうか?

以前作成した記事が古かったので、最新情報でアップデートしました。

本記事が皆様のお役に立てれば幸いです。

著者について
広野 祐司

AWS サーバーレスアーキテクチャと React を使用して社内向け e-Learning アプリ開発とコンテンツ作成に勤しんでいます。React でアプリを書き始めたら、快適すぎて他の言語には戻れなくなりました。近年は社内外への AWS 技術支援にも従事しています。AWS サービスには AWS が考える IT 設計思想が詰め込まれているので、いつも AWS を通して勉強させて頂いてまます。
取得資格:AWS 認定は15資格、IT サービスマネージャ、ITIL v3 Expert 等
2020 - 2025 Japan AWS Top Engineer 受賞
2022 - 2025 AWS Ambassador 受賞
2023 当社初代フルスタックエンジニア認定
好きなAWSサービス:AWS AppSync Events / AWS Step Functions / AWS CloudFormation

広野 祐司をフォローする

クラウドに強いによるエンジニアブログです。

SCSKクラウドサービス(AWS)は、企業価値の向上につながるAWS 導入を全面支援するオールインワンサービスです。AWS最上位パートナーとして、多種多様な業界のシステム構築実績を持つSCSKが、お客様のDX推進を強力にサポートします。

AWSクラウド
シェアする
タイトルとURLをコピーしました