こんにちは、広野です。
以前、以下の記事で紹介していた AWS Step Functions のジョブを最新化したので紹介します。今回のジョブは社内教育用の動画作成で活用しています。
※上記記事は古いので、お勧めしません。
動画は Amazon CloudFront 経由、Amazon Cognito 認証付きで配信しています。参考記事はこちら。
アーキテクチャ
基本的には以下の記事と同じですので、割愛します。
仕様
- MP4 ファイルを Amazon S3 バケットの input フォルダに配置したら、自動で AWS Step Functions ステートマシンが呼び出される。
- ステートマシンは、AWS Elemental MediaConvert を呼び出し、S3 上の MP4 ファイルを GIF に変換し S3 バケットの output フォルダに保存する。
- 最後に完了したことを Amazon SNS で通知する。
- コーデックは H.264 を採用。互換性重視のため。H.265 ではブラウザによっては再生できない。
- 画質は 360p と 720p の 2段階。アダプティブビットレート (ABR) 対応で視聴者のネットワーク状況に応じてプレーヤーが自動で画質を切り替えられるようにした。
インプット
以下のように、MP4 ファイルを S3 バケットに配置します。
アウトプット
以下のように、m3u8 ファイル群が生成されます。プレーヤーに、この例では sample-cat.m3u8 を指定することになります。
AWS CloudFormation テンプレート
実際にデプロイしたときに使用した AWS CloudFormation テンプレートを貼り付けます。詳細な設定はこちらをご覧ください。
AWSTemplateFormatVersion: 2010-09-09
Description: The CloudFormation template that creates a Step Functions state machine. It provides converting MP4 to HLS (H.264, AAC, ABR 360p/720p). The IAM role MediaConvert_Default_Role must be created in your AWS account before creating a job. Please refer https://docs.aws.amazon.com/mediaconvert/latest/ug/creating-the-iam-role-in-mediaconvert-configured.html for details.
# ------------------------------------------------------------#
# Input Parameters
# ------------------------------------------------------------#
Parameters:
SystemName:
Type: String
Description: System name. use lower case only. (e.g. example)
Default: example
MaxLength: 10
MinLength: 1
SubName:
Type: String
Description: System sub name. use lower case only. (e.g. prod or dev)
Default: dev
MaxLength: 10
MinLength: 1
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "General Configuration"
Parameters:
- SystemName
- SubName
Resources:
# ------------------------------------------------------------#
# S3
# ------------------------------------------------------------#
S3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
LifecycleConfiguration:
Rules:
- Id: AutoDelete
Status: Enabled
ExpirationInDays: 14
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
NotificationConfiguration:
EventBridgeConfiguration:
EventBridgeEnabled: true
Tags:
- Key: Cost
Value: !Sub ${SystemName}-${SubName}
# ------------------------------------------------------------#
# Elemental MediaConvert
# ------------------------------------------------------------#
MediaConvertQueue:
Type: AWS::MediaConvert::Queue
Properties:
Description: !Sub For ${SystemName}-${SubName}
Name: !Sub ${SystemName}-${SubName}-mp4-hls-conv
PricingPlan: ON_DEMAND
Status: ACTIVE
Tags:
Cost: !Sub ${SystemName}-${SubName}
MediaConvertJobTemplate:
Type: AWS::MediaConvert::JobTemplate
Properties:
Name: !Sub ${SystemName}-${SubName}-mp4-hls-abr-360-720
Description: !Sub The transcoding configuration for ${SystemName}-${SubName} (MP4 to HLS ABR 360p/720p)
Category: !Ref SystemName
AccelerationSettings:
Mode: DISABLED
Priority: 0
Queue: !GetAtt MediaConvertQueue.Arn
SettingsJson:
Inputs:
- VideoSelector:
ColorSpace: FOLLOW
SampleRange: FOLLOW
Rotate: DEGREE_0
AudioSelectors:
Audio Selector 1:
DefaultSelection: DEFAULT
FilterEnable: AUTO
PsiControl: IGNORE_PSI
FilterStrength: 0
DeblockFilter: DISABLED
DenoiseFilter: DISABLED
InputScanType: AUTO
TimecodeSource: ZEROBASED
OutputGroups:
- CustomName: HLS_output
Name: Apple HLS
Outputs:
# 360p video
- ContainerSettings:
Container: M3U8
NameModifier: _360p
VideoDescription:
Width: 640
Height: 360
ScalingBehavior: DEFAULT
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
RateControlMode: QVBR
MaxBitrate: 800000
CodecProfile: MAIN
CodecLevel: AUTO
FramerateControl: INITIALIZE_FROM_SOURCE
GopSize: 2
GopSizeUnits: SECONDS
GopClosedCadence: 1
GopBReference: DISABLED
EntropyEncoding: CABAC
AdaptiveQuantization: HIGH
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
SceneChangeDetect: TRANSITION_DETECTION
QualityTuningLevel: SINGLE_PASS_HQ
ParControl: INITIALIZE_FROM_SOURCE
# 720p video
- ContainerSettings:
Container: M3U8
NameModifier: _720p
VideoDescription:
Width: 1280
Height: 720
ScalingBehavior: DEFAULT
Sharpness: 50
CodecSettings:
Codec: H_264
H264Settings:
InterlaceMode: PROGRESSIVE
RateControlMode: QVBR
MaxBitrate: 2500000
CodecProfile: MAIN
CodecLevel: AUTO
FramerateControl: INITIALIZE_FROM_SOURCE
GopSize: 2
GopSizeUnits: SECONDS
GopClosedCadence: 1
GopBReference: DISABLED
EntropyEncoding: CABAC
AdaptiveQuantization: HIGH
SpatialAdaptiveQuantization: ENABLED
TemporalAdaptiveQuantization: ENABLED
FlickerAdaptiveQuantization: DISABLED
SceneChangeDetect: TRANSITION_DETECTION
QualityTuningLevel: SINGLE_PASS_HQ
ParControl: INITIALIZE_FROM_SOURCE
# Audio
- ContainerSettings:
Container: M3U8
NameModifier: _audio
AudioDescriptions:
- AudioTypeControl: FOLLOW_INPUT
AudioSourceName: Audio Selector 1
CodecSettings:
Codec: AAC
AacSettings:
Bitrate: 96000
RateControlMode: CBR
CodecProfile: LC
CodingMode: CODING_MODE_2_0
SampleRate: 48000
Specification: MPEG4
LanguageCodeControl: FOLLOW_INPUT
OutputGroupSettings:
Type: HLS_GROUP_SETTINGS
HlsGroupSettings:
SegmentLength: 10
MinSegmentLength: 0
Destination: !Sub s3://${S3Bucket}/output/
DestinationSettings:
S3Settings:
StorageClass: STANDARD
SegmentControl: SEGMENTED_FILES
ManifestCompression: NONE
DirectoryStructure: SINGLE_DIRECTORY
TimecodeConfig:
Source: ZEROBASED
StatusUpdateInterval: SECONDS_60
Tags:
Cost: !Sub ${SystemName}-${SubName}
DependsOn:
- MediaConvertQueue
# ------------------------------------------------------------#
# Step Functions State Machine
# ------------------------------------------------------------#
StateMachineMp4HlsConv:
Type: AWS::StepFunctions::StateMachine
Properties:
StateMachineName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
StateMachineType: STANDARD
DefinitionSubstitutions:
DsSystemName: !Ref SystemName
DsSubName: !Ref SubName
DSRegion: !Ref AWS::Region
DsAwsAccountId: !Ref AWS::AccountId
DsMediaConvertJobTemplateArn: !GetAtt MediaConvertJobTemplate.Arn
DsMediaConvertQueueArn: !GetAtt MediaConvertQueue.Arn
DsSnsTopicArn: !GetAtt SNSTopic.TopicArn
DefinitionString: |-
{
"StartAt": "CreateJob",
"States": {
"CreateJob": {
"Type": "Task",
"Resource": "arn:aws:states:::mediaconvert:createJob.sync",
"Arguments": {
"JobTemplate": "${DsMediaConvertJobTemplateArn}",
"Queue": "${DsMediaConvertQueueArn}",
"Role": "arn:aws:iam::${DsAwsAccountId}:role/service-role/MediaConvert_Default_Role",
"Settings": {
"Inputs": [
{
"FileInput": "{% 's3://' & $states.input.detail.bucket.name & '/' & $states.input.detail.object.key %}"
}
]
},
"Tags": {
"Cost": "${DsSystemName}-${DsSubName}"
}
},
"Next": "SnsPublish",
"QueryLanguage": "JSONata",
"TimeoutSeconds": 600
},
"SnsPublish": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"QueryLanguage": "JSONata",
"Arguments": {
"TopicArn": "${DsSnsTopicArn}",
"Message": {
"Input": "{% $states.input.Job.Settings.Inputs[0].FileInput %}",
"Status": "{% $states.input.Job.Status %}",
"Messages": "{% $states.input.Job.Messages %}"
}
},
"TimeoutSeconds": 30,
"End": true
}
},
"TimeoutSeconds": 660,
"Comment": "For converting a MP4 media to HLS (ABR 360p/720p)"
}
LoggingConfiguration:
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt LogGroupStateMachineMp4HlsConv.Arn
IncludeExecutionData: true
Level: ERROR
RoleArn: !GetAtt StateMachineMp4HlsConvRole.Arn
TracingConfiguration:
Enabled: false
Tags:
- Key: Cost
Value: !Sub ${SystemName}-${SubName}
DependsOn:
- LogGroupStateMachineMp4HlsConv
- StateMachineMp4HlsConvRole
- MediaConvertJobTemplate
# ------------------------------------------------------------#
# Step Functions State Machine LogGroup (CloudWatch Logs)
# ------------------------------------------------------------#
LogGroupStateMachineMp4HlsConv:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub /aws/vendedlogs/states/${SystemName}-${SubName}-mp4-hls-conv
RetentionInDays: 365
Tags:
- Key: Cost
Value: !Sub ${SystemName}-${SubName}
# ------------------------------------------------------------#
# Step Functions State Machine Execution Role (IAM)
# ------------------------------------------------------------#
StateMachineMp4HlsConvRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${SystemName}-${SubName}-StateMachineMp4HlsConvRole
Description: This role allows State Machines to invoke specified AWS resources.
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
states.amazonaws.com
Action:
- sts:AssumeRole
Path: /service-role/
ManagedPolicyArns:
- arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
- arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess
- arn:aws:iam::aws:policy/AWSElementalMediaConvertFullAccess
Policies:
- PolicyName: !Sub ${SystemName}-${SubName}-StateMachineMp4HlsConvPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- "s3:GetObject"
Resource:
- !Sub "${S3Bucket.Arn}/input/*"
- Effect: Allow
Action:
- "s3:PutObject"
Resource:
- !Sub "${S3Bucket.Arn}/output/*"
- Effect: Allow
Action:
- "events:PutTargets"
- "events:PutRule"
- "events:DescribeRule"
Resource:
- !Sub "arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/StepFunctionsGetEventsForMediaConvertJobRule"
- Effect: Allow
Action:
- "sns:Publish"
Resource:
- !GetAtt SNSTopic.TopicArn
DependsOn:
- S3Bucket
- SNSTopic
# ------------------------------------------------------------#
# EventBridge Rule for starting State Machine
# ------------------------------------------------------------#
EventBridgeRuleStartSfn:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${SystemName}-${SubName}-mp4-hls-conv-start-sfn
Description: !Sub This rule starts mp4 hls converter Sfn for ${SystemName}-${SubName}. The trigger is the S3 event notifications.
EventBusName: !Sub "arn:aws:events:${AWS::Region}:${AWS::AccountId}:event-bus/default"
EventPattern:
source:
- "aws.s3"
detail-type:
- "Object Created"
detail:
bucket:
name:
- !Ref S3Bucket
object:
key:
- wildcard: "input/*.mp4"
State: ENABLED
Targets:
- Arn: !GetAtt StateMachineMp4HlsConv.Arn
Id: !Sub ${SystemName}-${SubName}-mp4-hls-conv-start-sfn
RoleArn: !GetAtt EventBridgeRuleSfnRole.Arn
DependsOn:
- EventBridgeRuleSfnRole
# ------------------------------------------------------------#
# EventBridge Rule Invoke Step Functions State Machine Role (IAM)
# ------------------------------------------------------------#
EventBridgeRuleSfnRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub ${SystemName}-${SubName}-EventBridgeHlsSfnRole
Description: !Sub This role allows EventBridge to invoke mp4 hls converter Sfn for ${SystemName}-${SubName}.
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: !Sub ${SystemName}-${SubName}-EventBridgeHlsSfnPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- "states:StartExecution"
Resource:
- !GetAtt StateMachineMp4HlsConv.Arn
DependsOn:
- StateMachineMp4HlsConv
# ------------------------------------------------------------#
# SNS Topic
# ------------------------------------------------------------#
SNSTopic:
Type: AWS::SNS::Topic
Properties:
TracingConfig: PassThrough
DisplayName: !Sub ${SystemName}-${SubName}-mp4-hls-conv
FifoTopic: false
Tags:
- Key: Cost
Value: !Sub ${SystemName}-${SubName}
AWS Elemental MediaConvert は、前提として MediaConvert に割り当てる IAM ロールが必要になります。以下のドキュメント参考に作成が必要ですが、今回は雑に広い権限を作成をしています。以前は権限決め打ちで、かつ AWS アカウント共通で持たないといけなかった記憶がありますが、今は細かく定義できるようになっています。
AWSTemplateFormatVersion: 2010-09-09
Description: The CloudFormation template that creates a MediaConvert_Default_Role in your AWS account. It is needed when you create MediaConvert jobs.
Resources:
# ------------------------------------------------------------#
# Elemental MediaConvert Default Role (IAM)
# ------------------------------------------------------------#
MediaConvertDefaultRole:
Type: AWS::IAM::Role
Properties:
RoleName: MediaConvert_Default_Role
Description: This role allows MediaConvert to execute jobs.
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- mediaconvert.amazonaws.com
Action:
- sts:AssumeRole
Path: /service-role/
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonS3FullAccess
- arn:aws:iam::aws:policy/AmazonAPIGatewayInvokeFullAccess
まとめ
いかがでしたでしょうか?
以前作成した記事が古かったので、最新情報でアップデートしました。
本記事が皆様のお役に立てれば幸いです。





