Amazon MSKを使うときに知っておきたいベストプラクティス

そのクラスター構築では、下記を設定するのが一般的です。

Topic: MSKTutorialTopic TopicId: 8TdcHHTZRgaoF9JmsPS1yg PartitionCount: 1       ReplicationFactor: 3    Configs: min.insync.replicas=2,message.format.version=3.0-IV1,unclean.leader.election.enable=true

bootstrap.servers=b-1.example:xxxx,b-2.example:xxxx,b-3.example:xxxx

The number of partitions per broker is above the recommended limit. Add more brokers and rearrange the partitions per broker to be below the recommended limit, then retry the request

{
  "Sid": "VisualEditor1",
  "Effect": "Allow",
  "Action": [
  "kafka-cluster:*Topic*",
  "kafka-cluster:ReadData",
  "kafka-cluster:WriteData"
  ],
  "Resource": "arn:aws:kafka:*:123456789000:topic/*/*/*-dev"
}

sh-5.2$ ./bin/kafka-console-producer.sh --bootstrap-server <bootstrap-server>--topic SampleTopic001-dev --producer.config ./config/client.properties
>message1

sh-5.2$ ./bin/kafka-console-producer.sh --bootstrap-server <bootstrap-server>--topic SampleTopic001-stg --producer.config ./config/client.properties
>message1
[2025-11-28 09:15:09,879] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 5215 : {SampleTopic001-stg=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2025-11-28 09:15:09,885] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [SampleTopic001-stg] (org.apache.kafka.clients.Metadata)
[2025-11-28 09:15:09,887] ERROR Error when sending message to topic SampleTopic001-stg with key: null, value: 8 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [SampleTopic001-stg]

bootstrap.servers=b-1.example:xxxx,b-2.example:xxxx,b-3.example:xxxx