Question

You are working on a system that will use AWS Kinesis, and it is getting data from various log sources. You are looking at creating an initial number of shards for the Kinesis stream.

Which of the following can be used to calculate the initial number of shards for the Kinesis stream? Choose 2 answers from the options given below.

A. Incoming write bandwidth

B. Outgoing write bandwidth

C. Incoming read bandwidth

D. Outgoing read bandwidth.

Answer 1

Answer - A and D.

This is given in the AWS Documentation.

You can calculate the initial number of shards (

number_of_shards.

) that your stream needs by using the input values in the following formula:

number_of_shards = max(incoming_write_bandwidth_in_KiB/1024, outgoing_read_bandwidth_in_KiB/2048)

For more information on Amazon Kinesis streams, please refer to the below URL-

https://docs.aws.amazon.com/streams/latest/dev/amazon-kinesis-streams.html

When designing an Amazon Kinesis data stream, one of the most important factors to consider is the number of shards. A shard represents a unit of data capacity in a Kinesis stream, and it can process up to 1MB of data per second or 1000 write transactions per second. Therefore, the number of shards you create for a Kinesis stream determines the maximum amount of data that the stream can handle.

When deciding on the number of shards to create for a Kinesis stream, there are several factors to consider, such as the amount of incoming write bandwidth, the expected outgoing write bandwidth, and the incoming and outgoing read bandwidth. Based on these factors, you can use the following approaches to determine the initial number of shards for your Kinesis stream:

1. Incoming write bandwidth: One way to determine the initial number of shards for a Kinesis stream is to consider the expected incoming write bandwidth. To do this, you can estimate the total amount of data that the stream is expected to receive per second and divide it by the maximum amount of data that a single shard can handle (1 MB/s). For example, if you expect the stream to receive 5 MB/s of data, you would need to create five shards to handle the incoming write bandwidth.

2. Incoming and outgoing read bandwidth: Another approach to determine the initial number of shards for a Kinesis stream is to consider the expected incoming and outgoing read bandwidth. To do this, you can estimate the maximum number of read transactions per second that the stream is expected to receive and divide it by the maximum number of read transactions that a single shard can handle (5 read transactions/s). For example, if you expect the stream to receive 10,000 read transactions per second, you would need to create 2,000 shards to handle the incoming and outgoing read bandwidth.

Outgoing write bandwidth and outgoing read bandwidth are not used to calculate the initial number of shards for a Kinesis stream, as these factors are determined by the number of consumers and the amount of data they are consuming from the stream.

In summary, when determining the initial number of shards for a Kinesis stream, you should consider the expected incoming write bandwidth and the expected incoming and outgoing read bandwidth.