- Kinesis Streams
- Kinesis Firehose
- Kinesis Analytics
- Kinesis Streams
- Ingest and process streaming data with "custom applications"
- Producers put data into streams
- Consumers consume data and process them (fleet of instances)
- Shard
- It is a basic throughout unit of a stream
- Streaming data is in the form on Shards
- Default retention period is 1 day
- You can keep data till 7 days (168 hrs)
- Kinesis Firehose
- Capture, transform & load streaming data
- Deliver real time data to "AWS destinations" like S3, RedShift, ElasticSearch, Splunk etc.,
- You don't need to write applications or manage resources
- Using Lambda, you can transform data before delivering data
- You can compress/encrypt data there by saving storage cost and increasing security
- Scales automatically
- Kinesis Analytics
- Run SQL queries on Kinesis Streaming data
- Analyze and process streaming data from Kinesis Streaming/Firehose
Kinesis Limitations
- No of shards
- Upto 200
- Upto 500 in some regions
- No hard limit (AWS can lift off if u request)
- Data retention
- 1 day be default
- Upto 7 days
- Hard limit (AWS cannot lift off)
- Write limitations
- Shard throughput - upto 1 MB
- No of transactions - 1000 records/sec
- Read limitations
- One shard can return < 2 MB/sec
- No of transactions: 5 records/shard/sec
