AWS MSK Serverless Live Demo

This project consist of four projects. Each project has its own documentation in README.md

Getting started AWS MSK Serverless

• Kafka Publisher and WebSocket
• Kafka Steam Aggregator
• CDK for deployment
• Create Kafka topic (demo purpose)

Getting Started (Local Development)

# init local kafka
docker-compose up -d

Then start the application in the following sequence:

• createTopic
• ktor-consumer
• ktor-publisher

Deploy

# install AWS Cloud Development Kit
nvm install 16
nvm use 16
npm install -g aws-cdk

# deploy
cd cdk
cdk deploy EcrStack
cdk deploy VpcStack
cdk deploy FargateStack

Resources for learning

Always use jib to build java/kotlin image

https://cloud.google.com/java/getting-started/jib

Kotlin Coroutine (Actor model)

• https://kt.academy/article/cc-channel
• https://kotlinlang.org/docs/shared-mutable-state-and-concurrency.html

Ktor websocket

• https://ktor.io/docs/websocket.html

Kafka Stream

• https://docs.confluent.io/platform/current/streams/developer-guide/dsl-api.html

AWS Glue (Kafka Schema Registry)

• https://aws.amazon.com/blogs/big-data/introducing-protocol-buffers-protobuf-schema-support-in-amazon-glue-schema-registry/

GitHub

View Github