Apache Kafka Online Course

Apache Kafka Online Course

This course offers a comprehensive, step-by-step introduction to Apache Kafka, beginning with core concepts and progressing to practical implementation. You’ll start by gaining a solid theoretical understanding of Kafka’s architecture and internals, followed by hands-on exercises to reinforce your learning.

You'll learn how to develop a Kafka Producer using the Producer API to publish messages, explore key configuration options to ensure reliable message delivery, and then move on to building a Kafka Consumer using the Consumer API to read data from Kafka topics.

By the end of this course, you’ll have a clear and practical understanding of Apache Kafka and be capable of developing robust Kafka Consumer applications using Java.

Who Should Take This Course?

This course is perfect for both beginners and those with prior experience in Kafka. Whether you want to build Java applications using Kafka’s Producer and Consumer APIs or dive into more advanced Kafka operations, this course will provide the essential skills and knowledge you need.

Course Curriculum

Getting Started with the Course

• Course Introduction and Objectives
• Prerequisites

Getting Started with Kafka

• Introduction to Kafka
• Kafka Terminologies and Client APIs

Download and Install Kafka

• Download and Install Kafka

Understanding Kafka Components and its Internals - (Theory + Hands-On)

• Kafka Topics and Partitions - Theory
• Set Up a ZooKeeper/Kafka Broker in Local
• Create Topic, Produce and Consume Messages Using the CLI
• Produce and Consume Messages with Key
• Consumer Offsets
• Consumer Groups
• Commit Log and Retention Policy
• Kafka as a Distributed Streaming System
• Setting up a Kafka Cluster in Local with 3 Kafka Brokers
• How Kafka Cluster Distributes the Client Requests? Leader/Follower
• How Kafka Handles Data Loss? Replication and In-Sync-Replica(ISR)
• Fault Tolerance and Robustness in Kafka

Kafka Producer API - Hands-On

• Project Setup
• Sending ProducerRecord Using Producer API
• Build Kafka Producer - Send Messages Synchronously
• Add Logging Using LogBack
• Sending Messages Asynchronously
• Send Messages with Key
• Programmatic Interface to Produce Messages
• Kafka Producer API - Behind the Scenes

Kafka Producer API - Guaranteed Message Delivery Configurations

• Kafka Producer API Configurations
• Configuring Acks and min.insync.replicas – Hands-On
• Configuring retries and retry.backoff.ms – Hands-On

Kafka Consumer API – Hands-On

• Project Setup
• Consuming Messages using Consumer API - poll() loop
• Build Kafka Consumer Using the Consumer API
• auto.offset.reset - Earliest Versus Latest

Consumer Groups and Consumer Rebalance – Hands-On

• Scaling the Consumer Using Consumer Groups
• Consumer Rebalance
• Consumer Rebalance and max.poll.interval.ms

Consumer Offsets - Default and Manual Offset Management – Hands-On

• Committing Offsets - Default Offset Commit Strategy
• Manually Committing Offsets
• Committing Offsets Synchronously - Using commitSync()
• Committing Offsets Asynchronously - Using commitAsync()
• Committing Specific Offsets - using CommitSync()

Consumer Rebalance Listeners – Hands-On

• Introduction to Rebalance Listeners
• Implement ConsumerRebalanceListener in Kafka Consumer
• Committing Offsets Using ConsumerRebalanceListener

Kafka Consumer - seekToBeginning(), seekToEnd(), and seek() – Hands-On

• Introduction to seekToBeginning() or seekToEnd()
• seekToBeginning() and seekToEnd() – Hands-On
• Introduction to seek()
• Seek to a Specific Offset Using seek() – Hands-On

Custom Serializer and Deserializers in Kafka – Hands-On

• Why Custom Kafka Serializer/Deserializer?
• Build a Custom Serializer in Kafka Producer Using Jackson ObjectMapper
• Build Kafka Producer Using Custom Serializer
• Build a Custom DeSerializer in Kafka Producer Using Jackson ObjectMapper
• Build Kafka Consumer Using Custom Serializer
• Publish Custom Objects Using StringSerializer and Jackson ObjectMapper
• Consume Custom Objects using StringDeserializer and Jackson ObjectMapper