Administration of Confluent Apache Kafka Training Course

Module 1: Overview of Confluent Apache Kafka Architecture and Setup

• The role of Kafka in contemporary data pipelines
• Distinguishing features between Apache Kafka and Confluent Kafka
• Essential components: producers, consumers, brokers, topics, and partitions
• Deployment models for Kafka clusters and considerations for scaling

Module 2: Setting Up Zookeeper Quorums

• An introduction to Zookeeper
• The function of Zookeeper within a Kafka cluster
• Determining the appropriate Zookeeper Quorum size
• Configuring Zookeeper parameters
• Establishing SSH access on servers
• Practical exercise: Configuring Zookeeper as both a team and a service
• Utilizing the Zookeeper Command Line Interface (CLI)
• Practical exercise: Configuring Zookeeper Quorums
• Understanding the Zookeeper internal file system
• Performance metrics that influence Zookeeper
• Demonstration of Zookeeper and Zoonavigator management tools

Module 3: Configuring the Kafka Cluster

• Foundational Kafka concepts
• General Kafka configuration settings
• Practical exercise: Configuring Kafka brokers
• Practical exercise: Running Kafka commands
• Practical exercise: Setting up a Kafka Multi-Broker Cluster
• Practical exercise: Testing the Kafka cluster
• Verifying connectivity to the Kafka cluster
• Configuring Advertised.listeners: a critical setting
• Managing Topic configurations
• Settings for message ingestion and retrieval
• Practical exercise: Demonstrating Kafka resilience
• Kafka performance optimization: I/O
• Kafka performance optimization: Network (RED)
• Kafka performance optimization: RAM
• Kafka performance optimization: CPU
• Kafka performance optimization: Operating System (OS)
• Kafka performance: Other factors
• Practical exercise: Modifying Kafka broker configurations

Module 4: Advanced Kafka Configurations

• Configuring Landoop Kafka topic UI, Confluent REST Proxy, and Confluent Schema Registry
• Message transmission and reception via CLI, Java, and the Spring framework
• Monitoring metrics and utilizing tools (Confluent Control Center, Elasticsearch, etc.)
• Managing log files and offsets
• Strategies for high availability and disaster recovery
• Achieving high availability through replication
• Optimizing producer and consumer performance
• Developing disaster recovery plans
• Managing failover control and data recovery
• Configuring connectors
• Implementing Kafka Connect
• Implementing Kafka security features

Wrap-up and Recommended Next Steps