Kafka Introduction

Introduction

• Pub/Sub

• Used for near realtime processing - events

• Kafka began at LinkedIn - Jay Krep - I Love Logs - also Amazon Kinesis

• Solves 3 problems - unreliable receivers, scaling, messy communications

Dataflow for realtime event analytics

Why event driven

• compare to batch

• fraud detection

• IOT

• show architecture of stuff

• infinite sequence ....

What is Pub/Sub

• Producer/Consumer with a buffer

• Events are numbered and processed in order

Architecture

Kafka - put everything on the stream

Solves the Messy Communications Problem

https://www.slideshare.net/KaiWaehner/apache-kafka-as-eventdriven-open-source-streaming-platform-prague-meetup?qid=64dec3ab-f2a2-4ee5-86df-23feef24840d&v=&b=&from_search=2

Scaling

Open up another buffer or broker

Start more consumers

Replication for reliability

https://www.saowen.com/a/e25edaf7cf39676b6cb617bbb1e988a0ecf9fe3cc0275e85a2ff5d76bc59f14b

Brokers, Partitions, Topics

• Broker - URL

• Partition

• Topic

• http://codeflex.co/what-is-apache-kafka/