Kafka Notizen Cheat Sheet

# zookee­per.pr­ope­rties
# the directory where the snapshot is stored.
dataDi­r=/­tmp­/zo­okeeper
# the port at which the clients will connect
client­Por­t=2181
# disable the per-ip limit on the number of connec­tions since this is a non-pr­odu­ction config
maxCli­ent­Cnxns=0

# Zookeeper starten
bin/zo­oke­epe­r-s­erv­er-­sta­rt.sh config­/zo­oke­epe­r.p­rop­erties

# server.pr­ope­rties
# The minimum age of a log file to be eligible for deletion due to age
log.re­ten­tio­n.h­our­s=168
# A size-based retention policy for logs. Segments are pruned from the log unless the remaining
# segments drop below log.re­ten­tio­n.b­ytes. Functions indepe­ndently of log.re­ten­tio­n.h­ours.
#log.r­ete­nti­on.b­yt­es=­107­3741824
# The maximum size of a log segment file. When this size is reached a new log segment will be created.
log.se­gme­nt.b­yt­es=­107­3741824
# The interval at which log segments are checked to see if they can be deleted according
# to the retention policies
log.re­ten­tio­n.c­hec­k.i­nte­rva­l.m­s=3­00000
##### Zookeeper ######­###­###­###­###­###­###­#####
'­zoo­kee­per.co­nne­ct=­loc­alh­ost­:21­81'
# Timeout in ms for connecting to zookeeper
zookee­per.co­nne­cti­on.t­im­eou­t.m­s=6000

# Broker (Server) starten
bin/ka­fka­-se­rve­r-s­tart.sh config­/se­rve­r.p­rop­erties

# Topic test erstellen
bin/ka­fka­-to­pics.sh \
--create --zook­eeper localh­ost­:2181 \
--repl­ica­tio­n-f­actor 1 \
--part­itions 1 \
--topic test \

# Topics auflisten
bin/ka­fka­-to­pics.sh --list --zook­eeper localh­ost­:2181

Kafka wurde von LinkedIn entwickelt und ist inzwischen ein Open Source Projekt der Apache Software Founda­tion. Die Firma hinter Kafka heißt Confluent, sie wurde von den Entwickler aus LinkedIn ausgeg­ründet. Das bedeutet aber nicht, dass es nicht noch viele weitere Firmen gibt welche Teile des Kafka Universums bereit­ste­llen.

Quicks­tart: https:­//k­afk­a.a­pac­he.o­rg­/qu­ick­start
Download: https:­//w­ww.a­pa­che.or­g/d­yn/­clo­ser.cg­i?p­ath­=/k­afka/
Docker Container: https:­//h­ub.d­oc­ker.co­m/r­/co­nfl­uen­t/kafka
Ökosystem: https:­//c­wik­i.a­pac­he.o­rg­/co­nfl­uen­ce/­dis­pla­y/K­AFK­A/E­cos­ystem

Im Kafka Universum wird die Welt wie folgt von abstrakt zu konkret herunter gebrochen:

Cluster
DataCenter
Rack
Instanz