kafka 集群安装

🧙 Questions

安装kafka集群（2.12-2.6.2）

☄️ Ideas

CDH 安装kafka 如需外网访问一定要配置 advertised.host.name 192.168.16.75

前提

ip	name
121.199.75.185(172.30.31.225)	master
47.99.57.56(172.30.31.227)	slave1
101.37.77.107(172.30.31.226)	slave2

• zookeeper集群

下载安装

# 三台都要做
wget https://dlcdn.apache.org/kafka/2.6.2/kafka_2.12-2.6.2.tgz --no-check-certificate
tar -vzxf kafka_2.12-2.6.2.tgz -C /opt
ln -s /opt/kafka_2.12-2.6.2 /opt/kafka 

vim /etc/profile
# === vim /etc/profile ===
export KAFKA_HOME=/opt/kafka
export PATH=$PATH:$ZOOKEEPER_HOME/bin:$KAFKA_HOME/bin
# === vim /etc/profile ===
source /etc/profile

修改配置文件

mkdir -p /data/kafka/logs

# broker.id 必须唯一
# listeners 一定要是ip地址且要是内网 
# advertised.listeners 一定要是ip且要为外网
vim /opt/kafka/config/server.properties 

# ===  vim /opt/kafka/config/server.properties  ===
broker.id=1
listeners=PLAINTEXT://172.30.31.225:9092
advertised.listeners=PLAINTEXT://121.199.75.185:9092
log.dirs=/data/kafka/logs
zookeeper.connect=master:2181,slave1:2181,slave2:2181
delete.topic.enable=true
# ===  vim /opt/kafka/config/server.properties  ===

启动kafka集群

kafka-server-start.sh -daemon /opt/kafka/config/server.properties
kafka-server-stop.sh -daemon /opt/kafka/config/server.properties

验证kafka集群

# 查看端口是否开始
netstat -ntpl | grep 9092

# 验证kafka集群是否生效  在其他kafka上是否存在topic
kafka-topics.sh --create --zookeeper master:2181 --replication-factor 3 -partitions 10 --topic ispong_kafka
kafka-topics.sh --describe --zookeeper master:2181
kafka-topics.sh --delete --zookeeper master:2181 --topic ispong_kafka
# 一台输入，多台输出
kafka-console-producer.sh --broker-list master:9092,slave1:9092,slave2:9092 --topic ispong_kafka
kafka-console-consumer.sh --bootstrap-server master:9092 --topic ispong_kafka --from-beginning

强行删除topic

zkCli.sh -server localhost:2181
ls /brokers/topics
rmr /brokers/topics/ispong_kafka
deleteall /brokers/topics/ispong_kafka

🔗 Links

• kafka website
• kafka download