标签：__ V06 v01 offsets -- kafka topic consumer
[BigDataHadoop：Hadoop&kafka.V06] [BigDataHadoop.kafka][|章节一|Hadoop生态圈技术栈|kafka|kafka生产与消费|]
一、kakfa生产与消费：验证zookeeper和kafka状态
### --- 验证zookeeper状态
[root@hadoop ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/yanqi/servers/zookeeper/bin/../conf/zoo.cfg
Mode: standalone### --- 启动kafka
[root@hadoop ~]# kafka-server-start.sh -daemon /opt/yanqi/servers/kafka/config/server.properties### --- kafka-topics.sh 用于管理主题。
~~~ # 列出现有的主题
[root@hadoop ~]# kafka-topics.sh --list --zookeeper localhost:2181/myKafka
~~~ # 创建主题，该主题包含一个分区，该分区为Leader分区，它没有Follower分区副本。
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--create --topic topic_1 --partitions 1 --replication-factor 1~~~ # 查看分区信息
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka --list
~~~ 输出参数
topic_1
~~~ # 查看指定主题的详细信息
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--describe --topic topic_1
~~~ 输出参数
Topic:topic_1 PartitionCount:1 ReplicationFactor:1 Configs:
Topic: topic_1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0~~~ # 注
~~~ 当前的主题是：topic_1
~~~ 当前主题有几个分区：PartitionCount：1
~~~ 副本因子是：ReplicationFactor：1~~~ # 删除指定主题
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--delete --topic topic_1
~~~ 输出参数
Topic topic_1 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.~~~ # 创建topic_2主题，有1个副本5个分区
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--create --topic topic_2 --partitions 5 --replication-factor 1~~~ # 查看主题
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--describe --topic topic_2
Topic:topic_2 PartitionCount:5 ReplicationFactor:1 Configs:
Topic: topic_2 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 4 Leader: 0 Replicas: 0 Isr: 0### --- kafka-console-producer.sh用于生产消息：
~~~ # 开启生产者
[root@hadoop ~]# kafka-console-producer.sh --topic topic_1 \
--broker-list localhost:9092
~~写入参数
>yanqi——1
>yanqi——2
>yanqi——3### --- kafka-console-consumer.sh用于消费消息：
~~~ # 开启消费者
[root@hadoop ~]# kafka-console-consumer.sh --bootstrap-server localhost:9092 \
--topic topic_1
~~~ 输出参数
yanqi——1
yanqi——2
yanqi——3~~~ # 开启消费者方式二，从头消费，不按照偏移量消费
[root@hadoop ~]# kafka-console-consumer.sh --bootstrap-server localhost:9092 \
--topic topic_1 --from-beginning
~~~ 输出参数
hello world
hello world
yanqi——1
yanqi——2
yanqi——3### --- 在磁盘上查看生成消费的数据
~~~ # 该目录下打开了很多偏移量
~~~ 这些都是kafka生成消费的数据，包括主题分区详情。
[root@hadoop ~]# ls /opt/yanqi/servers/kafka/kafka-logs/
cleaner-offset-checkpoint __consumer_offsets-15 __consumer_offsets-22 __consumer_offsets-3 __consumer_offsets-37 __consumer_offsets-44 __consumer_offsets-7 topic_2-0
__consumer_offsets-0 __consumer_offsets-16 __consumer_offsets-23 __consumer_offsets-30 __consumer_offsets-38 __consumer_offsets-45 __consumer_offsets-8 topic_2-1
__consumer_offsets-1 __consumer_offsets-17 __consumer_offsets-24 __consumer_offsets-31 __consumer_offsets-39 __consumer_offsets-46 __consumer_offsets-9 topic_2-2
__consumer_offsets-10 __consumer_offsets-18 __consumer_offsets-25 __consumer_offsets-32 __consumer_offsets-4 __consumer_offsets-47 log-start-offset-checkpoint topic_2-3
__consumer_offsets-11 __consumer_offsets-19 __consumer_offsets-26 __consumer_offsets-33 __consumer_offsets-40 __consumer_offsets-48 meta.properties topic_2-4
__consumer_offsets-12 __consumer_offsets-2 __consumer_offsets-27 __consumer_offsets-34 __consumer_offsets-41 __consumer_offsets-49 recovery-point-offset-checkpoint
__consumer_offsets-13 __consumer_offsets-20 __consumer_offsets-28 __consumer_offsets-35 __consumer_offsets-42 __consumer_offsets-5 replication-offset-checkpoint
__consumer_offsets-14 __consumer_offsets-21 __consumer_offsets-29 __consumer_offsets-36 __consumer_offsets-43 __consumer_offsets-6 topic_1-0===============================END===============================
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor
来自为知笔记(Wiz)
标签：__,V06,v01,offsets,--,kafka,topic,consumer 来源： https://www.cnblogs.com/yanqivip/p/16121776.html
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。