css3
V2EX  ›  Kafka

Python 消费 ka-f-a-k 时无限挂起,如何定位原因

  •  
  •   css3 · Jun 18, 2021 via iPhone · 1628 views
    This topic created in 1792 days ago, the information mentioned may be changed or developed.

    直接通过ka,fka 命令行是可以消费的,但通过 python 消费时,一直消费不到,进程也不超时,更换消费组,消费lastet最新 msg 也无响应, consume.poll(1000, 1) 第一个参数的 timeout 仿佛无效,设置了也是一直等待消费

    import time
    from kafka import KafkaConsumer
    
    topic1 = 'additional_order'
    host = '192.168.14.55:9092'
    
    
    consume = KafkaConsumer(topic, group_id="group1", 
                                  bootstrap_servers=host, auto_offset_reset='latest',
                                  security_protocol='SASL_PLAINTEXT',
                                  sasl_mechanism='PLAIN',
                                  sasl_plain_username='admin',
                                  sasl_plain_password='$%<a'
                                  api_version=(0, 10)
                                  )
    
    
    while True:
        count += 1
        if time.time() - start_time &gt; 1 * 60:
            print('超时,退出')
            break
        msg = consume.poll(1000, 1)  # 看上去,进程是一直等候在这里
        print(count)
    
    5 replies    2021-06-18 22:51:52 +08:00
    freshgoose
        1
    freshgoose  
       Jun 18, 2021
    这也是敏感词吗……
    css3
        2
    css3  
    OP
       Jun 18, 2021 via iPhone
    @freshgoose 不是,自动分类到 ka 板块了,求助没人看😂
    ynkkdev
        3
    ynkkdev  
       Jun 18, 2021
    api_version=(0,1,0)?
    shicheng1993
        4
    shicheng1993  
       Jun 18, 2021
    用 RocketMQ 好像遇到过类似的问题,因为个人水平问题,用间接的方式解决了。
    子进程进行消费并记录最后消费时间,主进程死循环,查看每个子进程消费时间,超时则重启子进程。不求甚解可以试试,O(∩_∩)O~
    css3
        5
    css3  
    OP
       Jun 18, 2021
    @youngce 这个影响不大吧?
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   3041 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 39ms · UTC 05:39 · PVG 13:39 · LAX 22:39 · JFK 01:39
    ♥ Do have faith in what you're doing.