V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
V2EX 提问指南
yellowmarlboro
V2EX  ›  问与答

kafka 的连接问题...

  •  
  •   yellowmarlboro · 2018-08-21 12:40:02 +08:00 · 5957 次点击
    这是一个创建于 2289 天前的主题,其中的信息可能已经有所发展或是发生改变。

    目前是 我要通过我的服务器连远程服务器的 kafka,我的服务器只作 producer,远程服务器把我的 ip 加入白名单了, 我用 telnet 也能通,可是用代码(python: ptkafka\kafka-python)就报这种错:

    pykafka: 'Failed to connect newly created broker for ip:host'
    kafka-python: 'NoBrokersAvailable'
    

    目前是 broker 或者配置问题? 另外我服务器不需要安装 kafka 了吧?

    我才看了半天 kafka 菜鸡 求教大佬们

    第 1 条附言  ·  2018-08-21 17:06:31 +08:00

    后来加上api_version参数后就如下错误,找了好多说是kafka配置问题..

    INFO:kafka.client:Bootstrap succeeded: found 1 brokers and 35 topics.
    INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <connected> [IPv4 ('host', port)]>: Closing connection.
    DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <disconnecting> [IPv4 ('host', port)]>: reconnect backoff 0.05978240071782098 after 1 failures
    DEBUG:kafka.metrics.metrics:Added sensor with name bufferpool-wait-time
    DEBUG:kafka.metrics.metrics:Added sensor with name batch-size
    DEBUG:kafka.metrics.metrics:Added sensor with name compression-rate
    DEBUG:kafka.metrics.metrics:Added sensor with name queue-time
    DEBUG:kafka.metrics.metrics:Added sensor with name produce-throttle-time
    DEBUG:kafka.metrics.metrics:Added sensor with name records-per-request
    DEBUG:kafka.metrics.metrics:Added sensor with name bytes
    DEBUG:kafka.metrics.metrics:Added sensor with name record-retries
    DEBUG:kafka.metrics.metrics:Added sensor with name errors
    DEBUG:kafka.metrics.metrics:Added sensor with name record-size-max
    DEBUG:kafka.producer.sender:Starting Kafka producer I/O thread.
    DEBUG:kafka.producer.kafka:Kafka producer started
    DEBUG:kafka.producer.kafka:Requesting metadata update for topic enmonitor
    DEBUG:kafka.client:Initializing connection to node 0 for metadata request
    DEBUG:kafka.client:Initiating connection to node 0 at host_1:port
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-sent
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-received
    DEBUG:kafka.metrics.metrics:Added sensor with name node-0.latency
    
    第 2 条附言  ·  2018-08-21 17:06:57 +08:00
    附上一条附言
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 None]>: creating new socket
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
    ERROR:kafka.conn:Connection attempt to <BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]> timed out
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: Closing connection. KafkaConnectionError: timeout
    WARNING:kafka.client:Node 0 connection failed -- refreshing metadata
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnecting> [IPv4 ('host_1', port)]>: reconnect backoff 0.04183829594241324 after 1 failures
    DEBUG:kafka.client:Initializing connection to node 0 for metadata request
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: creating new socket
    DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
    INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
    Traceback (most recent call last):
    kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.
    3 条回复    2018-10-18 21:16:04 +08:00
    nilai
        1
    nilai  
       2018-08-21 12:44:16 +08:00
    advertised.listeners= 这个项有没有配置上??
    yellowmarlboro
        2
    yellowmarlboro  
    OP
       2018-08-21 17:08:52 +08:00
    @nilai 这个配置是在远程服务器 server.properties 配置的是吧.. 远程没配置好的问题么?
    我指定了 api_version 更改了以后就有附言的问题,远程服务器人告诉我是 api 版本问题 可是我模块已经最高了啊并且官网支持现在 kafka 版本的。。
    Pythonerxiaobai
        3
    Pythonerxiaobai  
       2018-10-18 21:16:04 +08:00
    @yellowmarlboro 楼主这个问题解决了么?我也遇到了相同的问题
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   1068 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 23ms · UTC 22:28 · PVG 06:28 · LAX 14:28 · JFK 17:28
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.