Kafka consumers logging "Auto offset commit failed" every few minutes -
i using kafka 0.9 "new" consumer api connect kafka 0.9.0.0. every couple of minutes, following error logged:
2016-01-27t16:38:14,584z [pool-1-thread-1] error o.a.k.c.c.i.consumercoordinator - auto offset commit failed. org.apache.kafka.clients.consumer.internals.sendfailedexception: null
here gist of broker , consumer configs.
there no corresponding/concerning log messages in broker logs, , records appear flowing through system fine. message annoyance, causing distracting alerts , noise.
i'm curious know exception means, real questions are:
- should concerned exception?
- if should not concerned, there way suppress tweaking consumer/broker configs?
- if answer 2 questions above "no", rule suppressing these messages (i can logback) don't miss legitimate
sendfailedexception
log messages
i have same error message in different place (heron , kafkaspout)! finding following:
if use enable.auto.commit = true
must not use consumer.commitasync();
Comments
Post a Comment