On Fri, Apr 17, 2015 at 9:05 AM, Tao Li <firstname.lastname@example.org> wrote:
Hi all:My use case is KafkaChannel + HDFSEventSink.I found that SinkRunner.PollingRunner will call HDFSEventSink.process() in a while loop. For example, a message in kafka contains dirty data, so HDFSEventSink.process() consume message from kafka, throws exception because of dirty data, and kafka offset doesn't commit. And the outer loop, will continue call HDFSEventSink.process(). Because the kafka offset doesn't change, so HDFSEventSink will consume the dirty data again. The bad loop is never stopped.I want to know that if we have a mechanism to cover this case? For example, we have a max retry num for a unique HDFSEventSink.process() call and give up when exceed max limit.