flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wenxing zheng <wenxing.zh...@gmail.com>
Subject Can HDFS sink support exactly once delivery to avoid the duplication of data
Date Thu, 23 Nov 2017 03:07:17 GMT
Dear experts,

When using the HDFS sinks with the KafkaChannel, we found that the data
might be duplicated due to the writing timeout.

Can the HDFS sink support the writing of the flume events in Kafka in
exactly once?

Appreciated for any advice.
Regards, Wenxing

View raw message