flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Estimating the event loss
Date Fri, 17 Jul 2015 18:20:29 GMT
Note that if you simply reconfigure (rather than kill the JVM and restart)
you would not lose data in the memory channel


On Fri, Jul 17, 2015 at 7:08 AM, Johny Rufus <jrufus@cloudera.com> wrote:

> When you restart the agent, all the events in the memory channel will be
> lost (if the memory channel was full and contained 100k events, they will
> be lost)
> File channel will not result in loss of events, on restart the channel's
> state will be restored (File channel always maintains the last two data
> files the events were written to, even if the events in them were
> completely drained, this may give the impression that it still has data
> left)
> Thanks,
> Rufus
> On Thu, Jul 16, 2015 at 11:10 PM, Buntu Dev <buntudev@gmail.com> wrote:
>> I'm using a Memory channel with capacity set to 100000. Does this mean
>> when the flume agent restarts, its possible that I loose about 100k events?
>> For any other durable channel say File channel I noticed .tmp files
>> created and written to but when I restart the agent these .tmp files are
>> left as-is. Since some events seem to be written to the .tmp files, should
>> I expect loss of events in this scenario?
>> Thanks!

View raw message