flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <cshannon...@gmail.com>
Subject Re: Flume Configuration & topology approach
Date Mon, 07 Apr 2014 15:38:24 GMT
On Apr 7, 2014 9:35 AM, "Jeff Lord" <jlord@cloudera.com> wrote:
> On Thu, Apr 3, 2014 at 11:27 AM, Mohit Durgapal <durgapalmohit@gmail.com>
>> Hi Jeff,
>> Yes, I am using the memory channel, and that's because I want it to be
more reliable and not miss any events/messages.
>> As I've read in flume documentation that the memory channel is fast but
there could be a chance of missing events if the in-memory buffer fills up.
> Memory channel is not reliable, meaning if the flume agent goes down or
is restarted while there are events in the channel than this data will be
> For reliability please use the file channel.


I am using an upstream agent with a spooling directory source and a memory
channel, and the downstream agent uses a memory channel and an HDFS sink.
If my downstream agent goes down for any reason, are the entries lost in
the downstream agent's memory channel still preserved in the memory channel
/ file directory of the upstream agent?

All the best, Chris

View raw message