flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Restarts without data loss
Date Mon, 09 Jul 2012 07:21:14 GMT
Unfortunately I don't have any specific numbers that I can share. It has been tested for extended
periods of time, and has proved durable and stable. Hopefully, someone will be able to run
perf tests on the channel soon.

Thanks
Hari

-- 
Hari Shreedharan


On Monday, July 9, 2012 at 12:12 AM, Senthilvel Rangaswamy wrote:

> Hari,
> 
> Would you have some numbers around the performance of FileChannel ? Like, how many events
you
> were able to process etc.,
> 
> --
> ..Senthil
> 
> On Mon, Jul 9, 2012 at 12:07 AM, Hari Shreedharan <hshreedharan@cloudera.com (mailto:hshreedharan@cloudera.com)>
wrote:
> > Senthil, 
> > 
> > Have you tried using it recently, with multiple data folders etc. In recent tests,
we have seen reasonably good performance. Of course, the performance of MemoryChannel would
be much better, since it is in-memory :-). You should try to use the FileChannel as much as
you can, else there is a risk of losing data. 
> > 
> > Thanks
> > Hari
> > 
> > -- 
> > Hari Shreedharan
> > 
> > 
> > On Monday, July 9, 2012 at 12:01 AM, Senthilvel Rangaswamy wrote:
> > 
> > > We do use persistent channel when there is overflow. Using FileChannel for
regular operations
> > > is too slow for us.
> > > 
> > > On Sun, Jul 8, 2012 at 11:58 PM, Brock Noland <brock@cloudera.com (mailto:brock@cloudera.com)>
wrote:
> > > > I am guessing you are aware, but you could use a persistent channel such
as file channel.  
> > > > 
> > > > -- 
> > > > Brock Noland
> > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> > > > 
> > > > 
> > > > On Monday, July 9, 2012 at 7:18 AM, Senthilvel Rangaswamy wrote:
> > > > 
> > > > > We are using Flume 1.2.0 with memory channel. When we rollout new
configs/decorators
> > > > > we may need to restart flume at which point any events in memory
channel is gone. Any
> > > > > ways to avoid this ?
> > > > > 
> > > > > Thanks,
> > > > > -- 
> > > > > ..Senthil
> > > > > 
> > > > > "If there's anything more important than my ego around, I want it

> > > > >  caught and shot now."
> > > > >                                                     - Douglas Adams.
> > > > > 
> > > > 
> > > 
> > > 
> > > 
> > > -- 
> > > ..Senthil
> > > 
> > > "If there's anything more important than my ego around, I want it 
> > >  caught and shot now."
> > >                                                     - Douglas Adams.
> > > 
> > 
> 
> 
> 
> -- 
> ..Senthil
> 
> "If there's anything more important than my ego around, I want it 
>  caught and shot now."
>                                                     - Douglas Adams.
> 


Mime
View raw message