flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Senthilvel Rangaswamy <senthil...@gmail.com>
Subject Re: Restarts without data loss
Date Mon, 09 Jul 2012 07:12:38 GMT
Hari,

Would you have some numbers around the performance of FileChannel ? Like,
how many events you
were able to process etc.,

--
..Senthil

On Mon, Jul 9, 2012 at 12:07 AM, Hari Shreedharan <hshreedharan@cloudera.com
> wrote:

> Senthil,
>
> Have you tried using it recently, with multiple data folders etc. In
> recent tests, we have seen reasonably good performance. Of course, the
> performance of MemoryChannel would be much better, since it is in-memory
> :-). You should try to use the FileChannel as much as you can, else there
> is a risk of losing data.
>
> Thanks
> Hari
>
> --
> Hari Shreedharan
>
> On Monday, July 9, 2012 at 12:01 AM, Senthilvel Rangaswamy wrote:
>
> We do use persistent channel when there is overflow. Using FileChannel for
> regular operations
> is too slow for us.
>
> On Sun, Jul 8, 2012 at 11:58 PM, Brock Noland <brock@cloudera.com> wrote:
>
>  I am guessing you are aware, but you could use a persistent channel such
> as file channel.
>
> --
> Brock Noland
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On Monday, July 9, 2012 at 7:18 AM, Senthilvel Rangaswamy wrote:
>
> We are using Flume 1.2.0 with memory channel. When we rollout new
> configs/decorators
> we may need to restart flume at which point any events in memory channel
> is gone. Any
> ways to avoid this ?
>
> Thanks,
> --
> ..Senthil
>
> "If there's anything more important than my ego around, I want it
>  caught and shot now."
>                                                     - Douglas Adams.
>
>
>
>
>
> --
> ..Senthil
>
> "If there's anything more important than my ego around, I want it
>  caught and shot now."
>                                                     - Douglas Adams.
>
>
>


-- 
..Senthil

"If there's anything more important than my ego around, I want it
 caught and shot now."
                                                    - Douglas Adams.

Mime
View raw message