flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincentius Martin <vincentiusmar...@gmail.com>
Subject Re: All channels in an agent get slower after a channel is full
Date Thu, 13 Nov 2014 20:43:29 GMT
it is between agents, I am using avro sinks and file channels while all of
those channels write the checkpoint to a disk.

For the rest, I am using default configuration.


Regards,
Vincentius Martin

On Fri, Nov 14, 2014 at 1:39 AM, Hari Shreedharan <hshreedharan@cloudera.com
> wrote:

> What does your configuration look like? What sink are you using?
>
> On Thu, Nov 13, 2014 at 8:23 AM, Vincentius Martin <
> vincentiusmartin@gmail.com> wrote:
>
>> Hi,
>>
>> In my cluster, I have an agent with one source connected to multiple
>> channels. Each channel connected to different sink (1 channel paired with 1
>> sink) which send events to different agents (like one to many relation).
>> Just like the multiplexing flow example in Flume user guide website.
>>
>> However, when a channel reaches its capacity (already full)  I see that
>> the agent performance gets slower.
>>
>> What I mean by getting slower is that, all other channel-sink pairs in
>> that agent also get slower when sending events to their destination. I can
>> understand if the overfilled channel-sink pair get slower, but why it
>> affects another channel-sink pairs in that agent? From what I see here, the
>> other pairs should be independent with the overfilled channel except that
>> they use the same source, right?
>>
>> Thanks!
>>
>> Regards,
>> Vincentius Martin
>>
>
>

Mime
View raw message