flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan" <hshreedha...@cloudera.com>
Subject Re: All channels in an agent get slower after a channel is full
Date Thu, 13 Nov 2014 20:47:59 GMT
Are all the channels sharing the same disk(s)? 


On Thu, Nov 13, 2014 at 12:44 PM, Vincentius Martin
<vincentiusmartin@gmail.com> wrote:

> it is between agents, I am using avro sinks and file channels while all of
> those channels write the checkpoint to a disk.
> For the rest, I am using default configuration.
> Regards,
> Vincentius Martin
> On Fri, Nov 14, 2014 at 1:39 AM, Hari Shreedharan <hshreedharan@cloudera.com
>> wrote:
>> What does your configuration look like? What sink are you using?
>> On Thu, Nov 13, 2014 at 8:23 AM, Vincentius Martin <
>> vincentiusmartin@gmail.com> wrote:
>>> Hi,
>>> In my cluster, I have an agent with one source connected to multiple
>>> channels. Each channel connected to different sink (1 channel paired with 1
>>> sink) which send events to different agents (like one to many relation).
>>> Just like the multiplexing flow example in Flume user guide website.
>>> However, when a channel reaches its capacity (already full)  I see that
>>> the agent performance gets slower.
>>> What I mean by getting slower is that, all other channel-sink pairs in
>>> that agent also get slower when sending events to their destination. I can
>>> understand if the overfilled channel-sink pair get slower, but why it
>>> affects another channel-sink pairs in that agent? From what I see here, the
>>> other pairs should be independent with the overfilled channel except that
>>> they use the same source, right?
>>> Thanks!
>>> Regards,
>>> Vincentius Martin
View raw message