flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arvind Prabhakar <arv...@apache.org>
Subject Re: Flume-OG Chokes alternatives in Flume-NG to control data transfer limit
Date Wed, 14 May 2014 07:36:13 GMT
At the moment, the only way to control the drain rate will be by the number
of active sinks. Within a sink though, there is no control the drain rate.
Having a drain rate throttle within the sink would be an overkill and may
not add much value since the sink itself is bound by batch size and single
thread transfer semantics.

Regards,
Arvind Prabhakar


On Thu, May 8, 2014 at 5:47 AM, pravesh suyal <pravesh.suyal@gmail.com>wrote:

> Thanx Arvind.
>
> Is the steady rate configurable through the flume configurations?
> Otherwise, that would mean we need to adjust through adjusting the total
> number of Sinks? In case I just need a single Source/Sink pairing and still
> would want to control the drain rate?
>
> Regards
> Pravesh
>
>
> On Tue, May 6, 2014 at 9:44 PM, Arvind Prabhakar <arvind@apache.org>wrote:
>
>> This is indirectly available in Flume (NG) via the number of sinks you
>> setup. Each sink operates on a single thread that drains the channel at a
>> steady rate.  If you want to have a lower drain rate, reduce the number of
>> sinks; and conversely increase them for higher drain rate.
>>
>> Regards,
>> Arvind Prabhakar
>>
>>
>> On Tue, May 6, 2014 at 12:05 AM, pravesh suyal <pravesh.suyal@gmail.com>wrote:
>>
>>>
>>> Dear Flume community,
>>>
>>> Is there is any alternative to the Flume-OG Chokes in the Flume-NG?
>>>
>>> Chokes served the purpose well to control the data transfer rate limit
>>> across the Sources-to-Sinks.
>>>
>>> Is this been dropped from Flume-NG purposely? OR there is an alternative?
>>>
>>> Regards
>>> --
>>> Pravesh Suyal
>>>
>>
>>
>
>
> --
> Pravesh Suyal
>

Mime
View raw message