flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: how to print the channel capacity
Date Wed, 15 May 2013 08:46:53 GMT
sorry pressed enter too soon

as for your question: how many events a flume agent can hold?
sorry but I don't think there is any direct answer to that.... .I may be
very well wrong there as I am myself pretty new with flume

there was a JIRA for the capacity of file channels FLUME-1571


On Wed, May 15, 2013 at 1:50 PM, Nitin Pawar <nitinpawar432@gmail.com>wrote:

> for maximum performance on your data flow two things which will matter
> most are: the channel and the transaction batch size.
> when you say losing data, are you using memory channel? or file channel?
>
> Flume can batch events. The batch size is the maximum number of events
> that a sink or client will attempt to take from a channel in a single
> transaction.
>
>  What is the channel type
> do you have a slow sink so the # events written out are less than # event
> incoming to channels so over time it piles up
>
> others may point out more things.
> Also your flume conf and if you are seeing any errors on flume then that
> will help people to find out the problem
>
>
> On Wed, May 15, 2013 at 11:07 AM, liuyongbo <liuyongbo@baidu.com> wrote:
>
>>  Hi:****
>>
>>          I’m using flume to pass log data to mongodb, but I find that
>> some data lose when the pressure is in high level, so I want to know the
>> max request that flume can hold and need to print the capacity.but I can
>> not find the proper way to do this instead of change the source code. Any
>> ideas?****
>>
>>          thanks****
>>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Mime
View raw message