flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Shannon <cshannon...@gmail.com>
Subject Re: fileChannel always reached it's capacity
Date Fri, 21 Mar 2014 12:39:45 GMT
You can add more sinks to the channel to work in parallel until the channel
stays drained. Just make sure you assign a unique prefix to each sink's
file name.
On Mar 21, 2014 4:03 AM, "Bean Edwards" <edwardsbean@gmail.com> wrote:

> I use hdfsSink and fileChannel, it seems hdfsSink consume too slow.here
> is my configuration:
> ...
> agent1.channels.thrift_ch2.capacity = 200000000
> agent1.channels.thrift_ch2.transactionCapacity = 100000
> ...
> agent1.sinks.hdfsSink.hdfs.batchSize = 100000
> agent1.sinks.hdfsSink.hdfs.threadsPoolSize = 40
>
> very appreciate for guiding !!!
>

Mime
View raw message