flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan" <hshreedha...@cloudera.com>
Subject Re: ElasticSearch workers thread number?
Date Wed, 15 Apr 2015 20:35:43 GMT
You can add multiple ES sinks to read from the same channel to parallelize writes. 

Thanks, Hari

On Wed, Apr 15, 2015 at 1:17 PM, Marina <ppine7@yahoo.com> wrote:

> One more question on Flume ElasticSearchSink. I see that for HDFS sink, you can control
the number of worker threads using this parameter: hdfs.threadsPoolSize. However, I do not
see any such parameter for ElasticSearchSink.... Does it mean that ElasticSsearchSink will
span only one thread to push events form Flume to ES? If I have rates of a few hundred events
per second (say, up to 3K) - this will become a performance bottleneck.
> thanks!Marina
View raw message