flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay Srivastava <Ajay.Srivast...@guavus.com>
Subject Re: Installing flume in distributed mode and HA
Date Mon, 18 Jan 2016 09:39:57 GMT
Thanks Gonzalo for quick reply.

By load balancer, do you mean load balancing group of flume agents ?
If yes, I do need to take care of HA for sources, channels and sinks too. Am I correct ?


On 18-Jan-2016, at 2:58 pm, Gonzalo Herreros <gherreros@gmail.com<mailto:gherreros@gmail.com>>

That Cloudera documentation is ancient and talks about the old Flume (that's why it differs
from what you see in the Apache website), the modern Flume (also called Flume-ng) doesn't
not have a master, to have HA you need several agents with the same configuration and a load
balancer in front.
In some cases specific cases you might need need to build a hierarchy and even add Kafka channels
you have true fail tolerances.


On 18 January 2016 at 09:21, Ajay Srivastava <Ajay.Srivastava@guavus.com<mailto:Ajay.Srivastava@guavus.com>>

I am new to flume and am looking for the documentation to configure flume in distributed mode.
There was no section present for this in user guide.
I came across this page on cloudera http://archive.cloudera.com/cdh/3/flume/UserGuide/ which
describes steps to configure flume in fully distributed mode. Since I could not find these
instructions on apache website, I am wondering if CDH has its own version of flume. Do these
configuration steps work well for apache flume as well ?


View raw message