flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: Installing flume in distributed mode and HA
Date Mon, 18 Jan 2016 09:49:54 GMT
I mean for instance if you use an http source then have a HW or SW load
balancer so it can call any of the agents.
Yes, you have to worry about HA at source, channel and sink. Depending on
the use case, it might be enough to have it at the source.

On 18 January 2016 at 09:39, Ajay Srivastava <Ajay.Srivastava@guavus.com>
wrote:

> Thanks Gonzalo for quick reply.
>
> By load balancer, do you mean load balancing group of flume agents ?
> If yes, I do need to take care of HA for sources, channels and sinks too.
> Am I correct ?
>
>
>
> Regards,
> Ajay
>
>
> On 18-Jan-2016, at 2:58 pm, Gonzalo Herreros <gherreros@gmail.com> wrote:
>
> That Cloudera documentation is ancient and talks about the old Flume
> (that's why it differs from what you see in the Apache website), the modern
> Flume (also called Flume-ng) doesn't not have a master, to have HA you need
> several agents with the same configuration and a load balancer in front.
> In some cases specific cases you might need need to build a hierarchy and
> even add Kafka channels you have true fail tolerances.
>
> Regards,
> Gonzalo
>
> On 18 January 2016 at 09:21, Ajay Srivastava <Ajay.Srivastava@guavus.com>
> wrote:
>
>> Hi,
>>
>> I am new to flume and am looking for the documentation to configure flume
>> in distributed mode. There was no section present for this in user guide.
>> I came across this page on cloudera
>> http://archive.cloudera.com/cdh/3/flume/UserGuide/ which describes steps
>> to configure flume in fully distributed mode. Since I could not find these
>> instructions on apache website, I am wondering if CDH has its own version
>> of flume. Do these configuration steps work well for apache flume as well ?
>>
>>
>>
>> Regards,
>> Ajay
>>
>
>
>

Mime
View raw message