flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Flume NG for enterprise environment
Date Wed, 03 Oct 2012 18:25:51 GMT
Hi,

I would use puppet, chef, scp, or Cloudera Manager to manage the
configuration files of the flume agents. No sense in writing something
custom just to copy files around.

Brock

On Wed, Oct 3, 2012 at 9:39 AM, Kim, Jongkook <jongkook.kim@citi.com> wrote:
> Hi Brock,
>
> Without a central place for agent config, how to update flume.conf files on hundreds
of agents?
>
> We are trying to give user an ability to update agent setup (previously a logical node)
real-time.
>
> About 10 - 30 logical nodes are created per agent and they can be updated or deleted
anytime.
>
> Do you have any architectural design of this?
>
>
> Thanks.
> -----Original Message-----
> From: Brock Noland [mailto:brock@cloudera.com]
> Sent: Wednesday, October 03, 2012 10:31 AM
> To: user@flume.apache.org
> Cc: *DE Global E2E Sponge Stakeholders
> Subject: Re: Flume NG for enterprise environment
>
> Hi,
>
> I can confirm that many organizations are using Flume NG in a production enterprise environment.
You are correct the concept of a master no longer exists.
>
> Brock
>
> On Tue, Oct 2, 2012 at 4:02 PM, Kim, Jongkook <jongkook.kim@citi.com> wrote:
>> Hi,
>>
>>
>>
>> I'm using flume 0.94 in CDH3 environment and thinking to migrate to CDH4.
>>
>>
>>
>> The old flume has a master instance to control agents and collectors,
>> so we developed a configuration UI  to communicate with a flume master.
>>
>>
>>
>> From this UI, user can add, edit, and delete logical nodes.
>>
>>
>>
>> Does anyone use flume NG on enterprise environment?
>>
>>
>>
>> Without the master concept, how do you manage all agents and collectors.
>>
>>
>>
>> If you need to read more files on a machine on flume NG, do you
>> manually update flume.conf file and restart the agent?
>>
>>
>>
>> Thanks.
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message