flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kim, Jongkook " <jongkook....@citi.com>
Subject RE: Flume NG for enterprise environment
Date Wed, 03 Oct 2012 18:33:33 GMT
Brock,

Thanks for your response.

We are using Cloudera Manager Free edition for our POC environment, and it doesn't have a
flume control.

Dose Enterprise version has it?

Thanks.

-----Original Message-----
From: Brock Noland [mailto:brock@cloudera.com] 
Sent: Wednesday, October 03, 2012 2:26 PM
To: Kim, Jongkook [CCC-OT_IT]
Cc: *DE Global E2E Sponge Stakeholders; user@flume.apache.org
Subject: Re: Flume NG for enterprise environment

Hi,

I would use puppet, chef, scp, or Cloudera Manager to manage the configuration files of the
flume agents. No sense in writing something custom just to copy files around.

Brock

On Wed, Oct 3, 2012 at 9:39 AM, Kim, Jongkook <jongkook.kim@citi.com> wrote:
> Hi Brock,
>
> Without a central place for agent config, how to update flume.conf files on hundreds
of agents?
>
> We are trying to give user an ability to update agent setup (previously a logical node)
real-time.
>
> About 10 - 30 logical nodes are created per agent and they can be updated or deleted
anytime.
>
> Do you have any architectural design of this?
>
>
> Thanks.
> -----Original Message-----
> From: Brock Noland [mailto:brock@cloudera.com]
> Sent: Wednesday, October 03, 2012 10:31 AM
> To: user@flume.apache.org
> Cc: *DE Global E2E Sponge Stakeholders
> Subject: Re: Flume NG for enterprise environment
>
> Hi,
>
> I can confirm that many organizations are using Flume NG in a production enterprise environment.
You are correct the concept of a master no longer exists.
>
> Brock
>
> On Tue, Oct 2, 2012 at 4:02 PM, Kim, Jongkook <jongkook.kim@citi.com> wrote:
>> Hi,
>>
>>
>>
>> I'm using flume 0.94 in CDH3 environment and thinking to migrate to CDH4.
>>
>>
>>
>> The old flume has a master instance to control agents and collectors, 
>> so we developed a configuration UI  to communicate with a flume master.
>>
>>
>>
>> From this UI, user can add, edit, and delete logical nodes.
>>
>>
>>
>> Does anyone use flume NG on enterprise environment?
>>
>>
>>
>> Without the master concept, how do you manage all agents and collectors.
>>
>>
>>
>> If you need to read more files on a machine on flume NG, do you 
>> manually update flume.conf file and restart the agent?
>>
>>
>>
>> Thanks.
>>
>>
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - 
> http://incubator.apache.org/mrunit/



--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message