flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tech Mail <ynerella...@gmail.com>
Subject Re: Flume NG and zookeeper
Date Fri, 08 Feb 2013 14:44:07 GMT
Hi,

I am looking for similar functionality, but we loose the flexibility.

Cannot add a new agent dynamically to the list of collectors.
Cannot move the agents from one host to other host or from one datacenter to other.
Entire list of agents has to be provided to all end user applications.

How to dynamically get the host/port of the agent from the client application?  
   LDAP server server.  
    Any other ways ?

Yogi

On Feb 7, 2013, at 7:50 PM, 吳瑞琳 <rlwu607@gmail.com> wrote:

> Mike,
> 
> Thanks a lot. It fits our need. I'll try your advice.
> 
> Regards,
> RL
> 
> 2013/2/8 Mike Percy <mpercy@apache.org>
>> Interesting use case. Have you considered using the existing polling-based load balancing
RPC client functionality?
>> 
>> http://flume.apache.org/FlumeDeveloperGuide.html#loadbalancing-rpc-client
>> 
>> Regards,
>> Mike
>> 
>> 
>> On Thu, Feb 7, 2013 at 7:07 PM, 吳瑞琳 <rlwu607@gmail.com> wrote:
>>> Thank you all.
>>> 
>>> I am trying configure multiple hosts running Flume to collect data from clients.
I want to use Zookeeper to track status of Flume hosts so that my clients can get the list
of available Flume host and send data to one of them. I would like to make Flume hosts to
register an ephemeral node under some Zookeeper node when it is up.  
>>> 
>>> Regards,
>>> RL
>>> 
>>> 
>>> 2013/2/8 Mike Percy <mpercy@apache.org>
>>>> Integrate in what way?
>>>> 
>>>> 
>>>> On Thu, Feb 7, 2013 at 6:36 PM, 吳瑞琳 <rlwu607@gmail.com> wrote:
>>>>> Hi all,
>>>>> 
>>>>> I am trying to integrate Flume NG and zookeeper. However, I did not find
any configuration about this in Flume NG. Could you please advise how to deal with this?
>>>>> 
>>>>> Thanks,
>>>>> RL
> 

Mime
View raw message