flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Flume dynamic configuraiton
Date Mon, 04 Aug 2014 07:22:22 GMT
Are you talking about FLUME-1491 patch 7 or some other patch? If yes, then
it does listen for change notifications from Zookeeper, via Curator's
NodeCache.



On Sun, Aug 3, 2014 at 10:21 PM, Paweł Róg <prog88@gmail.com> wrote:

> Hi,
> I fetched a patch and see that zookeeper configuration provider doesn't
> listen on zookeeper and doesn't emit events when somethings changes in
> zookeeper. Did I miss something?
>
> --
> Paweł
>
> 2014-08-01 13:16 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>
>
>>
>>
>> On Fri, Aug 1, 2014 at 4:40 PM, Paweł <prog88@gmail.com> wrote:
>>
>>> Hi,
>>> Thanks for information. Please tell me what happens with existing
>>> sources, channels and sinks when configuration is reloaded? Are they
>>> destroyed and started once again?
>>>
>>
>> During reload Flume shall stop existing ones and the reload the config
>> creating Components as specified in changed config.
>>
>> See org.apache.flume.lifecycle package for Life cycle stuff
>> and org.apache.flume.node package for how Agent starts and use
>> Configuration.
>>
>>
>>>
>>> Zookeeper would be nice here. Can you tell me if there are any estimates
>>> for time needed to close the issue or to have any stable version (no needed
>>> to be pushed into master)?
>>>
>>
>> Patch is attached, need to be battle tested.
>>
>>
>>>
>>> --
>>> Paweł
>>>
>>>
>>> 2014-08-01 12:59 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>>>
>>> If you change the configuration file, Flume reloads it, unless you tell
>>>> it not to by specifying no-reload-conf option.
>>>>
>>>>
>>>> On Fri, Aug 1, 2014 at 3:35 PM, Paweł <prog88@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>> Currently I see no option to configure Flume dynamically.
>>>>>
>>>>> I have a case where I have to be able to
>>>>> * add or remove Source
>>>>> * attach Source to a Channel
>>>>> * attach a Channel to a Sink or several Sinks
>>>>>
>>>>> All of this things are possible but only with configuration file which
>>>>> means restart of application each time when the anything changes.
>>>>>
>>>>> 1) Are there any plans to add dynamic configuration mechanisms?
>>>>>
>>>>
>>>> I think it's already supported. When you reconfigure, Flume Agents goes
>>>> through its Lifecycle events stop and then restart.
>>>>
>>>>
>>>>> 2) Do you think any REST API or JMX would be good in this place?
>>>>>
>>>>
>>>> There is JIRA for configuration using Zookeeper
>>>> https://issues.apache.org/jira/browse/FLUME-1491
>>>>
>>>>
>>>>> 3) How complicated it is to make Flume configurable? Does it men tons
>>>>> of coding and months to implement it or rather it is not so hard?
>>>>>
>>>>
>>>>
>>>>>
>>>>> --
>>>>> Paweł
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> thanks
>>>> ashish
>>>>
>>>> Blog: http://www.ashishpaliwal.com/blog
>>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>>>
>>>
>>>
>>
>>
>> --
>> thanks
>> ashish
>>
>> Blog: http://www.ashishpaliwal.com/blog
>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>
>
>


-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message