flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paweł Róg <pro...@gmail.com>
Subject Re: Flume dynamic configuraiton
Date Sun, 03 Aug 2014 16:51:45 GMT
Hi,
I fetched a patch and see that zookeeper configuration provider doesn't
listen on zookeeper and doesn't emit events when somethings changes in
zookeeper. Did I miss something?

--
Paweł

2014-08-01 13:16 GMT+02:00 Ashish <paliwalashish@gmail.com>:

>
>
>
> On Fri, Aug 1, 2014 at 4:40 PM, Paweł <prog88@gmail.com> wrote:
>
>> Hi,
>> Thanks for information. Please tell me what happens with existing
>> sources, channels and sinks when configuration is reloaded? Are they
>> destroyed and started once again?
>>
>
> During reload Flume shall stop existing ones and the reload the config
> creating Components as specified in changed config.
>
> See org.apache.flume.lifecycle package for Life cycle stuff
> and org.apache.flume.node package for how Agent starts and use
> Configuration.
>
>
>>
>> Zookeeper would be nice here. Can you tell me if there are any estimates
>> for time needed to close the issue or to have any stable version (no needed
>> to be pushed into master)?
>>
>
> Patch is attached, need to be battle tested.
>
>
>>
>> --
>> Paweł
>>
>>
>> 2014-08-01 12:59 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>>
>> If you change the configuration file, Flume reloads it, unless you tell
>>> it not to by specifying no-reload-conf option.
>>>
>>>
>>> On Fri, Aug 1, 2014 at 3:35 PM, Paweł <prog88@gmail.com> wrote:
>>>
>>>> Hi,
>>>> Currently I see no option to configure Flume dynamically.
>>>>
>>>> I have a case where I have to be able to
>>>> * add or remove Source
>>>> * attach Source to a Channel
>>>> * attach a Channel to a Sink or several Sinks
>>>>
>>>> All of this things are possible but only with configuration file which
>>>> means restart of application each time when the anything changes.
>>>>
>>>> 1) Are there any plans to add dynamic configuration mechanisms?
>>>>
>>>
>>> I think it's already supported. When you reconfigure, Flume Agents goes
>>> through its Lifecycle events stop and then restart.
>>>
>>>
>>>> 2) Do you think any REST API or JMX would be good in this place?
>>>>
>>>
>>> There is JIRA for configuration using Zookeeper
>>> https://issues.apache.org/jira/browse/FLUME-1491
>>>
>>>
>>>> 3) How complicated it is to make Flume configurable? Does it men tons
>>>> of coding and months to implement it or rather it is not so hard?
>>>>
>>>
>>>
>>>>
>>>> --
>>>> Paweł
>>>>
>>>
>>>
>>>
>>> --
>>> thanks
>>> ashish
>>>
>>> Blog: http://www.ashishpaliwal.com/blog
>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>>
>>
>>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>

Mime
View raw message