flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Connor Woodson <cwoodson....@gmail.com>
Subject Re: Parameters in Configuration File
Date Sat, 16 Mar 2013 00:07:26 GMT
There is some work on modifying the configuration system to make Flume work
with Zookeeper. I believe the way that's being done is by allowing for some
form of pluggable configuration system - you choose between the default
properties file or from something else. I'll look for the JIRA later; but
if you want, it should be possible to make a configuration mechanism to use
Spring instead of the properties file (I don't know enough about Spring to
say more about how it could be done).

Essentially, it should be possible to do, if you or someone else wants to
invest the time in making it work.

- Connor

On Fri, Mar 15, 2013 at 3:07 PM, Kenison, Matt <Matt.Kenison@disney.com>wrote:

> I understand you have to continue to support existing clients. However, I
> don't know that total backwards compatibility would be desireable in the
> long run. If you want to support both the existing property configuration
> and a new one, one way would be to expose setters for each attribute of
> selectors, sinks, etc. This would let them be set directly, and allow the
> configuration to be moved to a separate factory class which would be used
> by the existing property files.
> I have not dealt in detail with the configuration, though, so I'm not sure
> if there's something I'm overlooking.
> From: Mike Percy <mpercy@apache.org>
> Reply-To: "user@flume.apache.org" <user@flume.apache.org>
> Date: Fri, 15 Mar 2013 11:37:42 -0700
> To: "user@flume.apache.org" <user@flume.apache.org>
> Subject: Re: Parameters in Configuration File
> Hi Matt,
> Properties files were the simple way to get started, and now that Flume is
> on version 1.3.1 we can't just abandon the properties file format.
> I'm open to suggestions for adopting more full featured configuration
> functionality while maintaining backwards compatibility with the existing
> config files.
> Regards,
> Mike
> On Fri, Mar 15, 2013 at 10:55 AM, Kenison, Matt <Matt.Kenison@disney.com>wrote:
>> I'm sure this discussion has happened already, but is there a reason
>> Spring (or something similar) isn't used for configuration, rather than the
>> property files? It seems like there is a lot of boilerplate logic in the
>> various implementations of Configurable.configure(Context) that is just
>> completely unnecessary. In addition, you wouldn't have to spend time
>> building something like property substitution that's been done many times
>> before.
>> From: Mike Percy <mpercy@apache.org>
>> Reply-To: "user@flume.apache.org" <user@flume.apache.org>
>> Date: Fri, 15 Mar 2013 10:27:00 -0700
>> To: "user@flume.apache.org" <user@flume.apache.org>
>> Subject: Re: Parameters in Configuration File
>> Hey Connor,
>> Take a look at the discussion @
>> https://issues.apache.org/jira/browse/FLUME-1941
>> I you want to help work on this you are more than welcome to :)
>> Regards,
>> Mike
>> On Thu, Mar 14, 2013 at 5:09 PM, Connor Woodson <cwoodson.dev@gmail.com>wrote:
>>> Does the Flume configuration file support parameters, for instance in
>>> the way that you can specify them in a log4j conf file?
>>> - Connor

View raw message