flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Flume flume.master.savefile
Date Thu, 28 Jul 2011 09:29:13 GMT
Eric,

I think we need to keep it the 0.9.x series -- there was an incompatible
change made in the past and this was used in part of the recovery.  I do
think it should be removed if we make a new branch, or at least changed to
be a complete and explicit master backup and restore mechanism.

Jon.

On Mon, Jul 25, 2011 at 7:19 PM, Eric Sammer <esammer@cloudera.com> wrote:

> Jon:
>
> As a side note (and open to the community), do you think it makes
> sense to deprecate this? I saw it today and was surprised it's still
> around.
>
> On Mon, Jul 25, 2011 at 3:24 PM, Jonathan Hsieh <jon@cloudera.com> wrote:
> > [Please subscribe to new flume-user@incubator.apache.org list,
> > bcc flume-user@cloudera.org, cc flume-user@incubator.apache.org]
> > Prashanth,
> > Which mode is your master using?  If you are using any ZK mode the
> configs
> > should be persistent, and survive master restarts.
> > The flume.master.savefile is an old setting that doesn't support all the
> > things that the master now keeps track of.  It was used before the ZK
> > backend was added to the master.  It is still usable for basic
> > configrutions, and I believe only "auto-loads" when using memory backed
> > flume master (but I'd need to check -- its been a long time).
> > You could take the file that is created and execute a "exec load
> <filename>"
> > in the shell to read and load the configs from the file.
> > Jon.
> > On Tue, Jul 19, 2011 at 4:42 PM, Prashanth R <r.prashanth@gmail.com>
> wrote:
> >>
> >> A bit of a context:
> >>
> >> Right now, if the flume master is restarted, it loses all the
> >> configurations  / mappings that are set. I would ideally like it to
> >> pick these configurations from a file when it initializes. Is there a
> >> good way of doing this?
> >>
> >> - Prashanth
> >>
> >> On Mon, Jul 18, 2011 at 5:59 PM, Prashanth R <r.prashanth@gmail.com>
> >> wrote:
> >> > Does anyone know if the property 'flume.master.savefile' in
> >> > flume-site.xml is valid and can be safely used to initalize node
> >> > configurations  after a startup ? if so, the format of the commands
> >> > inside the file.
> >> >
> >> >
> >> >
> >> > --
> >> > - Prash
> >> >
> >>
> >>
> >>
> >> --
> >> - Prash
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // Software Engineer, Cloudera
> > // jon@cloudera.com
> >
> >
>
>
>
> --
> Eric Sammer
> twitter: esammer
> data: www.cloudera.com
>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
View raw message