flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alten-Lorenz <wget.n...@gmail.com>
Subject Re: How to make sure flume WAL is finished before shut it down
Date Thu, 06 Dec 2012 07:18:01 GMT
moving back to user@


You can monitor the directories (usually /tmp/flume*) for not delivered events, or using the
reports from the webUI.

Since FlumeOG (0.9x) will not actively developed anymore, we strongly suggest to move into
Flume 1.3.x and up. More informations you'll find in our wikie:


On Dec 5, 2012, at 11:07 AM, Shengjie Min <kelvin.msj@gmail.com> wrote:

> I've asked this question in the flume user group, didn't quite get the
> answer.
> we are running flume 0.9.4, We want to make sure flume's WAL is completed
> before we shut down the flume process. Is there anyway or any metrics
> exposed can indicate WAL is fully processed so that it's safe to bring it
> down gracefully? How do I know flume has finished all the data?
> In our case, we are switching to another input approach, so flume will not
> be getting any inputs, it only has a certain size backlog needs to be
> cleaned up. So it's more like making sure all the data is delivered before
> gracefully shutdown this flume agent forever.
> -- 
> All the best,
> Shengjie Min

Alexander Alten-Lorenz
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

View raw message