flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@apache.org>
Subject Re: Is Memory Channel data lost on process stop?
Date Wed, 28 May 2014 21:54:40 GMT
The traditional memory channel does not store anything to disk on process
shutdown, so you lose the data if you kill the process. However, a plain
reconfiguration will not reinitialize or clear the channel as long as the
channel name remains the same.

The recoverable memory channel thing (FLUME-896) was never stable and was
removed some time back.

As far as I know, the spillable memory channel does not today spill to disk
on shutdown, but one could conceive of adding this feature, since it uses
its own internal file channel instance to "spill" when its memory capacity
has been exceeded.


On Wed, May 28, 2014 at 1:44 PM, Christopher Shannon

> Good question; I look forward to someone cogently answering it.
> Related, and you probably know this already, the only signals that invoke
> the JVM shutdown hooks (clean shutdown) are HUP and TERM. (kill -HUP <PID>;
> kill -TERM <PID>).
> On May 28, 2014 3:37 PM, "Otis Gospodnetic" <otis.gospodnetic@gmail.com>
> wrote:
>> Hi,
>> What happens with events buffered in the Memory channel (say via the
>> Spillable version - https://issues.apache.org/jira/browse/FLUME-1227 or
>> via its Durable cousin - https://issues.apache.org/jira/browse/FLUME-896) when the
process is stopped?
>> Say the process is stopped gently and appropriately, via an init.d script
>> or a gentle kill (not kill -9).
>> Is the data that was in memory still lost?
>> Or is there a Shutdown hook that gets run to preserve data on disk?
>> Thanks,
>> Otis
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/

View raw message