flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Vila <av...@devlogic.eu>
Subject Re: Flume restart
Date Thu, 04 Dec 2014 21:40:49 GMT
Zdravo Bojane :)

Flume is well designed and it shouldn't eat up the memory. On the other
hand, miss-configuration can effectively bring server to a crawl and
eventually produce events loss.
Pasting your configuration in here, along with basic hardware info behind
it and a size of your single event in bytes would be helpful.

The most common things to blame is using inappropriately large memory
channel size for a given amount of memory, transaction size, HDFS sink
batch size etc. because all of them are stored in memory.

Anyway, you can achieve graceful restart by changing flume's configuration
file modification time - basically just touching it.
It will sense a change and as a result close sources, sinks and channels,
and start them again without overhead of booting up JVM.
That should trigger java garbage collector to clean up resources associated
to those closed instances of sources, sinks and channels.
As a result, you might loose some events if you're using memory channel
since I think it doesn't have a shutdown procedure. Sink should flush batch
to the HDFS, but that also should be tested.

Also, tweaking Java GC could be of a help, but I never had a need to do so
with Flume.


On Thu, Dec 4, 2014 at 9:18 PM, Bojan Kostić <blood9raven@gmail.com> wrote:

> Hi,
>
> I have a problem with my flume setup. Overtime they just take too much
> memory. And i need them to restart every now and then, I searched web and i
> did not found any clue how to fix this. Some people blame HDFS...
> For now i just kill process with TERM signal and then wait for couple of
> minutes to shutdown. Now i wish to do this automatically every day. But i
> don't want to lose logs. Is there a way to do this? I checked flume-ng
> script and there is only start. I could write my own sh script which will
> send TERM signal and then check for flume process and if there is none
> start again. But first i want to check is there some smarter way to do this.
>
> Best
> Bojan
>



-- 

Best regards,
Ahmed Vila | Senior software developer
DevLogic | Sarajevo | Bosnia and Herzegovina

Office : +387 33 942 123
Mobile: +387 62 139 348

Website: www.devlogic.eu
E-mail   : avila@devlogic.eu
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. This email contains confidential information. It should
not be copied, disclosed to, retained or used by, any party other than the
intended recipient. Any unauthorised distribution, dissemination or copying
of this E-mail or its attachments, and/or any use of any information
contained in them, is strictly prohibited and may be illegal. If you are
not an intended recipient then please promptly delete this e-mail and any
attachment and all copies and inform the sender directly via email. Any
emails that you send to us may be monitored by systems or persons other
than the named communicant for the purposes of ascertaining whether the
communication complies with the law and company policies.

-- 
---------------------------------------------------------------------
This e-mail and any attachment is for authorised use by the intended 
recipient(s) only. This email contains confidential information. It should 
not be copied, disclosed to, retained or used by, any party other than the 
intended recipient. Any unauthorised distribution, dissemination or copying 
of this E-mail or its attachments, and/or any use of any information 
contained in them, is strictly prohibited and may be illegal. If you are 
not an intended recipient then please promptly delete this e-mail and any 
attachment and all copies and inform the sender directly via email. Any 
emails that you send to us may be monitored by systems or persons other 
than the named communicant for the purposes of ascertaining whether the 
communication complies with the law and company policies.

Mime
View raw message