flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <esam...@cloudera.com>
Subject Re: Log4J appender
Date Wed, 30 Nov 2011 18:28:41 GMT

On Wed, Nov 30, 2011 at 12:23 AM, Srinivasan Subramanian <
ssrini_vasan@hotmail.com> wrote:

>  I was evaluating the log4j appender provided with Flume.  But there is
> one aspect I dont understand:
> The log4j appender makes a connection to teh flume-agent and retries a
> maximum of 10 times (default - configurable) if the connection is not made
> successfully.
> Questions:
> 1. When will the connection fail?  If the agent is not running on the
> node?  In that case given that the default implementation waits for 1
> second before each retry for a total of 10 retries, would this mean that
> each logging call from the application would be delayed by 10 seconds?
>  That would affect performance right?

Almost certainly, yes, assuming log4j is synchronous (I'm 99.9% sure it
is). Of course, synchronous logging is the only way to guarantee event
delivery in this context; if the application were to log the event and move
on without waiting for a response an event could get dropped and no one
would be responsible for retrying the send.

> 2. What happens to the log message when the agent is not available?  Is it
> lost?

If the log4j appender runs out of retries I believe I wrote it to throw an
exception. This would be the equivalent of using a standard file appender
and running out of disk space. In other words, the log call failed and
should be handled by the application.

Let me know if you have any other questions!

> I am a little confused with the implementation and any help in explaining
> this is appreciated.
> Regards
> Srini

Eric Sammer
twitter: esammer
data: www.cloudera.com

View raw message