flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Handling channel full exception
Date Tue, 20 Jan 2015 14:14:37 GMT
Not sure if I can answer this to your satisfaction, but let me try.
The reason why it logs as warning is, the HDFS can go down even while
the Agent is running and Flume shall keep on retrying to establish the
connection. So if you configuration is correct and HDFS is running, it
would establish the connection.

Second part, even if the Agent is running with connection established
you can still get channel full exception. This would occur if the
event push rate to channel is higher than event drain rate by HDFS

What do you plan to do by finding the connection state?

Hope it help!

On Mon, Jan 19, 2015 at 10:44 AM, mahendran m <mahendranec@hotmail.com> wrote:
> Hi ,
> I have started the flume agent with spooling directory source and HDFS sink.
> how can i know whether connection to HDFS is succeed or not . i have
> configured log4j.properties to print only error message . when connection to
> HDFS failed log4j through this exception as WARN so i cannot get this
> exception when set log4j threshold as ERROR.at some point it lead to channel
> full exception
> How can i find whether connection to HDFS is succeed or not  ?
> Thanks.


Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

View raw message