flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 周梦想 <abloz...@gmail.com>
Subject how to know which line the agent have sent ?
Date Wed, 06 Feb 2013 02:45:14 GMT

I'm using tailDirs('mydir') source of the agent to gather logs to hadoop
hdfs. I notice some documents advise that if the agent collapsed,  I have
to remove files in 'mydir' and clear flume.agent.logdir. Thus I will lose
some data or have duplicate data. And I don't know which line the agent
have sent to.

I'm worrying about the agent failure and resend or miss-send the content to
collector. I want to know how to check which line of log file the agent
have sent if the agent exit suddenly. The files in flute log dir, such as
sending,sent can't be read.

Please give some advise to process such situation.

Andy Zhou

View raw message