flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Stricks <...@wapolabs.com>
Subject Re: Strange Logging Behavior with v094
Date Fri, 09 Mar 2012 03:07:38 GMT
We have two logical nodes on the agent, and it turns out that one of the
log files is being written to exclusively by one of the node processes, and
the other log file is being written to by the other node process. The
master had log rotation on it still, the embarrassing truth.

On Tue, Mar 6, 2012 at 9:32 PM, Jay Stricks <jay@wapolabs.com> wrote:

> I use the flume-daemon.sh script to initialize my agents. In that script,
> I define the FLUME_LOG_DIR and FLUME_LOGFILE environment variables so that
> my log file ends up being in:
> /var/log/flume/flume-flume-node-ip-10-114-81-9.log.
> Daily, a new log file with the day prior's date extension is created (e.g,
> flume-flume-node-ip-10-114-81-9.log.2012-03-06). The original log file
> without the date extension is overwritten. And both files then have  a
> first entry with a time stamp of the new day (e.g., 2012-03-07
> 00:00:09,084).  Then, Flume begins log to both of these files concurrently.
> Running diff on them shows that they're different in many places, but not
> entirely so.
> I don't have any log rotation set up on the /var/log/flume directory, so
> it must be an automatic Flume setting to rotate the file daily. This
> started happening only after upgrading to 094-cdh3u2. I'm also seeing it
> happen on my collector servers, whereas on my master (which I initialize
> via a shell script with this command: /usr/local/flume/bin/flume master >>
> /var/log/flume/master_collector.log), nothing has been written to the file
> for several days and it has a 0 byte size--though the master seems to be
> running okay.
> Is anyone else experiencing this problem? Am I missing something about log
> rotation where this could be happening outside of Flume? Where would the
> setting to control this be, if Flume itself is responsible?
> Thanks,
> JS

View raw message