flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Stricks <...@wapolabs.com>
Subject Processing Data During Master Outage
Date Thu, 09 Feb 2012 15:02:47 GMT
Hi folks,

I'm having stability problems with my master, and until we get it resolved,
I am curious how you would recommend getting Flume to handle the data that
was generated on the agent 1) during the outage, and then 2) until the flow
configuration settings were processed by the master once it was back up. Is
this data actively being collected by the agents until it gets the configs
from the Master and then forwards it on to the collectors? If not, is there
a way to take access_log files and have them be processed from the missing
time stamp?  Any advice would be greatly appreciated.

The agents have the following untranslated sources and sinks:

*Source: *tail("/var/log/httpd/access_log","true")
*Sink: *{value("app","ngnaccesslogs") => autoE2EChain }

*Source: *syslogUdp(5140)
*Sink: *{value("app","ngnfb") => autoE2EChain }


Jay Stricks

View raw message