flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: about flume-ng agent
Date Thu, 25 Oct 2012 17:25:04 GMT

Questions inline:

On Thu, Oct 25, 2012 at 11:54 AM, Paul Chavez
<pchavez@verticalsearchworks.com> wrote:
> I have tried to go the syslogUDP route to get log files from a Windows server to a flume
agent, and did not find it an adequate solution.
> - We are seeing corruped events when sending IIS logs (known issue: https://issues.apache.org/jira/browse/FLUME-1365)
> - Our data is too large to fit in a 1500 byte ethernet frame so events are fragmented
and the syslogUDP source ignores the continuation packets, resulting in truncated events.

I am not a Syslog expert, is there a way to detect the continuation
packets? If so I think we should open a JIRA on this issue.

> I have just been able to build the flume-ng agent for Windows and am testing the avro-client
functionality on Windows. I think this will be the best bet for us in the short term, using
LogParser to incrementally create files via scheduled task for the avro client to send along.
> Long term we want to develop a .net Avro client library for our apps to use directly.
I suppose a log4net avro appender would be nice too.

I think a .NET version of the RPCClient and log4net appender would be
awesome.  If you do write them, would you considering pushing them
back to the flume community?


View raw message