flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: about flume-ng agent
Date Thu, 25 Oct 2012 17:25:04 GMT
Hi,

Questions inline:

On Thu, Oct 25, 2012 at 11:54 AM, Paul Chavez
<pchavez@verticalsearchworks.com> wrote:
> I have tried to go the syslogUDP route to get log files from a Windows server to a flume
agent, and did not find it an adequate solution.
>
> - We are seeing corruped events when sending IIS logs (known issue: https://issues.apache.org/jira/browse/FLUME-1365)
> - Our data is too large to fit in a 1500 byte ethernet frame so events are fragmented
and the syslogUDP source ignores the continuation packets, resulting in truncated events.

I am not a Syslog expert, is there a way to detect the continuation
packets? If so I think we should open a JIRA on this issue.


> I have just been able to build the flume-ng agent for Windows and am testing the avro-client
functionality on Windows. I think this will be the best bet for us in the short term, using
LogParser to incrementally create files via scheduled task for the avro client to send along.
>
> Long term we want to develop a .net Avro client library for our apps to use directly.
I suppose a log4net avro appender would be nice too.

I think a .NET version of the RPCClient and log4net appender would be
awesome.  If you do write them, would you considering pushing them
back to the flume community?

Brock

Mime
View raw message