flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Chavez <pcha...@verticalsearchworks.com>
Subject RE: about flume-ng agent
Date Thu, 25 Oct 2012 20:59:08 GMT
I was unaware of an HTTPSource, after reviewing the FLUME-1199 issue it may well be the best
use case for us.

That said, while I am manually building and testing the latest snapshot for the Windows side,
our actual Hadoop machines are running CDH4.1.1 which has flume-ng 1.2.0.

When a version of flume-ng containing HTTPSource is packaged along with the rest of the hadoop
distribution I will look at it. As a 'windoze guy' ;-) I do not manage the hadoop systems.

Thank you,
Paul Chavez

-----Original Message-----
From: Will McQueen [mailto:will@cloudera.com] 

Would the new HttpSource work for you?

On Oct 25, 2012, at 2:47 PM, Brock Noland <brock@cloudera.com> wrote:

> On Thu, Oct 25, 2012 at 1:16 PM, Paul Chavez 
> <pchavez@verticalsearchworks.com> wrote:
>> 
>> I am no expert either but the various syslog related RFC and RFC-type documentation
I can find recommends that messeages be kept small in order to avoid fragmentation.
> 
> Ahh OK.
> 
>> 
>> Testing with the syslogTCP source did not show any issues with fragmentation, but
the tool we are using to send syslog messages over TCP (LogParser) does not separate messages
with a carriage return so messages weren't parsed correctly by the flume source.
> 
> Yes TCP/IP will deliver the events in ordert.
> 
>> I would hope so but I am not in a position to make any guarantees on behalf of my
employer.
> 
> No worries, I understand!
> 
> Brock
> 
> --
> Apache MRUnit - Unit testing MapReduce - 
> http://incubator.apache.org/mrunit/

Mime
View raw message