The HTTPSource will be a part of Apache Flume-1.3.0. 


Hari Shreedharan

On Thursday, October 25, 2012 at 4:59 PM, Paul Chavez wrote:

I was unaware of an HTTPSource, after reviewing the FLUME-1199 issue it may well be the best use case for us.

That said, while I am manually building and testing the latest snapshot for the Windows side, our actual Hadoop machines are running CDH4.1.1 which has flume-ng 1.2.0.

When a version of flume-ng containing HTTPSource is packaged along with the rest of the hadoop distribution I will look at it. As a 'windoze guy' ;-) I do not manage the hadoop systems.

Thank you,
Paul Chavez

-----Original Message-----
From: Will McQueen []

Would the new HttpSource work for you?

On Oct 25, 2012, at 2:47 PM, Brock Noland <> wrote:

On Thu, Oct 25, 2012 at 1:16 PM, Paul Chavez

I am no expert either but the various syslog related RFC and RFC-type documentation I can find recommends that messeages be kept small in order to avoid fragmentation.

Ahh OK.

Testing with the syslogTCP source did not show any issues with fragmentation, but the tool we are using to send syslog messages over TCP (LogParser) does not separate messages with a carriage return so messages weren't parsed correctly by the flume source.

Yes TCP/IP will deliver the events in ordert.

I would hope so but I am not in a position to make any guarantees on behalf of my employer.

No worries, I understand!


Apache MRUnit - Unit testing MapReduce -