flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: flume-ng syslogtcp issue
Date Thu, 12 Jul 2012 06:27:01 GMT
You are right. It is an error. It should actually be in bold. Please feel free to submit a
patch :-) 

-- 
Hari Shreedharan


On Wednesday, July 11, 2012 at 11:21 PM, Stern, Mark wrote:

> That does not look quite right.
> 
> 'sink.directory' is not bold, which suggests that it is optional. But
> there is no default. So where do the files go if you do not specify it?
> 
> On Thu, 2012-07-12 at 15:15 +0900, Juhani Connolly wrote:
> > file_roll is detailed in the Flume user guide. It's generated in
> > target/docs when you do a full build. I've put it up at
> > http://people.apache.org/~juhanic/flume-docs/FlumeUserGuide.html#file-roll-sink
for convenience, but the version may not match yours.
> > 
> > If the docs are insufficient let us know and we'll try to address it.
> > 
> > On 07/12/2012 02:38 PM, Young Kim wrote:
> > 
> > > After some debugging, I found out that the syslogTcp is working as
> > > intended. I didn't realize that debug no longer shows the resulting
> > > syslog messages on success. On that note, this may be a bit
> > > off-topic, but is there documentation showing how file_roll should
> > > be used to write to local disk? 
> > > 
> > > 
> > > Thanks,
> > > Young
> > > On Wednesday, July 11, 2012 at 11:12 AM, Young Kim wrote:
> > > 
> > > > Hey Alex, 
> > > > 
> > > > 
> > > > The IP is correct, but I've altered it to say localhost. As for
> > > > the syslog tests, I've ran it verbatim, but it doesn't register.
> > > > However, when I alter /tmp/foo to include two new lines, flume
> > > > registers it as a partial event. 
> > > > 
> > > > 
> > > > It still seems odd to me that flume-ng isn't detecting the
> > > > syslogtcp whereas flume-og is doing it perfectly. Is it possible
> > > > that there is a bug in the way flume-ng is implementing it?
> > > > 
> > > > 
> > > > Thanks,
> > > > Young
> > > > 
> > > > 
> > > > On Tuesday, July 10, 2012 at 11:22 PM, alo alt wrote:
> > > > 
> > > > > HI,
> > > > > 
> > > > > 
> > > > > try these for syslog tests:
> > > > > echo "<13>Jun 20 12:12:12 host foo[345]: a syslog message"
> > > > > > /tmp/foo
> > > > > 
> > > > > nc -v FLUME SYSLOG AGENT 5140 < /tmp/foo
> > > > > 
> > > > > 
> > > > > > syslog-agent.sources.syslog.host = 172.17.1.92
> > > > > 
> > > > > 
> > > > > Means, will only listen only on this interface, is the IP
> > > > > correct?
> > > > > 
> > > > > 
> > > > > cheers,
> > > > > Alex
> > > > > 
> > > > > 
> > > > > On Jul 10, 2012, at 10:59 PM, Young Kim wrote:
> > > > > 
> > > > > 
> > > > > > Hey guys, 
> > > > > > 
> > > > > > 
> > > > > > I've been trying to use flume-ng (build in trunk) to read
> > > > > > syslogs through tcp then write to a flat file, as seen in this
> > > > > > configuration:
> > > > > > 
> > > > > > 
> > > > > > # Source, sink, and channel name below
> > > > > > syslog-agent.channels = rmc
> > > > > > syslog-agent.sources = syslog
> > > > > > syslog-agent.sinks = flat
> > > > > > 
> > > > > > 
> > > > > > # Configure channels
> > > > > > syslog-agent.channels.rmc.type = memory
> > > > > > 
> > > > > > 
> > > > > > # Configure sources
> > > > > > syslog-agent.sources.syslog.type = syslogTcp
> > > > > > syslog-agent.sources.syslog.port = 5140
> > > > > > syslog-agent.sources.syslog.host = 172.17.1.92
> > > > > > syslog-agent.sources.syslog.channels = rmc
> > > > > > 
> > > > > > 
> > > > > > # Configure sinks
> > > > > > syslog-agent.sinks.flat.type = FILE_ROLL
> > > > > > syslog-agent.sinks.flat.directory = /Users/ykim/Desktop/test
> > > > > > syslog-agent.sinks.flat.rollInterval = 0
> > > > > > syslog-agent.sinks.flat.channels = rmc
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > After a bit of wrangling with rsyslog configurations, it
> > > > > > appeared as if rsyslog wasn't sending any packets. However,
I
> > > > > > decided to try out flume-og with a simple dump syslogTcp, and
> > > > > > it correctly reads the syslog. At this point, I'm a bit lost
> > > > > > and not sure what else to look at. Hopefully, someone on here
> > > > > > can shed some light on what could be wrong. 
> > > > > > 
> > > > > > 
> > > > > > Thanks,
> > > > > > Young Kim
> > > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > --
> > > > > Alexander Alten-Lorenz
> > > > > http://mapredit.blogspot.com
> > > > > German Hadoop LinkedIn Group: http://goo.gl/N8pCF
> > > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> 
> 
> 



Mime
View raw message