flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiggins <keith.wigg...@genband.com>
Subject Taildir multiLineRegex patch FLUME-2800 help
Date Wed, 11 Nov 2015 16:01:53 GMT
Hi,

1.      Hi,
2.
3.     I am testing the FLUME-2800 multiLine for taildir and it doesn't seem to be working
for me.  Does anyone have any experience with this patch?  Does the config I am using look
ok?
4.
agent.sources.CDRS.type = TAILDIR
agent.sources.CDRS.channels = memoryChannel
agent.sources.CDRS.filegroups = f1
agent.sources.CDRS.filegroups.f1 = /opt/OneEMS_LOGS/G6MM.log
agent.sources.CDRS.batchSize = 100
agent.sources.CDRS.multiLineRegex = "^%{TIMESTAMP_ISO8601}"
agent.sources.CDRS.maxNumberLines = 4096

log line entry;

2015-08-05 13:46:21,030 INFO [main] [com.centeredlogic.ecx.tomcat.ServerLifecycleListener]
- Got lifecycle event: before_init

It seems to detect that I am asking for the ISO8601 but it doesn't seem to read the file as
nothing is coming out to my sink.  If I remove the multiLine it does write to my sink (file_roll).
  I am using the GROK expression as that is what I am using in Logstash multiline which works
with the file I am tailing.

2015-11-11 10:50:57,645 (conf-file-poller-0) [DEBUG - org.apache.flume.source.taildir.grok.dictionary.GrokDictionary.digestExpression(GrokDictionary.java:136)]
Digested ["^%{TIMESTAMP_ISO8601}"] into ["^(?>\d\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T
](?:2[0123]|[01]?[0-9]):?(?:[0-5][0-9])(?::?(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))?(?:Z|[+-](?:2[0123]|[01]?[0-9])(?::?(?:[0-5][0-9])))?"]
before compilation

2015-11-11 10:50:57,756 (conf-file-poller-0) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:42)]
Creating instance of sink: k1, type: file_roll


Keith Wiggins
Software Designer

500 Palladium Drive, Suite 2100
Ottawa ON Canada K2V 1C2
www.genband.com<http://www.genband.com/>
office:+1.343.883.2656

keith.wiggins@genband.com
[Description: http://www.genband.com/extras/email/email_signature_line_below.jpg]<http://www.genband.com/>
Connect with us:[Description: http://www.genband.com/extras/email/facebookBTN.gif]<http://www.facebook.com/genband>[Description:
http://www.genband.com/extras/email/twitterBTN.gif]<https://twitter.com/#!/genband>[Description:
http://www.genband.com/extras/email/linkedBTN.gif]<http://www.linkedin.com/company/genband>[Description:
http://www.genband.com/extras/email/youtubeBTN.gif]<http://www.youtube.com/GenbandClips>[Description:
http://www.genband.com/extras/email/blogBTN.gif]<http://www.genband.com/driving-innovation>


Mime
View raw message