flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chhaya Vishwakarma <Chhaya.Vishwaka...@lntinfotech.com>
Subject RE: hbase as sink
Date Fri, 27 Dec 2013 03:10:26 GMT

host1.sources = src1
host1.sinks = sink1
host1.channels = ch1

host1.sources.src1.type = exec
host1.sources.src1.command = tail -F /home/user/Flume/flume-ng/csa-node1.log
host1.sources.src1.channels = ch1
host1.sources.src1.restart = true

host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
host1.sinks.sink1.channel = ch1
host1.sinks.sink1.table = test10
host1.sinks.sink1.columnFamily = testing

host1.sinks.sink1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
host1.sinks.sink1.serializer.regex =(\.\w+)(?<=Exception)
host1.sinks.sink1.serializer.regexIgnoreCase = true
host1.sinks.sink1.serializer.colNames = exception


Chhaya Vishwakarma

From: Brock Noland [mailto:brock@cloudera.com]
Sent: Thursday, December 26, 2013 9:41 PM
To: user@flume.apache.org
Subject: Re: hbase as sink

Please share your sink configuration.

On Thu, Dec 26, 2013 at 4:05 AM, Chhaya Vishwakarma <Chhaya.Vishwakarma@lntinfotech.com<mailto:Chhaya.Vishwakarma@lntinfotech.com>>

I want to parse a log file to get the highlighted text.

Regular expression used to parse


But table in hbase shows no rows
The above Regex  is working on the below sample data in regex editor (gskinner)

13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - An Error has occured for com.marsh.framework.core.exception.MarshException:
Your session has timed out.
13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - handleException():com.marsh.framework.core.exception.MarshException:
Your session has timed out.
13 Aug 2013 15:45:12,061 [WebContainer : 2] ERROR - Remote Exception while updating CSA Details
java.rmi.ServerException: RemoteException occurred in server thread;nested exception is:

What notation does flume follows for regex? (for e.g pearl notation)

Chhaya Vishwakarma

The contents of this e-mail and any attachment(s) may contain confidential or privileged information
for the intended recipient(s). Unintended recipients are prohibited from taking action on
the basis of information in this e-mail and using or disseminating the information, and must
notify the sender and delete it from their system. L&T Infotech will not accept responsibility
or liability for the accuracy or completeness of, or the presence of any virus or disabling
code in this e-mail"

Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org

View raw message