flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chhaya Vishwakarma <Chhaya.Vishwaka...@lntinfotech.com>
Subject RE: hbase as sink
Date Fri, 27 Dec 2013 03:10:26 GMT
Hi

host1.sources = src1
host1.sinks = sink1
host1.channels = ch1

host1.sources.src1.type = exec
host1.sources.src1.command = tail -F /home/user/Flume/flume-ng/csa-node1.log
host1.sources.src1.channels = ch1
host1.sources.src1.restart = true

host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
host1.sinks.sink1.channel = ch1
host1.sinks.sink1.table = test10
host1.sinks.sink1.columnFamily = testing

host1.sinks.sink1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
host1.sinks.sink1.serializer.regex =(\.\w+)(?<=Exception)
host1.sinks.sink1.serializer.regexIgnoreCase = true
host1.sinks.sink1.serializer.colNames = exception

host1.channels.ch1.type=memory


Regards,
Chhaya Vishwakarma


From: Brock Noland [mailto:brock@cloudera.com]
Sent: Thursday, December 26, 2013 9:41 PM
To: user@flume.apache.org
Subject: Re: hbase as sink

Please share your sink configuration.

On Thu, Dec 26, 2013 at 4:05 AM, Chhaya Vishwakarma <Chhaya.Vishwakarma@lntinfotech.com<mailto:Chhaya.Vishwakarma@lntinfotech.com>>
wrote:
Hi,

I want to parse a log file to get the highlighted text.

Regular expression used to parse

(\.\w+)(?<=Exception)

But table in hbase shows no rows
The above Regex  is working on the below sample data in regex editor (gskinner)

13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - An Error has occured for com.marsh.framework.core.exception.MarshException:
Your session has timed out.
13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - handleException():com.marsh.framework.core.exception.MarshException:
Your session has timed out.
13 Aug 2013 15:45:12,061 [WebContainer : 2] ERROR - Remote Exception while updating CSA Details
java.rmi.ServerException: RemoteException occurred in server thread;nested exception is:

What notation does flume follows for regex? (for e.g pearl notation)

Regards,
Chhaya Vishwakarma


________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information
for the intended recipient(s). Unintended recipients are prohibited from taking action on
the basis of information in this e-mail and using or disseminating the information, and must
notify the sender and delete it from their system. L&T Infotech will not accept responsibility
or liability for the accuracy or completeness of, or the presence of any virus or disabling
code in this e-mail"



--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org

Mime
View raw message