flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chhaya Vishwakarma <Chhaya.Vishwaka...@lntinfotech.com>
Subject RE: flume agent not starting
Date Wed, 22 Jan 2014 04:50:03 GMT
Hi ,

What i should use then for interceptor to work shall I use CAT ??

-----Original Message-----
From: Wolfgang Hoschek [mailto:whoschek@cloudera.com] 
Sent: Tuesday, January 21, 2014 6:30 PM
To: user@flume.apache.org
Subject: Re: flume agent not starting

'tail' with exec source spits out one flume event per line into the interceptor. But readMultiLine
expects multiple lines per event, not one line per event. In other words, by the time the
data arrives in the interceptor it's already too late for readMultiLine to make sense.

Wolfgang.


On Jan 17, 2014, at 2:42 PM, Chhaya Vishwakarma wrote:

> Hi,
>  
> Agent is starting I am using morphline interceptor to parse events
>  
> Below is my morphline configuration
>  
> morphlines : [
> {
> id : morphline1
>  
> commands : [
> {
> readMultiLine {
>   regex : "Started session*"
>   what : next
>   charset : UTF-8
> }
> }
> ]
> }
> ]
>  
> This is my flume configuration file
>  
> host1.sources = src1
> host1.sinks = sink1
> host1.channels = ch1
>  
> host1.sources.src1.type = exec
> host1.sources.src1.command = tail -F 
> /home/user/apache-flume-1.4.0-bin/try.log
> host1.sources.src1.channels = ch1
> host1.sources.src1.restart = true
> #host1.sources.src1.batchsize=10
>  
> host1.sources.src1.interceptors = morphlineinterceptor 
> host1.sources.src1.interceptors.morphlineinterceptor.type = 
> org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder
> host1.sources.src1.interceptors.morphlineinterceptor.morphlineFile = 
> /home/user/apache-flume-1.4.0-bin/morphline.conf
>  
> host1.sinks.sink1.type = file_roll
> host1.sinks.sink1.channel = ch1
> host1.sinks.sink1.sink.directory = /home/user/apache-flume-1.4.0-bin 
> host1.sinks.sink1.sink.serializer= TEXT
> host1.sinks.sink1.sink.rollInterval=0
>  
> host1.channels.ch1.capacity=1000
> host1.channels.ch1.transactionCapacity=100
>  
> host1.channels.ch1.type=memory
>  
> input file content
>  
> Started session
> a
> b
> c
> Started session
> c
> d
> e
> Started session
>  
> But in /home/user/apache-flume-1.4.0-bin I can see a file named 
> 1389960784130-1 but it's not showing any content. I can see only new 
> line characters in output file
>  
> Regards,
> Chhaya Vishwakarma
>  
> 
> The contents of this e-mail and any attachment(s) may contain confidential or privileged
information for the intended recipient(s). Unintended recipients are prohibited from taking
action on the basis of information in this e-mail and using or disseminating the information,
and must notify the sender and delete it from their system. L&T Infotech will not accept
responsibility or liability for the accuracy or completeness of, or the presence of any virus
or disabling code in this e-mail"


Mime
View raw message