flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alireza <alireza.shahri...@gmail.com>
Subject help me, convert syslog to json or avro ,using morphlines??
Date Tue, 24 Jan 2017 08:04:01 GMT
HI
I have problem that i really need your help guys.I have topology that I
need to capture syslog messages with flume,ingest them to kafka topic and
then process them with Apache storm

So I have searched and find out I need use morphlines or use custom avro
serializer(correct me if im wrong?)
So i tried morphlines but it doesn’t work,what do i have to do?
in below you could see example of syslogs record:
for this i just need to extract timestamp and IP addresses in different
fields
<14>Jan  7 08:03:01 RT_FLOW: RT_FLOW_SESSION_CREATE: session created
10.250.191.216/49377->194.221.65.9/80 0x0 junos-http 5.213.21.241/10759->
194.221.65.9/80 0x0 source rule 1 N/A N/A 6 TCP_APP_ACCESS TRUST_INTERNET
UNTRUST_INTERNET 331629012 N/A(N/A) reth2.0 UNKNOWN UNKNOWN UNKNOWN

and this one ,i need to exatrct all filed in json format and serialize them
with avro
<134>Jan  7 11:33:21 TProbe radiusSniffer: SessionInfo: 3 10.142.75.205
432112005923973 989122112848 2017-01-07 11:32:48 013172007561580 4264 30063
10.10.218.89 10.10.109.70 mcinet 2 2017-01-07 11:33:21

what should i have to do? what  is proper morphlines config file?

Mime
View raw message