I have problem that i really need your help guys.I have topology that I need to capture syslog messages with flume,ingest them to kafka topic and then process them with Apache storm
So I have searched and find out I need use morphlines or use custom avro serializer(correct me if im wrong?)
So i tried morphlines but it doesn’t work,what do i have to do?
in below you could see example of syslogs record:
for this i just need to extract timestamp and IP addresses in different fields
<14>Jan 7 08:03:01 RT_FLOW: RT_FLOW_SESSION_CREATE: session created 10.250.191.216/49377-
0x0 junos-http 126.96.36.199/10759-
0x0 source rule 1 N/A N/A 6 TCP_APP_ACCESS TRUST_INTERNET UNTRUST_INTERNET 331629012 N/A(N/A) reth2.0 UNKNOWN UNKNOWN UNKNOWN
and this one ,i need to exatrct all filed in json format and serialize them with avro
<134>Jan 7 11:33:21 TProbe radiusSniffer: SessionInfo: 3 10.142.75.205 432112005923973 989122112848 2017-01-07 11:32:48 013172007561580 4264 30063 10.10.218.89 10.10.109.70 mcinet 2 2017-01-07 11:33:21
what should i have to do? what is proper morphlines config file?