flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mostafa T <mtpuls...@gmail.com>
Subject Extending AbstractAvroEventSerializer, Overriding convert(Event e)
Date Tue, 03 Mar 2015 15:45:41 GMT
Hello,

I am writing a custom event parser by extending AbstractAvroEventSerializer.

My question is about the "protected abstract T *convert
<https://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/AbstractAvroEventSerializer.html#convert%28org.apache.flume.Event%29>*
(Event
<https://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/Event.html>
 event)"

In the implementations I found online, the event is stringified then parsed

ex:
String logline = new String(event.getBody(), Charsets.UTF_8);
....
String dateTime = logline.substring(seek, nextMarker);
....
String logLevel = logline.substring(seek, nextMarker);
....
full example here:
https://github.com/DandyDev/flume-plugins/blob/master/src/main/java/nl/info/flume/serialization/JavaLogAvroEventSerializer.java


My problem with this approach is:
I believe that stringifying the event is probably not effecient.
Also my object is big and nested, it will be a nightmare to write then
maintain if I have to parse each field manually.

Any ideas how can I implement this in an efficient way both maintenance and
performance wise.

Thanks in advance!

Mime
View raw message