Maybe it make by 'tengine.sinks.hdfs4log.serializer = avro_event' , but still don't know why and howto ...


2013/5/15 higkoohk <higkoohk@gmail.com>
My flume.conf

tengine.sources = tengine
tengine.sources.tengine.type = exec
tengine.sources.tengine.command = tail -n +0 -F /data/log/tengine/access.log
tengine.sources.tengine.channels = file4log
tengine.sinks = hdfs4log
tengine.sinks.hdfs4log.type = hdfs
tengine.sinks.hdfs4log.channel = file4log
tengine.sinks.hdfs4log.serializer = avro_event
tengine.sinks.hdfs4log.hdfs.path = hdfs://hdfs.kisops.org:8020/flume/tengine
tengine.sinks.hdfs4log.hdfs.filePrefix = access
tengine.sinks.hdfs4log.hdfs.fileSuffix = .log
tengine.sinks.hdfs4log.hdfs.rollInterval = 0
tengine.sinks.hdfs4log.hdfs.rollCount = 0
tengine.sinks.hdfs4log.hdfs.rollSize = 134217728
tengine.sinks.hdfs4log.hdfs.batchSize = 1024
tengine.sinks.hdfs4log.hdfs.threadsPoolSize = 1
tengine.sinks.hdfs4log.hdfs.fileType = DataStream
tengine.sinks.hdfs4log.hdfs.writeFormat = Text
tengine.channels = file4log
tengine.channels.file4log.type = file
tengine.channels.file4log.capacity = 4096
tengine.channels.file4log.transactionCapacity = 1024
tengine.channels.file4log.checkpointDir = /data/log/hdfs
tengine.channels.file4log.dataDirs = /data/log/loadrunner

When I see the logs in hdfs , there are same headers in files which not creater by app :
Obj avro.codec null avro.schema� {"type":"record","name":"Event","fields":[{"name":"headers","type":{"type":"map","values":"string"}},{"name":"body","type":"bytes"}]}�" �,�)��E����5�Y� ��
�� agent25.kisops.org|10.20.216.20|1368610557.341|200|207|255|GET /status?00000005 HTTP/1.1|0.000|52033467��

See the image :


What does it mean , how to remove it or when and how to use this info ? 

Many thanks !