Hi shadyxu,

This is the repo: https://github.com/lem0na/flume
My paches are in "es-flume" branch
As i said there must be a better solution but this works.
Hope that in near future oficial ES sink will support JSON payload too.

best regards,
Nickolay Kolev

On Wed, Nov 19, 2014 at 11:02 AM, shadyxu <shadyxu@gmail.com> wrote:
Hi Nickolay,

Thank you and please upload the repo on Github.

2014-11-19 16:41 GMT+08:00 Nickolay Kolev <lem0na2@gmail.com>:
Hi,

It looks like ES sink is oriented to handle text payload and not json and there are problems with serialization.
We have the same problem and I have developed patch and custom version based on sources (two months ago)
I am not java developer so probably it is not the best solution but it works.
Now it is hosted on company's internal repo but if you have interest I can upload it on GitHub or Bitbucket

best regards,
Nickolay Kolev


On Mon, Nov 17, 2014 at 11:47 AM, shadyxu <shadyxu@gmail.com> wrote:
Hi everyone,

I am now using Flume to collect log into ElasticSearch. And the logs are in json format. However, when I check them in ES, it seems that Flume has put the entire json log in the @message attribute. Is there any config to do or need I do some coding to separate them into different columns as ES always does?

BTW, I found that ttl seems not working in ElasticSearch. I'm using Flume 1.5.0.1.

Any clue shall be appreciated.