If you submit your patch to Flume, we can hopefully commit it to the sink bundled with Flume


On Wed, Nov 19, 2014 at 12:42 AM, Nickolay Kolev <lem0na2@gmail.com> wrote:


It looks like ES sink is oriented to handle text payload and not json and there are problems with serialization.
We have the same problem and I have developed patch and custom version based on sources (two months ago)
I am not java developer so probably it is not the best solution but it works.
Now it is hosted on company's internal repo but if you have interest I can upload it on GitHub or Bitbucket

best regards,
Nickolay Kolev

On Mon, Nov 17, 2014 at 11:47 AM, shadyxu <shadyxu@gmail.com> wrote:
Hi everyone,

I am now using Flume to collect log into ElasticSearch. And the logs are in json format. However, when I check them in ES, it seems that Flume has put the entire json log in the @message attribute. Is there any config to do or need I do some coding to separate them into different columns as ES always does?

BTW, I found that ttl seems not working in ElasticSearch. I'm using Flume

Any clue shall be appreciated.