flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Flume ElasticSearch Sink not indexing JSON data
Date Wed, 09 Apr 2014 13:18:46 GMT
IMHO, checkout the trunk, build it and then just use the agent config from
old setup. Keep the old setup as is.

thanks
ashish


On Wed, Apr 9, 2014 at 6:17 PM, Deepak Subhramanian <
deepak.subhramanian@gmail.com> wrote:

> Thanks Otis. I will give it a try. Do I have to replace the flume jars or
> add the flume jar to the plugins directory to override the current flume
> version.
>
>
> On Wed, Apr 9, 2014 at 2:19 AM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
>> Hi,
>>
>> The ES Sink has been modified quite a bit in the trunk.  Would it make
>> sense to give that a try?
>>
>> Otis
>> --
>> Performance Monitoring * Log Analytics * Search Analytics
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>> On Mon, Apr 7, 2014 at 12:08 PM, Deepak Subhramanian <
>> deepak.subhramanian@gmail.com> wrote:
>>
>>> I am posting JSON data to ElasticSearch using Flume . But for some
>>> reason it is not loading the correct data in the message. Instead of JSON
>>> message it is storing message data as
>>> "org.elasticsearch.common.xcontent.XContentBuilder@32f4122e"
>>>
>>> I tried using the DynamicSerializer which determines the content-type.
>>> But no luck.
>>>
>>> Here is the sample data posted on Elastic Search.
>>>
>>> {
>>>
>>>   "_index": "test_flume-2014-04-07",
>>>
>>>   "_type": "logs",
>>>
>>>   "_id": "M9E-33RQTy2kA6QhW6mSUw",
>>>
>>>   "_score": null,
>>>
>>>   "_source": {
>>>
>>>     "@message": "
>>> org.elasticsearch.common.xcontent.XContentBuilder@58bf76d2",
>>>
>>>     "@timestamp": "2014-04-07T09:49:26.490Z",
>>>
>>>     "@fields": {
>>>
>>>       "timestamp": "1396864166490"
>>>
>>>     }
>>>
>>>   },
>>>
>>>   "sort": [
>>>
>>>     1396864166490,
>>>
>>>     1396864166490
>>>
>>>   ]
>>>
>>> }
>>>
>>> org.elasticsearch.common.xcontent.XContentBuilder
>>> --
>>> Deepak Subhramanian
>>>
>>
>>
>
>
> --
> Deepak Subhramanian
>



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message