flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From IT CTO <goi....@gmail.com>
Subject Re: Newbe question about Flume-ng - ElasticSearchSink
Date Tue, 08 Sep 2015 16:10:48 GMT
Thanks, FLUME-2787 and FLUME-2788 reported.
I will also try getting into the WIKI and learn about your contribution
process to see if I can help with it.
Do you have "HOWTO" for documentation contribution? :-)
Eran

On Tue, Sep 8, 2015 at 6:44 PM Hari Shreedharan <hshreedharan@cloudera.com>
wrote:

> Can you file separate jiras for each one here:
> https://issues.apache.org/jira/browse/FLUME
>
>
> Thanks,
> Hari
>
> On Tue, Sep 8, 2015 at 8:41 AM, IT CTO <goi.cto@gmail.com> wrote:
>
>> Hi,
>> I just started learning flume and using it to send events to our HDFS
>> cluster and log them in elasticsearch.
>> I am using the latest build
>>
>> Flume 1.6.0
>> Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
>> Revision: 2561a23240a71ba20bf288c7c2cda88f443c2080
>> Compiled by hshreedharan on Mon May 11 11:15:44 PDT 2015
>> From source with checksum b29e416802ce9ece3269d34233baf43f
>>
>> I found two issues which I think should be filed as bug:
>> 1) When using
>> agent.sinks.elastic-sink.serializer =
>> org.apache.flume.sink.elasticsearch.ElasticSearchDynamicSerializer
>> the event timestamp is stored as string and not dateOptionalTime if the
>> agent.sinks.elastic-sink.serializer is not set the code works as expected.
>>
>> 2) Documentation BUG - when using elasticsearchsink a timestamp field
>> need to be created mainly if we want to use it with kibana (as the
>> documentation say) so in order for it to work we should use the timestamp
>> interceptor. I think the documentation should note it in the
>> elasticearchsink section.
>>
>> Thanks,
>> Eran
>>
>>
>> --
>> Eran | "You don't need eyes to see, you need vision" (Faithless)
>>
>
> --
Eran | "You don't need eyes to see, you need vision" (Faithless)

Mime
View raw message