flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shady Xu <shad...@gmail.com>
Subject Re: How to customize the key in a HDFS SequenceFile sink
Date Tue, 08 Sep 2015 07:39:48 GMT
The way Gonzalo said is how we always do it too.

2015-09-08 15:29 GMT+08:00 Gonzalo Herreros <gherreros@gmail.com>:

> I would implement a custom serializer and configure it in the standard
> Hdfs sink.
> That way you control how you build the key for each event.
>
> Regards,
> Gonzalo
>
> On 8 September 2015 at 06:42, <Thomas.Beer@continental-corporation.com>
> wrote:
>
>>
>> Hello,
>>
>> I'm using Flume's HDFS SequenceFile sink for writing data to HDFS. I'm
>> looking for a possibility to create "custom keys". Per default, Flume is
>> using the Timestamp as key within a SequenceFile. However, in my usecase I
>> would like to use a customized string as key (instead of the timestamp).
>>
>> What are best practices for implementing/configuring such a "custom key"
>> within Flume?
>>
>> Best, Thomas
>>
>>
>

Mime
View raw message