flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Parquet buffering capability
Date Sat, 23 Aug 2014 16:22:37 GMT
Missed the link:
https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLUME-2439

On Saturday, August 23, 2014, Hari Shreedharan <hshreedharan@cloudera.com>
wrote:

> This was added recently and has not yet made into a release:
>
> On Saturday, August 23, 2014, Kristoffer Sjögren <stoffe@gmail.com
> <javascript:_e(%7B%7D,'cvml','stoffe@gmail.com');>> wrote:
>
>> Hi
>>
>> Does flume have support for buffering/staging avro events locally on disk
>> and storing them in hdfs as parquet files?
>>
>> Cloudera CDK explains [1] how to do this method manually but ideally I
>> want this process directly integrated into the flume runtime.
>>
>> Cheers,
>> -Kristoffer
>>
>> 1. https://github.com/cloudera/cdk-examples/tree/master/dataset-staging
>>
>

Mime
View raw message