flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Parquet buffering capability
Date Sat, 23 Aug 2014 16:22:15 GMT
This was added recently and has not yet made into a release:

On Saturday, August 23, 2014, Kristoffer Sjögren <stoffe@gmail.com> wrote:

> Hi
>
> Does flume have support for buffering/staging avro events locally on disk
> and storing them in hdfs as parquet files?
>
> Cloudera CDK explains [1] how to do this method manually but ideally I
> want this process directly integrated into the flume runtime.
>
> Cheers,
> -Kristoffer
>
> 1. https://github.com/cloudera/cdk-examples/tree/master/dataset-staging
>

Mime
View raw message