flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johny Rufus <jru...@cloudera.com>
Subject Re: Flume Roll File Sink - unable to create YY/mm/dd directory
Date Sun, 24 May 2015 14:22:59 GMT
The Roll File Sink does not support escape sequences in the directory
parameter and it also does not support  filePrefix config.

Thanks,
Rufus

On Sun, May 24, 2015 at 6:00 AM, rafeeq shanavaz <rafeeqshanavaz@gmail.com>
wrote:

> Hi,
>
> I am using Flume Roll-File-Sink to write data in text file on local
> directory.
>
> I am trying to use /%Y/%m/%d  in directory to maintain my data in daily
> basis.
>
> *Agent.sinks.rollSink.sink.directory =
> /home/cloudera/flume_events/%Y/%m/%d/*
>
> I am using timeStampInterceptor to achive above directory structure as
> below:
>
> Agent.sources.sample.interceptors = i2
> Agent.sources.sample.interceptors.i2.type =
> org.apache.flume.interceptor.TimestampInterceptor$Builder
> #Agent.sources.sample.interceptors.i2.type = timestamp
> Agent.sources.rollSink.filePrefix = FlumeData.%Y/%m/%d
>
> But I am unable to create directory based on the timestamp and also i am
> sending timestamp with event header.
>
>
> Below exception is thrown:
>
> Caused by: java.io.FileNotFoundException:
> /home/cloudera/flume_events/%Y/%m/%d/1432468505170-1 (No such file or
> directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
>
>
> *Please help me to create directory based on the timestamp in Roll File
> Sink.*
>
> Regards,
> Rafeeq Shanavaz
>
>
> “The virtue of a man ought to be measured, not by his extraordinary
> exertions, but by his everyday conduct.” Blaise Pascal
>

Mime
View raw message