flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rafeeq shanavaz <rafeeqshana...@gmail.com>
Subject Re: Flume Roll File Sink - unable to create YY/mm/dd directory
Date Sun, 24 May 2015 14:43:32 GMT
To achieve it, which sink need to be used. I want to write my events on
text file on a local directory not on hdfs.

Please suggest your valuable answer.


Regards,
Rafeeq Shanavaz


“The virtue of a man ought to be measured, not by his extraordinary
exertions, but by his everyday conduct.” Blaise Pascal

On Sun, May 24, 2015 at 5:22 PM, Johny Rufus <jrufus@cloudera.com> wrote:

> The Roll File Sink does not support escape sequences in the directory
> parameter and it also does not support  filePrefix config.
>
> Thanks,
> Rufus
>
> On Sun, May 24, 2015 at 6:00 AM, rafeeq shanavaz <rafeeqshanavaz@gmail.com
> > wrote:
>
>> Hi,
>>
>> I am using Flume Roll-File-Sink to write data in text file on local
>> directory.
>>
>> I am trying to use /%Y/%m/%d  in directory to maintain my data in daily
>> basis.
>>
>> *Agent.sinks.rollSink.sink.directory =
>> /home/cloudera/flume_events/%Y/%m/%d/*
>>
>> I am using timeStampInterceptor to achive above directory structure as
>> below:
>>
>> Agent.sources.sample.interceptors = i2
>> Agent.sources.sample.interceptors.i2.type =
>> org.apache.flume.interceptor.TimestampInterceptor$Builder
>> #Agent.sources.sample.interceptors.i2.type = timestamp
>> Agent.sources.rollSink.filePrefix = FlumeData.%Y/%m/%d
>>
>> But I am unable to create directory based on the timestamp and also i am
>> sending timestamp with event header.
>>
>>
>> Below exception is thrown:
>>
>> Caused by: java.io.FileNotFoundException:
>> /home/cloudera/flume_events/%Y/%m/%d/1432468505170-1 (No such file or
>> directory)
>> at java.io.FileOutputStream.open(Native Method)
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
>>
>>
>> *Please help me to create directory based on the timestamp in Roll File
>> Sink.*
>>
>> Regards,
>> Rafeeq Shanavaz
>>
>>
>> “The virtue of a man ought to be measured, not by his extraordinary
>> exertions, but by his everyday conduct.” Blaise Pascal
>>
>
>

Mime
View raw message