It does http://flume.apache.org/releases/1.1.0.html but the root of the issue is that the source doesn't add the timestamp header to the event. Also 1.2 has just been released which has Interceptors which I think can add the timestamp for you.

Brock

On Thu, Jul 26, 2012 at 12:59 PM, khadar basha <khadarskb@gmail.com> wrote:
Hi Brock.

flume-1.1.0 doesn't support this escape sequence options?

-Khadar

On Thu, Jul 26, 2012 at 10:34 PM, Brock Noland <brock@cloudera.com> wrote:
The flume user doc here http://flume.apache.org/releases/1.2.0.html  has many examples.

Brock


On Thu, Jul 26, 2012 at 11:37 AM, khadar basha <khadarskb@gmail.com> wrote:
Hi Brock,

could you please provide me the sample configuration?

Thanks,
Khadar


On Thu, Jul 26, 2012 at 7:23 PM, Brock Noland <brock@cloudera.com> wrote:
If you want to use a bucketed paths your events need to have a "timestamp" header which tail source does not appear to add.

Brock

On Thu, Jul 26, 2012 at 6:32 AM, khadar basha <khadarskb@gmail.com> wrote:

Hi 

we are using flume-1.1.0-cdh4.0.1 . we have the following configurations in flume-conf.properties.

agent1Test1.sources = seqGenSrc
agent1Test1.channels = memoryChannel
agent1Test1.sinks = loggerSink

# For each one of the sources, the type is defined
agent1Test1.sources.seqGenSrc.type = exec
agent1Test1.sources.seqGenSrc.command= tail -f /tomcat/logs/catalina.out


# The channel can be defined as follows.
agent1Test1.sources.seqGenSrc.channels = memoryChannel

# Each sink's type must be defined
agent1Test1.sinks.loggerSink.type = hdfs
agent1Test1.sinks.loggerSink.hdfs.path = hdfs://hdfshost:port/data/%Y/%m/%{host}/mylogs

agent1Test1.sinks.loggerSink.hdfs.fileType = DataStream

#Specify the channel the sink should use
agent1Test1.sinks.loggerSink.channel = memoryChannel

# Each channel's type is defined.
agent1Test1.channels.memoryChannel.type = memory

# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
agent1Test1.channels.memoryChannel.capacity = 1000


I am getting the following exception. Anybody facing this issue.


2012-07-26 13:07:52,002 ERROR hdfs.HDFSEventSink: process failed
java.lang.NumberFormatException: null
        at java.lang.Long.parseLong(Long.java:375)
        at java.lang.Long.valueOf(Long.java:525)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:662)
2012-07-26 13:07:52,003 ERROR flume.SinkRunner: Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NumberFormatException: null
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:469)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NumberFormatException: null
        at java.lang.Long.parseLong(Long.java:375)
        at java.lang.Long.valueOf(Long.java:525)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:220)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:310)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:402)
        ... 3 more

--
Thanks,
Khadar




--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



--
Thanks,
Khadar




--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



--
Thanks,
Khadar




--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/