The completed filename will always contain the epochTimestamp/counter added to it (this is to uniquely distinguish the rolled files)


On Fri, May 29, 2015 at 10:46 AM, Guyle M. Taber <> wrote:
Ok I figured this out by using the %{basename} placeholder.

However I’m trying to figure out how to prevent the epoch suffix from being applied to every file as it’s written to hdfs.


How do I prevent the epoch timestamp from being appended to every file name?

> On May 28, 2015, at 3:23 PM, <> wrote:
> I’m using the %{file} var to hold and preserve the file/log name as it’s stored in HDFS, but it seems to be recreating the entire directory structure from the source side.
> How can I simply write the filename as-is into the HDFS path specified?
> dp1.sinks.sinkSG.hdfs.filePrefix = %{file}  # Just want the file name and not the entire path+filename.
> dp1.sinks.sinkSG.hdfs.path = hdfs://{host}/%y-%m-%d