flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Turcsanyi <turcsa...@cloudera.com>
Subject Re: Query regarding customizing baseNameHeader value
Date Thu, 28 Jun 2018 09:13:08 GMT
Hi Jyotsna,

I'm afraid you need to write a custom interceptor to replace characters in
the event header fields (similar to SearchAndReplaceInterceptor, but this
works only with the event body).

Regards,
Peter Turcsanyi


On Wed, Jun 27, 2018 at 2:00 PM, Jyotsna G <jyotsna.g@gmail.com> wrote:

> Hello,
>
> I have a spoolDir flume agent configured and the source path contains ":"
> colon character in it which results in the below exception while writing in
> to HDFS.
>
> org.apache.flume.EventDeliveryException: java.lang.IllegalArgumentException:
> Pathname /TPA_PAAS/20180626.01:21:14/IDCS-CertificateAnalysis.csv.1530084044272.tmp
> from hdfs://slc15zcc:8020/TPA_PAAS/*20180626.01:21:14*/IDCS-
> CertificateAnalysis.csv.1530084044272.tmp* is not a valid DFS filename.*
>
> https://issues.apache.org/jira/browse/HADOOP-14217
>
> What's the best way to work around this issue?
>
> 1) Is there a flume interceptor which I can configure to edit the
> baseNameHeader value in the header?
>
> 2) Should I build a custom interceptor to remove/replace the colon
> character in the basenameHeader?
>
> Thanks & Regards,
> Jyotsna
>

Mime
View raw message