flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyotsna G <jyotsn...@gmail.com>
Subject Re: Query regarding customizing baseNameHeader value
Date Fri, 29 Jun 2018 09:49:25 GMT
Thank you Peter.
Followed your advice and implemented a custom interceptor to remove the ":"
character.

Regards,
Jyotsna

On Thu, Jun 28, 2018 at 2:43 PM, Peter Turcsanyi <turcsanyi@cloudera.com>
wrote:

> Hi Jyotsna,
>
> I'm afraid you need to write a custom interceptor to replace characters in
> the event header fields (similar to SearchAndReplaceInterceptor, but this
> works only with the event body).
>
> Regards,
> Peter Turcsanyi
>
>
> On Wed, Jun 27, 2018 at 2:00 PM, Jyotsna G <jyotsna.g@gmail.com> wrote:
>
>> Hello,
>>
>> I have a spoolDir flume agent configured and the source path contains ":"
>> colon character in it which results in the below exception while writing in
>> to HDFS.
>>
>> org.apache.flume.EventDeliveryException: java.lang.IllegalArgumentException:
>> Pathname /TPA_PAAS/20180626.01:21:14/IDCS-CertificateAnalysis.csv.1530084044272.tmp
>> from hdfs://slc15zcc:8020/TPA_PAAS/*20180626.01:21:14*/IDCS-Certifi
>> cateAnalysis.csv.1530084044272.tmp* is not a valid DFS filename.*
>>
>> https://issues.apache.org/jira/browse/HADOOP-14217
>>
>> What's the best way to work around this issue?
>>
>> 1) Is there a flume interceptor which I can configure to edit the
>> baseNameHeader value in the header?
>>
>> 2) Should I build a custom interceptor to remove/replace the colon
>> character in the basenameHeader?
>>
>> Thanks & Regards,
>> Jyotsna
>>
>
>

Mime
View raw message