flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyotsna G <jyotsn...@gmail.com>
Subject Issue similar to Flume-2052
Date Thu, 21 Jun 2018 12:54:37 GMT
Hi All,
Am currently using Flume 1.7.0.

While processing a set of files, am encountering the below exception and
Flume hangs unpredictably.

18/06/21 05:40:40 ERROR source.SpoolDirectorySource: FATAL: Spool Directory
source src-tpa_idoru: { spoolDir: /tmp/tpa-idoru/ }: Uncaught exception in
SpoolDirectorySource thread. Restart or reconfigure Flume to continue
processing.
java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at
org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:283)
at
org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:132)
at
org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:70)
at
org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:89)
at
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readDeserializerEvents(ReliableSpoolingFileEventReader.java:343)
at
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:331)
at
org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:250)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


This issue looks similar to https://jira.apache.org/jira/browse/FLUME-2052

Wouldn't this fix be already included in 1.7? How do I enable flume to skip
these errors and proceed further?

Thanks,
Jyotsna

Mime
View raw message