flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Alfeld <jalf...@gmail.com>
Subject File Channel Issue
Date Thu, 12 Nov 2015 14:43:04 GMT
I am trying to set up two file channels on a  server to forward log
information from two different spool directory sources, this configuration
was previously working using a spillable memory channel. We needed to shift
to a pure file channel to prevent data loss in the event of a server
issue.  When attempting to start the service I receive the following errors
in the log file:

10 Nov 2015 16:13:25,784 ERROR [lifecycleSupervisor-1-0]
(org.apache.flume.channel.file.Log.replay:492)  - Failed to initialize Log
on [channel=bluecoat-channel]
java.io.EOFException
        at java.io.RandomAccessFile.readInt(RandomAccessFile.java:827)
        at java.io.RandomAccessFile.readLong(RandomAccessFile.java:860)
        at
org.apache.flume.channel.file.EventQueueBackingStoreFactory.get(EventQueueBackingStoreFactory.java:80)
        at org.apache.flume.channel.file.Log.replay(Log.java:426)
        at
org.apache.flume.channel.file.FileChannel.start(FileChannel.java:290)
        at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
10 Nov 2015 16:13:25,786 ERROR [lifecycleSupervisor-1-0]
(org.apache.flume.channel.file.FileChannel.start:301)  - Failed to start
the file channel [channel=bluecoat-channel]
java.io.EOFException
        at java.io.RandomAccessFile.readInt(RandomAccessFile.java:827)
        at java.io.RandomAccessFile.readLong(RandomAccessFile.java:860)
        at
org.apache.flume.channel.file.EventQueueBackingStoreFactory.get(EventQueueBackingStoreFactory.java:80)
        at org.apache.flume.channel.file.Log.replay(Log.java:426)
        at
org.apache.flume.channel.file.FileChannel.start(FileChannel.java:290)
        at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)


The same errors then  repeat for the second file channel.

Here is the  configuration for one of the channels, the second channel is
the same (with the exception of the data and checkpoint directories):

agent.channels.bluecoat-channel.type = file
agent.channels.bluecoat-channel.capacity = 100000000
agent.channels.bluecoat-channel.checkpointDir =
/Dropbox/flume_tmp/bluecoat-channel/checkpoint
agent.channels.bluecoat-channel.dataDirs =
/Dropbox/flume_tmp/bluecoat-channel/data



Any suggestions on where to start to correct this issue so I can move this
into production?

Jeff Alfeld

Mime
View raw message