flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Gujjar <arungujjart...@yahoo.com>
Subject Sink fails to read channel data files created by other flume instances
Date Tue, 11 Nov 2014 23:35:46 GMT

I have 2 flume instances running (1.4 version), each instance has a configuration to rotate
HDFS sink file per hour. I get 24 HDFS sink files created / day from each instance and 48
from 2 instances.
Instead these 48 files, I want only 24 files created, so I tried these below option:
In both flume instances I configured only source and channel, so that both the instances will
write to same channel then, I created a one more instance that has only channel and sink which
suppose to read all the data from channel and write it into sink, but this did not work. This
new sink channel was able to read the existing data files from channel but did not pick any
of the new data files created from other flume instances. 
Can someone please help on how to fix this issue?

View raw message