flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh B <venkateshkava...@gmail.com>
Subject Flume custom source stopping after some time when started through java program.
Date Mon, 14 Jul 2014 08:18:46 GMT

I'm running flume in Cent os 6.3 version.
Apache flume version 1.3

I had written many FLUME CUSTOM SOURCES,
All sources are working fine, except two sources.

When I start flume i.e.,  flume-ng ...... command from shell it is working
fine, flume is collecting data continuously for 6hours I tested it for many
times about 10 times.

But when I start flume from java program, i.e., In one ".sh" file I will
write "flume-ng --conf path to conf folder -f flume configuration file path
-n agent" command, And from java program I will run this .sh file by
calling  Runtime.getRuntime().exec(.sh file path).waitFor().

In flume custom source I have written logic for collecting data from web
and then I will write that data to channel processor. Here sink is hdfs.

Here the PROBLEM is flume is collecting data about 7 minutes and then the
not collecting when I start flume from java program.

When I search the running processes it is showing that flume is running but
my custom sources is not running.
I implemented logging for the flume custom source by logging what I came to
know is that flume custom source is not running at the same time no error
is caught.

Here flume custom source is stopping without any intimation of error.

What is the difference between running flume directly from shell and
through java program?
Why it is stopping did anyone face this problem?

What actually my flume custom source code looks is
in start() method I'm creating a thread and this thread will collect data.

Thanking you..

View raw message