flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SaravanaKumar TR <saran0081...@gmail.com>
Subject Re: Flume custom source stopping after some time when started through java program.
Date Mon, 14 Jul 2014 08:37:09 GMT

My issue looks little similar to your.My exec source stops collecting data
after some time.

But I like to know how you could say flume is running but ,your custom
source is not running.Let me know how you identify that.

Because in running process "ps" i could see a single process i.e. flume
agent running.How we identity if source is running or not.


On 14 July 2014 13:48, Venkatesh B <venkateshkavayah@gmail.com> wrote:

> Hello,
> I'm running flume in Cent os 6.3 version.
> Apache flume version 1.3
> I had written many FLUME CUSTOM SOURCES,
> All sources are working fine, except two sources.
> When I start flume i.e.,  flume-ng ...... command from shell it is working
> fine, flume is collecting data continuously for 6hours I tested it for many
> times about 10 times.
> But when I start flume from java program, i.e., In one ".sh" file I will
> write "flume-ng --conf path to conf folder -f flume configuration file path
> -n agent" command, And from java program I will run this .sh file by
> calling  Runtime.getRuntime().exec(.sh file path).waitFor().
> In flume custom source I have written logic for collecting data from web
> and then I will write that data to channel processor. Here sink is hdfs.
> Here the PROBLEM is flume is collecting data about 7 minutes and then the
> not collecting when I start flume from java program.
> When I search the running processes it is showing that flume is running
> but my custom sources is not running.
> I implemented logging for the flume custom source by logging what I came
> to know is that flume custom source is not running at the same time no
> error is caught.
> Here flume custom source is stopping without any intimation of error.
> What is the difference between running flume directly from shell and
> through java program?
> Why it is stopping did anyone face this problem?
> What actually my flume custom source code looks is
> in start() method I'm creating a thread and this thread will collect data.
> Thanking you..

View raw message