flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Natkins <na...@streamsets.com>
Subject Re: Flume custom source get BLOCKED automatically after some time
Date Tue, 15 Jul 2014 17:27:36 GMT
Hi Venkatesh,

Does it reliably stop processing events after about 7 minutes, or does it
happen randomly, and just quickly? Does the program immediately start up
the Flume agent?

Have you looked at a thread dump from the program, at all? You can use
`jstack -F <pid>` to produce a stacktrace of all the threads in the
program, which might have some useful information. Can you post a thread
dump of the agent process that is spawned from your program?


On Tue, Jul 15, 2014 at 8:44 AM, Venkatesh B <venkateshkavayah@gmail.com>

> Hello,
> I have written flume custom source and in custom source in start() method
> I'm initiating a thread. This thread "data_thread" will collect data from
> web and writes to channel processor.
> My data collection thread is some thing like thread starts collecting data
> from web and when data is over then sleeps for some time and then again
> fetches data from web, this process is continuous. Here the control will be
> inside the thread "data_thread" all the time, since I have used while(true)
> inside thread logic.
> Here thread "data_thread" is collecting data upto 7mins and then it stops
> collecting data.
> I implemented logging and there by I came to know that  thread
>  "data_thread" state is "BLOCKED". And not returning for ever.
> In my custom source I tried in both ways by synchronizing all methods and
> also by non synchronizing all methods but no use.
> This blocking occurs when I start flume agent from java program, but when
> I start flume agent from shell it is working fine data is collected
> continuously for days without any error.
> Any idea on why thread is going to BLOCKED state??
> And also I want to know about flume custom source environment
> Thanking you
> Venkatesh

View raw message