flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh B <venkateshkava...@gmail.com>
Subject Flume custom source get BLOCKED automatically after some time
Date Tue, 15 Jul 2014 15:44:39 GMT

I have written flume custom source and in custom source in start() method
I'm initiating a thread. This thread "data_thread" will collect data from
web and writes to channel processor.

My data collection thread is some thing like thread starts collecting data
from web and when data is over then sleeps for some time and then again
fetches data from web, this process is continuous. Here the control will be
inside the thread "data_thread" all the time, since I have used while(true)
inside thread logic.

Here thread "data_thread" is collecting data upto 7mins and then it stops
collecting data.
I implemented logging and there by I came to know that  thread
 "data_thread" state is "BLOCKED". And not returning for ever.
In my custom source I tried in both ways by synchronizing all methods and
also by non synchronizing all methods but no use.

This blocking occurs when I start flume agent from java program, but when I
start flume agent from shell it is working fine data is collected
continuously for days without any error.

Any idea on why thread is going to BLOCKED state??

And also I want to know about flume custom source environment

Thanking you

View raw message