flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Jiang <it.mjji...@gmail.com>
Subject Re: flume agent not work
Date Tue, 25 Oct 2011 00:49:13 GMT

First of all, what do you mean all agents are alive. Are their daemons
running or blocked or sleeping? For example, if a daemon is too busy with
GC, it may not serve any function at all.

Hope this already fixed. But in general, I guess there are couple of places
to look at for clues.

1st, there is a web interface you can check status of flume services,
including agents.

2nd, check agent log for clues. Also check collectors so that together will
provide a complete view of possible problems.

3rd, check master for configuration. I guess its low probability that
configuration got altered unexpectedly. But check it as a routine wont hurt
nobody :)

4th, check network connection between agent and collector, and between
collector and hdfs. This may include both hardware, flume configuration and
system network configurations (e.g. any firewall or dns update recently?).

5th, check os log for abnormalities.

By all these means, you may want to reduce problem to a small area, e.g., an
agent may send data to collectors, but it is a collector that fails to relay
the data to hdfs. etc.

Hope this helps.


On Sat, Oct 8, 2011 at 12:51 AM, hao.wang <hao.wang@ipinyou.com> wrote:

> **
> Hi,All:
>     I have a problem about flume. In our production environment, We use
> flume to transfer logs from web servers to HDFS. We have 3 flume agents. But
> sometimes, only 1 agent works, the others can not work. I checked the status
> of flume agents. They are all alived. Does anybody know why?
> regards
> 2011-10-08
> ------------------------------
> hao.wang

View raw message