flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Kiran Aita <raviki...@eiqnetworks.com>
Subject RE: Monitoring Flume
Date Thu, 21 Jan 2016 13:49:13 GMT

We can enable monitoring for Flume Nodes by starting the agent with below configuration
Invoke the agent with setting --> -Dflume.monitoring.port=34545
Access the stats from url --> http://<ip>:34545/metrics<http://%3cip%3e:34545/metrics>

We get the stats in json format which can be further processed. Probably this may help you.

Ravikiran A

From: Guillermo Ortiz [mailto:konstt2000@gmail.com]
Sent: Thursday, January 21, 2016 7:15 PM
To: user@flume.apache.org
Subject: Re: Monitoring Flume

When you talk about ruby or python, you mean reading the log and waiting some pathers and
if they don't happen to launch an "action"?

2016-01-21 12:11 GMT+01:00 Simone Roselli <simone.roselli@plista.com<mailto:simone.roselli@plista.com>>:

in the case of "process up but not working" which can be considered a particular case, I would
setup a custom script (Bash? Python?) and use Nagios/Icinga for the alerting

For the collection of the metrics, I would connect Collectd to JMX, and then link Collectd
directly on Grafana (http://grafana.org/). The usual annoying part is to build the Collectd
configuration for the interested MBeans; but probably someone has done this already

I also would be interested in a generic good practice to query JMX, since I didn't find a
human way to do it using Python or Ruby ..

Simone Roselli
ITE Sysadmin

----- Original Message -----
From: "Guillermo Ortiz" <konstt2000@gmail.com<mailto:konstt2000@gmail.com>>
To: "user" <user@flume.apache.org<mailto:user@flume.apache.org>>
Sent: Thursday, January 21, 2016 11:54:34 AM
Subject: Monitoring Flume


I've looking for solutions to monitor a Flume Agent. Sometimes the process
is running but it isn't available to process any message because some error
happened. So, the process is running but Flume doesn't work.

How are people monitoring Flume in general?

We thought to create a JMX Source and send all data to some sink and do the
monitoring based on the JMX logs though Solr or whatever.

Any other solution or advise about monitoring Flume?

View raw message