flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Wise <m...@nextdoor.com>
Subject Re: how to print the channel capacity
Date Wed, 15 May 2013 17:10:39 GMT
We do the same thing, but with Collectd as our graphing/collection mechanism. I am actually
going to do a blog post in the next day or two with the code to our flume data collection
script, and some example graphs/etc. We've done a similar thing with Zookeeper monitoring


On May 15, 2013, at 9:36 AM, Paul Chavez <pchavez@verticalsearchworks.com> wrote:

> There are a few ways to monitor flume in operation. We use the JSON reporting, which
is available via 'http://<agent address>:<port>/metrics'. You need to start the
agent with the following parameters to get this interface:
> -Dflume.monitoring.type=http -Dflume.monitoring.port=34545
> We use cacti to graph channel size both as a percentage of maximum and absolute number
of events in channel. This provides warning if the sinks cannot keep up with the sources.

> We also graph ingress/egress event counts, much like a network bandwidth graph, for some
channels to get an idea of the throughput and to see if sources/sinks are running at same
> From: liuyongbo [mailto:liuyongbo@baidu.com] 
> Sent: Tuesday, May 14, 2013 10:38 PM
> To: user@flume.apache.org
> Subject: how to print the channel capacity 
> Hi:
>          I’m using flume to pass log data to mongodb, but I find that some data lose
when the pressure is in high level, so I want to know the max request that flume can hold
and need to print the capacity.but I can not find the proper way to do this instead of change
the source code. Any ideas?
>          thanks

View raw message