There are a few ways to monitor flume in operation. We use the
JSON reporting, which is available via 'http://<agent
address>:<port>/metrics'. You need to start the agent with the
following parameters to get this interface:
We use cacti to graph channel size both as a percentage of maximum and absolute number of events in channel. This provides warning if the sinks cannot keep up with the sources.
We also graph ingress/egress event counts, much like a network bandwidth graph, for some channels to get an idea of the
throughput and to see if sources/sinks are running at same
I’m using flume to
pass log data to mongodb, but I find that some data lose when the pressure is in
high level, so I want to know the max request that flume can hold and need to
print the capacity.but I can not find the proper way to do this instead of
change the source code. Any ideas?