flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Lord <jl...@cloudera.com>
Subject Re: Configuring flume agents remotely
Date Fri, 03 Jan 2014 14:34:03 GMT
Monitoring and configuration are two separate things here.
Flume is typically monitored using either ganglia or http/json.
Both methods are documented here:
http://flume.apache.org/FlumeUserGuide.html#monitoring

As for configuration management and changes a common way of handling this
would be to
use a configuration management tool such as chef, puppet or ansible.
The agent polls the conf file every 30 seconds and if changes are detected
than the necessary components will be dynamically reloaded.

Not sure why Kafka is being mentioned as that is an entirely separate
project which has nothing to do with flume.


On Fri, Jan 3, 2014 at 3:19 AM, yavuz gokirmak <ygokirmak@gmail.com> wrote:

> Hi all,
>
> We need to monitor and configure flume agents from a centralized tool. In
> previous versions as I know flume-master is used for such kind of
> requirements. Flume-master does not exist in flume-ng project.
>
> What is the adviced way for monitoring and configuring flume agents
> remotely *
>
> Assume I don't have ssh access to agent machines and therefore I need to
> get summary information such as #of events processed, #of files read etc..
> from running agents continuously
>
> And change agent configurations dynamically&remotely.
>
> regards
>

Mime
View raw message