flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlotta Hicks <Carlotta.Hi...@sas.com>
Subject RE: Monitoring the progress of events
Date Mon, 26 Jan 2015 22:17:02 GMT
I've created a simple agent with a source, a channel and a sink.  How can an application know
when all events sent to the source have been processed by the sink?

-----Original Message-----
From: Joey Echeverria [mailto:joey@cloudera.com] 
Sent: Monday, January 26, 2015 5:06 PM
To: user@flume.apache.org
Subject: Re: Monitoring the progress of events

@Seshu: If you emailed user-unsubscribe@flume.apache.org and it still didn't unsubscribe you
you should file a JIRA with Apache's infrastructure team:



What kind of monitoring are you looking for? I don't believe there is event-level monitoring
as this would be very expensive. What I've seen some folks do is send a canary event to verify
when processing of some batch has completed. This won't work if you have fan out/fan in in
the flow as the ordering between multiple Flume agent's isn't guaranteed.


On Mon, Jan 26, 2015 at 12:52 PM, Seshu V <sesh12@gmail.com> wrote:
> Please take my id off of this group.  Unsubscribe doesn't seem to be 
> working.
> thanks.
> On Mon, Jan 26, 2015 at 2:48 PM, Carlotta Hicks 
> <Carlotta.Hicks@sas.com>
> wrote:
>> Is there a way to monitor the progress of events from the time they 
>> are sent to the agent until they have been processed by the sink?  Is 
>> there a status for an agent?

Joey Echeverria
View raw message