Are these the counters from MonitoredCounterGroup? What is the scope of these counters? Can you reset these counters?
From: Joey Echeverria [mailto:firstname.lastname@example.org]
Sent: Monday, January 26, 2015 5:20 PM
Subject: Re: Monitoring the progress of events
Assuming you're using the AvroSource, the application will have to do something like this:
1. Send all of the events to the source
2. Send a specially crafted canary event to the source 3. Monitor the destination where the sink writes looking for the canary event
You might also be able to monitor the counters that the agent maintains to see if the expected number of events have been processed by the sink.
On Mon, Jan 26, 2015 at 2:17 PM, Carlotta Hicks <Carlotta.Hicks@sas.com> wrote:
> I've created a simple agent with a source, a channel and a sink. How can an application know when all events sent to the source have been processed by the sink?
> -----Original Message-----
> From: Joey Echeverria [mailto:email@example.com]
> Sent: Monday, January 26, 2015 5:06 PM
> To: firstname.lastname@example.org
> Subject: Re: Monitoring the progress of events
> @Seshu: If you emailed email@example.com and it still didn't unsubscribe you you should file a JIRA with Apache's infrastructure team:
> What kind of monitoring are you looking for? I don't believe there is event-level monitoring as this would be very expensive. What I've seen some folks do is send a canary event to verify when processing of some batch has completed. This won't work if you have fan out/fan in in the flow as the ordering between multiple Flume agent's isn't guaranteed.
> On Mon, Jan 26, 2015 at 12:52 PM, Seshu V <firstname.lastname@example.org> wrote:
>> Please take my id off of this group. Unsubscribe doesn't seem to be
>> On Mon, Jan 26, 2015 at 2:48 PM, Carlotta Hicks
>>> Is there a way to monitor the progress of events from the time they
>>> are sent to the agent until they have been processed by the sink?
>>> Is there a status for an agent?
> Joey Echeverria