flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bao Thai Ngo <baothai...@gmail.com>
Subject Re: E2E mode with decorators
Date Wed, 17 Aug 2011 09:39:33 GMT

As far as I understand ACK mechanism should work regardless any decorator
deployed at Collector as Mingje said. I developed and deployed several
plug-ins (decorators) that filter out events at Collector side and they work
well with ACK. Another thing I can suggest is: do not try to develop an ACK
events part in your decorator.

@Felix: Some advantages for deploying a decorator at collector side are:
- do not depend on agent side
- collect data we need and save other data for future needs (what we need is
just a small part of a very huge data)

just my 2cent.


On Tue, Aug 16, 2011 at 9:52 PM, Joe Crobak <joecrow@gmail.com> wrote:

> According to the Flume FAQ [1], Flume ack's events from the CollectorSink
> in E2E mode.  If I have a Decorator running on the Collector that filters
> out events (or transforms them or something), does that mean those events
> won't get ACK'd and thus will delivery will be retried for them
> indefinitely? IOW, is E2E mode unsupported in this situation -- or maybe is
> there a way for me to ACK events that I want to filter from the Decorator
> itself?
> Thanks,
> Joe
> [1] https://github.com/cloudera/flume/wiki/FAQ

View raw message