flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Giguere Villegas <felix.gigu...@mate1inc.com>
Subject Re: E2E mode with decorators
Date Tue, 16 Aug 2011 14:59:35 GMT
Maybe I'm missing something, but why don't you put your filtering Decorator
on the agent/source instead?

What's the point of sending those events all the way to the CollectorSink if
they're going to be filtered out in the end? The only reason I can see is if
the only place where you can easily determine which events to filter out IS
at the end of the flow, but I can't think of a reason why that would be the

I don't know the answer to your specific question (and I'd be curious to
find out as well), so I'm sorry if my comment doesn't help :) ...


On Tue, Aug 16, 2011 at 10:52 AM, Joe Crobak <joecrow@gmail.com> wrote:

> According to the Flume FAQ [1], Flume ack's events from the CollectorSink
> in E2E mode.  If I have a Decorator running on the Collector that filters
> out events (or transforms them or something), does that mean those events
> won't get ACK'd and thus will delivery will be retried for them
> indefinitely? IOW, is E2E mode unsupported in this situation -- or maybe is
> there a way for me to ACK events that I want to filter from the Decorator
> itself?
> Thanks,
> Joe
> [1] https://github.com/cloudera/flume/wiki/FAQ

View raw message