flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeong-shik Jang <jsja...@gmail.com>
Subject Re: Interceptors on a replicated source fan-out
Date Thu, 16 Jun 2016 10:10:20 GMT
Hi Ha Thai,

I think implementing a sink wrapper which allows you to use interceptors in
sink side can be an option.
Actually, I implemented one and have been using it successfully in our
Possible performance gain by leveraging multi-threaded processing of sinks
is also a benefit of this approach, which was the motivation that drove me
to implement it.

If you want to try to implement by yourself, I can give you some idea; my
company has plan to make it open source; but it doesn't happen yet.


2016-06-16 17:53 GMT+09:00 Chris Horrocks <chris@hor.rocks>:

> Interceptors are configured per source, to achieve the result you describe
> it would be easier to apply the interceptor to the source of the receiving
> agent. The alternative would be to use the morphline interceptor and write
> some conditional logic, but I'm not sure if the current interceptor
> implementations would allow you to conditionally bind certain events to a
> specific channel. I suppose you could always write a custom interceptor to
> do it.
> --
> Chris Horrocks
> From: Hai Thai <hai@mailtrust.com> <hai@mailtrust.com>
> Reply: user@flume.apache.org <user@flume.apache.org>
> <user@flume.apache.org>
> Date: 15 June 2016 at 22:40:07
> To: user@flume.apache.org <user@flume.apache.org> <user@flume.apache.org>
> Subject:  Interceptors on a replicated source fan-out
> When doing a replicated fan-out of a source, is there any way to apply an
> i=
> nterceptor to just 1 destination channel of that fan out while having the
> o=
> ther channels unaffected?
> -H

View raw message