flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ketaki Gadre." <ketaki_ga...@yahoo.co.in>
Subject Re: Complex topology in flume: reducing no. of channels and sinks
Date Thu, 05 Dec 2013 07:51:59 GMT

Just wanted to check if anybody has any suggestion on this.

Ketaki Gadre

On Friday, 29 November 2013 4:11 PM, Ketaki Gadre. <ketaki_gadre@yahoo.co.in> wrote:

In my use-case I need to send n log files on a machine (sender) to m different machines (receiver).
These receiver machines process each file differently and there are multiple such senders.
As I read in the User Guide, currently Flume supports channel selector i.e. data can be multiplexed/replicated
at the source level not at the channel level. This means that, to support above scenario,
I will need to have n sources, n*m channels and n*m sinks in the flume agent at each sender
machine. I want to reduce the number of channels as they will have replicated data and will
take a lot of space.

Does the current version of Flume supportĀ multiplexing/replicating at channel level so that
I can have reduce no. of channels? Also is there a way in which I can reduce no. of sinks
as well in this scenario?

Can you please suggest?

Ketaki Gadre
View raw message