flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Ortiz <konstt2...@gmail.com>
Subject Re: Load Balancing.
Date Fri, 12 Sep 2014 06:37:11 GMT
Any documentation or more details of how works??

2014-09-11 13:53 GMT+02:00 Guillermo Ortiz <konstt2000@gmail.com>:

> I guess that there're many open files open in the same time. What means
> that each event it's delivered some sink randomly.
>
> 2014-09-11 10:27 GMT+02:00 Guillermo Ortiz <konstt2000@gmail.com>:
>
>> I checked, you're right, is there more documentation about it somewhere?
>> How does it work? I mean, if I choose random, it choose randomly a sink
>> and when it finishes to generate a file and choose another sink or it send
>> each event for each sink and really there are many sink generating files
>> simultaneously?
>>
>>
>> 2014-09-11 3:08 GMT+02:00 Ashish <paliwalashish@gmail.com>:
>>
>>> IMHO,
>>> http://flume.apache.org/FlumeUserGuide.html#load-balancing-sink-processor
>>> explains it well. Failover Sink processor is used in case of failure event,
>>> but Load balancing Sink processor would be used to choose the Sink for
>>> sending Events.
>>>
>>> Can try this with a simple topology with two File Roll sinks, configured
>>> to use Load Balancing Sink Processor.
>>>
>>> On Wed, Sep 10, 2014 at 9:22 PM, Guillermo Ortiz <konstt2000@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I want to use the load balancer in Flume. After reading Flume
>>>> documentation about loading balancing I think that it's not really load
>>>> balacing, if I understood well, when a sink fail, it chooses a new sink
>>>> with ronud-robin or in a random way. But it doesn't distributed on this way
>>>> all the time among the differents sink.
>>>>
>>>> Is there any possiblity of distributed the load among all the sinks?
>>>> Because if I undertood well, the load balancing option doesn't do it.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> thanks
>>> ashish
>>>
>>> Blog: http://www.ashishpaliwal.com/blog
>>> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>>>
>>
>>
>

Mime
View raw message