flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajalakshmi Ramesh <rrajalaks...@gatech.edu>
Subject autoChains and autoCollectorSources load balancing
Date Thu, 16 Feb 2012 17:57:56 GMT
Hi All,
I am trying to understand the behaviour of autoChains and
autoCollectorSource. I understand that the agents are assigned to
collectors by choosing a collector to connect to using consistent hashing
technique. I was wondering if what I am about to describe actually happens
with the current design.

Say I have 3 agents and 2 collectors. The agents are configured as
autoBEChain and the collectors as autoCollectorSouce. Say I add two more
collectors to the flow with autoCollectorSouce. Please note that none of
the collectors have actually failed. I am just increasing the number of
collectors from 2 to 4.  Do the existing agents( a subset of agents) change
their connections to the new collectors? If it does switch collectors, does
this happen for every agent, everytime a new collector is added? . If it
does not switch collectors, then does that mean that the new collectors are
under-utilized.

I was unable to check the behavior myself because of the limited number of
machines i have. I am not able to see a generic pattern because of this
limitation.
Thanks in advance for your help.

Thanks & Regards
Rajalakshmi (Raji) Ramesh
-------------------------------------------------------
Graduate Student
Georgia Institute of Technology
Atlanta, GA
-------------------------------------------------------

Mime
View raw message