flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bhaskar V. Karambelkar" <bhaska...@gmail.com>
Subject Of BatchSize / Channel Capacity / Transaction Capacity
Date Tue, 08 Jan 2013 18:46:17 GMT
Can some one explain the importance of the following
1) Batch Size
  1.a) When configured by client code using the flume-core-sdk , to send
events to flume avro source.
  1.b) When set as a parameter on HDFS sink (or other sinks which support
BatchSize parameter)
  2.a) Channel Capacity
  2.b) Channel Transaction Capacity.

Under which conditions should these params be set to high values, and under
which conditions should they be set to low values.

How will setting these parameters to different values, affect throughput,
latency in event flow.
Specifically if we have clients with varying frequency of event generation,
i.e. some clients generating thousands of events/sec, while
others at a much slower rate, what effect will different values of these
params have on these clients ?


View raw message