Can some one explain the importance of the following
1) Batch Size
1.a) When configured by client code using the flume-core-sdk , to send events to flume avro source.
1.b) When set as a parameter on HDFS sink (or other sinks which support BatchSize parameter)
2.a) Channel Capacity
2.b) Channel Transaction Capacity.
Under which conditions should these params be set to high values, and under which conditions should they be set to low values.
How will setting these parameters to different values, affect throughput, latency in event flow.
Specifically if we have clients with varying frequency of event generation, i.e. some clients generating thousands of events/sec, while
others at a much slower rate, what effect will different values of these params have on these clients ?