flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mahendran m <mahendra...@hotmail.com>
Subject Question on flume performance tuning
Date Fri, 24 Apr 2015 04:41:30 GMT
Hi ,
I am consuming the message from ActiveMQ source using flume JMS source . and i am cousimg
the message from two queue say SlowerQueue and FasterQueue
SlowerQueue Message incoming rate is 10m/s 
Now i am configure the flume agent with two different configuration
Configuration1	Java memory - 100MB -250MB        Source batch size- 50 event	capacity - 5000event
T.Capacity - 200event	HDFS batch size - 100event
Configuration2
	Java memory - 500MB -700MB
        Source batch size- 100 event	capacity - 10000event
	T.Capacity - 1000event
	HDFS batch size - 500eventQ: for Slower queue which configuration provides better performance
result ?
FasterQueue Message incoming rate is 1000m/s 
Now i am configure the flume agent with two different configuration
Configuration1	Java memory - 100MB -250MB        Source batch size- 50 event	capacity - 5000event
T.Capacity - 200event	HDFS batch size - 100event
Configuration2
	Java memory - 500MB -700MB
        Source batch size- 100 event	capacity - 10000event
	T.Capacity - 1000event
	HDFS batch size - 500eventQ: for Faster queue which configuration provides better performance
result ?
Regards,Mahendran 		 	   		  
Mime
View raw message