flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mahendran m <mahendra...@hotmail.com>
Subject Perfromacnce issue in file channel
Date Sun, 09 Nov 2014 15:48:18 GMT
Hi All ,I have tested the file channel and file channel performance. below the code snippet
to generate 1000 event per sec 
while (watch.Elapsed.TotalSeconds < sec){	if (counter <= 1000)	{		uuid = uuid + 1;
		var text = HttpUtility.UrlEncode("HI , How are you", Encoding.UTF8);
		var flumeEvent = new ThriftFlumeEvent();		flumeEvent.Headers = new Dictionary<string,
string>() { { "Host", "http://www.bing.com/" } };		flumeEvent.Body = Encoding.ASCII.GetBytes(text);
	Client.append(flumeEvent);		counter += 1;	}}
i succeed in sending the 1000 event per sec when using memory channel , but in case if i used
file channel  i can  only reach upto 40 to 50 events per second
config for memory channel# Use memory channela1.channels.c1.type = memorya1.channels.c1.capacity
= 1000000a1.channels.c1.transactionCapacity = 1000a1.channels.c1.byteCapacityBufferPercentage
= 10a1.channels.c1.byteCapacity = 5368709120

config for file channel
#using the file channela1.channels.c1.type = filea1.channels.c1.capacity = 500000a1.channels.c1.transactionCapacity
= 10000a1.channels.c1.useDualCheckpoints = truea1.channels.c1.backupCheckpointDir = F:/.checkPoint
Can anyone explain why file channel is this much channel ? . if i used file channel in server
for directly logging each request is it better option as my server receive 100 to 200 request
per second
Regards,Mahendran

 		 	   		   		 	   		  
Mime
View raw message