flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mahendran m <mahendra...@hotmail.com>
Subject Channel full exception
Date Tue, 21 Apr 2015 07:06:06 GMT
Hi , 
I am using the flume JMS source to dequeue message from ActiveMQ and convert this message
into `List<Event>` using custom converter
Channel configuration
    agent.channels.c1.type = memory    agent.channels.c1.capacity = 1000000    agent.channels.c1.transactionCapacity
= 1500
when size of the `List<Event>` is less than or equal to 1500 (channel transaction capacity),then
flume write the events to channel , but if the event size is greater than 1500 then  i am
getting the below exception 
Error log
    21 Apr 2015 12:19:28,245 WARN  [PollableSourceRunner-JMSSource-s1] (org.apache.flume.source.jms.JMSSource.doProcess:263)
 - Error appending event to channel. Channel might be full. Consider increasing the channel
capacity or make sure the sinks perform faster.    org.apache.flume.ChannelException: Unable
to put batch on required channel: org.apache.flume.channel.MemoryChannel{name: c1}    	at
org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:200)   
	at org.apache.flume.source.jms.JMSSource.doProcess(JMSSource.java:257)    	at org.apache.flume.source.AbstractPollableSource.process(AbstractPollableSource.java:54)
   	at org.apache.flume.source.PollableSourceRunner$PollingRunner.run(PollableSourceRunner.java:139)
   	at java.lang.Thread.run(Thread.java:722)    Caused by: org.apache.flume.ChannelException:
Put queue for MemoryTransaction of capacity 1500 full, consider committing more frequently,
increasing capacity or increasing thread count    	at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doPut(MemoryChannel.java:84)
   	at org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
   	at org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80) 
  	at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
   	... 4 more
How to solve this problem?  Note: Event size varies dynamically basedon ActiveMQ message 
Thanks, 		 	   		  
Mime
View raw message