flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Lord <jl...@cloudera.com>
Subject Re: Need for UDP / Multicast Source
Date Thu, 17 Jan 2013 17:59:29 GMT
Hi Andrew,

You may try lowering transactionCapacity here.
The transactionCapacity should be set to the value of the largest batch
size that will be used to store or remove events from that channel. You
currently have it equal to the capacity of the channel. So essentially the
channel *could be* filled with one transaction depending on how you are
batching with your client.

Also it may be useful to turn up jmx monitoring and watch the channel
counters using jconsole. This way you can see exactly how many events are
placed in the channel.

To do this you will need to Set the following Java system properties
located at,


You should than be able to connect with
jconsole hostname:8081


On Thu, Jan 17, 2013 at 9:36 AM, Andrew Otto <otto@wikimedia.org> wrote:

> > I took those two border sequence numbers and extracted all of the log
> lines generated by cp1044 on the nearby host (not using Flume).  I should
> be able to compare the number of lines here with the number of lines in the
> 12 files I extracted from HDFS and Flume.  If they are the same, then Flume
> and UDPSource is working!
> Oh, I meant to link to a Gist with my current flume.conf and the commands
> I executed to investigate this.  Here it is:
> https://gist.github.com/4557178

View raw message