flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jimmy <jimmyj...@gmail.com>
Subject Re: JMS source and full file channel
Date Tue, 18 Mar 2014 18:08:47 GMT
Thank you, this is very helpful



On Monday, March 17, 2014, Brock Noland <brock@cloudera.com> wrote:

> When the channel is full, the JMSSource returns "BACKOFF" to
> PollableSourceRunner which implements backoff logic which can be seen here:
>
>
> https://github.com/apache/flume/blob/trunk/flume-ng-core/src/main/java/org/apache/flume/source/PollableSourceRunner.java#L139
>
>
> On Mon, Mar 17, 2014 at 7:38 PM, Jimmy <jimmyjack@gmail.com<javascript:_e(%7B%7D,'cvml','jimmyjack@gmail.com');>
> > wrote:
>
>> I have JMS source and file channel with HDFS sink... so far works good
>>
>> lets assume I hit channel capacity (for whatever reason e.g. HDFS down)
>> and channel stops accepting new messages from source....
>>
>> JMS is going to pull a message from queue and will try to delivery to
>> channel, but cannot, it will rollback. Again, works...
>>
>> Now if the channel is full for a while, am I going to see JMS pulling
>> messages from queue like crazy monkey just to roll them back few ms later
>> or is there some timeout-retry window e.g. "if JMS source cannot delivery 3
>> messages to channel, wait 30 seconds they retry again"
>>
>> thank you
>>
>>
>>
>>
>

Mime
View raw message