flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Westmacott <peter.westmac...@plumbee.co.uk>
Subject Re: Problem decreasing File Channel capacity
Date Wed, 18 Nov 2015 11:34:49 GMT
Many thanks Ahmed for your swift response.
We'll give that a try.

Peter

On 18 November 2015 at 11:12, Ahmed Vila <avila@devlogic.eu> wrote:

> Hi Peter,
>
> You're right... replaying more events than it can fit to a new channel
> size.
>
> Changing channel size is uncommon configuration change and surely it will
> have it's effects.
> For that reason, I would suggest completely draining the channel and after
> the Flume shut down, remove data and checkpoint dirs for the sake of fresh
> start.
>
>
>
> On Wed, Nov 18, 2015 at 12:06 PM, Peter Westmacott <
> peter.westmacott@plumbee.co.uk> wrote:
>
>> Hello,
>>
>> We’ve got a couple of flume agents running happily (1.5.0). We recently
>> decided to reduce the File Channel capacity (we had way more capacity than
>> we were regularly using and we have plenty of upstream buffering).
>>
>> On one agent this was fine, we adjusted the configuration and restarted
>> the agent. Apart from occasional log messages telling us that the channel
>> was full, we had no issues. (If the sink slows down then we want to
>> communicate the back pressure quickly - so this is by design. Is there a
>> way to tell Flume not to worry about this?)
>>
>> The other agent failed to restart. The error we saw was:
>>
>> 2015-10-05 10:37:27,169 [lifecycleSupervisor-1-0] ERROR
>> (org.apache.flume.channel.file.Log.replay:481)  - Failed to initialize Log
>> on [channel=ebs-1]
>>
>> java.lang.IllegalStateException: Unable to add FlumeEventPointer
>> [fileID=39020, offset=44376807]. Queue depth = 3000, Capacity = 3000
>>
>>         at
>> org.apache.flume.channel.file.ReplayHandler.processCommit(ReplayHandler.java:395)
>>
>>         at
>> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:335)
>>
>>         at org.apache.flume.channel.file.Log.doReplay(Log.java:508)
>>
>>         at org.apache.flume.channel.file.Log.replay(Log.java:462)
>>
>>         at
>> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:279)
>>
>> ...
>>
>> We hypothesised that this was because replaying the Write-Ahead-Log
>> caused the Queue to exceed its new capacity. (This agent had higher
>> throughput than the first.) Our question is this: Is there a recommended
>> approach to reducing a FileChannel capacity with a low (near-zero) chance
>> of error?
>>
>> In the meantime we will attempt to work around this by draining the
>> channel, and waiting for a checkpoint to occur on the empty channel state
>> before reducing the channel capacity.
>>
>> Any thoughts on any of this would be gratefully received. Apologies if we
>> missed a previous answer to any of this in the mailing list archives.
>>
>> Peter Westmacott
>>
>
>
>
> --
>
> Best regards,
> Ahmed Vila | Senior software developer
> DevLogic | Sarajevo | Bosnia and Herzegovina
>
> Office : +387 33 942 123
> Mobile: +387 62 139 348
>
> Website: www.devlogic.eu
> E-mail   : avila@devlogic.eu
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. This email contains confidential information. It should
> not be copied, disclosed to, retained or used by, any party other than the
> intended recipient. Any unauthorised distribution, dissemination or copying
> of this E-mail or its attachments, and/or any use of any information
> contained in them, is strictly prohibited and may be illegal. If you are
> not an intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender directly via email. Any
> emails that you send to us may be monitored by systems or persons other
> than the named communicant for the purposes of ascertaining whether the
> communication complies with the law and company policies.
>
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. This email contains confidential information. It should
> not be copied, disclosed to, retained or used by, any party other than the
> intended recipient. Any unauthorised distribution, dissemination or copying
> of this E-mail or its attachments, and/or any use of any information
> contained in them, is strictly prohibited and may be illegal. If you are
> not an intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender directly via email. Any
> emails that you send to us may be monitored by systems or persons other
> than the named communicant for the purposes of ascertaining whether the
> communication complies with the law and company policies.

Mime
View raw message