flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Delete individual message from queue
Date Wed, 04 Feb 2015 13:32:34 GMT
Is it possible to extend File Channel Integrity tool to support
filtering out corrupt events? Something like once we get a record and
it's not corrupt, provide a placeholder function to validate Event
implemented by user. I still don't have too much insight into
FileChannel implementation.

On Tue, Feb 3, 2015 at 12:49 AM, Hari Shreedharan
<hshreedharan@cloudera.com> wrote:
> Currently, no - there is no such tool, but this is a request that has come
> up time and again. Can you file a jira for this? If someone has time, they’d
> probably pick it up
> Thanks,
> Hari
> On Mon, Feb 2, 2015 at 10:52 AM, Charles McLaughlin <charles@nextdoor.com>
> wrote:
>> Hello,
>> We had a situation where one of our Flume agents got stuck on a message
>> due to unexpected format.  To get things moving again, I stopped the Flume
>> agent, moved the file backed channel data out of the way and re-started the
>> Flume agent.  I'd like to pop the bad message from the queue data on disk...
>> are there any tools or recommended ways to do this?
>> Thanks,
>> Charles


Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

View raw message