flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Natkins <na...@streamsets.com>
Subject Re: Flume stops processing event after a while
Date Mon, 14 Jul 2014 17:24:07 GMT
Hi Saravana,

What does your sink configuration look like?

Thanks,
Natty


On Fri, Jul 11, 2014 at 11:05 PM, SaravanaKumar TR <saran0081986@gmail.com>
wrote:

> Assuming each line in the logfile is considered as a event for flume ,
>
> 1.Do we have any maximum size of event defined for memory/file
> channel.like any maximum no of characters in a line.
> 2.Does flume supports all formats of data to be processed as events or do
> we have any limitation.
>
> I am just still trying to understanding why the flume stops processing
> events after sometime.
>
> Can someone please help me out here.
>
> Thanks,
> saravana
>
>
> On 11 July 2014 17:49, SaravanaKumar TR <saran0081986@gmail.com> wrote:
>
>> Hi ,
>>
>> I am new to flume and  using Apache Flume 1.5.0. Quick setup explanation
>> here.
>>
>> Source:exec , tail –F command for a logfile.
>>
>> Channel: tried with both Memory & file channel
>>
>> Sink: HDFS
>>
>> When flume starts , processing events happens properly and its moved to
>> hdfs without any issues.
>>
>> But after sometime flume suddenly stops sending events to HDFS.
>>
>>
>>
>> I am not seeing any errors in logfile flume.log as well.Please let me
>> know if I am missing any configuration here.
>>
>>
>> Below is the channel configuration defined and I left the remaining to be
>> default values.
>>
>>
>> a1.channels.c1.type = FILE
>>
>> a1.channels.c1.transactionCapacity = 100000
>>
>> a1.channels.c1.capacity = 10000000
>>
>> Thanks,
>> Saravana
>>
>>
>>
>>
>>
>

Mime
View raw message