flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander C.H. Lorenz" <wget.n...@googlemail.com>
Subject Re: Flume collectors started crashing regularly all of a sudden
Date Tue, 22 Nov 2011 16:07:51 GMT
That exception will be send when the master-RPC is'nt reachable:
https://svn.apache.org/repos/asf/incubator/flume/trunk/flume-core/src/main/java/com/cloudera/flume/handlers/endtoend/CollectorAckListener.java

- Alex

On Tue, Nov 22, 2011 at 4:58 PM, Jonathan Meed <jmeed@umich.edu> wrote:

> It appears that S3 is working. I can see new flume events getting added in
> my S3 bucket.
>
> Jonathan Meed
> University of Michigan
> School of Engineering Class of 2013
> jmeed@umich.edu
> 917-880-7974
>
>
>
> On Tue, Nov 22, 2011 at 10:55 AM, Alexander C.H. Lorenz <
> wget.null@googlemail.com> wrote:
>
>> Yes, looks like an issue in your S3 instance. Are they running and
>> available?
>>
>> - alex
>>
>>
>> On Tue, Nov 22, 2011 at 4:52 PM, Jonathan Meed <jmeed@umich.edu> wrote:
>>
>>> Hi,
>>>
>>>  I am actually using a multi-sink for HDFS and S3. Could that be the
>>> issues. The config I sued is below.
>>>
>>>
>>> config [beacon_flume02use, autoCollectorSource, [collectorSink("hdfs://
>>> 107.20.248.101/user/flume/beaconlog","syslog" , 30000),
>>> collectorSink("s3n://hooklogic-data-east/flume/%Y/%m/%d/%H","syslog") ],
>>> exec, config, beacon_flume01use, autoCollectorSource,
>>> [collectorSink("hdfs://ip-address/user/flume/beaconlog","syslog" , 30000),
>>> collectorSink("s3n://east/flume/%Y/%m/%d/%H","syslog") ]]
>>>
>>> Thanks
>>>
>>> Jonathan Meed
>>> University of Michigan
>>> School of Engineering Class of 2013
>>> jmeed@umich.edu
>>>  917-880-7974
>>>
>>>
>>>
>>> On Tue, Nov 22, 2011 at 10:50 AM, Alexander C.H. Lorenz <
>>> wget.null@googlemail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> runs on Amazon's S3?
>>>>
>>>> org.jets3t.service.impl.rest.httpclient.RestS3Service: Response
>>>> '/flume%2F2011%2F11%2F20%2F03%2Fsyslog20111121-193628975-0500.2960659853113025.00000025_%24folder%24'
>>>> - Unexpected response code 404, expected 200
>>>>
>>>> Check if the trackers are running.
>>>>
>>>> - alex
>>>>
>>>>
>>>> On Tue, Nov 22, 2011 at 4:45 PM, Jonathan Meed <jmeed@umich.edu> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I had a flume cluster operating pretty well over the last few weeks.
>>>>> In the past two days 1 of my 2 flume collectors started erroring ever
few
>>>>> minutes. It would work fine after a restart moving files for only a few
>>>>> minutes before stopping with the same error. Here's a link to the weblog.
>>>>> Any help would be greatly appreciated.
>>>>>
>>>>> http://pastebin.com/fspsNdYC
>>>>>
>>>>> Jonathan Meed
>>>>> University of Michigan
>>>>> College of Engineering Class of 2013
>>>>> jmeed@umich.edu
>>>>> 917-880-7974
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Alexander Lorenz
>>>> http://mapredit.blogspot.com
>>>>
>>>> *P **Think of the environment: please don't print this email unless
>>>> you really need to.*
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> *P **Think of the environment: please don't print this email unless you
>> really need to.*
>>
>>
>>
>


-- 
Alexander Lorenz
http://mapredit.blogspot.com

*P **Think of the environment: please don't print this email unless you
really need to.*

Mime
View raw message