flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan <jonny2...@gmail.com>
Subject Re: Having trouble writing to HDFS
Date Wed, 28 Sep 2011 15:47:05 GMT
If It helps, here is my log file,
2011-09-28 11:31:38,923 INFO com.cloudera.flume.handlers.rolling.RollSink:
Created RollSink: trigger=[TimeTrigger: maxAge=30000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@162e07d]
checkPeriodM$
2011-09-28 11:31:43,933 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 1 heartbeats
2011-09-28 11:31:48,943 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 2 heartbeats
2011-09-28 11:31:48,943 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 3 heartbeats
2011-09-28 11:31:53,953 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 4 heartbeats
2011-09-28 11:31:53,953 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 5 heartbeats
2011-09-28 11:31:58,963 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 6 heartbeats
2011-09-28 11:31:58,963 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 7 heartbeats
2011-09-28 11:31:59,473 WARN com.cloudera.flume.conf.FlumeBuilder:
Deprecated syntax: Expected a format spec but instead had a (String)
avrojson
2011-09-28 11:32:03,973 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 8 heartbeats
2011-09-28 11:32:03,973 WARN com.cloudera.flume.agent.LivenessManager:
Heartbeats are backing up, currently behind by 9 heartbeats
2011-09-28 11:32:08,932 ERROR com.cloudera.flume.agent.LogicalNode: Forcing
driver to exit uncleanly
2011-09-28 11:32:08,932 ERROR
com.cloudera.flume.core.connector.DirectDriver: Closing down due to
exception during append calls
java.io.IOException: Waiting for queue element was interrupted! null
        at
com.cloudera.flume.handlers.thrift.ThriftEventSource.next(ThriftEventSource.java:222)
        at
com.cloudera.flume.collector.CollectorSource.next(CollectorSource.java:72)
        at
com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:105)
Caused by: java.lang.InterruptedException
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2038)
        at
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
        at
com.cloudera.flume.handlers.thrift.ThriftEventSource.next(ThriftEventSource.java:209)
        ... 2 more
2011-09-28 11:32:08,933 INFO com.cloudera.flume.core.connector.DirectDriver:
Connector logicalNode collector1-2137 exited with error: Waiting for queue
element was interrupted! null
java.io.IOException: Waiting for queue element was interrupted! null
        at
com.cloudera.flume.handlers.thrift.ThriftEventSource.next(ThriftEventSource.java:222)
        at
com.cloudera.flume.collector.CollectorSource.next(CollectorSource.java:72)
        at
com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:105)
Caused by: java.lang.InterruptedException
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2038)
        at
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
        at
com.cloudera.flume.handlers.thrift.ThriftEventSource.next(ThriftEventSource.java:209)
        ... 2 more
2011-09-28 11:32:08,933 INFO com.cloudera.flume.collector.CollectorSource:
closed
2011-09-28 11:32:08,933 INFO com.cloudera.flume.agent.LogicalNode: Node
config successfully set to com.cloudera.flume.conf.FlumeConfigData@1348e2f
2011-09-28 11:32:08,936 INFO
com.cloudera.flume.handlers.thrift.ThriftEventSource: Closed server on port
35853...
2011-09-28 11:32:08,936 INFO
com.cloudera.flume.handlers.thrift.ThriftEventSource: Queue still has 0
elements ...
2011-09-28 11:32:08,936 INFO com.cloudera.flume.handlers.rolling.RollSink:
closing RollSink 'escapedCustomDfs("hdfs://
ec2-174-129-89-0.compute-1.amazonaws.com:54310/user/flume/","syslog%{rolltag}"
)'
2011-09-28 11:32:08,937 ERROR
com.cloudera.flume.core.connector.DirectDriver: Exiting driver logicalNode
collector1-2137 in error state CollectorSource | Collector because Waiting
for queue element was inte$
2011-09-28 11:32:08,937 INFO com.cloudera.flume.collector.CollectorSource:
opened
2011-09-28 11:32:08,937 INFO
com.cloudera.flume.handlers.thrift.ThriftEventSource: Starting blocking
thread pool server on port 35853...
2011-09-28 11:32:08,937 INFO com.cloudera.flume.handlers.rolling.RollSink:
opening RollSink
'escapedCustomDfs("hdfs://10.116.98.79:54310/user/flume/","syslog%{rolltag}"
)'
2011-09-28 11:32:08,940 WARN com.cloudera.flume.conf.FlumeBuilder:
Deprecated syntax: Expected a format spec but instead had a (String)
avrojson
2011-09-28 11:32:08,940 INFO
com.cloudera.flume.handlers.debug.InsistentOpenDecorator: Opened
MaskDecorator on try 0

Jonathan


On Wed, Sep 28, 2011 at 11:16 AM, Jonathan <jonny2112@gmail.com> wrote:

> To the best of my knowledge they are both running the same hdfs version. I
> installed both from Cloudera's CDH
>
> Jonathan
>
>
>
> On Wed, Sep 28, 2011 at 11:12 AM, Justin Workman <justinjworkman@gmail.com
> > wrote:
>
>> I have also seem this happen if the hadoop version on the collector nodes
>> is a different version than the hdfs version you are writhing to.
>>
>> Sent from my iPhone
>>
>> On Sep 28, 2011, at 8:40 AM, Jonathan <jonny2112@gmail.com> wrote:
>>
>> Hi,
>>
>> Yeah, that doesn't seem to help. Thanks for trying though.
>>
>> Jonathan
>>
>>
>> On Wed, Sep 28, 2011 at 10:35 AM, steve layland <<stevieplayland@gmail.com>
>> stevieplayland@gmail.com> wrote:
>>
>>> 54310
>>
>>
>>
>

Mime
View raw message