flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yogendra reddy <yogendra...@gmail.com>
Subject Flume log4j Appender issue
Date Wed, 25 Nov 2015 08:33:05 GMT
Hi All,

I'm trying to configure flume to write hadoop service logs to a common sink.

Here's what I have added to hdfs log4j.properties

# Define the root logger to the system property "hadoop.root.logger".
log4j.rootLogger=${hadoop.root.logger}, flume

#Flume Appender
log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = localhost
log4j.appender.flume.Port = 41414

and when I run sample pi job I get this error

$ hadoop jar hadoop-mapreduce-examples.jar pi 10 10
log4j:ERROR Could not find value for key log4j.appender.flume.layout
15/11/25 07:23:26 WARN api.NettyAvroRpcClient: Using default maxIOWorkers
log4j:ERROR RPC client creation failed! NettyAvroRpcClient { host:
localhost, port: 41414 }: RPC connection error
Exception in thread "main" java.lang.ExceptionInInitializerError
        at org.apache.hadoop.util.RunJar.run(RunJar.java:200)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.commons.logging.LogConfigurationException:
User-specified log class 'org.apache.commons.logging.impl.Log4JLogger'
cannot be found or is not useable.
        at
org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:804)
        at
org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541)
        at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292)
        at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657)
        at
org.apache.hadoop.util.ShutdownHookManager.<clinit>(ShutdownHookManager.java:44)
        ... 2 more

I have added these jars to hadoop-hdfs lib -

avro-ipc-1.7.3.jar
flume-avro-source-1.5.2.2.2.7.1-33.jar
flume-ng-log4jappender-1.5.2.2.2.7.1-33.jar
flume-hdfs-sink-1.5.2.2.2.7.1-33.jar
flume-ng-sdk-1.5.2.2.2.7.1-33.jar

and I do have the commons-logging( commons-logging-1.1.3.jar) and
log4j(1.2.17) jars present in the hdfs lib. Any pointers to debug this
issue?

Thanks,
Yogendra

Mime
View raw message