flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arvind Prabhakar <arv...@apache.org>
Subject Re: what are the libraries needed for flume log4jappender
Date Wed, 23 Jan 2013 22:06:56 GMT
I used the standard maven dependency plugin to get this information.

$ mvn dependency:tree

This will list out all the dependencies no matter what their scope is. But
you can safely ignore dependencies that have a test scope.

Regards,
Arvind Prabhakar

On Wed, Jan 23, 2013 at 1:39 PM, yogi nerella <ynerella999@gmail.com> wrote:

> Arvind,
>
> Thanks for the information, is there any dependency walker tool or do we
> have to do this manually?
> I have gone thru the pom.xml, which lists junit and others also as
> dependency, so how to avoid some of them which are unnecessary.
>
> Yogi
>
>
> On Wed, Jan 23, 2013 at 1:17 PM, Arvind Prabhakar <arvind@apache.org>wrote:
>
>> Here are the dependencies that maven shows for Flume's log4j appender
>> (generated from the 1.3.0 branch):
>>
>> org.apache.flume.flume-ng-clients:flume-ng-log4jappender:jar:1.3.1:
>>
>>    - log4j:log4j:jar:1.2.16
>>    - org.apache.flume:flume-ng-sdk:jar:1.3.1
>>       - org.apache.avro:avro:jar:1.7.2
>>          - org.codehaus.jackson:jackson-core-asl:jar:1.9.3
>>          - org.codehaus.jackson:jackson-mapper-asl:jar:1.9.3
>>          - com.thoughtworks.paranamer:paranamer:jar:2.3
>>          - org.xerial.snappy:snappy-java:jar:1.0.4.1
>>          - org.slf4j:slf4j-api:jar:1.6.1
>>       -  org.apache.avro:avro-ipc:jar:1.7.2
>>          - org.mortbay.jetty:jetty:jar:6.1.26
>>          - org.mortbay.jetty:jetty-util:jar:6.1.26
>>          - org.apache.velocity:velocity:jar:1.7
>>             - commons-collections:commons-collections:jar:3.2.1
>>             - commons-lang:commons-lang:jar:2.5
>>          - io.netty:netty:jar:3.4.0.Final
>>
>>
>> Regards,
>> Arvind Prabhakar
>>
>> On Wed, Jan 23, 2013 at 12:44 PM, Mike Percy <mpercy@apache.org> wrote:
>>
>>> If you want only the required library files, use Maven to pull in the
>>> exact set of dependencies.
>>>
>>>
>>> On Wed, Jan 23, 2013 at 12:24 PM, yogi nerella <ynerella999@gmail.com>wrote:
>>>
>>>> Mike,
>>>>
>>>> When I include all the libraries it is not giving compilation problem,
>>>> but log4jappender documentation says it only needs flume-ng-sdk.jar file.
>>>>
>>>> In that case, if I want to ship flume log4jappender, should I have to
>>>> ship all these jar files in flume/lib directory?
>>>>
>>>> Yogi
>>>>
>>>>
>>>> On Wed, Jan 23, 2013 at 12:08 PM, Mike Percy <mpercy@apache.org> wrote:
>>>>
>>>>> I don't use Eclipse but my understanding is that mvn eclipse:eclipse
>>>>> will create a usable Eclipse project file which you can import. the
>>>>> AvroFlumeEvent class is automatically generated at build time by the
maven
>>>>> avro compiler.
>>>>>
>>>>> If you want to avoid maven entirely, you have the option of pulling in
>>>>> all the jar files in the lib directory of the Flume binary archive. Those
>>>>> are all the dependencies Flume needs to run.
>>>>>
>>>>> Hope that helps!
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 23, 2013 at 12:03 PM, yogi nerella <ynerella999@gmail.com>wrote:
>>>>>
>>>>>> Mike,
>>>>>>
>>>>>> I am importing the project into eclipse using maven, but the project
>>>>>> has issues in the legacy-sources module.
>>>>>>
>>>>>> Description Resource Path Location Type
>>>>>> AvroFlumeEvent cannot be resolved to a type NettyAvroRpcClient.java
>>>>>> /flume-ng-sdk/src/main/java/org/apache/flume/api line 189 Java
>>>>>> Problem
>>>>>>
>>>>>>
>>>>>> Runtime issues are with 1.3.1
>>>>>> 1.   If I include just ng-core, I run into issues of FlumeException
>>>>>> not found.
>>>>>>
>>>>>> Classpath:
>>>>>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar
>>>>>>  log4j:WARN Error during default initialization
>>>>>> java.lang.NoClassDefFoundError: org/apache/flume/FlumeException
>>>>>> at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>>>>  at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
>>>>>> at java.lang.Class.getConstructor0(Unknown Source)
>>>>>>  at java.lang.Class.newInstance0(Unknown Source)
>>>>>>
>>>>>>
>>>>>> 2.   If I include ng-core, and ng-sdk, I run into the following
>>>>>> exception.
>>>>>> Classpath:
>>>>>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-sdk-1.3.1.jar
>>>>>>  log4j:WARN Error during default initialization
>>>>>> java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
>>>>>> at
>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
>>>>>>  at
>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
>>>>>> at
>>>>>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
>>>>>>
>>>>>>
>>>>>> Appreciate your help,
>>>>>> Yogi
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 23, 2013 at 11:54 AM, Mike Percy <mpercy@apache.org>wrote:
>>>>>>
>>>>>>> Yogi, Flume has lots of dependencies. You can't just drop in
one
>>>>>>> jar. That is what Maven is for.
>>>>>>>
>>>>>>> Regards
>>>>>>> Mike
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 23, 2013 at 11:51 AM, yogi nerella <
>>>>>>> ynerella999@gmail.com> wrote:
>>>>>>>
>>>>>>>> HI,
>>>>>>>>
>>>>>>>> I have tried the 1.3.0 and ran into issues, so I am building
from
>>>>>>>> the latest source and still has the same issue.
>>>>>>>>
>>>>>>>> Yogi
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jan 23, 2013 at 11:36 AM, Mike Percy <mpercy@apache.org>wrote:
>>>>>>>>
>>>>>>>>> What version of Flume are you using? Are you using Maven
for your
>>>>>>>>> build?
>>>>>>>>>
>>>>>>>>> You should be able to get away with just flume-ng-core.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jan 23, 2013 at 10:02 AM, yogender nerella <
>>>>>>>>> ynerella@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I would like to make my app directly write events
to an flume
>>>>>>>>>> agent.
>>>>>>>>>>
>>>>>>>>>> What are the libraries needed to add into my application?
>>>>>>>>>>
>>>>>>>>>> I have tried to add the following three but runs
into class not
>>>>>>>>>> found errors.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> log4j.properties are as follows:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> log4j.rootLogger=TRACE, flume
>>>>>>>>>> log4j.appender.flume =
>>>>>>>>>> org.apache.flume.clients.log4jappender.Log4jAppender
>>>>>>>>>> log4j.appender.flume.Hostname = example.com
>>>>>>>>>> log4j.appender.flume.Port = 44444
>>>>>>>>>> log4j.appender.flume.layout=org.apache.log4j.TTCCLayout
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Error In Initialization
>>>>>>>>>>
>>>>>>>>>> Classpath:
>>>>>>>>>> C:\\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\test\flume\flume-ng-core-1.4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-log4jappender-1.4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-sdk-1.4.0-SNAPSHOT.jar
>>>>>>>>>> log4j:WARN Error during default initialization
>>>>>>>>>> java.lang.NoClassDefFoundError:
>>>>>>>>>> org/jboss/netty/channel/ChannelFactory
>>>>>>>>>> at
>>>>>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
>>>>>>>>>> at
>>>>>>>>>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:195)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>>>>>>>>>> at
>>>>>>>>>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
>>>>>>>>>> at
>>>>>>>>>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
>>>>>>>>>> at
>>>>>>>>>> org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
>>>>>>>>>> at
>>>>>>>>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
>>>>>>>>>>  at
>>>>>>>>>> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
>>>>>>>>>> at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
>>>>>>>>>>  at org.apache.log4j.Logger.getLogger(Logger.java:117)
>>>>>>>>>> at LogTest.<init>(LogTest.java:7)
>>>>>>>>>>  at LogTest.main(LogTest.java:11)
>>>>>>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>>>>>> org.jboss.netty.channel.ChannelFactory
>>>>>>>>>> at java.net.URLClassLoader$1.run(Unknown Source)
>>>>>>>>>>  at java.net.URLClassLoader$1.run(Unknown Source)
>>>>>>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>>  at java.net.URLClassLoader.findClass(Unknown Source)
>>>>>>>>>> at java.lang.ClassLoader.loadClass(Unknown Source)
>>>>>>>>>>  at sun.misc.Launcher$AppClassLoader.loadClass(Unknown
Source)
>>>>>>>>>> at java.lang.ClassLoader.loadClass(Unknown Source)
>>>>>>>>>>  ... 16 more
>>>>>>>>>> log4j:WARN No appenders could be found for logger
(LogTest).
>>>>>>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>>>>>>> log4j:WARN See
>>>>>>>>>> http://logging.apache.org/log4j/1.2/faq.html#noconfig
for more
>>>>>>>>>> info.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Appreciate your help,
>>>>>>>>>> Yogi
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message