From user-return-3476-apmail-flume-user-archive=flume.apache.org@flume.apache.org Wed Jan 23 20:45:12 2013 Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3AD8AED26 for ; Wed, 23 Jan 2013 20:45:12 +0000 (UTC) Received: (qmail 26547 invoked by uid 500); 23 Jan 2013 20:45:12 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 26502 invoked by uid 500); 23 Jan 2013 20:45:11 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 26493 invoked by uid 99); 23 Jan 2013 20:45:11 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 20:45:11 +0000 Received: from localhost (HELO mail-la0-f48.google.com) (127.0.0.1) (smtp-auth username mpercy, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 20:45:11 +0000 Received: by mail-la0-f48.google.com with SMTP id fq13so4221558lab.35 for ; Wed, 23 Jan 2013 12:45:09 -0800 (PST) X-Received: by 10.112.8.129 with SMTP id r1mr1272013lba.127.1358973909134; Wed, 23 Jan 2013 12:45:09 -0800 (PST) MIME-Version: 1.0 Received: by 10.114.16.195 with HTTP; Wed, 23 Jan 2013 12:44:28 -0800 (PST) In-Reply-To: References: From: Mike Percy Date: Wed, 23 Jan 2013 12:44:28 -0800 Message-ID: Subject: Re: what are the libraries needed for flume log4jappender To: "user@flume.apache.org" Content-Type: multipart/alternative; boundary=e0cb4efe330c1ceea004d3fac509 --e0cb4efe330c1ceea004d3fac509 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable If you want only the required library files, use Maven to pull in the exact set of dependencies. On Wed, Jan 23, 2013 at 12:24 PM, yogi nerella wrote= : > Mike, > > When I include all the libraries it is not giving compilation problem, bu= t > log4jappender documentation says it only needs flume-ng-sdk.jar file. > > In that case, if I want to ship flume log4jappender, should I have to shi= p > all these jar files in flume/lib directory? > > Yogi > > > On Wed, Jan 23, 2013 at 12:08 PM, Mike Percy wrote: > >> I don't use Eclipse but my understanding is that mvn eclipse:eclipse wil= l >> create a usable Eclipse project file which you can import. the >> AvroFlumeEvent class is automatically generated at build time by the mav= en >> avro compiler. >> >> If you want to avoid maven entirely, you have the option of pulling in >> all the jar files in the lib directory of the Flume binary archive. Thos= e >> are all the dependencies Flume needs to run. >> >> Hope that helps! >> >> >> >> On Wed, Jan 23, 2013 at 12:03 PM, yogi nerella wr= ote: >> >>> Mike, >>> >>> I am importing the project into eclipse using maven, but the project ha= s >>> issues in the legacy-sources module. >>> >>> Description Resource Path Location Type >>> AvroFlumeEvent cannot be resolved to a type NettyAvroRpcClient.java >>> /flume-ng-sdk/src/main/java/org/apache/flume/api line 189 Java Problem >>> >>> >>> Runtime issues are with 1.3.1 >>> 1. If I include just ng-core, I run into issues of FlumeException not >>> found. >>> >>> Classpath: >>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.ja= r;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\ap= ache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar >>> log4j:WARN Error during default initialization >>> java.lang.NoClassDefFoundError: org/apache/flume/FlumeException >>> at java.lang.Class.getDeclaredConstructors0(Native Method) >>> at java.lang.Class.privateGetDeclaredConstructors(Unknown Source) >>> at java.lang.Class.getConstructor0(Unknown Source) >>> at java.lang.Class.newInstance0(Unknown Source) >>> >>> >>> 2. If I include ng-core, and ng-sdk, I run into the following >>> exception. >>> Classpath: >>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.ja= r;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\ap= ache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar;C:\apache\apache-= flume-1.3.1-bin\lib\flume-ng-sdk-1.3.1.jar >>> log4j:WARN Error during default initialization >>> java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory >>> at >>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFacto= ry.java:168) >>> at >>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFacto= ry.java:128) >>> at >>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Lo= g4jAppender.java:184) >>> >>> >>> Appreciate your help, >>> Yogi >>> >>> >>> >>> On Wed, Jan 23, 2013 at 11:54 AM, Mike Percy wrote: >>> >>>> Yogi, Flume has lots of dependencies. You can't just drop in one jar. >>>> That is what Maven is for. >>>> >>>> Regards >>>> Mike >>>> >>>> >>>> On Wed, Jan 23, 2013 at 11:51 AM, yogi nerella = wrote: >>>> >>>>> HI, >>>>> >>>>> I have tried the 1.3.0 and ran into issues, so I am building from the >>>>> latest source and still has the same issue. >>>>> >>>>> Yogi >>>>> >>>>> >>>>> On Wed, Jan 23, 2013 at 11:36 AM, Mike Percy wrote= : >>>>> >>>>>> What version of Flume are you using? Are you using Maven for your >>>>>> build? >>>>>> >>>>>> You should be able to get away with just flume-ng-core. >>>>>> >>>>>> >>>>>> On Wed, Jan 23, 2013 at 10:02 AM, yogender nerella < >>>>>> ynerella@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I would like to make my app directly write events to an flume agent= . >>>>>>> >>>>>>> What are the libraries needed to add into my application? >>>>>>> >>>>>>> I have tried to add the following three but runs into class not >>>>>>> found errors. >>>>>>> >>>>>>> >>>>>>> log4j.properties are as follows: >>>>>>> >>>>>>> >>>>>>> log4j.rootLogger=3DTRACE, flume >>>>>>> log4j.appender.flume =3D >>>>>>> org.apache.flume.clients.log4jappender.Log4jAppender >>>>>>> log4j.appender.flume.Hostname =3D example.com >>>>>>> log4j.appender.flume.Port =3D 44444 >>>>>>> log4j.appender.flume.layout=3Dorg.apache.log4j.TTCCLayout >>>>>>> >>>>>>> >>>>>>> Error In Initialization >>>>>>> >>>>>>> Classpath: >>>>>>> C:\\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\test\= flume\flume-ng-core-1.4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-log4jappender= -1.4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-sdk-1.4.0-SNAPSHOT.jar >>>>>>> log4j:WARN Error during default initialization >>>>>>> java.lang.NoClassDefFoundError: >>>>>>> org/jboss/netty/channel/ChannelFactory >>>>>>> at >>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientF= actory.java:168) >>>>>>> at >>>>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientF= actory.java:128) >>>>>>> at >>>>>>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOption= s(Log4jAppender.java:195) >>>>>>> at >>>>>>> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java= :307) >>>>>>> at >>>>>>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter= .java:172) >>>>>>> at >>>>>>> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter= .java:104) >>>>>>> at >>>>>>> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigu= rator.java:809) >>>>>>> at >>>>>>> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigu= rator.java:735) >>>>>>> at >>>>>>> org.apache.log4j.PropertyConfigurator.configureRootCategory(Propert= yConfigurator.java:615) >>>>>>> at >>>>>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigura= tor.java:502) >>>>>>> at >>>>>>> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigura= tor.java:547) >>>>>>> at >>>>>>> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionC= onverter.java:483) >>>>>>> at org.apache.log4j.LogManager.(LogManager.java:127) >>>>>>> at org.apache.log4j.Logger.getLogger(Logger.java:117) >>>>>>> at LogTest.(LogTest.java:7) >>>>>>> at LogTest.main(LogTest.java:11) >>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>> org.jboss.netty.channel.ChannelFactory >>>>>>> at java.net.URLClassLoader$1.run(Unknown Source) >>>>>>> at java.net.URLClassLoader$1.run(Unknown Source) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(Unknown Source) >>>>>>> at java.lang.ClassLoader.loadClass(Unknown Source) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) >>>>>>> at java.lang.ClassLoader.loadClass(Unknown Source) >>>>>>> ... 16 more >>>>>>> log4j:WARN No appenders could be found for logger (LogTest). >>>>>>> log4j:WARN Please initialize the log4j system properly. >>>>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfi= gfor more info. >>>>>>> >>>>>>> >>>>>>> Appreciate your help, >>>>>>> Yogi >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > --e0cb4efe330c1ceea004d3fac509 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
If you want only the required library files, use Maven to = pull in the exact set of dependencies.

=
On Wed, Jan 23, 2013 at 12:24 PM, yogi nerel= la <ynerella999@gmail.com> wrote:
Mike,

Wh= en I include all the libraries it is not giving compilation problem, but lo= g4jappender documentation says it only needs flume-ng-sdk.jar file. =C2=A0= =C2=A0

In that case, if I want to ship flume log4jappender, sh= ould I have to ship all these jar files in flume/lib directory? =C2=A0=C2= =A0

Yogi


On Wed, Jan 23, 2013 at 12:08 PM, Mike Percy <= mpercy@apache.org> wrote:
I don't use Eclipse but= my understanding is that mvn eclipse:eclipse will create a usable Eclipse = project file which you can import. the AvroFlumeEvent class is automaticall= y generated at build time by the maven avro compiler.

If you want to avoid maven entirely, you have the option of = pulling in all the jar files in the lib directory of the Flume binary archi= ve. Those are all the dependencies Flume needs to run.

Hope that helps!



On Wed, Jan 23, 2013 at 12:03 = PM, yogi nerella <ynerella999@gmail.com> wrote:
Mike,

I = am importing the project into eclipse using maven, but the project has issu= es in the legacy-sources module.

Description Resource Path Location <= /span>Type
AvroFlumeEvent cannot be resolved to a type NettyAvroRpcClient.java /flume-ng-sdk/src/main/java/org/apache/flume/api line 189 Java Problem


Runtime issues are with 1.3.1
= 1. =C2=A0 If I include just ng-core, I run into issues of FlumeException no= t found.

Classpath: C:\ServiceManager\workspa= ce\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.= 1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flum= e-ng-log4jappender-1.3.1.jar
log4j:WARN Error during default initialization
java.la= ng.NoClassDefFoundError: org/apache/flume/FlumeException
at java.lang.Class.getDeclaredConstruc= tors0(Native Method)
at java.lang.Class.privat= eGetDeclaredConstructors(Unknown Source)
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.newIns= tance0(Unknown Source)


2. =C2=A0 If= I include ng-core, and ng-sdk, I run into the following exception.
Classpath: C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\tes= t\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1= .jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar;= C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-sdk-1.3.1.jar
log4j:WARN Error during default initialization
java.lang.NoC= lassDefFoundError: org/jboss/netty/channel/ChannelFactory
at org.apache.flume.api.RpcClientFact= ory.getDefaultInstance(RpcClientFactory.java:168)
at org.apache.flume.api.R= pcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
at org.apache.flume.clients= .log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)


Appreciate your help,
<= div>Yogi



On Wed, Jan 23, 2013 at 11:54 AM, Mik= e Percy <mpercy@apache.org> wrote:
Yogi,=C2=A0Flume has lots o= f dependencies. You can't just drop in one jar. That is what Maven is f= or.

Regards
Mike


On Wed, Jan 23, 2013 at 11:51 AM, yogi nerella <ynerella999@gmail.com<= /a>> wrote:
HI,

I have tried the 1.3.0 and ran into= issues, so I am building from the latest source and still has the same iss= ue.

Yogi


On Wed, Jan 23, 2013 at 11:36 AM, Mike P= ercy <mpercy@apache.org> wrote:
What version of Flume are you using? Are you using Maven f= or your build?

You should be able to get away with just = flume-ng-core.


On Wed, Jan 23, 2013 at 10:02 AM, yogend= er nerella <ynerella@gmail.com> wrote:
Hi,

I would like to make my app directly write events to= an flume agent.

What are the libraries needed to = add into my application?

I have tried to add the f= ollowing three but runs into class not found errors.


log4j.properties are as follows:


log4j.rootLogger=3DTRACE, flume
=
log4j.appender.flume =3D org.apache.flume.clients.log4jappender.Log4jA= ppender
log4j.appender.flume.Hostname =3D example.com
log4j.appender.flume.Port =3D 44444<= /div>
log4j.appender.flume.layout=3Dorg.apache.log4j.TTCCLayout


Error In Initialization

Classp= ath: C:\\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\test\flu= me\flume-ng-core-1.4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-log4jappender-1.= 4.0-SNAPSHOT.jar;C:\test\flume\flume-ng-sdk-1.4.0-SNAPSHOT.jar
log4j:WARN Error during default initialization
java.lang.NoC= lassDefFoundError: org/jboss/netty/channel/ChannelFactory
at org.apache.flume.api.RpcClientFact= ory.getDefaultInstance(RpcClientFactory.java:168)
at org.apache.flume.api.R= pcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
at org.apache.flume.clients.log4j= appender.Log4jAppender.activateOptions(Log4jAppender.java:195)
at org.apache.log4j.confi= g.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.se= tProperties(PropertySetter.java:172)
at org.apache.log4j.confi= g.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurato= r.parseAppender(PropertyConfigurator.java:809)
at org.apache.log4j.Prope= rtyConfigurator.parseCategory(PropertyConfigurator.java:735)
at org.apache.log4j.PropertyConfig= urator.configureRootCategory(PropertyConfigurator.java:615)
at org.apache.log4j.Prope= rtyConfigurator.doConfigure(PropertyConfigurator.java:502)
at org.apache.log4j.PropertyConfigur= ator.doConfigure(PropertyConfigurator.java:547)
at org.apache.log4j.helpe= rs.OptionConverter.selectAndConfigure(OptionConverter.java:483)
<= span style=3D"white-space:pre-wrap"> at org.apache.log4j.LogManager.= <clinit>(LogManager.java:127)
at org.apache.log4j.Logge= r.getLogger(Logger.java:117)
at LogTest.<init>(LogTest.java:7)
at LogTest.main(LogTest.j= ava:11)
Caused by: java.lang.ClassNotFoundException: org.jboss.ne= tty.channel.ChannelFactory
= at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoade= r$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoade= r.findClass(Unknown Source)
= at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppC= lassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 16 more
log= 4j:WARN No appenders could be found for logger (LogTest).
log4j:W= ARN Please initialize the log4j system properly.
=C2=A0 =C2=A0 =C2=A0

Appreciate your help,
Yogi







--e0cb4efe330c1ceea004d3fac509--