flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vijay k <k.vija...@gmail.com>
Subject Re: Flume agent failure
Date Fri, 06 Jul 2012 06:59:40 GMT
Yes Mike you are correct, i have executed the "$HADOOP_HOME/bin/hadoop
classpath" command, i am getting following error.

hduser@md-trngpoc1:/usr/local/hadoop$ bin/hadoop classpath
Exception in thread "main" java.lang.NoClassDefFoundError: classpath
Caused by: java.lang.ClassNotFoundException: classpath
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: classpath.  Program will exit.
that is the reason, I could not able to run the flume-ng?. any way i am
going to install hadoop stable 1.0.3 version.

Thanks a lot for your help.
Vijay


On Fri, Jul 6, 2012 at 12:11 AM, Mike Percy <mpercy@cloudera.com> wrote:

> Vijay, I agree with Alex & Mohammed, your Hadoop installation seems messed
> up. Either that, or you have a copy & paste error in your output...
>
> Also, apparently you are trying to run Hadoop 0.20.204? It would be much
> better for you to either use a stable version of Hadoop, such as 1.0.3, or
> use the Hadoop version from CDH 3 or 4. I think 0.20.204 may even be
> missing hsync (I know it's there in 0.20.205), and if so then the Flume
> HDFS Sink will not work.
>
> At a minimum, ensure that your HADOOP_HOME is pointing to a location so
> that you can execute "$HADOOP_HOME/bin/hadoop classpath" and it will output
> some jars, etc. The output you showed below looks like it should be the
> output of "which hadoop"... if it's not then the hadoop classpath command
> is broken in your version of hadoop and since Flume depends on it working,
> you are kind of out of luck without hacking up the start script or putting
> jars and config files into the Flume lib directory.
>
> Regards,
> Mike
>
>
> On Thu, Jul 5, 2012 at 5:08 AM, Mohammad Tariq <dontariq@gmail.com> wrote:
>
>> If you are using CDH then it's always better to uninstall the previous
>> version. If it is Apache Hadoop then it's just a matter of changing
>> the directory.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Thu, Jul 5, 2012 at 5:28 PM, vijay k <k.vijay52@gmail.com> wrote:
>> > Hi,
>> >
>> > with out removing older version, shall i install hadoop 0.20.204 in
>> another
>> > folder?
>> >
>> > Thanks,
>> > vijay
>> >
>> > On Wed, Jul 4, 2012 at 7:26 PM, alo alt <wget.null@gmail.com> wrote:
>> >>
>> >> Yes.
>> >>
>> >> Follow these link:
>> >> http://hadoop.apache.org/common/docs/r0.20.204.0/
>> >>
>> >> to install a single node or cluster. Note, you need on the sink side
>> the
>> >> same hdfs version as you've in your cluster.
>> >>
>> >> - Alex
>> >>
>> >>
>> >> On Jul 4, 2012, at 3:51 PM, vijay k wrote:
>> >>
>> >> > I have already downloaded the hadoop-0.20.204.0.tar.gz,
>> >> >
>> >> > shall i go ahead  with hadoop 0.20.204 ver installation. please
>> confirm
>> >> > it
>> >> > is compatible with flume 1.2.0 version?
>> >> >
>> >> > Thanks,
>> >> > Vijay
>> >> >
>> >> > On Wed, Jul 4, 2012 at 7:03 PM, alo alt <wget.null@gmail.com>
wrote:
>> >> >
>> >> >> 1. Flume uses the libs from the classpath
>> >> >> 2. remove the installed hadoop version (apt-get remove hadoop*)
>> >> >> 3. http://hadoop.apache.org/common/docs/   => click at the version
>> you
>> >> >> want to install and follow the steps described there
>> >> >>
>> >> >> - Alex
>> >> >>
>> >> >> On Jul 4, 2012, at 2:36 PM, vijay k wrote:
>> >> >>
>> >> >>> Hi Alex,
>> >> >>>
>> >> >>> I have installed hadoop via tar xzf hadoop-0.20.2.zip in Ubuntu,
>> but i
>> >> >>> could not able to find the *hadoop-install* directory.
>> >> >>> Please let me know the following query's
>> >> >>>
>> >> >>> 1. which latest HADOOP version is compatible with FLUME 1.2.0
.
>> >> >>> 2. if need to remove my hadoop 0.20.0 version, how to remove
the
>> older
>> >> >>> version.
>> >> >>> 3. and provide the hadoop latest version installation steps.
>> >> >>>
>> >> >>> Thanks a lot for your comments.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Wed, Jul 4, 2012 at 5:28 PM, alo alt <wget.null@gmail.com>
>> wrote:
>> >> >>>
>> >> >>>> U're on Ubuntu / Debian?
>> >> >>>>
>> >> >>>> If yes, remove the hadoop installation via apt, you have
an really
>> >> >>>> old
>> >> >> one
>> >> >>>> installed. Take a look into the /usr/local/hadoop/lib directory
>> (as
>> >> >>>> you
>> >> >>>> posted before):
>> >> >>>>
>> >> >>>> -rw-r--r-- 1 hduser hadoop   15010 2010-02-19 13:25
>> xmlenc-0.52.jar
>> >> >>>> -rw-r--r-- 1 hduser hadoop    8601 2010-02-19 13:25
>> >> >> slf4j-log4j12-1.4.3.jar
>> >> >>>>
>> >> >>>> and so on. From 2010 - pretty old.
>> >> >>>>
>> >> >>>> How do you installed hadoop? I think you've done already
from the
>> >> >> tarball
>> >> >>>> into /hadoop-install/hadoop - there should be a ../lib
dir too.
>> >> >>>> Export
>> >> >>>> these and try again.
>> >> >>>>
>> >> >>>> - Alex
>> >> >>>>
>> >> >>>>
>> >> >>>> On Jul 4, 2012, at 1:50 PM, vijay k wrote:
>> >> >>>>
>> >> >>>>> Hi Alex,
>> >> >>>>>
>> >> >>>>> I have set the HADOOP_CLASSPATH=/usr/local/hadoop/lib
only, but
>> >> >>>>> still
>> >> >> am
>> >> >>>>> unable to proceed further.
>> >> >>>>> Please find the attached flume-ng script. Please guide
me if i
>> did
>> >> >>>> anything
>> >> >>>>> wrong in the script
>> >> >>>>>
>> >> >>>>> Thanks a lot for your helping to proceed further.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> On Wed, Jul 4, 2012 at 5:08 PM, alo alt <wget.null@gmail.com>
>> wrote:
>> >> >>>>>
>> >> >>>>>> Hi Vijay,
>> >> >>>>>>
>> >> >>>>>> but you've set the path to:
>> >> >>>>>>> hadoop classpath        -- /usr/local/hadoop/bin/hadoop
>> >> >>>>>>
>> >> >>>>>> should be: /usr/local/hadoop/lib
>> >> >>>>>>
>> >> >>>>>> Flume reads out the variable and look into that
path for all
>> needed
>> >> >>>>>> classes / jars. But in /usr/local/hadoop/bin/hadoop
flume found
>> >> >> nothing.
>> >> >>>>>>
>> >> >>>>>> - Alex
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> On Jul 4, 2012, at 1:33 PM, vijay k wrote:
>> >> >>>>>>
>> >> >>>>>>> Hi Alex,
>> >> >>>>>>>
>> >> >>>>>>> Hadoop lib are installed following path
>> >> >>>>>>>
>> >> >>>>>>> root@md-trngpoc1
>> >> >>>>>> :/usr/local/flume_dir/flume/flume-1.2.0-incubating-SNAPSHOT/bin#
>> >> >>>>>>> cd /usr/local/hadoop/lib
>> >> >>>>>>> root@md-trngpoc1:/usr/local/hadoop/lib# ls
-lrt
>> >> >>>>>>> total 8428
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> --
>> >> >>>>>> Alexander Alten-Lorenz
>> >> >>>>>> http://mapredit.blogspot.com
>> >> >>>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>> <flume-ng script.txt>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> Alexander Alten-Lorenz
>> >> >>>> http://mapredit.blogspot.com
>> >> >>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >> >>>>
>> >> >>>>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Alexander Alten-Lorenz
>> >> >> http://mapredit.blogspot.com
>> >> >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >> >>
>> >> >>
>> >>
>> >>
>> >> --
>> >> Alexander Alten-Lorenz
>> >> http://mapredit.blogspot.com
>> >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>> >>
>> >
>>
>
>

Mime
View raw message