phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: Phoenix - loading via mapreduce
Date Wed, 10 Dec 2014 19:52:12 GMT
Hi Rama,

Sorry, I lost track of this.

The steps to set up your environment to run mapreduce will depend on
which version of Hadoop you're using, as well as which distribution
(i.e. the base Apache release, CDH, HDP, or something else).

If you're running the base Apache release, then the docs here should
help you get things set up:
http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html#Pseudo-Distributed_Operation

If you're running a vendor distribution such as CDH or HDP, their
documentation should help you to get started.

It sounds like you're talking about running things on a single node,
in which case you won't get too much advantage by running the
mapreduce-based importer -- you're probably better off just importing
data via psql.py (as explained here:
http://phoenix.apache.org/bulk_dataload.html).

- Gabriel


On Thu, Dec 4, 2014 at 10:09 PM, Rama Ramani <rama.ramani@live.com> wrote:
> Thanks Gabriel.
>
> Yes, I am running on Windows. Tried setting the variable and it does not
> help. I do see that the machine has mapreduce logs of jobs.
>
> I am now setting up a UBUNTU 14.10 machine and setting it up to access the
> HBase cluster (running on Windows). I downloaded the phoenix binary from
> http://mirror.symnds.com/software/Apache/phoenix/phoenix-4.1.0/bin/ and
> installed Java on it. I am able to run psql.py against the cluster
>
> Can you please send me the instructions to setup the necessary libs for
> mapreduce on the Ubuntu machine?
>
> thanks
> Rama
>> Date: Thu, 4 Dec 2014 09:00:11 +0100
>> Subject: Re: Phoenix - loading via mapreduce
>> From: gabriel.reid@gmail.com
>> To: user@phoenix.apache.org
>
>>
>> Thanks for pointing out PHOENIX-976 James (I had lost track of that
>> one), but I think that this is a different issue.
>>
>> @Rama, I see you're running on Windows. Can you confirm that you're
>> able to start (non-Phoenix) MapReduce jobs from your Windows machine?
>>
>> In any case, the configuration parameter that determines where the job
>> jar is unzipped locally is hadoop.tmp.dir, so setting this to
>> something else should change the location where things are being
>> unzipped. However, I wouldn't be surprised if other issues come up
>> with starting the job due to running on Windows.
>>
>> - Gabriel
>>
>>
>> On Thu, Dec 4, 2014 at 5:51 AM, James Taylor <jamestaylor@apache.org>
>> wrote:
>> > See https://issues.apache.org/jira/browse/PHOENIX-976
>> >
>> > On Wed, Dec 3, 2014 at 5:38 PM, Rama Ramani <rama.ramani@live.com>
>> > wrote:
>> >> Sorry, send to dev alias by mistake, sending to the user DL
>> >>
>> >> When running the Mapreduce command
>> >> fromhttp://phoenix.apache.org/bulk_dataload.html, I am getting an
>> >> Access
>> >> denied exception.
>> >>
>> >> Is there an environment variable or a cmdline option to specify a
>> >> different
>> >> folder for the \apps folder?
>> >>
>> >>
>> >> C:\apps\dist\phoenix-4.0.0.2.1.8.0-2176>hadoop jar
>> >> phoenix-4.0.0.2.1.8.0-2176-client.jar
>> >> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table test2 --input
>> >> /data/test2.csv
>> >>
>> >> Exception in thread "main" java.io.FileNotFoundException:
>> >> \apps\temp\hdfs\tmp\ha
>> >> doop-unjar4246468824119614050\META-INF\LICENSE (Access is denied)
>> >> at java.io.FileOutputStream.open(Native Method)
>> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> >> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> >> at org.apache.hadoop.util.RunJar.unJar(RunJar.java:88)
>> >> at org.apache.hadoop.util.RunJar.unJar(RunJar.java:64)
>> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:188)
>> >>
>> >>
>> >> thanks
>> >> Rama

Mime
View raw message