phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Ervits <artemerv...@gmail.com>
Subject RE: bulk loading using OOZIE
Date Tue, 07 Oct 2014 13:42:19 GMT
Try to run an example shell script without phoenix. There's a sample in
oozie directory. Also a good place to find issues is in oozie logs. Try
running your process manually without Oozie. That way you can single out
what the issue is. I'm not an expert but I think Oozie doesn't find HBase
jars.
On Oct 7, 2014 8:41 AM, "Bulvik, Noam" <Noam.Bulvik@teoco.com> wrote:

>  I do not understand how it is related, we run CSV bulk loader as Ozzie
> shell script tasks, from the logs I see that when the script starts all the
> needed env' variable including the classpath that points to CDH libs is ok.
> Then from the CSV bulk loading it tries to access some file and fail with
> the message that file not found
>
>
>
> org.apache.phoenix.mapreduce.CsvBulkLoadTool: Import job on table=CDRINFO
> failed due to exception:java.io.FileNotFoundException: File does not exist:
> hdfs://dc50-dev-bd01:8020/opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/hbase/lib/hbase-server-0.98.1-cdh5.1.0.jar
>
>
>
> do I need to add all CDH file to share lib for the script to work – it
> does not make sense
>
>
>
>
>
>
>
> *From:* Artem Ervits [mailto:artemervits@gmail.com]
> *Sent:* Tuesday, October 7, 2014 2:25 PM
> *To:* user@phoenix.apache.org
> *Subject:* RE: bulk loading using OOZIE
>
>
>
> Is your oozie sharelib setup? Read about it, it will explain what Oozie
> expects.
>
> On Oct 7, 2014 7:18 AM, "Bulvik, Noam" <Noam.Bulvik@teoco.com> wrote:
>
>  I checked  the file exists on all cluster machine with full permission
> (it is part of CDH files )
>
>
>
> *From:* Artem Ervits [mailto:artemervits@gmail.com]
> *Sent:* Tuesday, October 7, 2014 1:37 PM
> *To:* user@phoenix.apache.org
> *Subject:* Re: bulk loading using OOZIE
>
>
>
> from one of your earlier log posts, it says it can't find a jar in the
> directory, please check the jar is there.
>
>
>
> On Tue, Oct 7, 2014 at 1:53 AM, Bulvik, Noam <Noam.Bulvik@teoco.com>
> wrote:
>
>  We login to ozzie using mapred user.
>
> It looks like the ozzie task runs the script as user nobody , when I
> executed the command line of the script after loing as nobody it works fine
> so I do not think we need to  explicit permission on some files
>
>
>
> I also do not think that we need to upload the CDH jars to ozzie (AFAIK) –
> do you?
>
>
>
> *From:* Artem Ervits [mailto:artemervits@gmail.com]
> *Sent:* Tuesday, October 7, 2014 12:51 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: bulk loading using OOZIE
>
>
>
> Double check that oozie user has permissions to read the required files in
> hdfs, for example, user running oozie wf is oozie then you should upload
> the jar and any property files to /user/oozie.
>
> On Oct 5, 2014 2:41 AM, "Bulvik, Noam" <Noam.Bulvik@teoco.com> wrote:
>
>  Hi,
>
>
>
> We are trying to do periodic bulk loading using OOZIE as scheduler.  We
> impalement script task that should call the bulk loading command line with
> all needed parameter. When we run the script directly from Linux  console
> it works fine but when we run the OOZIE task it fail saying that jar is
> missing or we do not have permission to read it. We are using CDH5.1 and
> the jar it companies about is one of the CDh jars that we checked and it is
> exists on all machines of the cluster
>
>
>
> Any idea ?
>
>
>
> Regards,
>
> Noam
>
> Information in this e-mail and its attachments is confidential and
> privileged under the TEOCO confidentiality terms that can be reviewed here
> <http://www.teoco.com/email-disclaimer>.
>
>
>
>
>
> --
> Artem
>
>

Mime
View raw message