Hi Pari,

AFAIK, Oozie does not supports HBase out of the box. Does the script runs fine without oozie?
You will need to do some ugly hacks at your end to get this going. We dont use Pig with HBase so i wont be able to tell you exact solution.

PS: At my workplace, we have had many challenges in integration of HBase and Avro with Oozie4.2.

Thanks,
Anil Gupta

On Fri, Aug 14, 2015 at 7:24 AM, Pariksheet Barapatre <pbarapatre@gmail.com> wrote:
Hi Ravi/All,

When I use PhoenixPigStorage to load HBase and try to run job through Oozie, I am getting below error -
Pig logfile dump:

Pig Stack Trace
---------------
ERROR 2043: Unexpected error during execution.

org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1333)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
    at org.apache.pig.PigServer.execute(PigServer.java:1297)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
    at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
    at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
    at org.apache.pig.Main.run(Main.java:478)
    at org.apache.pig.PigRunner.run(PigRunner.java:49)
    at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:286)
    at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:226)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
    at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.io.IOException: No FileSystem for scheme: hbase
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.createSuccessFile(MapReduceLauncher.java:688)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:461)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
    ... 27 more


Table is getting loaded but at the end oozie job failing with above error. Any workaround/known issues.

Thanks for your help in advance.

Regards
Pari



On 12 August 2015 at 17:59, Pariksheet Barapatre <pbarapatre@gmail.com> wrote:
Many Thanks Ravi.

Your solution worked. Let me get a JIRA.

Thanks
Pari

On 10 August 2015 at 20:08, Ravi Kiran <maghamravikiran@gmail.com> wrote:
Hi Pari,
   I wrote a quick test and there indeed seems to be an issue when SALT BUCKETS are mentioned on the table.  Can you please raise a JIRA ticket. 

 In the mean while, can you try the following to get over the issue.

raw = LOAD 'hbase://query/SELECT CLIENTID,EMPID,NAME FROM HIRES' USING org.apache.phoenix.pig.PhoenixHBaseLoader('localhost')';
grpd = GROUP raw BY CLIENTID; 
cnt = FOREACH grpd GENERATE group AS CLIENT,COUNT(raw);
DUMP cnt;

Let me know how it goes.

Regards
Ravi

On Mon, Aug 10, 2015 at 1:21 AM, Pariksheet Barapatre <pbarapatre@gmail.com> wrote:
Here are the steps -

-- Phoenix version - 4.2.2
-- CDH -- 5.3

-- SQLLINE
CREATE TABLE HIRES_SALTED( CLIENTID INTEGER NOT NULL, EMPID INTEGER NOT NULL, NAME VARCHAR CONSTRAINT pk PRIMARY KEY(CLIENTID,EMPID)) SALT_BUCKETS=2;

UPSERT INTO HIRES_SALTED VALUES(10,100,'ABC');
UPSERT INTO HIRES_SALTED VALUES(11,101,'XYZ');

-- PIG
register '/opt/phoenix-4.2.2-bin/phoenix-pig-4.2.2.jar';
register '/opt/phoenix-4.2.2-bin/phoenix-4.2.2-client.jar';

raw = LOAD 'hbase://table/HIRES_SALTED' USING org.apache.phoenix.pig.PhoenixHBaseLoader('localhost');
dump raw

Error -
2015-08-10 13:43:56,036 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: null
Details at logfile: /../pig_1439194425512.log

Pig Stack Trace
---------------
ERROR 1200: null


Failed to parse: null
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1648)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1621)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
        at org.apache.pig.Main.run(Main.java:541)
        at org.apache.pig.Main.main(Main.java:156)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.NullPointerException
        at org.apache.phoenix.pig.util.PhoenixPigSchemaUtil.getResourceSchema(PhoenixPigSchemaUtil.java:67)
        at org.apache.phoenix.pig.PhoenixHBaseLoader.getSchema(PhoenixHBaseLoader.java:220)
        at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
        at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
        at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
        at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
        ... 15 more
================================================================================

If we create table with SALT everything works without any issue.

Can anyone please help ?


Many Thanks
Pari

On 10 August 2015 at 13:12, Pariksheet Barapatre <pbarapatre@gmail.com> wrote:
Hi Russell,

below error I am getting in pig.log
Failed to parse: null
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
    at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1648)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1621)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
    at org.apache.pig.Main.run(Main.java:541)
    at org.apache.pig.Main.main(Main.java:156)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.NullPointerException
    at org.apache.phoenix.pig.util.PhoenixPigSchemaUtil.getResourceSchema(PhoenixPigSchemaUtil.java:67)
    at org.apache.phoenix.pig.PhoenixHBaseLoader.getSchema(PhoenixHBaseLoader.java:220)
    at org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
    at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
    at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
    at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
    at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
    at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
    at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
    at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
    at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
    ... 15 more

I am registering below two jars -
register '/home/yumecorp/phoenix-4.2.0-bin/phoenix-pig-4.2.0.jar';
register '/home/yumecorp/phoenix-4.2.0-bin/phoenix-4.2.0-client.jar';

Thanks
Pari

On 10 August 2015 at 12:50, Russell Jurney <russell.jurney@gmail.com> wrote:
There shouldn't be a limitation, no. I've stored to/read from Phoenix tables from Pig on CDH 5.

Can you paste the error?


On Monday, August 10, 2015, Pariksheet Barapatre <pbarapatre@gmail.com> wrote:
Hi All,

I am trying to run Pig script on Phoenix table.

I am using same example given in documentation.
CREATE TABLE HIRES( CLIENTID INTEGER NOT NULL, EMPID INTEGER NOT NULL, NAME VARCHAR CONSTRAINT pk PRIMARY KEY(CLIENTID,EMPID));

raw = LOAD 'hbase://table/HIRES USING org.apache.phoenix.pig.PhoenixHBaseLoader('localhost')';
grpd = GROUP raw BY CLIENTID;
cnt = FOREACH grpd GENERATE group AS CLIENT,COUNT(raw);
DUMP cnt;

The code above works without any issue, but if I CREATE HIRES tables with SALTs script failed with QueryParse.Error.

Is there any limitation of Pig integration with SALTED table.



Cheers,
Pari


--
Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com



--
Cheers,
Pari



--
Cheers,
Pari




--
Cheers,
Pari



--
Cheers,
Pari



--
Thanks & Regards,
Anil Gupta