phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Kiran <maghamraviki...@gmail.com>
Subject Re: Phoenix Pig integration
Date Mon, 10 Aug 2015 14:38:49 GMT
Hi Pari,
   I wrote a quick test and there indeed seems to be an issue when SALT
BUCKETS are mentioned on the table.  Can you please raise a JIRA ticket.

 In the mean while, can you try the following to get over the issue.

raw = LOAD 'hbase://*query/SELECT CLIENTID,EMPID,NAME FROM HIRES*' USING
org.apache.phoenix.pig.PhoenixHBaseLoader('localhost')';
grpd = GROUP raw BY CLIENTID;
cnt = FOREACH grpd GENERATE group AS CLIENT,COUNT(raw);
DUMP cnt;

Let me know how it goes.

Regards
Ravi

On Mon, Aug 10, 2015 at 1:21 AM, Pariksheet Barapatre <pbarapatre@gmail.com>
wrote:

> Here are the steps -
>
> -- Phoenix version - 4.2.2
> -- CDH -- 5.3
>
> -- SQLLINE
> CREATE TABLE HIRES_SALTED( CLIENTID INTEGER NOT NULL, EMPID INTEGER NOT
> NULL, NAME VARCHAR CONSTRAINT pk PRIMARY KEY(CLIENTID,EMPID))
> SALT_BUCKETS=2;
>
> UPSERT INTO HIRES_SALTED VALUES(10,100,'ABC');
> UPSERT INTO HIRES_SALTED VALUES(11,101,'XYZ');
>
> -- PIG
> register '/opt/phoenix-4.2.2-bin/phoenix-pig-4.2.2.jar';
> register '/opt/phoenix-4.2.2-bin/phoenix-4.2.2-client.jar';
>
> raw = LOAD 'hbase://table/HIRES_SALTED' USING
> org.apache.phoenix.pig.PhoenixHBaseLoader('localhost');
> dump raw
>
> Error -
> 2015-08-10 13:43:56,036 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1200: null
> Details at logfile: /../pig_1439194425512.log
>
> Pig Stack Trace
> ---------------
> ERROR 1200: null
>
>
> Failed to parse: null
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
>         at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1648)
>         at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1621)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
>         at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>         at org.apache.pig.Main.run(Main.java:541)
>         at org.apache.pig.Main.main(Main.java:156)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: java.lang.NullPointerException
>         at
> org.apache.phoenix.pig.util.PhoenixPigSchemaUtil.getResourceSchema(PhoenixPigSchemaUtil.java:67)
>         at
> org.apache.phoenix.pig.PhoenixHBaseLoader.getSchema(PhoenixHBaseLoader.java:220)
>         at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
>         at
> org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
>         at
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
>         ... 15 more
>
> ================================================================================
>
> If we create table with SALT everything works without any issue.
>
> Can anyone please help ?
>
>
> Many Thanks
> Pari
>
> On 10 August 2015 at 13:12, Pariksheet Barapatre <pbarapatre@gmail.com>
> wrote:
>
>> Hi Russell,
>>
>> below error I am getting in pig.log
>> Failed to parse: null
>>     at
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
>>     at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1648)
>>     at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1621)
>>     at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1093)
>>     at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
>>     at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
>>     at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>     at org.apache.pig.Main.run(Main.java:541)
>>     at org.apache.pig.Main.main(Main.java:156)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>     at java.lang.reflect.Method.invoke(Method.java:606)
>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> Caused by: java.lang.NullPointerException
>>     at
>> org.apache.phoenix.pig.util.PhoenixPigSchemaUtil.getResourceSchema(PhoenixPigSchemaUtil.java:67)
>>     at
>> org.apache.phoenix.pig.PhoenixHBaseLoader.getSchema(PhoenixHBaseLoader.java:220)
>>     at
>> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
>>     at
>> org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
>>     at
>> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:853)
>>     at
>> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
>>     at
>> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
>>     at
>> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
>>     at
>> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
>>     at
>> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
>>     at
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
>>     ... 15 more
>>
>> I am registering below two jars -
>> register '/home/yumecorp/phoenix-4.2.0-bin/phoenix-pig-4.2.0.jar';
>> register '/home/yumecorp/phoenix-4.2.0-bin/phoenix-4.2.0-client.jar';
>>
>> Thanks
>> Pari
>>
>> On 10 August 2015 at 12:50, Russell Jurney <russell.jurney@gmail.com>
>> wrote:
>>
>>> There shouldn't be a limitation, no. I've stored to/read from Phoenix
>>> tables from Pig on CDH 5.
>>>
>>> Can you paste the error?
>>>
>>>
>>> On Monday, August 10, 2015, Pariksheet Barapatre <pbarapatre@gmail.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I am trying to run Pig script on Phoenix table.
>>>>
>>>> I am using same example given in documentation.
>>>>
>>>> CREATE TABLE HIRES( CLIENTID INTEGER NOT NULL, EMPID INTEGER NOT NULL, NAME
VARCHAR CONSTRAINT pk PRIMARY KEY(CLIENTID,EMPID));
>>>>
>>>>
>>>> raw = LOAD 'hbase://table/HIRES USING
>>>> org.apache.phoenix.pig.PhoenixHBaseLoader('localhost')';
>>>> grpd = GROUP raw BY CLIENTID;
>>>> cnt = FOREACH grpd GENERATE group AS CLIENT,COUNT(raw);
>>>> DUMP cnt;
>>>>
>>>> The code above works without any issue, but if I CREATE HIRES tables
>>>> with SALTs script failed with QueryParse.Error.
>>>>
>>>> Is there any limitation of Pig integration with SALTED table.
>>>>
>>>>
>>>>
>>>> Cheers,
>>>> Pari
>>>>
>>>
>>>
>>> --
>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome
>>> .com
>>>
>>
>>
>>
>> --
>> Cheers,
>> Pari
>>
>
>
>
> --
> Cheers,
> Pari
>

Mime
View raw message