phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: issue with batch upsert for tables with secondary indexes
Date Mon, 17 Mar 2014 16:59:46 GMT
Good find, Skanda. In the case of a table with immutable rows, we don't
need to do that delete. In the general case, though, we do, as by setting
the column to null, the prior value needs to be removed.

Would you mind filing a JIRA?

Thanks,
James


On Mon, Mar 17, 2014 at 7:20 AM, Skanda <skanda.ganapathy@gmail.com> wrote:

> Hi,
>
> I just went through the code and found that phoneix sets a delete
> mutation, if a column is null for a particular rowkey.
>
>  if (rowEntry.getValue() == null) { // means delete
>                 row.delete();
>             }
>
> I don't understand why a column has to be deleted when its value is null
> as part of a upsert. According to my understanding, an exception can be
> thrown or the column can be skipped.
> Please correct me if i am wrong.
>
> Regards,
> Skanda
>
>
> On Mon, Mar 17, 2014 at 3:09 PM, Skanda <skanda.ganapathy@gmail.com>wrote:
>
>> Hi,
>>
>> I have a table with immutable secondary indexes. When I do a batch
>> upsert, I get the following exception:
>>
>> org.apache.phoenix.schema.IllegalDataException: java.sql.SQLException: ERROR 1027
(42Y86): Delete not allowed on a table with IMMUTABLE_ROW with non PK column in index. tableName=uh_repo
>> 	at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:198)
>> 	at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:179)
>> 	at org.apache.phoenix.execute.MutationState.commit(MutationState.java:333)
>> 	at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:293)
>> 	at com.yumecorp.pig.udf.storage.phoenix.YumePhoenixHBaseStorage$PhoenixOutputFormat$1.commit(YumePhoenixHBaseStorage.java:449)
>> 	at com.yumecorp.pig.udf.storage.phoenix.YumePhoenixHBaseStorage$PhoenixOutputFormat$1.commitTask(YumePhoenixHBaseStorage.java:400)
>> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.commitTask(PigOutputCommitter.java:277)
>> 	at org.apache.hadoop.mapred.Task.commit(Task.java:1014)
>> 	at org.apache.hadoop.mapred.Task.done(Task.java:884)
>> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
>> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>> 	at java.security.AccessController.doPrivileged(Native Method)
>> 	at javax.security.auth.Subject.doAs(Subject.java:396)
>> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>> 	at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> Caused by: java.sql.SQLException: ERROR 1027 (42Y86): Delete not allowed on a table
with IMMUTABLE_ROW with non PK column in index. tableName=uh_repo
>> 	at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:136)
>> 	at org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:182)
>> 	at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:196)
>> 	... 14 more
>>
>> I don't understand why phoenix has to perform a delete while doing a batch upsert.
Pls note that I'm using PhoenixStorage in Pig which does a batch upsert of records into hbase.
>>
>> Further, I tried doing a count(*) in both the main table and the secondary index
tables. They don't match. (ideally it should have been the same). Is it the reason behind
phoenix trying to perform a delete to keep the main and secondary tables in sync?
>>
>>
>> Regards,
>>
>> Skanda
>>
>>
>>
>>
>

Mime
View raw message