phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 丁桂涛(桂花) <dinggui...@baixing.com>
Subject UPSERT SELECT slow and crashes
Date Tue, 31 Mar 2015 01:48:48 GMT
Hi all,

I'm new to phoenix and found the UPSERT SELECT query ran very slowly and
failed after about 1 hour (I have set phoenix.query.timeoutMs to 6000000).

*Versions:*

   - Phoenix: 4.3.0

   - HBase: 0.98.5-hadoop2

   - Hive: 0.14.0


*Work Flow:*

   - create HBase table in hbase shell:  create 'H_TABLE', 'CF',
   {NUMREGIONS => 21, SPLITALGO => 'HexStringSplit'}

   - load data to HBase table in hive
   via org.apache.hadoop.hive.hbase.HBaseStorageHandler

   - create Phoenix view mapping to the HBase table H_TABLE: create view
   P_VIEW (... ...);

   - create a locally indexed Phoenix table: create table P_TABLE (... ...)
   SALT_BUCKETS = 42; create local index idx on P_TABLE (...);

   - insert data into P_TABLE: upsert into P_TABLE select * from
P_VIEW; There're
   total about 40M records and this query failed after about 1 hour. Below
   is the error msg. Any suggestions?


Error:  (state=08000,code=101)
                            [10/1869]
org.apache.phoenix.exception.PhoenixIOException
        at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
        at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:589)
        at
org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
        at
org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
        at
org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
        at
org.apache.phoenix.compile.UpsertCompiler$2.execute(UpsertCompiler.java:679)
        at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:280)
        at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:272)
        at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
        at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:270)
        at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1063)
        at sqlline.Commands.execute(Commands.java:822)
        at sqlline.Commands.sql(Commands.java:732)
        at sqlline.SqlLine.dispatch(SqlLine.java:808)
        at sqlline.SqlLine.begin(SqlLine.java:681)
        at sqlline.SqlLine.start(SqlLine.java:398)
        at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: java.lang.UnsupportedOperationException
        at java.util.AbstractList.add(AbstractList.java:148)
        at java.util.AbstractList.add(AbstractList.java:108)
        at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:535)
        ... 15 more

Mime
View raw message