phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 丁桂涛(桂花) <dinggui...@baixing.com>
Subject Re: UPSERT SELECT slow and crashes
Date Tue, 31 Mar 2015 02:49:57 GMT
Thanks Taylor for you kind reply.

Actually I'm using the phoenix console (i.e. ./sqlline.py xxxx). Is auto
commit on the default behaviour? Or how should I set this in the console?

On Tue, Mar 31, 2015 at 10:05 AM, James Taylor <jamestaylor@apache.org>
wrote:

> I suspect you may not have auto commit on, in which case Phoenix would
> attempt to buffer the results of the select in memory so that you
> could commit it when you choose. Try setting auto commit on
> (connection.setAutoCommit(true)) for your connection before issuing
> the UPSERT SELECT statement.
>
> On Mon, Mar 30, 2015 at 6:48 PM, 丁桂涛(桂花) <dingguitao@baixing.com>
wrote:
> > Hi all,
> >
> > I'm new to phoenix and found the UPSERT SELECT query ran very slowly and
> > failed after about 1 hour (I have set phoenix.query.timeoutMs to
> 6000000).
> >
> > Versions:
> >
> > Phoenix: 4.3.0
> >
> > HBase: 0.98.5-hadoop2
> >
> > Hive: 0.14.0
> >
> >
> > Work Flow:
> >
> > create HBase table in hbase shell:  create 'H_TABLE', 'CF', {NUMREGIONS
> =>
> > 21, SPLITALGO => 'HexStringSplit'}
> >
> > load data to HBase table in hive via
> > org.apache.hadoop.hive.hbase.HBaseStorageHandler
> >
> > create Phoenix view mapping to the HBase table H_TABLE: create view
> P_VIEW
> > (... ...);
> >
> > create a locally indexed Phoenix table: create table P_TABLE (... ...)
> > SALT_BUCKETS = 42; create local index idx on P_TABLE (...);
> >
> > insert data into P_TABLE: upsert into P_TABLE select * from P_VIEW;
> There're
> > total about 40M records and this query failed after about 1 hour. Below
> is
> > the error msg. Any suggestions?
> >
> >
> > Error:  (state=08000,code=101)
> > [10/1869]
> > org.apache.phoenix.exception.PhoenixIOException
> >         at
> >
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
> >         at
> >
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:589)
> >         at
> >
> org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
> >         at
> >
> org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
> >         at
> >
> org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
> >         at
> >
> org.apache.phoenix.compile.UpsertCompiler$2.execute(UpsertCompiler.java:679)
> >         at
> >
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:280)
> >         at
> >
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:272)
> >         at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> >         at
> >
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:270)
> >         at
> >
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1063)
> >         at sqlline.Commands.execute(Commands.java:822)
> >         at sqlline.Commands.sql(Commands.java:732)
> >         at sqlline.SqlLine.dispatch(SqlLine.java:808)
> >         at sqlline.SqlLine.begin(SqlLine.java:681)
> >         at sqlline.SqlLine.start(SqlLine.java:398)
> >         at sqlline.SqlLine.main(SqlLine.java:292)
> > Caused by: java.lang.UnsupportedOperationException
> >         at java.util.AbstractList.add(AbstractList.java:148)
> >         at java.util.AbstractList.add(AbstractList.java:108)
> >         at
> >
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:535)
> >         ... 15 more
> >
> >
> >
>

Mime
View raw message