phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hengesbach, Martin" <Martin.Hengesb...@fiz-Karlsruhe.de>
Subject Call queue too small
Date Mon, 26 Mar 2018 06:18:21 GMT
Hi,

I tried to copy round about 1 million records from one table to another with this sql:
UPSERT INTO New_Table (ID, data) SELECT ID, data FROM OLD_Table WHERE id >= '2010000000'
AND ID < '2011000000';

With smaller number of records it works, but with this sql I get the following exception:

18/03/26 07:54:02 WARN client.AsyncProcess: #1, table=New_Table, attempt=35/35 failed=28363ops,
last exception: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ipc.RpcServer$CallQueueTooBigException):
Call queue is full on /x.x.x.x:16020, is hbase.ipc.server.max.callqueue.size too small? on
dn13,16020,1521542835853, tracking started Mon Mar 26 07:44:16 CEST 2018; not retrying 28363
- final failure
Error: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 28363 actions:
RemoteWithExtrasException: 28363 times, (state=,code=0)
org.apache.phoenix.execute.CommitException: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
Failed 28363 actions: RemoteWithExtrasException: 28363 times,
        at org.apache.phoenix.execute.MutationState.send(MutationState.java:985)
        at org.apache.phoenix.execute.MutationState.send(MutationState.java:1327)
        at org.apache.phoenix.execute.MutationState.commit(MutationState.java:1159)
        at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:528)
        at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:525)
        at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
        at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:525)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:346)
        at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:331)
        at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
        at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:329)
        at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1440)
        at sqlline.Commands.execute(Commands.java:822)
        at sqlline.Commands.sql(Commands.java:732)
        at sqlline.SqlLine.dispatch(SqlLine.java:808)
        at sqlline.SqlLine.begin(SqlLine.java:681)
        at sqlline.SqlLine.start(SqlLine.java:398)
        at sqlline.SqlLine.main(SqlLine.java:292)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 28363
actions: RemoteWithExtrasException: 28363 times,
        at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:234)
        at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:214)
        at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.getErrors(AsyncProcess.java:1623)
        at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:909)
        at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:923)
        at org.apache.phoenix.execute.MutationState.send(MutationState.java:952)
        ... 17 more

I have searched a lot for correct / better values for hbase.ipc.server.max.callqueue.size
but I don't find anything. And in our configuration there is no entry for that.

I tried this on a Hortonworks 2.5.3 with Phoenix 4.7.

Any ideas where I must tune to get this sql running?


Thank's in advance
Martin

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message