phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "金砖"<jinzh...@wacai.com>
Subject drop phoenix index cause high cpu
Date Tue, 21 Feb 2017 02:59:44 GMT
Hi,
I have a big table with 10Million rows and 60fields. This table already created 8 indexes,
and yesterday I created another index.
create index table_another_index on table(another_field) async.
right after this index was created I realized is not needed. So I droppped it.
drop index table_another_index on table;
data was stilling writing in the mean time.


Here comes the problem:
load ofeveryregionserver node rised up from 1~2 to 50. cpu user time up to 80-90. thisphenomenon
continued 5minutes.


Question:
What happened after async index was dropped ?
I am running phoenix-4.8.0-1.0-hbase with hbase 1.1.3.


PS:

  some exception loged in regionserver log:
ERROR [B.defaultRpcServer.handler=253,queue=19,port=16020] parallel.BaseTaskRunner: Found
a failed task because: org.apache.phoenix.hbase.index.exception.SingleIndexWriteFailureException:
Failed 1 action: Table ’TABLE_ANOTHER_INDEX' was not found, got: TABLE_INDEX.: 1 time,
java.util.concurrent.ExecutionException: org.apache.phoenix.hbase.index.exception.SingleIndexWriteFailureException:
Failed 1 action: Table ’TABLE_ANOTHER_INDEX' was not found, got: TABLE_INDEX.: 1 time
 



some exception loged in client:
o.a.p.execute.MutationState - Swallowing exception and retrying after clearing meta cache
on connection. java.sql.SQLException: ERROR 2008 (INT10): Unable to find cached index metadata.
ERROR 2008 (INT10): ERROR 2008 (INT10): Unable to find cached index metadata. key=-615404062862804744
region=TABLE,xxx,1476675558366.cf334587bcdc120cf27e00d5d44ca0a6. Index update failed
Mime
View raw message