phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "金砖"<>
Subject system.stats and zk connection problem
Date Mon, 09 Oct 2017 07:35:33 GMT
hi, ALL:
 we are useing phoenix-4.7 with hbase-1.1.3, running a 17 nodes cluster with 200+ tables and
3500 regions.

Recently there are weird phenomenon happened and I’m certain that have not done anything
to the cluster:

  the RS that holding system.catalog region comes up lots of zk connection create and close
logs, alost 1 million times a day. eg:
  2017-10-09 15:26:12,656 INFO [B.defaultRpcServer.handler=129,queue=17,port=16020-SendThread(zk1:2181)]
zookeeper.ClientCnxn: Session establishment complete on server zk1/zk1:2181, sessionid = 0x258206f4e13c7ea,
negotiated timeout = 60000
2017-10-09 15:26:13,131 INFO [B.defaultRpcServer.handler=129,queue=17,port=16020] client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x258206f4e13c7ea
2017-10-09 15:26:13,136 INFO [B.defaultRpcServer.handler=129,queue=17,port=16020] zookeeper.ZooKeeper:
Session: 0x258206f4e13c7ea closed

  system.stats table reads is extremely high, about 60K qps

Have anyone met this before?  Or how can I fix this problem?

View raw message