phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Boado <pedro.bo...@gmail.com>
Subject Potential causes for very slow DELETEs?
Date Fri, 18 Aug 2017 11:46:07 GMT
Hi all,

We have two HBase 1.0 clusters running the same process in parallel
-effectively keeps the same data in both Phoenix tables-

This process feeds data into Phoenix 4.5 via HFile and once the data is
loaded a Spark process deletes a few thousand rows from the tables
-secondary indexing is disabled in our installation- .

After an HBase restart -no config changes involved-, one of the clusters
have started running these deletes too slowly (the fast run is taking 5min
and the slow one around 1h). And more worryingly while the process is
running Phoenix queries are taking hundreds of seconds instead of being sub
second (even opening sqlline is very slow).

We've almost run out of ideas trying to find the cause of this behaviour.
There are no evident GC pauses, CPU usage,  Hdfs IO is normal, Memory usage
is normal, etc.

As soon as the delete process finishes Phoenix goes back to normal
behaviour.

Does anybody have any ideas for potential causes of this behaviour?

Many thanks!!

Pedro.

Mime
View raw message