phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vasudevan, Ramkrishna S" <>
Subject RE: Update statistics made query 2-3x slower
Date Fri, 13 Feb 2015 05:51:00 GMT
Ya . I agree with Mujtaba here.  We can dig in more to know the reason why your queries are
getting slower.  I believe the query is getting executed as number of smaller chunks than
optimal way may be because of the data size/width.?


From: Mujtaba Chohan []
Sent: Friday, February 13, 2015 1:50 AM
Subject: Re: Update statistics made query 2-3x slower

Constantin - If possible can you please share your schema, approx. row/columns width, number
of region servers in your cluster plus their heap size, HBase/Phoenix version and any default
property overrides so we can identify why stats are slowing things down in your case.


On Thu, Feb 12, 2015 at 12:56 AM, Ciureanu, Constantin (GfK) <<>>
It worked!
Without stats it’s again faster (2-3x times) – but I do understand that all other normal
queries might benefit from the stats.

Thank you Mujtaba for the info,
Thank you Vasudevan for the explanations, I already used HBase and I agree it’s hard to
have a counter for the table rows (especially if the tombstones for deleted rows are still
there – ie. not compacted yet).


From: Mujtaba Chohan [<>]
Sent: Wednesday, February 11, 2015 8:54 PM
Subject: Re: Update statistics made query 2-3x slower

To compare performance without stats, try deleting related rows from SYSTEM.STATS or an easier
way, just truncate SYSTEM.STATS table from HBase shell and restart your region servers.

On Wed, Feb 11, 2015 at 10:29 AM, Vasudevan, Ramkrishna S <<>>
Hi Constantin

Before I could explain on the slowness part let me answer your 2nd question,

Phoenix is on top of HBase. HBase is a distributed NoSQL DB. So the data that is residing
inside logical entities called regions are spread across different nodes (region servers).
 There is nothing like a table that is in one location where you can keep updating the count
of rows that is getting inserted.

Which means that when you need  count(*) you may have to aggregate the count from every region
distributed across region servers. So in other words a table is not a single entity it is
a collection of regions.

Coming to your slowness in query, the update statistics query allows you to parallelize the
query into logical chunks on a single region.  Suppose there are 100K rows in a region the
statistics collected would allow you to run a query parallely for eg say execute parallely
on 10 equal chunks of 10000 rows within that region.

Have you modified any of the parameters related to statistics like this one ‘phoenix.stats.guidepost.width’.

From: Ciureanu, Constantin (GfK) [<>]
Sent: Wednesday, February 11, 2015 2:51 PM
Subject: Update statistics made query 2-3x slower

Hello all,

1.     Is there a good explanation why updating the statistics:
update statistics tableX;

made this query 2x times slower?   (it was 27 seconds before, now it’s somewhere between
60 – 90 seconds)
select count(*) from tableX;
|                 COUNT(1)                 |
| 5786227                                  |
1 row selected (62.718 seconds)

(If possible ☺ ) how can I “drop” those statistics?

2. Why there is nothing (like a counter / attribute for the table) to obtain the number of
rows in one table fast?

Thank you,

View raw message