I don’t think the HBase row_counter job is going to be faster than a count(*) query. Both require a full table scan, so neither will be particularly fast.

A couple of alternatives if you’re ok with an approximate count: 1) enable stats collection (but you can leave off usage to parallelize queries) and the do a SUM over the size column for the table using stats table directly, or 2) do a count(*) using TABLESAMPLE clause (again enabling stats as described above) to prevent a full scan.

On Thu, Feb 1, 2018 at 8:11 AM Flavio Pompermaier <pompermaier@okkam.it> wrote:
Hi Anil,
Obviously I'm not using HBase just for the count query..Most of the time I do INSERT and selective queries, I was just trying to figure out if my HBase + Phoenix installation is robust enough to deal with a huge amount of data..

On Thu, Feb 1, 2018 at 5:07 PM, anil gupta <anilgupta84@gmail.com> wrote:
Hey Flavio,

IMHO, If most of your app is just doing full table scans then i am not really sure HBase(or any other NoSql) will be a good fit for your solution.(building an OLAP system?) If you have point lookups and short range scans then HBase/Phoenix will work well.
Also, if you wanna do select count(*). The HBase row_counter job will be much faster than phoenix queries.

Thanks,
Anil Gupta

On Thu, Feb 1, 2018 at 7:35 AM, Flavio Pompermaier <pompermaier@okkam.it> wrote:
I was able to make it work changing the following params (both on server and client side and restarting hbase) and now the query answers in about 6 minutes:

hbase.rpc.timeout (to 600000)
phoenix.query.timeoutMs (to 600000)
hbase.client.scanner.timeout.period (from 1 m to 10m)
hbase.regionserver.lease.period (from 1 m to 10m)

However I'd like to know id those performances could be easily improved or not. Any ideas?

On Thu, Feb 1, 2018 at 4:30 PM, Vaghawan Ojha <vaghawan781@gmail.com> wrote:
I've the same problem, even after I increased the hbase.rpc.timeout the result is same. The difference is that I use 4.12.


On Thu, Feb 1, 2018 at 8:23 PM, Flavio Pompermaier <pompermaier@okkam.it> wrote:
Hi to all,
I'm trying to use the brand-new Phoenix 4.13.2-cdh5.11.2 over HBase and everything was fine until the data was quite small (about few millions). As I inserted 170 M of rows in my table I cannot get the row count anymore (using ELECT COUNT) because of org.apache.hbase.ipc.CallTimeoutException (operationTimeout 60000 expired).
How can I fix this problem? I could increase the hbase.rpc.timeout parameter but I suspect I could improve a little bit the HBase performance first..the problem is that I don't know how.

Thanks in advance,
Flavio





--
Thanks & Regards,
Anil Gupta