phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Batyrshin Alexander <0x62...@gmail.com>
Subject Re: Concurrent phoenix queries throw unable to create new native thread error
Date Thu, 11 Oct 2018 09:19:54 GMT
Problem comes from that "phoenix.query.threadPoolSize" can not be set via connection settings.
You must set this props via hbase-site.xml.
Looks like thread pool initialized before any connections inside JDBC driver.

I think that this moment must be clarified in documentation http://phoenix.apache.org/tuning.html
<http://phoenix.apache.org/tuning.html>
 

> On 10 Oct 2018, at 18:59, Hemal Parekh <hemal@bitscopic.com> wrote:
> 
> We have an analytical application running concurrent phoenix queries against Hortonworks
HDP 2.6 cluster. Application uses phoenix JDBC connection to run queries. Often times, concurrent
queries fail with "java.lang.OutOfMemoryError: unable to create new native thread" error.
JDBC connection sets following phoenix properties. 
> 
> connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
> connectionProps.setProperty("phoenix.query.querySize", "40000")
> 
> Phoenix version is 4.7 and Hbase version is 1.1.2, The HDP cluster has six regionservers
on six data nodes. Concurrent queries run against different phoenix tables, some are small
having few million records and some are big having few billions records. Most of the queries
do not have joins,  where clause includes conditions on rowkey and few nonkey columns. Queries
with joins (which are on small tables) have used USE_SORT_MERGE_JOIN hint. 
> 
> Are there other phoenix properties which need to be set on JDBC connection? Are above
values for phoenix.query.threadPoolSize and phoenix.query.querySize enough to handle concurrent
query use case? We have changed these two properties couple of times to increase their values
but the error still remains the same.
> 
> 
> Thanks,
> 
> Hemal Parekh
> 
> 
> 


Mime
View raw message