phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemal Parekh <he...@bitscopic.com>
Subject Re: Concurrent phoenix queries throw unable to create new native thread error
Date Thu, 11 Oct 2018 13:37:28 GMT
I have already set phoenix.query.threadPoolSize in hbase-site.xml which
resides on server side. I think phoenix.query.threadPoolSize is a client
side property and so should be set from the client application making
phoenix connection. Are you suggesting that there should also be a copy of
hbase-site.xml on the client machine?

On Thu, Oct 11, 2018 at 5:20 AM Batyrshin Alexander <0x62ash@gmail.com>
wrote:

> Problem comes from that "phoenix.query.threadPoolSize" can not be set via
> connection settings. You must set this props via hbase-site.xml.
> Looks like thread pool initialized before any connections inside JDBC
> driver.
>
> I think that this moment must be clarified in documentation
> http://phoenix.apache.org/tuning.html
>
>
> On 10 Oct 2018, at 18:59, Hemal Parekh <hemal@bitscopic.com> wrote:
>
> We have an analytical application running concurrent phoenix queries
> against Hortonworks HDP 2.6 cluster. Application uses phoenix JDBC
> connection to run queries. Often times, concurrent queries fail with
> "java.lang.OutOfMemoryError: unable to create new native thread" error.
> JDBC connection sets following phoenix properties.
>
> connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
> connectionProps.setProperty("phoenix.query.querySize", "40000")
>
> Phoenix version is 4.7 and Hbase version is 1.1.2, The HDP cluster has six
> regionservers on six data nodes. Concurrent queries run against different
> phoenix tables, some are small having few million records and some are big
> having few billions records. Most of the queries do not have joins,  where
> clause includes conditions on rowkey and few nonkey columns. Queries with
> joins (which are on small tables) have used USE_SORT_MERGE_JOIN hint.
>
> Are there other phoenix properties which need to be set on JDBC
> connection? Are above values for phoenix.query.threadPoolSize and phoenix.query.querySize
> enough to handle concurrent query use case? We have changed these two
> properties couple of times to increase their values but the error still
> remains the same.
>
>
> Thanks,
>
> Hemal Parekh
>
>
>
>
>

-- 

Hemal Parekh
Senior Data Warehouse Architect
m. 240.449.4396
[image: Bitscopic Inc] <http://bitscopic.com>

Mime
View raw message