phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaanai Zhang <cloud.pos...@gmail.com>
Subject Re: Concurrent phoenix queries throw unable to create new native thread error
Date Thu, 11 Oct 2018 01:37:53 GMT
>
> Often times, concurrent queries fail with "java.lang.OutOfMemoryError:
> unable to create new native thread
>

You can try to adjust VM options which your client program.

connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
> connectionProps.setProperty("phoenix.query.querySize", "40000")


Did you try to decrease values of the above configurations?
----------------------------------------
   Jaanai Zhang
   Best regards!



Hemal Parekh <hemal@bitscopic.com> 于2018年10月11日周四 上午1:18写道:

> limits.conf has following which I thought were sufficient. I will check if
> these limits are getting exceeded.
>
> *   -   nofile   32768
> *   -   nproc   65536
>
>
>
> Thanks,
> Hemal
>
>
> On Wed, Oct 10, 2018 at 12:40 PM Pedro Boado <pedro.boado@gmail.com>
> wrote:
>
>> Are you reaching any of the ulimits for the user running your application?
>>
>> On Wed, 10 Oct 2018, 17:00 Hemal Parekh, <hemal@bitscopic.com> wrote:
>>
>>> We have an analytical application running concurrent phoenix queries
>>> against Hortonworks HDP 2.6 cluster. Application uses phoenix JDBC
>>> connection to run queries. Often times, concurrent queries fail with
>>> "java.lang.OutOfMemoryError: unable to create new native thread" error.
>>> JDBC connection sets following phoenix properties.
>>>
>>> connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
>>> connectionProps.setProperty("phoenix.query.querySize", "40000")
>>>
>>> Phoenix version is 4.7 and Hbase version is 1.1.2, The HDP cluster has
>>> six regionservers on six data nodes. Concurrent queries run against
>>> different phoenix tables, some are small having few million records and
>>> some are big having few billions records. Most of the queries do not have
>>> joins,  where clause includes conditions on rowkey and few nonkey columns.
>>> Queries with joins (which are on small tables) have used
>>> USE_SORT_MERGE_JOIN hint.
>>>
>>> Are there other phoenix properties which need to be set on JDBC
>>> connection? Are above values for phoenix.query.threadPoolSize and phoenix.query.querySize
>>> enough to handle concurrent query use case? We have changed these two
>>> properties couple of times to increase their values but the error still
>>> remains the same.
>>>
>>>
>>> Thanks,
>>>
>>> Hemal Parekh
>>>
>>>
>>>
>>>
>
> --
>
> Hemal Parekh
> Senior Data Warehouse Architect
> m. 240.449.4396
> [image: Bitscopic Inc] <http://bitscopic.com>
>
>

Mime
View raw message