phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemal Parekh <he...@bitscopic.com>
Subject Re: Concurrent phoenix queries throw unable to create new native thread error
Date Thu, 11 Oct 2018 02:18:43 GMT
Yes, the values were initially set to 128 for phoenix.query.threadPoolSize
and 5000 for phoenix.query.queueSize. After getting thread error, these
values were increased to higher value few times but thread error continued.


Thanks,
Hemal


On Wed, Oct 10, 2018 at 9:38 PM Jaanai Zhang <cloud.poster@gmail.com> wrote:

> Often times, concurrent queries fail with "java.lang.OutOfMemoryError:
>> unable to create new native thread
>>
>
> You can try to adjust VM options which your client program.
>
> connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
>> connectionProps.setProperty("phoenix.query.querySize", "40000")
>
>
> Did you try to decrease values of the above configurations?
> ----------------------------------------
>    Jaanai Zhang
>    Best regards!
>
>
>
> Hemal Parekh <hemal@bitscopic.com> 于2018年10月11日周四 上午1:18写道:
>
>> limits.conf has following which I thought were sufficient. I will check
>> if these limits are getting exceeded.
>>
>> *   -   nofile   32768
>> *   -   nproc   65536
>>
>>
>>
>> Thanks,
>> Hemal
>>
>>
>> On Wed, Oct 10, 2018 at 12:40 PM Pedro Boado <pedro.boado@gmail.com>
>> wrote:
>>
>>> Are you reaching any of the ulimits for the user running your
>>> application?
>>>
>>> On Wed, 10 Oct 2018, 17:00 Hemal Parekh, <hemal@bitscopic.com> wrote:
>>>
>>>> We have an analytical application running concurrent phoenix queries
>>>> against Hortonworks HDP 2.6 cluster. Application uses phoenix JDBC
>>>> connection to run queries. Often times, concurrent queries fail with
>>>> "java.lang.OutOfMemoryError: unable to create new native thread" error.
>>>> JDBC connection sets following phoenix properties.
>>>>
>>>> connectionProps.setProperty("phoenix.query.threadPoolSize", "2000")
>>>> connectionProps.setProperty("phoenix.query.querySize", "40000")
>>>>
>>>> Phoenix version is 4.7 and Hbase version is 1.1.2, The HDP cluster has
>>>> six regionservers on six data nodes. Concurrent queries run against
>>>> different phoenix tables, some are small having few million records and
>>>> some are big having few billions records. Most of the queries do not have
>>>> joins,  where clause includes conditions on rowkey and few nonkey columns.
>>>> Queries with joins (which are on small tables) have used
>>>> USE_SORT_MERGE_JOIN hint.
>>>>
>>>> Are there other phoenix properties which need to be set on JDBC
>>>> connection? Are above values for phoenix.query.threadPoolSize and phoenix.query.querySize
>>>> enough to handle concurrent query use case? We have changed these two
>>>> properties couple of times to increase their values but the error still
>>>> remains the same.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Hemal Parekh
>>>>
>>>>
>>>>
>>>>
>>
>> --
>>
>> Hemal Parekh
>> Senior Data Warehouse Architect
>> m. 240.449.4396
>> [image: Bitscopic Inc] <http://bitscopic.com>
>>
>>

-- 

Hemal Parekh
Senior Data Warehouse Architect
m. 240.449.4396
[image: Bitscopic Inc] <http://bitscopic.com>

Mime
View raw message