phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mujtaba Chohan <mujt...@apache.org>
Subject Re: Is first query to a table region way slower?
Date Tue, 30 Jan 2018 00:57:02 GMT
Just to remove one variable, can you repeat the same test after truncating
Phoenix Stats table? (either truncate SYSTEM.STATS from HBase shell or use
sql: delete from SYSTEM.STATS)

On Mon, Jan 29, 2018 at 4:36 PM, Pedro Boado <pedro.boado@gmail.com> wrote:

> Yes there is a rs.next().
>
> In fact if I run this SELECT * FROM table LIMIT 1 in a loop for four
> different tables in the same cluster I get relatively consistent response
> times across iterations, but same pattern if I execute the code over and
> over again. So basically first call per table is way slower.
>
> And for some reason call to TABLE4 is way slower than the others ( only
> difference is this table being quite big compared to the others ) .
>
> By hooking a jmeter to the vm I see new threads being created and
> destroyed in both hconnection and phoenix threadpools per loop ( I am not
> pooling connections ) , and quite a lot of network IO in the IPC Network
> thread to one of the RS during the 4 seconds the first query takes (
> basically this thread is doing Net IO during 60-70% of the 4200 msec ) .
>
>
>  Starting healthcheck '1'
>  Checking table TABLE1 state took 874 msec.
>  Checking table TABLE2 state took 471 msec.
>  Checking table TABLE3 state took 844 msec.
>  Checking table TABLE4 state took 4234 msec.
>  Starting healthcheck '2'
>  Checking table TABLE1 state took 103 msec.
>  Checking table TABLE2 state took 98 msec.
>  Checking table TABLE3 state took 78 msec.
>  Checking table TABLE4 state took 148 msec.
>  Starting healthcheck '3'
>  Checking table TABLE1 state took 351 msec.
>  Checking table TABLE2 state took 108 msec.
>  Checking table TABLE3 state took 84 msec.
>  Checking table TABLE4 state took 137 msec.
>  Starting healthcheck '4'
>  Checking table TABLE1 state took 102 msec.
>  Checking table TABLE2 state took 94 msec.
>  Checking table TABLE3 state took 77 msec.
>  Checking table TABLE4 state took 138 msec.
>  Starting healthcheck '5'
>  Checking table TABLE1 state took 103 msec.
>  Checking table TABLE2 state took 93 msec.
>  Checking table TABLE3 state took 77 msec.
>  Checking table TABLE4 state took 142 msec.
> ...
>
>
> Any other idea maybe?
>
>
>
>
>
> On 29 Jan 2018 01:55, "James Taylor" <jamestaylor@apache.org> wrote:
>
>> Did you do an rs.next() on the first query? Sounds related to HConnection
>> establishment. Also, least expensive query is SELECT 1 FROM T LIIMIT 1.
>>
>> Thanks,
>> James
>>
>> On Sun, Jan 28, 2018 at 5:39 PM Pedro Boado <pedro.boado@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm running into issues with a java springboot app that ends up querying
>>> a Phoenix cluster (from out of the cluster) through the non-thin client.
>>>
>>> Basically this application has a high latency - around 2 to 4 seconds -
>>> for the first query per  primary key to each region of a table with 180M
>>> records ( and 10 regions ) . Following calls - for different keys within
>>> the same region - have an average response time of ~60-80ms. No secondary
>>> indexes involved. No writes to the table at all during these queries.
>>>
>>> I don't think it's related to HConnection establishing as it was already
>>> stablished before the query ran ( a SELECT * FROM table LIMIT 1 is executed
>>> as soon as the datasource is created )
>>>
>>> I've been doing some quick profiling and almost all the time is spent
>>> inside the actual jdbc call.
>>>
>>> So here's the question: in your experience, is this normal behaviour -
>>> so I have to workaround the problem from application code warming up
>>> connections during app startup -  or is it something unusual? Any
>>> experience reducing first query latencies?
>>>
>>> Thanks!
>>>
>>>

Mime
View raw message