phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mac Fang <mac.fangzhen....@gmail.com>
Subject Re: Weird High Read throuput of SYSTEM.STATS
Date Tue, 25 Apr 2017 14:43:50 GMT
Is there any configuration that I can disable / decrease the frequencies
 of cache refreshing. I was noticed by a "phoenix.stats.updateFrequency",
but i am not 100% sure if it is reasonable to change this which can avoid
frequent cache refresh

On Fri, Apr 21, 2017 at 1:32 PM, Ankit Singhal <ankitsinghal59@gmail.com>
wrote:

> During bulkload, compaction might be getting triggered which is generating
> new stats ( and delete the stale one's with a scan) then the table
> cache(containg stats) will also get refreshed (with another scan).
> Depending upon the no. of time the compactions is triggered and no. of
> times the stats needs to be cached again might be correlated with your
> observation.
>
> On Wed, Apr 19, 2017 at 4:10 AM, Sergey Soldatov <sergeysoldatov@gmail.com
> > wrote:
>
>> The bulk load itself should not cause any read activity on SYSTEM.STATS.
>> But if you run something like select count(1) from table to check whether
>> the bulk load was successful, it may cause read activity on SYSTEM.STATS
>> table.
>>
>> Thanks,
>> Sergey
>>
>> On Wed, Apr 12, 2017 at 10:38 PM, Mac Fang <mac.fangzhen.dev@gmail.com>
>> wrote:
>>
>>> Josh,
>>>
>>> Thanks for the reply,
>>> It is a CDH vendor version (4.7.0_1.3.0). Mutations are reasonable when
>>> we do bulk load. Question is why we have the high read load of the
>>> SYSTEM.STATS table.
>>>
>>>
>>> On Thu, Apr 13, 2017 at 11:42 AM, Josh Elser <josh.elser@gmail.com>
>>> wrote:
>>>
>>>> What version of Phoenix are you using? (an Apache release? some
>>>> vendors' packaging?)
>>>>
>>>> Academically speaking, when you bulk load some data, the stats table
>>>> should get updated (otherwise the stats are wrong until a compaction
>>>> occurs), but I can't specifically point you at a line of code that is doing
>>>> this (nor am I 100% positive it happens).
>>>>
>>>>
>>>> On Wed, Apr 12, 2017 at 3:12 AM, Mac Fang <mac.fangzhen.dev@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, Guys,
>>>>>
>>>>>
>>>>> We are notices some weird high read throuput of the HBase,
>>>>>
>>>>> The HBase Read Rate
>>>>>
>>>>>
>>>>> ​
>>>>> And the SYSTEM.STATS Read Rate
>>>>>
>>>>>
>>>>> ​
>>>>> During that time frame, the system did not have a high QPS. However,
>>>>> it did import some data (some millions) data via the "Phoenix Bulk Load".
>>>>>
>>>>> The question is what does phoenix do with the SYSTEM.STATS table when
>>>>> we do "Bulk Load"?
>>>>>
>>>>> We did not find any clues while we looked into the code. Any hints ?
>>>>>
>>>>>
>>>>> --
>>>>> regards
>>>>> macf
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> regards
>>> macf
>>>
>>
>>
>


-- 
regards
macf

Mime
View raw message