phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: hotspot in System.catalog table
Date Fri, 13 Apr 2018 06:33:19 GMT
It’s client side, but that’ll only impact newly created tables. You’ll need
to use the ALTER TABLE command for existing tables.

On Thu, Apr 12, 2018 at 11:30 PM anil gupta <anilgupta84@gmail.com> wrote:

> I have set phoenix.default.update.cache.frequency=60000 in hbase-site.xml
> via ambari(we barely alter schema). Is this a client or server side
> property?
>
> On Thu, Apr 12, 2018 at 11:14 PM, anil gupta <anilgupta84@gmail.com>
> wrote:
>
>> I c. As per documentation[1], even for commits of upsert system.catalog
>> is called. IMO, ALWAYS seems to be really aggressive. Is there any reason
>> UPDATE_CACHE_FREQUENCY is set to ALWAYS by default? Do we plan to change
>> the default value to 5 or 10 sec? Thanks for your help.
>>
>>
>> PS: we were running into a lot of Phoenix scalability issues due to this.
>>
>> [1] https://phoenix.apache.org/language/index.html#options
>>
>> On Thu, Apr 12, 2018 at 11:06 PM, James Taylor <jamestaylor@apache.org>
>> wrote:
>>
>>> No, that won’t make a difference.
>>>
>>> On Thu, Apr 12, 2018 at 10:51 PM anil gupta <anilgupta84@gmail.com>
>>> wrote:
>>>
>>>> Thanks for quick reply, James. We will look into UPDATE_CACHE_FREQUENCY
>>>> property. If we just replace PS with Statement, will it fix the
>>>> problem(AFAIK, Statement is not compiled)?
>>>>
>>>> On Thu, Apr 12, 2018 at 10:43 PM, James Taylor <jamestaylor@apache.org>
>>>> wrote:
>>>>
>>>>> Try setting the UPDATE_CACHE_FREQUENCY table property (and configuring
>>>>> the phoenix.default.update.cache.frequency system-wide property). That'll
>>>>> prevent pinging the region hosting SYSTEM.CATALOG every time a query
is
>>>>> compiled. We've found value of even 5 seconds makes a big difference.
For
>>>>> more on that, see here[1] and here[2].
>>>>>
>>>>> In the future, we'll let the SYSTEM.CATALOG table span multiple
>>>>> regions - keep an eye on PHOENIX-3534.
>>>>>
>>>>> Thanks,
>>>>> James
>>>>>
>>>>> [1] https://phoenix.apache.org/#Altering
>>>>> [2] https://phoenix.apache.org/language/index.html#options
>>>>>
>>>>> On Thu, Apr 12, 2018 at 10:32 PM, anil gupta <anilgupta84@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> System.catalog table seems to be single region table(correct?). We
>>>>>> are currently facing a problem of hotspot on System.catalog table.
>>>>>> One of our app does around 4-5k select queries/sec. And, It is
>>>>>> creating a new preparedstatement everytime. I suspect that while
>>>>>> instantiating a new preparedstatement(contrary to Statement),
>>>>>> system.catalog table is queried first. Hence, it is resulting into
>>>>>> hotspotting. Is my analysis correct?
>>>>>>
>>>>>> (I have already suggested my colleagues to try using Statement
>>>>>> instead of PS if they have to create a new one everytime.)
>>>>>>
>>>>>> --
>>>>>> Thanks & Regards,
>>>>>> Anil Gupta
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks & Regards,
>>>> Anil Gupta
>>>>
>>>
>>
>>
>> --
>> Thanks & Regards,
>> Anil Gupta
>>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>

Mime
View raw message