phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: Phoenix upsert query time
Date Wed, 03 Aug 2016 15:23:11 GMT
Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY feature,
you can try setting the CURRENT_SCN property to Long.MAX_VALUE when you
connect. Another alternative would be to set it to one more than the
creation time of your tables. You can control the timestamp your tables are
created at by setting the CURRENT_SCN property on the connection making the
CREATE TABLE call. More on that here:
https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API

Thanks,
James

On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <anilgupta84@gmail.com> wrote:

> How many nodes you have in cluster? How many regions in that phoenix
> table? Can you do batch upserts?
> If Phoenix is querying for MetaData for every upsert in a
> preparedStatement then it definitely sounds like a bug/performance problem.
> IMO, 30 ms is not really that horrible of a performance given that you
> also have a secondary index. Have you tried increasing number of write
> clients?
>
> On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <anu1307@gmail.com>
> wrote:
>
>> I don't have option to update my CDH5.7. My upsert query is taking 30ms
>> with one fully covered index on table.
>>
>> I am using Spring JDBC template which uses prepared statement internally.
>>
>>
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.
>> getNamespaceName());
>> jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to
>> use explicit prepared statement? Link to code
>> <https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>
>>
>> On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <anilgupta84@gmail.com>
>> wrote:
>>
>>> Are you using a prepared statement for upserts? IMO, query should be
>>> compiled only once when prepared statement is used.
>>>
>>> Sent from my iPhone
>>>
>>> On Aug 2, 2016, at 7:56 AM, Samarth Jain <samarth@apache.org> wrote:
>>>
>>> Best bet is to updgrade your cloudera version to cdh5.7. It supports
>>> phoenix 4.7. See -
>>>
>>> http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351
>>>
>>>
>>> On Tuesday, August 2, 2016, anupama agarwal <anu1307@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We need to insert to do 10K upserts per second in our phoenix table. We
>>>> are using phoenix 4.5 with CDH5.4.4.  Problem is each upsert query is
>>>> making rpc call to read metadata about the table.
>>>>
>>>> "o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved
>>>> stale table"
>>>>
>>>> Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7
>>>> version. Any way to apply this in Phoenix 4.5 by caching metadata at client
>>>> side? My table schema will change only once in 6 months. This would be
>>>> really helpful for me.
>>>>
>>>> Regards
>>>> Anupama
>>>>
>>>>
>>>>
>>>>
>>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>

Mime
View raw message