phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Heather, James (ELS)" <james.heat...@elsevier.com>
Subject Re: Phoenix upsert query time
Date Wed, 03 Aug 2016 18:17:30 GMT
Can there be disastrous consequences of a high value for UPDATE_CACHE_FREQUENCY? What happens
if a node pushes a change when it's got a stale cache?

On 3 August 2016 16:23:15 James Taylor <jamestaylor@apache.org> wrote:

Short of upgrading to 4.7 to leverage the UPDATE_CACHE_FREQUENCY feature, you can try setting
the CURRENT_SCN property to Long.MAX_VALUE when you connect. Another alternative would be
to set it to one more than the creation time of your tables. You can control the timestamp
your tables are created at by setting the CURRENT_SCN property on the connection making the
CREATE TABLE call. More on that here: https://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API

Thanks,
James

On Tue, Aug 2, 2016 at 2:59 PM, anil gupta <anilgupta84@gmail.com<mailto:anilgupta84@gmail.com>>
wrote:
How many nodes you have in cluster? How many regions in that phoenix table? Can you do batch
upserts?
If Phoenix is querying for MetaData for every upsert in a preparedStatement then it definitely
sounds like a bug/performance problem.
IMO, 30 ms is not really that horrible of a performance given that you also have a secondary
index. Have you tried increasing number of write clients?

On Tue, Aug 2, 2016 at 10:54 AM, anupama agarwal <anu1307@gmail.com<mailto:anu1307@gmail.com>>
wrote:
I don't have option to update my CDH5.7. My upsert query is taking 30ms with one fully covered
index on table.

I am using Spring JDBC template which uses prepared statement internally.

<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>


        NamedParameterJdbcTemplate jdbcTemplate = jdbcTemplateMap.get(event.getNamespaceName());
        jdbcTemplate.update(upsertQuery, event.getFieldMapPair()); Do I need to use explicit
prepared statement? Link to code<https://github.com/Flipkart/aesop/blob/master/data-layers/data-layer-hbase/src/main/java/com/flipkart/aesop/hbasedatalayer/upsert/HBaseUpsertDataLayer.java>

On Tue, Aug 2, 2016 at 10:31 PM, Anil Gupta <anilgupta84@gmail.com<mailto:anilgupta84@gmail.com>>
wrote:
Are you using a prepared statement for upserts? IMO, query should be compiled only once when
prepared statement is used.

Sent from my iPhone

On Aug 2, 2016, at 7:56 AM, Samarth Jain <samarth@apache.org<mailto:samarth@apache.org>>
wrote:

Best bet is to updgrade your cloudera version to cdh5.7. It supports phoenix 4.7. See -
http://community.cloudera.com/t5/Cloudera-Labs/ANNOUNCE-Third-installment-of-Cloudera-Labs-packaging-of-Apache/m-p/42351#U42351


On Tuesday, August 2, 2016, anupama agarwal <anu1307@gmail.com<mailto:anu1307@gmail.com>>
wrote:
Hi,

We need to insert to do 10K upserts per second in our phoenix table. We are using phoenix
4.5 with CDH5.4.4.  Problem is each upsert query is making rpc call to read metadata about
the table.

"o.a.phoenix.compile.FromCompiler.createTableRef 446 - Re-resolved stale table"

Phoenix have added feature to specify "UPDATE_CACHE_FREQUENCY" in 4.7 version. Any way to
apply this in Phoenix 4.5 by caching metadata at client side? My table schema will change
only once in 6 months. This would be really helpful for me.

Regards
Anupama







--
Thanks & Regards,
Anil Gupta


________________________________

Elsevier Limited. Registered Office: The Boulevard, Langford Lane, Kidlington, Oxford, OX5
1GB, United Kingdom, Registration No. 1982084, Registered in England and Wales.

Mime
View raw message