phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: Bad performance of the first resultset.next()
Date Wed, 19 Apr 2017 16:41:33 GMT
I'm guessing that you're using a version of HDP? If you're using those 
versions from Apache, please update as they're dreadfully out of date.

What is the DDL of the table you're reading from? Do you have any 
secondary indexes on this table (if so, on what columns)? What kind of 
query are you running? What is the output of `EXPLAIN <sql>` for these 
queries?

For example, this could be easily explained if Phoenix is reading the 
data table and post-filtering records. It could take significant amounts 
of time to read data that does not satisfy your query until you get to 
some data which does...

Lee wrote:
> Hi all,
>
> currently I am struggling with a performance issue in my Rest API. The API
> receives loads of requests coming from frontend in parallel, makes SQL
> queries using Phoenix JDBC driver to fetch data from HBase. For each
> request, the api makes only 1 query to phoenix/hbase.
>
> I find out, that the very first ResultSet.next() always take long time to
> get data from hbase. As far as I know, it gets data in batch, stores them in
> main memory, enables the following next() to get data directly from main
> memory and thus save up the network overload. The following next() takes
> usually less than 10 ms to finish.
>
> Sometimes this first next() takes more than 10 seconds and gets increasing
> from time to time to 30 or even 40 secs. For each query we expect maximal
> 25000 rows.
> What can be here the bottleneck for this behaviour?
>
> Some information regarding my setup:
> Hadoop: 2.7.1
> HBase: 1.1.2
> Phoenix: 4.4.0 Hbase 1.1
> Table has 605M rows - salted in 7 buckets - 26 regions across 10 region
> servers
> phoenix.query.threadPoolSize = 128 (default)
> phoenix.query.queueSize = 5000 (default)
>
> Thanks!
> Lee
>
>
>
> --
> View this message in context: http://apache-phoenix-user-list.1124778.n5.nabble.com/Bad-performance-of-the-first-resultset-next-tp3424.html
> Sent from the Apache Phoenix User List mailing list archive at Nabble.com.

Mime
View raw message