phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: resultSet.next() is slow
Date Wed, 12 Mar 2014 20:37:20 GMT
Your first call to ResultSet.next does all the job (loading all 10000 rows
from HBase) and probably not in a optimal way? One row at a time?
I do not know if default scan caching setting affects Phoenix or Phoenix
uses its own scan caching - this is question to Phoenix developers.

-Vladimir


On Wed, Mar 12, 2014 at 8:21 AM, alex kamil <alex.kamil@gmail.com> wrote:

> same result
>
> Execute Query : 401ms
>
> meta.getColumnCount : 1ms
>
> Time taken for first getNext() : 22593ms
>
> Time taken to  loop through : 22871ms
>
> Execute Query : 14ms
>
> meta.getColumnCount : 0ms
>
> Time taken for first getNext() : 21892ms
>
> Time taken to  loop through : 22041ms
>
>
> On Tue, Mar 11, 2014 at 7:15 PM, James Taylor <jamestaylor@apache.org>wrote:
>
>> What does the time look like if you run your query twice in the same JVM,
>> as I suspect you may be hitting a one-time cost:
>>
>> for (int i =0; i < 2; i++) {
>>
>> Statement stmt  = con.createStatement();
>>
>>   ResultSet resultSet =stmt.executeQuery(query);
>>
>>  Object o = null; int i=0;boolean isFirst = true;
>>
>>  long start = System.currentTimeMillis();
>>
>> * while(resultSet.next())*{
>>
>>   if (isFirst){
>>
>>   System.out.println("*Time taken for first getNext() *: " +
>> (System.currentTimeMillis() - start));
>>
>>   isFirst = false;
>>
>>   }
>>
>>   for (i = 0; i < colmax; ++i)
>>
>>   {
>>
>>   o = resultSet.getObject(i + 1);
>>
>>   if (o != null){
>>
>>    //rowKeys.add(o.toString());
>>
>>   }
>>
>>   }
>>
>>  }
>>
>>     System.out.println("Time taken to  loop through : " +
>> (System.currentTimeMillis() - start));
>>
>> }
>>
>>
>>
>>
>>
>> On Tue, Mar 11, 2014 at 12:57 PM, alex kamil <alex.kamil@gmail.com>wrote:
>>
>>> running the query on a table with 3 salt buckets (previously 128)
>>> brought down the time to single digits sec, what could be the possible
>>> explanation?
>>>
>>>
>>> On Tue, Mar 11, 2014 at 3:30 PM, alex kamil <alex.kamil@gmail.com>wrote:
>>>
>>>> running the below query in sqlline takes less than 4 sec  but when
>>>> calling from Java client the first resultSet.next() is much slower (>20
>>>> sec), any ideas?
>>>>
>>>>
>>>> client code
>>>>
>>>> String JDBC_DRIVER = "jdbc:phoenix:myip";
>>>>
>>>>  String query = "SELECT ROWKEY FROM mytable LIMIT 10000";
>>>>
>>>> //ROWKEY size=32bytes, mytable size=4M rows (128 regions on 3 servers
>>>> with 40GB heap)
>>>>
>>>>  Class.forName("com.salesforce.phoenix.jdbc.PhoenixDriver");
>>>>
>>>> DriverManager.getConnection(JDBC_DRIVER);
>>>>
>>>>    Connection con = DriverManager.getConnection(JDBC_DRIVER);
>>>>
>>>>  Statement stmt  = con.createStatement();
>>>>
>>>>  ResultSet resultSet =stmt.executeQuery(query);
>>>>
>>>>  Object o = null; int i=0;boolean isFirst = true;
>>>>
>>>>  long start = System.currentTimeMillis();
>>>>
>>>> * while(resultSet.next())*{
>>>>
>>>>   if (isFirst){
>>>>
>>>>   System.out.println("*Time taken for first getNext() *: " +
>>>> (System.currentTimeMillis() - start));
>>>>
>>>>   isFirst = false;
>>>>
>>>>   }
>>>>
>>>>   for (i = 0; i < colmax; ++i)
>>>>
>>>>   {
>>>>
>>>>   o = resultSet.getObject(i + 1);
>>>>
>>>>   if (o != null){
>>>>
>>>>    //rowKeys.add(o.toString());
>>>>
>>>>   }
>>>>
>>>>   }
>>>>
>>>>  }
>>>>
>>>>     System.out.println("Time taken to  loop through : " +
>>>> (System.currentTimeMillis() - start));
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message