phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Nigam <sumit_o...@yahoo.com>
Subject Re: ResultSet size
Date Tue, 06 Oct 2015 17:33:53 GMT
Thanks Samarth and Jesse.
So, in effect setting the batch size (say, stmt.setFetchSize()) ensures that only that much
data is copied over the wire en-mass? And 'behind-the-scenes', Phoenix driver would fetch
next batch as each fetch size is exhausted?
Thanks,Sumit 
      From: Samarth Jain <samarth.jain@gmail.com>
 To: "user@phoenix.apache.org" <user@phoenix.apache.org> 
Cc: Sumit Nigam <sumit_only@yahoo.com> 
 Sent: Tuesday, October 6, 2015 9:20 PM
 Subject: Re: ResultSet size
   
To add to what Jesse said, you can override the default scanner fetch size programmatically via
Phoenix by calling statement.setFetchSize(int).
On Tuesday, October 6, 2015, Jesse Yates <jesse.k.yates@gmail.com> wrote:



So HBase (and by extension, Phoenix) does not do true "streaming" of rows - rows are copied
into memory from the HFiles and then eventually copied en-mass onto the wire. On the client
they are pulled off in chunks and paged through by the client scanner. You can control the
batch size (amount of rows in each 'page') via the usual HBase client configurations

On Tue, Oct 6, 2015 at 8:38 AM Sumit Nigam <sumit_only@yahoo.com> wrote:

Hi,
Does Phoenix buffer the result set internally? I mean when I fire a huge skip scan IN clause,
then the data being returned may be too huge to contain in memory. So, ideally I'd like to
stream data through the resultset.next() method. So, my question is does Phoenix really stream
results? 
And if so, is there a way to control how much is loads in one time in client side before its
next() fetches next batch of data from region servers to client?
Best regards,Sumit



  
Mime
View raw message