phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil Gupta <anilgupt...@gmail.com>
Subject Re: Secondary index memory footprint
Date Tue, 22 Mar 2016 17:17:35 GMT
I dont know the meaning in_memory=false in context of phoenix. In context of hbase, that entire
data will be kept in memory( not just block cache). I m guessing, Phoenix is also doing same
thing.
By data size on disk, you can roughly estimate memory footprint. For example, You can load
one index into memory by setting in_memory flag and see how much memory it took. Then u can
estimate memory usage for other index on basis of your experiment.

Sent from my iPhone

> On Mar 22, 2016, at 9:05 AM, Sumit Nigam <sumit_only@yahoo.com> wrote:
> 
> Thank you Anil.
> 
> Any inputs on what does IN_MEMORY = false do at index creation time?
> 
> I get the separate table part. But not sure I understood why would memory footprint be
equal to data size. Would this secondary index reside entirely in memory? I guess only parts
of it would reside on block cache, right?
> 
> Thanks for such a prompt reply.
> Sumit
> 
> 
> 
> From: anil gupta <anilgupta84@gmail.com>
> To: "user@phoenix.apache.org" <user@phoenix.apache.org>; Sumit Nigam <sumit_only@yahoo.com>

> Sent: Tuesday, March 22, 2016 9:11 PM
> Subject: Re: Secondary index memory footprint
> 
> Global indexes are stored in a separate hbase table. So, you can estimate memory footprint
by looking at the data size of that index currently.
> 
> HTH,
> Anil Gupta
> 
> On Tue, Mar 22, 2016 at 7:19 AM, Sumit Nigam <sumit_only@yahoo.com> wrote:
> Hi,
> 
> I am trying to estimate what (if any) are the implications of accumulating data in phoenix
secondary index. I have a secondary index on 3 columns and would like to know if anyone has
an idea of how to estimate memory footprint of secondary index (if any) based on number of
entries in data table. The index is global and not local.
> 
> Plus, what is the impact of setting IN_MEMORY=true when index is created? 
> 
> Thanks,
> Sumit
> 
> 
> 
> -- 
> Thanks & Regards,
> Anil Gupta
> 
> 

Mime
View raw message