phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanooj Padmakumar <p.san...@gmail.com>
Subject Re: Scanning big region parallely
Date Thu, 20 Oct 2016 06:01:44 GMT
Hi James,

We are loading data from Phoenix tables into in-memory database. Based on
the query we are finding the number of phoenix input splits (similar to
what happens inside phoenix MR) and loads the data into in-memory database
in parallel. So we are looking for ways to further parallelize the scan of
a larger region.

As you mentioned phoenix does this for all its queries. Can you please
provide pointers to the phoenix code where this happens ?

Thanks for the prompt response.

Thanks
Sanooj Padmakumar

On Wed, Oct 19, 2016 at 11:22 PM, James Taylor <jamestaylor@apache.org>
wrote:

> Hi Sanooj,
> I'm not sure what you mean by "loading data in our HBase table into
> in-memory", but Phoenix queries tables in parallel, even within a region
> depending on how you've configured statistics and guideposts as described
> here: http://phoenix.apache.org/update_statistics.html
>
> Thanks,
> James
>
>
> On Wednesday, October 19, 2016, Sanooj Padmakumar <p.sanooj@gmail.com>
> wrote:
>
>> Hi All
>>
>>
>> We are are loading data in our HBase table into in-memory. For this we
>> provide a start row and end row and scan the hbase regions. Is there a way
>> we can scan a big region in parallel to fasten this whole process ? Any
>> help/pointers on this will be of great help.
>>
>> --
>> Thanks,
>> Sanooj Padmakumar
>>
>


-- 
Thanks,
Sanooj Padmakumar

Mime
View raw message