phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vivek Paranthaman <vivek.parantha...@bizruntime.com>
Subject Re: Scanning big region parallely
Date Thu, 20 Oct 2016 06:11:12 GMT
Hi,

How to get the Schema out of Hbase using Phoenix driver?
metadata.getSchemas() produce..
http://stackoverflow.com/questions/40123547/phoenixioexception-nomethodexception-deferredfileoutputstramstring-string-fi

On Thu, Oct 20, 2016 at 11:31 AM, Sanooj Padmakumar <p.sanooj@gmail.com>
wrote:

> Hi James,
>
> We are loading data from Phoenix tables into in-memory database. Based on
> the query we are finding the number of phoenix input splits (similar to
> what happens inside phoenix MR) and loads the data into in-memory database
> in parallel. So we are looking for ways to further parallelize the scan of
> a larger region.
>
> As you mentioned phoenix does this for all its queries. Can you please
> provide pointers to the phoenix code where this happens ?
>
> Thanks for the prompt response.
>
> Thanks
> Sanooj Padmakumar
>
> On Wed, Oct 19, 2016 at 11:22 PM, James Taylor <jamestaylor@apache.org>
> wrote:
>
>> Hi Sanooj,
>> I'm not sure what you mean by "loading data in our HBase table into
>> in-memory", but Phoenix queries tables in parallel, even within a region
>> depending on how you've configured statistics and guideposts as described
>> here: http://phoenix.apache.org/update_statistics.html
>>
>> Thanks,
>> James
>>
>>
>> On Wednesday, October 19, 2016, Sanooj Padmakumar <p.sanooj@gmail.com>
>> wrote:
>>
>>> Hi All
>>>
>>>
>>> We are are loading data in our HBase table into in-memory. For this we
>>> provide a start row and end row and scan the hbase regions. Is there a way
>>> we can scan a big region in parallel to fasten this whole process ? Any
>>> help/pointers on this will be of great help.
>>>
>>> --
>>> Thanks,
>>> Sanooj Padmakumar
>>>
>>
>
>
> --
> Thanks,
> Sanooj Padmakumar
>

Mime
View raw message