phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <mich.talebza...@gmail.com>
Subject Re: Index in Phoenix view on Hbase is not updated
Date Sat, 22 Oct 2016 18:22:22 GMT
A workout I deployed was to rebuild the index immediately after bulk load
of data into Hbase table

ALTER INDEX MARKETDATAHBASE_IDX1 ON "marketDataHbase" REBUILD;



Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 22 October 2016 at 18:25, Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> Sorry Ted,
>
> This is the syntax for view
>
> create view "marketDataHbase" (PK VARCHAR PRIMARY KEY,
> “price_info”.”ticker” VARCHAR,"price_info"."timecreated" VARCHAR,
> "price_info"."price" VARCHAR);
>
> Thanks James for clarification.
>
> My understanding is that when one creates an index on a Phoenix view on an
> Hbase table,  a local index file is created in Phoenix with data as is and
> that Phoenix table has no way of knowing if new data is added to Hbase
> table.
>
> From an operational point of view unless index in Phoenix is immediately
> updated after any append to Hbase, then it is pretty useless.
>
> In my case data is inserted to Hbase table. I am just using Phoenix for
> data queries (DQ) as opposed to inserts.
>
> Regards,
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 22 October 2016 at 17:59, James Taylor <jamestaylor@apache.org> wrote:
>
>> Hi Mich,
>> Phoenix indexes are only updated if you use Phoenix APIs to input the
>> data.
>> Thanks,
>> James
>>
>>
>> On Saturday, October 22, 2016, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> The first statement creates index, not view.
>>>
>>> Can you check ?
>>>
>>> Cheers
>>>
>>> > On Oct 22, 2016, at 1:51 AM, Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I have a Hbase table that is populated via
>>> > org.apache.hadoop.hbase.mapreduce.ImportTsv
>>> > through bulk load ever 15 minutes. This works fine.
>>> >
>>> > In Phoenix I created a view on this table
>>> >
>>> > jdbc:phoenix:rhes564:2181> create index marketDataHbase_idx on
>>> > "marketDataHbase" ("price_info"."ticker", "price_info"."price",
>>> > "price_info"."timecreated");
>>> >
>>> > This also does what is supposed to do and shows correct count.
>>> >
>>> > I then created an index in Phoenix as below
>>> >
>>> > create index index_dx1 on "marketDataHbase"
>>> > ("price_info"."timecreated","price_info"."ticker",
>>> "price_info"."price");
>>> >
>>> > that showed the records OK at that time. I verified this using explain
>>> >
>>> >
>>> > 0: jdbc:phoenix:rhes564:2181> explain select count(1) from
>>> > "marketDataHbase";
>>> > +---------------------------------------------------------+
>>> > |                          PLAN                           |
>>> > +---------------------------------------------------------+
>>> > | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER INDEX_DX1  |
>>> > |     SERVER FILTER BY FIRST KEY ONLY                     |
>>> > |     SERVER AGGREGATE INTO SINGLE ROW                    |
>>> > +---------------------------------------------------------+
>>> >
>>> > Now the issue is that the above does not show new data since build in
>>> Hbase
>>> > table unless I do the following:
>>> >
>>> > 0: jdbc:phoenix:rhes564:2181> alter index INDEX_DX1 on
>>> "marketDataHbase"
>>> > rebuild;
>>> >
>>> >
>>> > Which is not what an index should do (The covered index should be
>>> > maintained automatically).
>>> > The simple issue is how to overcome this problem?
>>> >
>>> > As I understand the index in Phoenix ia another file independent of the
>>> > original phoenix view so I assume that this index file is not updated
>>> for
>>> > one reason or other?
>>> >
>>> > Thanks
>>>
>>
>

Mime
View raw message