phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Shen <>
Subject Re: Local Index data not replicating for older HBase versions
Date Thu, 23 May 2019 18:29:54 GMT
Thank you for clarifying Thomas!

On Thu, May 23, 2019 at 11:27 AM Thomas D'Silva <>

> The CDH branches have been maintained by Pedro, I am don't know what is
> the release plan for those branches.
> On Tue, May 21, 2019 at 11:13 PM William Shen <>
> wrote:
>> Hieu,
>> You're welcome to file a JIRA and submit a fix for the CDH branches
>> (4.14-cdh5.11, 4.x-cdh5.15, etc). I think while 4.x-HBase-1.2 is EOL, the
>> CDH branches are not. CDH 5.x is based on HBase-1.2, but it also contains a
>> concoction of 1.2.x, 1.3, 1.4 patches back-ported by Cloudera. (Thomas,
>> feel free to correct me if I misunderstood... but when we EOL'd HBase-1.2,
>> I was under the impression that the CDH branches will live on).
>> On Tue, May 21, 2019 at 9:24 PM Thomas D'Silva <>
>> wrote:
>>> Your approach seems like the correct thing to do. HBase has stopped
>>> supporting the 1.2 branch, so we also EOL'ed it, there will not be any more
>>> releases targeting HBase 1.2. I would suggest that you upgrade to a later
>>> version.
>>> On Tue, Apr 30, 2019 at 8:55 PM Hieu Nguyen <> wrote:
>>>> Hi,
>>>> We are on Phoenix 4.14-cdh5.11.  We are experiencing an issue where
>>>> local index data is not being replicated through HBase replication.  As
>>>> suggested in a previous email thread (
>>>> we have enabled replication for the local indexes (the "L#0" column family
>>>> on the same table).  We wrote an integration test to demonstrate this issue
>>>> on top of 4.14-cdh5.11 branch (
>>>> ).
>>>> After some investigation and debugging, we discovered the following:
>>>> 1. Commit a2f4d7eebec621b58204a9eb78d552f18dcbcf24 (PHOENIX-3827) fixed
>>>> the issue, but only in Phoenix for HBase1.3+.  It uses the
>>>> miniBatchOp.addOperationsFromCP() API introduced in HBase1.3.
>>>> Unfortunately, for the time being, we are stuck on cdh5.11 (based on
>>>> HBase1.2).
>>>> 2. IndexUtil.writeLocalUpdates() is called in both implementations of
>>>> IndexCommitter, both taking skipWAL=true.  It seems like we'd actually want
>>>> to not skip WAL to ensure that local-index updates are replicated correctly
>>>> (since, as mentioned in the above email thread, "HBase-level replication
>>>> the data table will not trigger index updates").  After changing the
>>>> skipWAL flag to false, the above integration test passes.
>>>> Would it make sense to fix local-index replication for Phoenix versions
>>>> on <= HBase1.2 by setting skipWAL to false for writeLocalUpdates?  As
>>>> as I can tell, it is not a perfect solution compared to using
>>>> addOperationsFromCP() because the local index update still wouldn't be
>>>> atomic.  But at least local-index updates can be replicated.  Let me know
>>>> if I'm missing something.  Happy to file a JIRA or submit a fix.
>>>> Thanks,
>>>> -Hieu

View raw message