phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Shen <>
Subject Re: Local Index data not replicating for older HBase versions
Date Wed, 22 May 2019 06:13:29 GMT

You're welcome to file a JIRA and submit a fix for the CDH branches
(4.14-cdh5.11, 4.x-cdh5.15, etc). I think while 4.x-HBase-1.2 is EOL, the
CDH branches are not. CDH 5.x is based on HBase-1.2, but it also contains a
concoction of 1.2.x, 1.3, 1.4 patches back-ported by Cloudera. (Thomas,
feel free to correct me if I misunderstood... but when we EOL'd HBase-1.2,
I was under the impression that the CDH branches will live on).

On Tue, May 21, 2019 at 9:24 PM Thomas D'Silva <>

> Your approach seems like the correct thing to do. HBase has stopped
> supporting the 1.2 branch, so we also EOL'ed it, there will not be any more
> releases targeting HBase 1.2. I would suggest that you upgrade to a later
> version.
> On Tue, Apr 30, 2019 at 8:55 PM Hieu Nguyen <> wrote:
>> Hi,
>> We are on Phoenix 4.14-cdh5.11.  We are experiencing an issue where local
>> index data is not being replicated through HBase replication.  As suggested
>> in a previous email thread (
>> we have enabled replication for the local indexes (the "L#0" column family
>> on the same table).  We wrote an integration test to demonstrate this issue
>> on top of 4.14-cdh5.11 branch (
>> ).
>> After some investigation and debugging, we discovered the following:
>> 1. Commit a2f4d7eebec621b58204a9eb78d552f18dcbcf24 (PHOENIX-3827) fixed
>> the issue, but only in Phoenix for HBase1.3+.  It uses the
>> miniBatchOp.addOperationsFromCP() API introduced in HBase1.3.
>> Unfortunately, for the time being, we are stuck on cdh5.11 (based on
>> HBase1.2).
>> 2. IndexUtil.writeLocalUpdates() is called in both implementations of
>> IndexCommitter, both taking skipWAL=true.  It seems like we'd actually want
>> to not skip WAL to ensure that local-index updates are replicated correctly
>> (since, as mentioned in the above email thread, "HBase-level replication of
>> the data table will not trigger index updates").  After changing the
>> skipWAL flag to false, the above integration test passes.
>> Would it make sense to fix local-index replication for Phoenix versions
>> on <= HBase1.2 by setting skipWAL to false for writeLocalUpdates?  As far
>> as I can tell, it is not a perfect solution compared to using
>> addOperationsFromCP() because the local index update still wouldn't be
>> atomic.  But at least local-index updates can be replicated.  Let me know
>> if I'm missing something.  Happy to file a JIRA or submit a fix.
>> Thanks,
>> -Hieu

View raw message