phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas D'Silva" <tdsi...@salesforce.com>
Subject Re: Local Index data not replicating for older HBase versions
Date Wed, 22 May 2019 04:14:10 GMT
Your approach seems like the correct thing to do. HBase has stopped
supporting the 1.2 branch, so we also EOL'ed it, there will not be any more
releases targeting HBase 1.2. I would suggest that you upgrade to a later
version.

On Tue, Apr 30, 2019 at 8:55 PM Hieu Nguyen <hieu@box.com> wrote:

> Hi,
>
> We are on Phoenix 4.14-cdh5.11.  We are experiencing an issue where local
> index data is not being replicated through HBase replication.  As suggested
> in a previous email thread (
> https://lists.apache.org/thread.html/984fba3c8abd944846deefb3ea285195e0436b9181b9779feac39b59@%3Cuser.phoenix.apache.org%3E),
> we have enabled replication for the local indexes (the "L#0" column family
> on the same table).  We wrote an integration test to demonstrate this issue
> on top of 4.14-cdh5.11 branch (
> https://github.com/hnguyen08/phoenix/commit/3589cb45d941c6909fb3deb5f5abb0f8dfa78dd7
> ).
>
> After some investigation and debugging, we discovered the following:
> 1. Commit a2f4d7eebec621b58204a9eb78d552f18dcbcf24 (PHOENIX-3827) fixed
> the issue, but only in Phoenix for HBase1.3+.  It uses the
> miniBatchOp.addOperationsFromCP() API introduced in HBase1.3.
> Unfortunately, for the time being, we are stuck on cdh5.11 (based on
> HBase1.2).
> 2. IndexUtil.writeLocalUpdates() is called in both implementations of
> IndexCommitter, both taking skipWAL=true.  It seems like we'd actually want
> to not skip WAL to ensure that local-index updates are replicated correctly
> (since, as mentioned in the above email thread, "HBase-level replication of
> the data table will not trigger index updates").  After changing the
> skipWAL flag to false, the above integration test passes.
>
> Would it make sense to fix local-index replication for Phoenix versions on
> <= HBase1.2 by setting skipWAL to false for writeLocalUpdates?  As far as I
> can tell, it is not a perfect solution compared to using
> addOperationsFromCP() because the local index update still wouldn't be
> atomic.  But at least local-index updates can be replicated.  Let me know
> if I'm missing something.  Happy to file a JIRA or submit a fix.
>
> Thanks,
> -Hieu
>
>

Mime
View raw message