phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marica Tan <marica....@gmail.com>
Subject How to manually generate a salted row key?
Date Tue, 13 Sep 2016 05:59:51 GMT
Hi,

We have a table created via phoenix with salt bucket, but we're using HBase
API to insert records since we need to manually set the HBase version and I
believe that it isn't possible via phoenix.

Our table has a composite key (firstKey varchar, secondKey varchar,
thirdKey varchar) and when we do a select query with a where condition on
the firstKey, not all records are retrieved.

We checked the value of the firstKey and it should return 10 records, but
we're only getting 7.

If we do a where firstKey = 'someValue' we get 7
If we do a where firstKey like '%someValue' we get 10

So we think the main culprit is the way we generate the row key. Here's the
code:

 def generateRowKey(nBuckets: Integer, compositeKeys: String*): Array[Byte]
= {

  val keys = compositeKeys.tail.foldLeft(ArrayBuffer[Array[Byte]](convertToByteArray(compositeKeys.head)))((a,
b) => {
    a += QueryConstants.SEPARATOR_BYTE_ARRAY
    a += convertToByteArray(b)
  })

  val rowKey = ByteUtil.concat(QueryConstants.SEPARATOR_BYTE_ARRAY,
keys.toSeq: _*)

  updateSaltingByte(rowKey, nBuckets)

  rowKey
}

def convertToByteArray(key: String): Array[Byte] = key match {
  case x if StringUtils.isNotBlank(x) => Bytes.toBytes(x)
  case _ => ByteUtil.EMPTY_BYTE_ARRAY
}

def updateSaltingByte(rowKey: Array[Byte], nBuckets: Integer): Unit = {
  if (nBuckets > 0) {
    rowKey(0) = SaltingUtil.getSaltingByte(rowKey,
      SaltingUtil.NUM_SALTING_BYTES, rowKey.length -
SaltingUtil.NUM_SALTING_BYTES, nBuckets)
  }
}


Btw, we're using phoenix 4.4


Thanks,
--
Marica Tan

Mime
View raw message