phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: Numbers low-level format in Phoenix
Date Thu, 08 Jan 2015 16:29:37 GMT
Hi Lavrenty,

Thanks for posting that hbase shell session info.

You're currently inserting strings like "[B@1e0f477f" into HBase. It's
actually not possible to insert binary (byte array) values into HBase
using the HBase shell like you're doing. Internally, the HBase shell
calls the equivalent to toString().getBytes() on any value that you
write to HBase.

If you want to write binary-encoded data to directly to HBase (without
using Phoenix) you'll need to use the Java API, and not the HBase
shell, or at least not the typical built-in shell commands.

- Gabriel


On Thu, Jan 8, 2015 at 1:45 PM, Lavrenty Eskin
<lavrenty.eskin@netcracker.com> wrote:
> There are full table scan from hbase shell. Table has been created from phoenix. Two
first values (UL=Unsigned Long) has been upserted from phoenix via jdbc driver (each upsert
generate two KV pairs).
>
> Another KV pairs has been inserted into the same table via hbase shell the way like this:
>
> put 'TEST_EVENTS', '913912383771315636525190071419507789000', 'ref:type_id', Bytes.toBytes(Long.parseLong("1234567890",
10)), 1419507789742
> Each of conversation try are explained in key string. No one of them can read correctly
then from phoenix. It receives ambigous numbers instead of 1234567890
>
> hbase(main):026:0> scan 't3_lavr'
> ROW                                             COLUMN+CELL
>  key-PHOENIX-UL-1234567890                      column=ref:_0, timestamp=1419935384987,
value=
>  key-PHOENIX-UL-1234567890                      column=ref:type_id, timestamp=1419935384987,
value=\x00\x00\x00\x00I\x96\x02\xD2
>
> key-PHOENIX-UL-1234567891                      column=ref:_0, timestamp=1419935641902,
value=
> key-PHOENIX-UL-1234567891                      column=ref:type_id, timestamp=1419935641902,
value=\x00\x00\x00\x00I\x96\x02\xD3
>
>  key-hbase-Bytes.fromHex(0x00000000499602D2)    column=ref:type_id, timestamp=1419507789742,
value=[B@6b3c7a2c
>  key-hbase-Bytes.fromHex(0x499602D2)            column=ref:type_id, timestamp=1419507789742,
value=[B@5385550c
>  key-hbase-Bytes.fromHex(0x80000000499602D2)    column=ref:type_id, timestamp=1419507789742,
value=[B@2c9a49b3
>  key-hbase-Bytes.fromHex(499602D2)              column=ref:type_id, timestamp=1419507789742,
value=[B@1e0f477f
>  key-hbase-Bytes.fromHex(D2029649)              column=ref:type_id, timestamp=1419507789742,
value=[B@722bfe37
>  key-hbase-Bytes.toStringBinary(Bytes.toBytes(l column=ref:type_id, timestamp=1419507789742,
value=\x5Cx00\x5Cx00\x5Cx00\x5Cx00I\x5Cx96\x5Cx02\x5CxD2
>  on))
>  key-hbase-Bytes.vintToBytes(bi)                column=ref:type_id, timestamp=1419507789742,
value=[B@16a0c14b
>  key-hbase-Bytes.vintToBytes(lon)               column=ref:type_id, timestamp=1419507789742,
value=[B@4b2d49e2
>  key-hbase-[B@499602D2                          column=ref:type_id, timestamp=1419507789742,
value=[B@499602D2
>  key-hbase-bytes.tobytes(1234567890)            column=ref:type_id, timestamp=1419507789742,
value=[B@337f23d0
>  key-hbase-bytes.tobytes(long)                  column=ref:type_id, timestamp=1419507789742,
value=[B@263da67d
>  key-hbase-bytes.tobytes(long.valueOf)          column=ref:type_id, timestamp=1419507789742,
value=[B@73e8c7d7
>  key-hbase-bytes.tobytes(long.valueOf10)        column=ref:type_id, timestamp=1419507789742,
value=[B@4778d705
>  key-hbase-bytes.tobytes(long.valueOf16)        column=ref:type_id, timestamp=1419507789742,
value=[B@5a83fb14
>
>
> -----Original Message-----
> From: Gabriel Reid [mailto:gabriel.reid@gmail.com]
> Sent: Thursday, January 08, 2015 3:13 PM
> To: user@phoenix.apache.org
> Subject: Re: Numbers low-level format in Phoenix
>
> It sounds like you might be storing the toString() representation of a byte array of
HBase.
>
> Could you post an example snippet of the code you're using to store things in HBase,
as well as a snippet of how you're reading this data in the HBase shell (or wherever you're
reading it).
>
> On Thu, Jan 8, 2015 at 12:44 PM, Lavrenty Eskin <lavrenty.eskin@netcracker.com>
wrote:
>> Hi Gabriel,
>>
>> But why then I receive in HBase shell two different string representation of the
byte array?
>> For byte arrays stored from phoenix - \x00\x00\x00\x00I\x96\x02\xD2 and [B@13217cf6
for stored from HBase.
>> The same time phoenix have wrong understanding of "[B@13217cf6" and
>> receives -323837278362736236786-like value instead 1234567890 I have to understand
the way to store values via hbase API but read from phoenix then correctly.
>>
>> Thanks
>>
>> -----Original Message-----
>> From: Gabriel Reid [mailto:gabriel.reid@gmail.com]
>> Sent: Thursday, January 08, 2015 2:09 PM
>> To: user@phoenix.apache.org
>> Subject: Re: Numbers low-level format in Phoenix
>>
>> Hi Lavrenty,
>>
>> Phoenix actually does store numerical data using byte arrays, in a
>> similar fashion to what the HBase bytes class does. There's more
>> information on the various types and their underlying encoding
>> available here: http://phoenix.apache.org/language/datatypes.html
>>
>> I'm guessing you got the string representation
>> ("\x00\x00\x00\x00I\x96\x02\xD2") from the HBase shell -- this is a string representation
of the byte array (containing 8 bytes) containing the serialized value of 1234567890. The
strings you posted like "[B@13217cf6" are the default string representation of byte arrays
in java. To convert these to a human-readable value (like what the HBase shell does), you
could do the following:
>>
>>     Bytes.toStringBinary(Bytes.toBytes(1234567890L));
>>
>> - Gabriel
>>
>> On Thu, Jan 8, 2015 at 9:44 AM, Lavrenty Eskin <lavrenty.eskin@netcracker.com>
wrote:
>>> Helo all,
>>> I'm surprised that phoenix store numbers not in HBase 'Byte' format. Looks like
a big overhead there, isn't it?
>>> Just takes 1234567890 value (0х499602D2):
>>> Phoenix stores that as string '\x00\x00\x00\x00I\x96\x02\xD2'
>>> But why it cannot store as in HBase format value=[B@499602d2 ?
>>>
>>> Another issue is why it write wrong bytes if you write from HBase shell? :
>>> Bytes.toBytes(1234567890)       -->>    value=[B@13217cf6,
>>> Bytes.toBytes(1234567890L)      -->>    value=[B@3caab4f
>>
>>
>> ________________________________
>> The information transmitted herein is intended only for the person or entity to which
it is addressed and may contain confidential, proprietary and/or privileged material. Any
review, retransmission, dissemination or other use of, or taking of any action in reliance
upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer.
>
>
> ________________________________
> The information transmitted herein is intended only for the person or entity to which
it is addressed and may contain confidential, proprietary and/or privileged material. Any
review, retransmission, dissemination or other use of, or taking of any action in reliance
upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer.

Mime
View raw message