phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lavrenty Eskin <lavrenty.es...@NetCracker.com>
Subject RE: Numbers low-level format in Phoenix
Date Thu, 08 Jan 2015 12:45:29 GMT
There are full table scan from hbase shell. Table has been created from phoenix. Two first
values (UL=Unsigned Long) has been upserted from phoenix via jdbc driver (each upsert generate
two KV pairs).

Another KV pairs has been inserted into the same table via hbase shell the way like this:

put 'TEST_EVENTS', '913912383771315636525190071419507789000', 'ref:type_id', Bytes.toBytes(Long.parseLong("1234567890",
10)), 1419507789742
Each of conversation try are explained in key string. No one of them can read correctly then
from phoenix. It receives ambigous numbers instead of 1234567890

hbase(main):026:0> scan 't3_lavr'
ROW                                             COLUMN+CELL
 key-PHOENIX-UL-1234567890                      column=ref:_0, timestamp=1419935384987, value=
 key-PHOENIX-UL-1234567890                      column=ref:type_id, timestamp=1419935384987,
value=\x00\x00\x00\x00I\x96\x02\xD2

key-PHOENIX-UL-1234567891                      column=ref:_0, timestamp=1419935641902, value=
key-PHOENIX-UL-1234567891                      column=ref:type_id, timestamp=1419935641902,
value=\x00\x00\x00\x00I\x96\x02\xD3

 key-hbase-Bytes.fromHex(0x00000000499602D2)    column=ref:type_id, timestamp=1419507789742,
value=[B@6b3c7a2c
 key-hbase-Bytes.fromHex(0x499602D2)            column=ref:type_id, timestamp=1419507789742,
value=[B@5385550c
 key-hbase-Bytes.fromHex(0x80000000499602D2)    column=ref:type_id, timestamp=1419507789742,
value=[B@2c9a49b3
 key-hbase-Bytes.fromHex(499602D2)              column=ref:type_id, timestamp=1419507789742,
value=[B@1e0f477f
 key-hbase-Bytes.fromHex(D2029649)              column=ref:type_id, timestamp=1419507789742,
value=[B@722bfe37
 key-hbase-Bytes.toStringBinary(Bytes.toBytes(l column=ref:type_id, timestamp=1419507789742,
value=\x5Cx00\x5Cx00\x5Cx00\x5Cx00I\x5Cx96\x5Cx02\x5CxD2
 on))
 key-hbase-Bytes.vintToBytes(bi)                column=ref:type_id, timestamp=1419507789742,
value=[B@16a0c14b
 key-hbase-Bytes.vintToBytes(lon)               column=ref:type_id, timestamp=1419507789742,
value=[B@4b2d49e2
 key-hbase-[B@499602D2                          column=ref:type_id, timestamp=1419507789742,
value=[B@499602D2
 key-hbase-bytes.tobytes(1234567890)            column=ref:type_id, timestamp=1419507789742,
value=[B@337f23d0
 key-hbase-bytes.tobytes(long)                  column=ref:type_id, timestamp=1419507789742,
value=[B@263da67d
 key-hbase-bytes.tobytes(long.valueOf)          column=ref:type_id, timestamp=1419507789742,
value=[B@73e8c7d7
 key-hbase-bytes.tobytes(long.valueOf10)        column=ref:type_id, timestamp=1419507789742,
value=[B@4778d705
 key-hbase-bytes.tobytes(long.valueOf16)        column=ref:type_id, timestamp=1419507789742,
value=[B@5a83fb14


-----Original Message-----
From: Gabriel Reid [mailto:gabriel.reid@gmail.com]
Sent: Thursday, January 08, 2015 3:13 PM
To: user@phoenix.apache.org
Subject: Re: Numbers low-level format in Phoenix

It sounds like you might be storing the toString() representation of a byte array of HBase.

Could you post an example snippet of the code you're using to store things in HBase, as well
as a snippet of how you're reading this data in the HBase shell (or wherever you're reading
it).

On Thu, Jan 8, 2015 at 12:44 PM, Lavrenty Eskin <lavrenty.eskin@netcracker.com> wrote:
> Hi Gabriel,
>
> But why then I receive in HBase shell two different string representation of the byte
array?
> For byte arrays stored from phoenix - \x00\x00\x00\x00I\x96\x02\xD2 and [B@13217cf6 for
stored from HBase.
> The same time phoenix have wrong understanding of "[B@13217cf6" and
> receives -323837278362736236786-like value instead 1234567890 I have to understand the
way to store values via hbase API but read from phoenix then correctly.
>
> Thanks
>
> -----Original Message-----
> From: Gabriel Reid [mailto:gabriel.reid@gmail.com]
> Sent: Thursday, January 08, 2015 2:09 PM
> To: user@phoenix.apache.org
> Subject: Re: Numbers low-level format in Phoenix
>
> Hi Lavrenty,
>
> Phoenix actually does store numerical data using byte arrays, in a
> similar fashion to what the HBase bytes class does. There's more
> information on the various types and their underlying encoding
> available here: http://phoenix.apache.org/language/datatypes.html
>
> I'm guessing you got the string representation
> ("\x00\x00\x00\x00I\x96\x02\xD2") from the HBase shell -- this is a string representation
of the byte array (containing 8 bytes) containing the serialized value of 1234567890. The
strings you posted like "[B@13217cf6" are the default string representation of byte arrays
in java. To convert these to a human-readable value (like what the HBase shell does), you
could do the following:
>
>     Bytes.toStringBinary(Bytes.toBytes(1234567890L));
>
> - Gabriel
>
> On Thu, Jan 8, 2015 at 9:44 AM, Lavrenty Eskin <lavrenty.eskin@netcracker.com>
wrote:
>> Helo all,
>> I'm surprised that phoenix store numbers not in HBase 'Byte' format. Looks like a
big overhead there, isn't it?
>> Just takes 1234567890 value (0х499602D2):
>> Phoenix stores that as string '\x00\x00\x00\x00I\x96\x02\xD2'
>> But why it cannot store as in HBase format value=[B@499602d2 ?
>>
>> Another issue is why it write wrong bytes if you write from HBase shell? :
>> Bytes.toBytes(1234567890)       -->>    value=[B@13217cf6,
>> Bytes.toBytes(1234567890L)      -->>    value=[B@3caab4f
>
>
> ________________________________
> The information transmitted herein is intended only for the person or entity to which
it is addressed and may contain confidential, proprietary and/or privileged material. Any
review, retransmission, dissemination or other use of, or taking of any action in reliance
upon, this information by persons or entities other than the intended recipient is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer.


________________________________
The information transmitted herein is intended only for the person or entity to which it is
addressed and may contain confidential, proprietary and/or privileged material. Any review,
retransmission, dissemination or other use of, or taking of any action in reliance upon, this
information by persons or entities other than the intended recipient is prohibited. If you
received this in error, please contact the sender and delete the material from any computer.
Mime
View raw message