phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abe Weinograd <...@flonet.com>
Subject Re: Date deserialization change in 3.0
Date Fri, 04 Apr 2014 15:43:42 GMT
Also, it turns out Squirrel and other tools we are using aren't terribly
happy with the UNSIGNED_DATE data type.  If we wanted to keep using DATE,
but when we had to serialize to bytes ourselves, it seems we have to flip
the bit for the sign using the decodeLong, correct?

Abe


On Fri, Apr 4, 2014 at 11:21 AM, Abe Weinograd <abe@flonet.com> wrote:

> HI James,
>
> Thanks for the feedback.  That's great.  We were blasting everything out
> and starting from scratch each time we went back and forth.  I will test it
> out.  We will be doing both M/R and interacting with JDBC.
>
> Thanks!
> Abe
>
>
> On Fri, Apr 4, 2014 at 11:16 AM, James Taylor <jamestaylor@apache.org>wrote:
>
>> Existing date/time types are now represented by UNSIGNED_DATE,
>> UNSIGNED_TIME, and UNSIGNED_TIMESTAMP. If you have code that matches on
>> DATE, TIME, or TIMESTAMP, it should be changed to use the UNSIGNED
>> equivalent.
>>
>> Also, not sure if you've done this yet, but you should run your tables
>> through our upgrade process:
>> http://phoenix.incubator.apache.org/upgrade_from_2_2.html
>>
>> This will (among other things), change the metadata as described above
>> (so if you're not explicitly referencing PDataType date/time types, but
>> driving off of the DatabaseMetaData calls, no change would be necessary).
>>
>> The reason we made this change, is so that we can support negative
>> date/time values (which we've had a few requests for).
>>
>> Thanks,
>> James
>>
>>
>>
>>
>> On Fri, Apr 4, 2014 at 8:03 AM, Abe Weinograd <abe@flonet.com> wrote:
>>
>>> We are bulk loading rows into our HBase tables directly via M/R ->
>>> HFiles.  This worked well in 2.2.3 and has broken with 3.0.
>>>
>>> We have a 3 column PK (int, date, int).  When I insert directly via the
>>> JDBC driver, everything works well.  When loading via the HFile, logging
>>> the Hex String from our M/R job and doing a scan on the table in the hbase
>>> shell, it shows this:
>>>
>>> The first 12 bytes (int then date) of our key looks like this on the
>>> scan:
>>> \x00\x00\x00\x06\x00\x00\x1D\x8F\xD5&\x84\x00
>>>
>>> x00\x00\x1D\x8F\xD5&\x84\x00 was supposed to represent 12/31/2999
>>> 00:00:00 UTC
>>> The long value is 32503593600000L
>>>
>>> We are using hbase client util Bytes.toBytes(32503593600000L) to get our
>>> value.  It seems like a change in the PDataType code causes this not to
>>> work anymore with 3.0.
>>>
>>> Does this make sense?  Is there an easy way for us to fix it?  I know
>>> the Bytes.toBytes() representation looks funny with the & in it, but it
>>> seems to work across the board.
>>>
>>> Any ideas how we can work around this?
>>>
>>> Thanks,
>>> Abe
>>>
>>
>>
>

Mime
View raw message