phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abe Weinograd <...@flonet.com>
Subject Re: Date deserialization change in 3.0
Date Fri, 04 Apr 2014 15:21:38 GMT
HI James,

Thanks for the feedback.  That's great.  We were blasting everything out
and starting from scratch each time we went back and forth.  I will test it
out.  We will be doing both M/R and interacting with JDBC.

Thanks!
Abe


On Fri, Apr 4, 2014 at 11:16 AM, James Taylor <jamestaylor@apache.org>wrote:

> Existing date/time types are now represented by UNSIGNED_DATE,
> UNSIGNED_TIME, and UNSIGNED_TIMESTAMP. If you have code that matches on
> DATE, TIME, or TIMESTAMP, it should be changed to use the UNSIGNED
> equivalent.
>
> Also, not sure if you've done this yet, but you should run your tables
> through our upgrade process:
> http://phoenix.incubator.apache.org/upgrade_from_2_2.html
>
> This will (among other things), change the metadata as described above (so
> if you're not explicitly referencing PDataType date/time types, but driving
> off of the DatabaseMetaData calls, no change would be necessary).
>
> The reason we made this change, is so that we can support negative
> date/time values (which we've had a few requests for).
>
> Thanks,
> James
>
>
>
>
> On Fri, Apr 4, 2014 at 8:03 AM, Abe Weinograd <abe@flonet.com> wrote:
>
>> We are bulk loading rows into our HBase tables directly via M/R ->
>> HFiles.  This worked well in 2.2.3 and has broken with 3.0.
>>
>> We have a 3 column PK (int, date, int).  When I insert directly via the
>> JDBC driver, everything works well.  When loading via the HFile, logging
>> the Hex String from our M/R job and doing a scan on the table in the hbase
>> shell, it shows this:
>>
>> The first 12 bytes (int then date) of our key looks like this on the
>> scan:
>> \x00\x00\x00\x06\x00\x00\x1D\x8F\xD5&\x84\x00
>>
>> x00\x00\x1D\x8F\xD5&\x84\x00 was supposed to represent 12/31/2999
>> 00:00:00 UTC
>> The long value is 32503593600000L
>>
>> We are using hbase client util Bytes.toBytes(32503593600000L) to get our
>> value.  It seems like a change in the PDataType code causes this not to
>> work anymore with 3.0.
>>
>> Does this make sense?  Is there an easy way for us to fix it?  I know the
>> Bytes.toBytes() representation looks funny with the & in it, but it seems
>> to work across the board.
>>
>> Any ideas how we can work around this?
>>
>> Thanks,
>> Abe
>>
>
>

Mime
View raw message