phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abe Weinograd <...@flonet.com>
Subject Re: Date deserialization change in 3.0
Date Fri, 04 Apr 2014 18:38:08 GMT
Can we pull Phoenix jars in our POM and reference the encode/decode long?
 Is it something we can/should rely on?

Abe


On Fri, Apr 4, 2014 at 11:59 AM, James Taylor <jamestaylor@apache.org>wrote:

> Yes, correct. You can use CAST to cast back and forth between signed and
> unsigned date/time types (or back and forth to BIGINT as well).
>
>
> On Fri, Apr 4, 2014 at 8:43 AM, Abe Weinograd <abe@flonet.com> wrote:
>
>> Also, it turns out Squirrel and other tools we are using aren't terribly
>> happy with the UNSIGNED_DATE data type.  If we wanted to keep using DATE,
>> but when we had to serialize to bytes ourselves, it seems we have to flip
>> the bit for the sign using the decodeLong, correct?
>>
>> Abe
>>
>>
>> On Fri, Apr 4, 2014 at 11:21 AM, Abe Weinograd <abe@flonet.com> wrote:
>>
>>> HI James,
>>>
>>> Thanks for the feedback.  That's great.  We were blasting everything out
>>> and starting from scratch each time we went back and forth.  I will test it
>>> out.  We will be doing both M/R and interacting with JDBC.
>>>
>>> Thanks!
>>> Abe
>>>
>>>
>>> On Fri, Apr 4, 2014 at 11:16 AM, James Taylor <jamestaylor@apache.org>wrote:
>>>
>>>> Existing date/time types are now represented by UNSIGNED_DATE,
>>>> UNSIGNED_TIME, and UNSIGNED_TIMESTAMP. If you have code that matches on
>>>> DATE, TIME, or TIMESTAMP, it should be changed to use the UNSIGNED
>>>> equivalent.
>>>>
>>>> Also, not sure if you've done this yet, but you should run your tables
>>>> through our upgrade process:
>>>> http://phoenix.incubator.apache.org/upgrade_from_2_2.html
>>>>
>>>> This will (among other things), change the metadata as described above
>>>> (so if you're not explicitly referencing PDataType date/time types, but
>>>> driving off of the DatabaseMetaData calls, no change would be necessary).
>>>>
>>>> The reason we made this change, is so that we can support negative
>>>> date/time values (which we've had a few requests for).
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Apr 4, 2014 at 8:03 AM, Abe Weinograd <abe@flonet.com> wrote:
>>>>
>>>>> We are bulk loading rows into our HBase tables directly via M/R ->
>>>>> HFiles.  This worked well in 2.2.3 and has broken with 3.0.
>>>>>
>>>>> We have a 3 column PK (int, date, int).  When I insert directly via
>>>>> the JDBC driver, everything works well.  When loading via the HFile,
>>>>> logging the Hex String from our M/R job and doing a scan on the table
in
>>>>> the hbase shell, it shows this:
>>>>>
>>>>> The first 12 bytes (int then date) of our key looks like this on the
>>>>> scan:
>>>>> \x00\x00\x00\x06\x00\x00\x1D\x8F\xD5&\x84\x00
>>>>>
>>>>> x00\x00\x1D\x8F\xD5&\x84\x00 was supposed to represent 12/31/2999
>>>>> 00:00:00 UTC
>>>>> The long value is 32503593600000L
>>>>>
>>>>> We are using hbase client util Bytes.toBytes(32503593600000L) to get
>>>>> our value.  It seems like a change in the PDataType code causes this
not to
>>>>> work anymore with 3.0.
>>>>>
>>>>> Does this make sense?  Is there an easy way for us to fix it?  I know
>>>>> the Bytes.toBytes() representation looks funny with the & in it,
but it
>>>>> seems to work across the board.
>>>>>
>>>>> Any ideas how we can work around this?
>>>>>
>>>>> Thanks,
>>>>> Abe
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message