phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marek Wiewiorka <marek.wiewio...@gmail.com>
Subject Re: hive external table
Date Mon, 06 Apr 2015 20:31:54 GMT
Thanks Nicolas - could you just let me know what branch should I use to
build your project against Phoenix 4.3.0?
Master of your repo seems to use 4.2.2...
Sure I can live using epoch secs/millis instead of timestamp - however it
would be nice to have timestamp mapping implemented.


Regards,
Marek

2015-04-06 22:25 GMT+02:00 Nicolas Maillard <nmaillard@hortonworks.com>:

>  Hello Marek
>
>
>  your error seems to point to a conversion mismatch when bringing back
> the timestamp type from Phoenix, essentially the timestamp is being brought
> back as a Text value in the the Hive world, I'll fix that asap.
>
> Not nice workaorund try using a long it should work but I will fix the
> issue at hand
>
>
>  Concerning your table no problem in your declaration, make sure your
> znode is hbase-unsecure though, it usually is hbase (the default ) but in
> my hortonworks sandbox it is hbase-unsecure this is why I specify it.
>
>
>
>
>      [image: Hortonworks, Inc.] <http://hortonworks.com/>
> *Nicolas  <mjohansson@hortonworks.com>Maillard*  *Solutions Engineer -
> EMEA* <https://www.linkedin.com/pub/mats-johansson/0/64b/9>
>   +33 (0) 668176605|
> Skype: nicolas.maillard.hortonworks
>       ------------------------------
> *From:* Marek Wiewiorka <marek.wiewiorka@gmail.com>
> *Sent:* Monday, April 06, 2015 10:05 PM
> *To:* user@phoenix.apache.org
> *Subject:* Re: hive external table
>
>  Hi Nicolas - thanks for your response!
>
> Where can I find the latest branch - I used master from this repo:
> https://github.com/nmaillard/Phoenix-Hive
>  and it is built againts 4.2.2 - I changed dependency in pom and tried to
> build it using 4.3 I got series of errors like these:
>  [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[29,33]
> cannot find symbol
>   symbol:   class PDataType
>   location: package org.apache.phoenix.schema
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[93,19]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[102,19]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[116,19]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[128,19]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[164,19]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/util/HiveConfigurationUtil.java:[216,13]
> cannot find symbol
>   symbol:   class PDataType
>   location: class org.apache.phoenix.hive.util.HiveConfigurationUtil
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/PhoenixMetaHook.java:[41,33]
> cannot find symbol
>   symbol:   class PDataType
>   location: package org.apache.phoenix.schema
> [ERROR]
> /home/mw/Phoenix-Hive/src/main/java/org/apache/phoenix/hive/PhoenixSerde.java:[41,33]
> cannot find symbol
>
>
>  Could you also please advise how should I define an external table in
> Hive using Phoenix-hive for a table created in Phoenix as follows:
>
>  create table se_dwh.homes_usage_hour
> (hu_ts time not null ,
> hu_ho_id integer not null ,
> hu_stream_id integer not null,
> hu_usage double constraint pk PRIMARY KEY(hu_ts,hu_ho_id,hu_stream_id) );
>
>  I'm not sure how to set mapping columns correctly - I tried something
> like this:
> CREATE EXTERNAL TABLE phx_usage_hour(
>      hu_ts timestamp,
>     hu_ho_id int,
>     hu_stream_id int,
>     hu_usage double)
> STORED BY  "org.apache.phoenix.hive.PhoenixStorageHandler"
> TBLPROPERTIES(
>     'phoenix.hbase.table.name'='se_dwh.homes_usage_hour',
>     'phoenix.zookeeper.znode.parent'='hbase-unsecure',
>     'phoenix.rowkeys'='hu_ts,hu_ho_id,hu_stream_id'
> );
>
>  It was created without a problem - but when I run a simple query:
>
>  hive> select * from phx_usage_hour limit 1;
> OK
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to
> org.apache.hadoop.hive.serde2.io.TimestampWritable
> Time taken: 0.084 seconds
>
>  I compiled your project using Phoenix 4.2.2 but I got 4.3 version
> installed in HBase - could it be the root cause of the exception ?
>
>  Thanks!
> Marek
>
>
>
>
>
>
>
>
>
> 2015-04-06 20:57 GMT+02:00 Nicolas Maillard <nmaillard@hortonworks.com>:
>
>>  Hello Marek
>>
>>
>>  There are a couple ways of reaching to phoenix through Hive
>>
>> - One is calling out directly the hbase layer with the Hive Hbase
>> connector but this has some caveats.
>>
>> - Second is this project I am working on, the latest branch is built
>> against phoenix 4.3, but building it against 4.2.2 should not create any
>> problems from a usage part , if it does I need to fix it.
>>
>> Could you explain the issue or file an issue on the project.
>>
>> - Third if it is urgent you could dump your Hive data in a csv format and
>> use the phoenix Bulkloader.
>>
>>
>>  Whatever the way you decide to go please do forward some details about
>> your error so I can fix if needed
>>
>>
>>
>>      [image: Hortonworks, Inc.] <http://hortonworks.com/>
>> *Nicolas  <mjohansson@hortonworks.com>Maillard*  *Solutions Engineer -
>> EMEA* <https://www.linkedin.com/pub/mats-johansson/0/64b/9>
>>   +33 (0) 668176605|
>> Skype: nicolas.maillard.hortonworks
>>       ------------------------------
>> *From:* Marek Wiewiorka <marek.wiewiorka@gmail.com>
>> *Sent:* Monday, April 06, 2015 5:45 PM
>> *To:* user@phoenix.apache.org
>> *Subject:* hive external table
>>
>>   Hi All,
>> I trying to create an external table mapped to a Phoenix table and I
>> cannot make it working.
>> I tried this project:
>> https://github.com/nmaillard/Phoenix-Hive
>>  but it fails to build against Phoenix 4.3.0.
>> I compiled it using 4.2.2 but still it's throwing some exception while
>> trying to access
>> an external table.
>>
>>  Before I try to dig deeper - I would like to ask if  there is  any
>> other way of using Phoenix in Hive environment?
>>  Maybe you recommend a different approach of transferring data from
>> Phoenix
>> to Hive?
>>
>>  Many thanks in advance!
>>
>>  Marek
>>
>
>

Mime
View raw message