phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject Re: Kerberos Secure cluster and phoenix
Date Mon, 01 Sep 2014 23:11:49 GMT
Hi Deepak,

What version of phoenix you are using? Phoenix 3.1 and 4.1 support
connecting to secure Hadoop/HBase cluster out of the box(Phoenix-19). Are
you running HBase on a fully distributed cluster?

I would recommend you to use
*phoenix-*-client-without-hbase.jar file.*

Use following command to invoke sqlline:
sqlline.sh <zk>:<zk_port>:<root_dir>:<principal>:<keytab>

Last week i used 3.1 release to connect to a secure HBase cluster running
cdh4.6. Here is the bash script with modified classpath:
---------------------------------------------------------------------------------------


















*#!/bin/bashcurrent_dir=$(cd $(dirname
$0);pwd)phoenix_jar_path="$current_dir/.."phoenix_client_jar=$(find
$phoenix_jar_path/phoenix-*-client-without-hbase.jar)if [ -z "$1" ]   then
echo -e "Zookeeper not specified. \nUsage: sqlline.sh <zookeeper>
<optional_sql_file> \nExample: \n 1. sqlline.sh localhost \n 2. sqlline.sh
localhost ../examples/stock_symbol.sql";  exit; fiif [ "$2" ]   then
sqlfile="--run=$2";fiecho Phoenix_Client_Jar=$phoenix_client_jarjava -cp
"/etc/hbase/conf:.:../sqlline-1.1.2.jar:../jline-2.11.jar:/opt/cloudera/parcels/CDH/lib/hbase/hbase-0.94.15-cdh4.6.0-security.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop/*:/opt/cloudera/parcels/CDH/lib/hadoop/lib/*:../phoenix-core-3.1.0.jar:$phoenix_client_jar"
-Dlog4j.configuration=file:$current_dir/log4j.properties sqlline.SqlLine -d
org.apache.phoenix.jdb c.PhoenixDriver -u jdbc:phoenix:$1 -n none -p none
--color=true --fastConnect=false --verbose=true
--isolation=TRANSACTION_READ_COMMITTED $sqlfile*
---------------------------------------------------------------------------------------

Just modify above script as per 5.1 release of CDH and your environment
setup.
Let us know if it doesn't works.


Thanks,
Anil Gupta


On Mon, Sep 1, 2014 at 3:12 PM, Alex Kamil <alex.kamil@gmail.com> wrote:

> Deepak,
>
> also I'd check first if hbase is working and accessible in secure mode
> with the same kerberos principal you use for phoenix client
> -  start hbase shell and see if you can run some commands in secure mode
> - verify hbase, hadoop, zookeeper running in secure mode, are there any
> exceptions in server logs
> - can you execute command in hdfs shell and with zookeeper client
> - run kinit as shown in cdh security guide for hbase, what do you see when
> you run klist
> - enable kerberos debug mode in sqlline.py, with something like
>
> kerberos="-Djava.security.auth.login.config=/myapp/phoenix/bin/zk-jaas.conf
> -Dsun.security.krb5.*debug=true* -Djava.security.krb5.realm=MYDOMAIN
> -Djava.security.krb5.kdc=MYKDC -Djava.security.krb5.conf=/etc/krb5.conf"
>
> java_cmd = 'java ' + *kerberos* + ' -classpath ".' + os.pathsep
> +extrajars+ os.pathsep+ phoenix_utils.phoenix_client_jar + \
>
> Alex
>
>
> On Mon, Sep 1, 2014 at 6:09 PM, James Taylor <jamestaylor@apache.org>
> wrote:
>
>> Please try with the 4.1 jars in our binary distribution here:
>>
>> http://phoenix.apache.org/download.html
>>
>> Make sure to use the jars for the client and server in the hadoop2
>> directory.
>>
>> Then follow the directions that Alex posted here:
>>
>>
>> http://bigdatanoob.blogspot.com/2013/09/connect-phoenix-to-secure-hbase-cluster.html
>>
>>
>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Security-Guide/CDH5-Security-Guide.html
>>
>> It sounds to me like there's a mismatch between your client and server
>> jars.
>>
>> Thanks,
>> James
>>
>> On Mon, Sep 1, 2014 at 2:43 PM,  <Deepak_Gattala@dell.com> wrote:
>> > I am getting this following error really appreciate any comments. please
>> >
>> > Error: com.google.protobuf.ServiceException: java.io.IOException: Call
>> to ausgtmhadoop10.us-poclab.dellpoc.com/192.168.1.100:60000 failed on
>> local exception: java.io.EOFException (state=08000,code=101)
>> > org.apache.phoenix.exception.PhoenixIOException:
>> com.google.protobuf.ServiceException: java.io.IOException: Call to
>> ausgtmhadoop10.us-poclab.dellpoc.com/192.168.1.100:60000 failed on local
>> exception: java.io.EOFException
>> >         at
>> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:101)
>> >        at
>> org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:846)
>> >         at
>> org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1057)
>> >         at
>> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1156)
>> >         at
>> org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:422)
>> >         at
>> org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
>> >         at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:226)
>> >         at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:908)
>> >         at
>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1452)
>> >         at
>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:131)
>> >         at
>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:112)
>> >         at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650)
>> >         at
>> sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701)
>> >         at sqlline.SqlLine$Commands.connect(SqlLine.java:3942)
>> >         at sqlline.SqlLine$Commands.connect(SqlLine.java:3851)
>> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >         at java.lang.reflect.Method.invoke(Method.java:606)
>> >         at
>> sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810)
>> >         at sqlline.SqlLine.dispatch(SqlLine.java:817)
>> >         at sqlline.SqlLine.initArgs(SqlLine.java:633)
>> >         at sqlline.SqlLine.begin(SqlLine.java:680)
>> >         at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441)
>> >         at sqlline.SqlLine.main(SqlLine.java:424)
>> > Caused by: org.apache.hadoop.hbase.MasterNotRunningException:
>> com.google.protobuf.ServiceException: java.io.IOException: Call to
>> ausgtmhadoop10.us-poclab.dellpoc.com/192.168.1.100:60000 failed on local
>> exception: java.io.EOFException
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1650)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1676)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1884)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:2671)
>> >         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:397)
>> >         at
>> org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:402)
>> >         at
>> org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:772)
>> >         ... 23 more
>> > Caused by: com.google.protobuf.ServiceException: java.io.IOException:
>> Call to ausgtmhadoop10.us-poclab.dellpoc.com/192.168.1.100:60000 failed
>> on local exception: java.io.EOFException
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
>> >         at
>> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1687)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1596)
>> >         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1622)
>> >         ... 29 more
>> > Caused by: java.io.IOException: Call to
>> ausgtmhadoop10.us-poclab.dellpoc.com/192.168.1.100:60000 failed on local
>> exception: java.io.EOFException
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1485)
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1457)
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657)
>> >         ... 34 more
>> > Caused by: java.io.EOFException
>> >         at java.io.DataInputStream.readInt(DataInputStream.java:392)
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1072)
>> >         at
>> org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:728)
>> > sqlline version 1.1.2
>> >
>> > -----Original Message-----
>> > From: James Taylor [mailto:jamestaylor@apache.org]
>> > Sent: Monday, September 1, 2014 4:35 PM
>> > To: user
>> > Subject: Re: Kerberos Secure cluster and phoenix
>> >
>> > In addition to the above, in our 3.1/4.1 release, you can pass through
>> the principal and keytab file on the connection URL to connect to different
>> secure clusters, like this:
>> >
>> >
>> DriverManager.getConnection("jdbc:phoenix:h1,h2,h3:2181:user/principal:/user.keytab");
>> >
>> > The full URL is now of the form
>> > jdbc:phoenix:<quorom>:<port>:<rootNode>:<principal>:<keytabFile>
>> >
>> > where <port> and <rootNode> may be absent. We determine that <port>
is
>> present if it's a number and <rootNode> if it begins with a '/'.
>> >
>> > One other useful feature from this work, not related to connecting to a
>> secure cluster, you may specify only the <principal> which would cause a
>> different HConnection to be used (per unique principal per cluster). In
>> this way, you can pass through different HBase properties that apply to the
>> HConnection (such as timeout parameters).
>> >
>> > For example:
>> >
>> > DriverManager.getConnection("jdbc:phoenix:h1:longRunning", props);
>> >
>> > where props would contain the HBase config parameters and values for
>> timeouts in a "longRunning" connection which could be completely different
>> than connection gotten through this URL:
>> >
>> > DriverManager.getConnection("jdbc:phoenix:h1:shortRunning", props);
>> >
>> > Thanks,
>> > James
>> >
>> > On Mon, Sep 1, 2014 at 2:13 PM, Alex Kamil <alex.kamil@gmail.com>
>> wrote:
>> >> see
>> >> http://bigdatanoob.blogspot.com/2013/09/connect-phoenix-to-secure-hbas
>> >> e-cluster.html
>> >>
>> >> and
>> >> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/la
>> >> test/CDH5-Security-Guide/CDH5-Security-Guide.html
>> >>
>> >>
>> >> On Mon, Sep 1, 2014 at 5:02 PM, <Deepak_Gattala@dell.com> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>>
>> >>>
>> >>> Any one has success doing a Phoenix connection  to a secure Hbase
>> >>> Hadoop cluster, if yes can you please kindly let me know the steps
>> >>> taken, I am on the recent version of phoenix and using Cloudera CDH
>> 5.1 with hbase 0.98.
>> >>>
>> >>>
>> >>>
>> >>> Appreciate your help.
>> >>>
>> >>>
>> >>>
>> >>> Thanks
>> >>>
>> >>> Deepak Gattala
>> >>
>> >>
>>
>
>


-- 
Thanks & Regards,
Anil Gupta

Mime
View raw message