phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mallieswari Dineshbabu <dmalliesw...@gmail.com>
Subject Azure Blob storage backed secure cluster not working with Phoenix
Date Mon, 30 Oct 2017 11:43:54 GMT
Hi All,



I have a Hadoop(version 2.7.2) - HBase(version 1.2.5) Kerberos enabled
secure cluster configured with default file system as Azure Blob Storage
<https://hadoop.apache.org/docs/r2.7.2/hadoop-azure/index.html>* (WASBS)*
in windows environment. In this cluster I try to integrate Phoenix(4.12.0).
I could not establish sqline thin client connection successfully. Phoenix
thin client and thick client is working fine with Hadoop secure cluster
configured with *local file system* but it is failing in *Azure blob file
system* alone.



*Query*:

Kindly update, if I am missing anything else or please update whether Azure
Blob storage backed secure cluster is supported by Phoenix or not.



*Configuration Changes:*

   1. *Referred phoenix-4.12.0-HBase-1.2-client.jar and
   phoenix-4.12.0-HBase-1.2-thin-client.jar in hbase/lib*



   1. *Added following Properties for phoenix in hbase/conf/Hbase-site.xml
   (secure cluster) : *



phoenix.queryserver.withRemoteUserExtractor

true

Phoenix.queryserver.keytab.file

Create HTTP user and keytab

Phoenix.queryserver.kerberos.principal

Refer the principal of HTTP user



   1. *Set the Environment variable:*

set JAVA_HOME= C:\Hadoop\Java\jdk1.7.0_51

set HBASE_HOME= C:\Hadoop\ HBase

set HADOOP_HOME=C:\Hadoop\Hadoop

set Python=C:\Hadoop\WinPython\python-2.7.10.amd64

set path=%JAVA_HOME%\bin;%HBASE_HOME%\conf;%HADOOP_HOME%\etc\hadoop;%Python%

set HADOOP_CONF_DIR=C:\Hadoop\Hadoop\etc\hadoop

set HBASE_CONF_DIR=C:\Hadoop\HBase\conf

set HADOOP_CLASSPATH=C:\Hadoop\Hadoop\share\hadoop



   1. *Command used to start phoenix thin client*

Ø  python sqlline-thin.py
http://namenode.AD.ONMICROSOFT.COM:8765;authentication=SPNEGO;principal=phoenixclient@AD.ONMICROSOFT.COM;keytab=C:\\Hadoop\keytabs\phoenixclient.keytab;serialization=PROTOBUF
<http://namenode.AD.ONMICROSOFT.COM:8765;authentication=SPNEGO;principal=phoenixclient@AD.ONMICROSOFT.COM;keytab=C:/Hadoop/keytabs/phoenixclient.keytab;serialization=PROTOBUF>
;





No exception produced while starting *Query server*.



*Exception details from Sqline thin client connection:*

SQLException: ERROR 103 (08004): Unable to establish connection. ->
SQLException: ERROR 103 (08004): Unable to establish connection. ->
IOException: java.lang.reflect.InvocationTargetException ->
InvocationTargetException: (null exception message) ->
ExceptionInInitializerError: (null exception message) -> RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.azure.NativeAzureFileSystem not found ->
ClassNotFoundException: Class
org.apache.hadoop.fs.azure.NativeAzureFileSystem not found. Error -1
(00000) null

java.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004):
Unable to establish connection.

        at
org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(JdbcMeta.java:621)

        at
org.apache.calcite.avatica.remote.LocalService.apply(LocalService.java:285)

        at
org.apache.calcite.avatica.remote.Service$OpenConnectionRequest.accept(Service.java:1771)

        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

        at
org.apache.phoenix.queryserver.server.QueryServer$PhoenixDoAsCallback.doAsRemoteUser(QueryServer.java:463)

        at
org.apache.calcite.avatica.server.HttpServer$Builder$1.doAsRemoteUser(HttpServer.java:725)

        at
org.apache.calcite.avatica.server.AvaticaProtobufHandler.handle(AvaticaProtobufHandler.java:120)

        at
org.apache.phoenix.shaded.org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542)

        at
org.apache.phoenix.shaded.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)

Caused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish
connection.

        at
org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:489)

        at
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)

        at
org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:418)

        at
org.apache.phoenix.query.ConnectionQueryServicesImpl.access$500(ConnectionQueryServicesImpl.java:257)

        at
org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)

        at
org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2360)

        at
org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)

Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)

        at
org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:431)

        at
org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)

        at
org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:409)

        ... 35 more

Caused by: java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)

        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

        at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)

        ... 40 more

Caused by: java.lang.ExceptionInInitializerError

        at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)

        at
org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)

        at
org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)

        at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905)

        at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:648)

       ... 45 more

Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class org.apache.hadoop.fs.azure.NativeAzureFileSystem not found

        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)

        at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2638)

        at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)

        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)

        at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)

        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)

        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)

        at
org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:120)

        at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:243)

        ... 50 more

Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.azure.NativeAzureFileSystem not found

        at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)

        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)

        ... 60 more

sqlline version 1.2.0

0: jdbc:phoenix:thin:url=http://namenode1.TIM>



Please help me resolve this issue.



Regards,

Mallieswari D

Mime
View raw message