phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rafa <raf...@gmail.com>
Subject Re: Azure Blob storage backed secure cluster not working with Phoenix
Date Mon, 30 Oct 2017 11:57:58 GMT
Hi Mallieswari,

I have no experience at all with Azure, but the excepction points to
missing classes in your classpath.

Perhaps you need to add the proper jars to be able to interact with an
Azure FS:
https://hadoop.apache.org/docs/stable/hadoop-azure/index.html

There is an interesting article in :  http://beadooper.com/?p=409

Regards,
rafa


On Mon, Oct 30, 2017 at 12:43 PM, Mallieswari Dineshbabu <
dmallieswari@gmail.com> wrote:

> Hi All,
>
>
>
> I have a Hadoop(version 2.7.2) - HBase(version 1.2.5) Kerberos enabled
> secure cluster configured with default file system as Azure Blob Storage
> <https://hadoop.apache.org/docs/r2.7.2/hadoop-azure/index.html>* (WASBS)*
> in windows environment. In this cluster I try to integrate Phoenix(4.12.0).
> I could not establish sqline thin client connection successfully. Phoenix
> thin client and thick client is working fine with Hadoop secure cluster
> configured with *local file system* but it is failing in *Azure blob file
> system* alone.
>
>
>
> *Query*:
>
> Kindly update, if I am missing anything else or please update whether
> Azure Blob storage backed secure cluster is supported by Phoenix or not.
>
>
>
> *Configuration Changes:*
>
>    1. *Referred phoenix-4.12.0-HBase-1.2-client.jar and
>    phoenix-4.12.0-HBase-1.2-thin-client.jar in hbase/lib*
>
>
>
>    1. *Added following Properties for phoenix in
>    hbase/conf/Hbase-site.xml (secure cluster) : *
>
>
>
> phoenix.queryserver.withRemoteUserExtractor
>
> true
>
> Phoenix.queryserver.keytab.file
>
> Create HTTP user and keytab
>
> Phoenix.queryserver.kerberos.principal
>
> Refer the principal of HTTP user
>
>
>
>    1. *Set the Environment variable:*
>
> set JAVA_HOME= C:\Hadoop\Java\jdk1.7.0_51
>
> set HBASE_HOME= C:\Hadoop\ HBase
>
> set HADOOP_HOME=C:\Hadoop\Hadoop
>
> set Python=C:\Hadoop\WinPython\python-2.7.10.amd64
>
> set path=%JAVA_HOME%\bin;%HBASE_HOME%\conf;%HADOOP_HOME%\etc\
> hadoop;%Python%
>
> set HADOOP_CONF_DIR=C:\Hadoop\Hadoop\etc\hadoop
>
> set HBASE_CONF_DIR=C:\Hadoop\HBase\conf
>
> set HADOOP_CLASSPATH=C:\Hadoop\Hadoop\share\hadoop
>
>
>
>    1. *Command used to start phoenix thin client*
>
> Ø  python sqlline-thin.py http://namenode.AD.ONMICROSOFT.COM:8765;
> authentication=SPNEGO;principal=phoenixclient@AD.
> ONMICROSOFT.COM;keytab=C:\\Hadoop\keytabs\phoenixclient.
> keytab;serialization=PROTOBUF
> <http://namenode.AD.ONMICROSOFT.COM:8765;authentication=SPNEGO;principal=phoenixclient@AD.ONMICROSOFT.COM;keytab=C:/Hadoop/keytabs/phoenixclient.keytab;serialization=PROTOBUF>
> ;
>
>
>
>
>
> No exception produced while starting *Query server*.
>
>
>
> *Exception details from Sqline thin client connection:*
>
> SQLException: ERROR 103 (08004): Unable to establish connection. ->
> SQLException: ERROR 103 (08004): Unable to establish connection. ->
> IOException: java.lang.reflect.InvocationTargetException ->
> InvocationTargetException: (null exception message) ->
> ExceptionInInitializerError: (null exception message) -> RuntimeException:
> java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem
> not found -> ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem
> not found. Error -1 (00000) null
>
> java.lang.RuntimeException: java.sql.SQLException: ERROR 103 (08004):
> Unable to establish connection.
>
>         at org.apache.calcite.avatica.jdbc.JdbcMeta.openConnection(
> JdbcMeta.java:621)
>
>         at org.apache.calcite.avatica.remote.LocalService.apply(
> LocalService.java:285)
>
>         at org.apache.calcite.avatica.remote.Service$
> OpenConnectionRequest.accept(Service.java:1771)
>
>         at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1657)
>
>         at org.apache.phoenix.queryserver.server.QueryServer$
> PhoenixDoAsCallback.doAsRemoteUser(QueryServer.java:463)
>
>         at org.apache.calcite.avatica.server.HttpServer$Builder$1.
> doAsRemoteUser(HttpServer.java:725)
>
>         at org.apache.calcite.avatica.server.AvaticaProtobufHandler.
> handle(AvaticaProtobufHandler.java:120)
>
>         at org.apache.phoenix.shaded.org.eclipse.jetty.security.
> SecurityHandler.handle(SecurityHandler.java:542)
>
>         at org.apache.phoenix.shaded.org.eclipse.jetty.server.
> HttpChannel.handle(HttpChannel.java:311)
>
> Caused by: java.sql.SQLException: ERROR 103 (08004): Unable to establish
> connection.
>
>         at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.
> newException(SQLExceptionCode.java:489)
>
>         at org.apache.phoenix.exception.SQLExceptionInfo.buildException(
> SQLExceptionInfo.java:150)
>
>         at org.apache.phoenix.query.ConnectionQueryServicesImpl.
> openConnection(ConnectionQueryServicesImpl.java:418)
>
>         at org.apache.phoenix.query.ConnectionQueryServicesImpl.
> access$500(ConnectionQueryServicesImpl.java:257)
>
>         at org.apache.phoenix.util.PhoenixContextExecutor.call(
> PhoenixContextExecutor.java:76)
>
>         at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> ConnectionQueryServicesImpl.java:2360)
>
>         at org.apache.phoenix.jdbc.PhoenixDriver.
> getConnectionQueryServices(PhoenixDriver.java:255)
>
> Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
> at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(
> ConnectionFactory.java:240)
>
>         at org.apache.hadoop.hbase.client.ConnectionManager.
> createConnection(ConnectionManager.java:431)
>
>         at org.apache.phoenix.query.HConnectionFactory$
> HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)
>
>         at org.apache.phoenix.query.ConnectionQueryServicesImpl.
> openConnection(ConnectionQueryServicesImpl.java:409)
>
>         ... 35 more
>
> Caused by: java.lang.reflect.InvocationTargetException
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>
>         at org.apache.hadoop.hbase.client.ConnectionFactory.
> createConnection(ConnectionFactory.java:238)
>
>         ... 40 more
>
> Caused by: java.lang.ExceptionInInitializerError
>
>         at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
>
>         at org.apache.hadoop.hbase.zookeeper.ZKClusterId.
> readClusterIdZNode(ZKClusterId.java:75)
>
>         at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(
> ZooKeeperRegistry.java:105)
>
>         at org.apache.hadoop.hbase.client.ConnectionManager$
> HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905)
>
>         at org.apache.hadoop.hbase.client.ConnectionManager$
> HConnectionImplementation.<init>(ConnectionManager.java:648)
>
>        ... 45 more
>
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.azure.NativeAzureFileSystem not found
>
>         at org.apache.hadoop.conf.Configuration.getClass(
> Configuration.java:2195)
>
>         at org.apache.hadoop.fs.FileSystem.getFileSystemClass(
> FileSystem.java:2638)
>
>         at org.apache.hadoop.fs.FileSystem.createFileSystem(
> FileSystem.java:2651)
>
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
>
>         at org.apache.hadoop.fs.FileSystem$Cache.getInternal(
> FileSystem.java:2687)
>
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
>
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
>
>         at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(
> DynamicClassLoader.java:120)
>
>         at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>
> (ProtobufUtil.java:243)
>
>         ... 50 more
>
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.azure.NativeAzureFileSystem not found
>
>         at org.apache.hadoop.conf.Configuration.getClassByName(
> Configuration.java:2101)
>
>         at org.apache.hadoop.conf.Configuration.getClass(
> Configuration.java:2193)
>
>         ... 60 more
>
> sqlline version 1.2.0
>
> 0: jdbc:phoenix:thin:url=http://namenode1.TIM>
>
>
>
> Please help me resolve this issue.
>
>
>
> Regards,
>
> Mallieswari D
>
>
>
>
>

Mime
View raw message