phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From C Reid <reidddc...@outlook.com>
Subject Phoenix connection to kerberized hbase fails
Date Wed, 19 Apr 2017 09:44:53 GMT
Hi group,


I used the following command to execute sqlline.py:

bin/sqlline.py hadoop-offline034.dx.momo.com,hadoop-offline035.dx.momo.com,hadoop-offline036.dx.momo.com:2181:/hbase:phoenix/hadoop-offline032.dx.momo.com@MOMO.OFFLINE:/opt/hadoop/etc/hadoop/security/phoenix.keytab


from the debug, login successfully

17/04/19 15:02:31 INFO jdbc.PhoenixEmbeddedDriver$ConnectionInfo: Trying to connect to a secure
cluster as phoenix/hadoop-offline032.dx.momo.com@MOMO.OFFLINE with keytab /opt/hadoop/etc/hadoop/security/phoenix.keytab
17/04/19 15:02:31 INFO jdbc.PhoenixEmbeddedDriver$ConnectionInfo: Successful login to secure
cluster

then zookeeper connection made:
17/04/19 15:02:31 INFO zookeeper.ClientCnxn: Session establishment complete on server hadoop-offline035.dx.momo.com/10.87.202.94:2181,
sessionid = 0x25b60572c9e001d, negotiated timeout = 90000
17/04/19 15:02:32 INFO query.ConnectionQueryServicesImpl: HConnnection established. Details:
hconnection-0x7e7ce49c

but when phoenix tried to make connection to hbase, the credentials was gone.
17/04/19 15:02:32 WARN ipc.RpcClientImpl: Exception encountered while connecting to the server
: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid
credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
17/04/19 15:02:32 FATAL ipc.RpcClientImpl: SASL authentication failed. The most likely cause
is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:34094)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:394)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:203)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:64)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any
Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 26 more

another information i found may be useful for group to diagnose this problem.
17/04/19 15:03:10 INFO client.RpcRetryingCaller: Call exception, tries=10, retries=35, started=38599
ms ago, cancelled=false, msg=row 'SYSTEM:CATALOG,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740,
hostname=hadoop-offline033.dx.momo.com,16020,1492508866054, seqNum=0

I'm not sure it's a bug or mistake of my usage, hopes to get some helps from group.
Thanks in advance.

Best regards,
---R



Mime
View raw message