livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsch, Tim" <Tim.Har...@Teradata.com>
Subject Re: user impersonation in Livy not working
Date Fri, 20 Jul 2018 16:22:00 GMT
Thanks Jeff,

Getting back to this now.  I will try your suggestion when I get my environment running again.
  I did run into something today that I missed earlier and now I have some questions:


I noticed a setting in livy-client.conf.template for livy.rsc.proxy-user::

# The user that should be impersonated when requesting a Livy session
# livy.rsc.proxy-user =


What effect would that property have when I'm doing user impersonation?  Is it necessary?
  Also, I see this at the top of livy-client.conf.template.  Does that mean I should put the
file in the 'conf' folder or some other folder accessible to the spark executors, like repl_2.10-jars
?


# Configurations for a Livy Client, any configurations set here will override any

# livy or spark-default configurations.
#
# Before a Livy Client is able to load these configurations the folder containing
# this file must be added to the application classpath
#


________________________________
From: Jeff Zhang <zjffdu@gmail.com>
Sent: Tuesday, July 10, 2018 5:27:52 PM
To: user@livy.incubator.apache.org
Subject: Re: user impersonation in Livy not working


Make sure you enable hiveContext. set livy.repl.enable-hive-context to true in livy.conf



Harsch, Tim <Tim.Harsch@teradata.com<mailto:Tim.Harsch@teradata.com>>于2018年7月10日周二
下午11:53写道:

Hi,

I have been unable to get proxyUsers to work with Livy.   I seem to have Kerberos configured
correctly, as evidenced by the last beeline command.  When I start a livy session the server
log will usually say something like: “owner: kylo, proxyUser: Some(dladmin)” so it seems
the server should know what to do.   But no matter what I try I always get “org.apache.spark.sql.AnalysisException:
Table not found”



# Ambari config

[/var/folders/_n/k2nq_2fx5t5__fcxqdl2tc6j_p8_px/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/p1426]



# Livy Configuration

livy.impersonation.enabled = true

livy.server.auth.type = kerberos

livy.server.launch.kerberos.principal = kylo/sandbox.kylo.io@KYLO

livy.server.launch.kerberos.keytab = /etc/security/keytabs/kylo.keytab

livy.server.auth.kerberos.principal = HTTP/sandbox.kylo.io@KYLO

livy.server.auth.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab

livy.server.access_control.enabled = true

livy.server.access_control.users = kylo,dladmin

livy.server.access_control.modify-users = kylo

livy.superusers = kylo



# Curl connect

kinit -kt /etc/security/keytabs/kylo.keytab kylo/sandbox.kylo.io@KYLO

klist

curl --negotiate -u : -s -X POST --data '{"kind": "spark", "proxyUser": "dladmin"}' -H "Content-Type:
application/json" sandbox.kylo.io:8998/sessions<http://sandbox.kylo.io:8998/sessions>

curl --negotiate -u : -s -X GET sandbox.kylo.io:8998/sessions<http://sandbox.kylo.io:8998/sessions>

* NOTE: verify proxyUser="dladmin" in response.  it will be null if impersonation is not enabled



# submit this code to Livy:

sqlContext.sql("select * from default.d4").show()



# Response:

{

"code": "sqlContext.sql(\"select * from default.d4\").show()",

"id": 0,

"output": {

"ename": "Error",

"evalue": "org.apache.spark.sql.AnalysisException: Table not found: `default`.`d4`;",

....



# Can "dladmin" see table data?

[root@sandbox more]# beeline -u "jdbc:hive2://localhost:10000/;principal=hive/sandbox.kylo.io@KYLO;;hive.server2.proxy.user=dladmin"

Connecting to jdbc:hive2://localhost:10000/;principal=hive/sandbox.kylo.io@KYLO;;hive.server2.proxy.user=dladmin

Connected to: Apache Hive (version 1.2.1000.2.5.6.0-40)

Driver: Hive JDBC (version 1.2.1000.2.5.6.0-40)

Transaction isolation: TRANSACTION_REPEATABLE_READ

Beeline version 1.2.1000.2.5.6.0-40 by Apache Hive

0: jdbc:hive2://localhost:10000/> select * from default.d4;

+--------+--+

| d4.d4 |

+--------+--+

| d4 |

+--------+--+

1 row selected (0.132 seconds)









Mime
View raw message