livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srungarapu vamsi <srungarapu1...@gmail.com>
Subject Re: Impersonation Error
Date Tue, 21 Aug 2018 06:45:51 GMT
Hi Tim,

Yes, I have referred the thread on user impersonation from Tuesday, July
10, 2018 (
https://www.mail-archive.com/user@livy.incubator.apache.org/msg00294.html )
Yes, I have enabled hive context.
Please find below configurations set in livy.conf .

livy.keystore = /path/to/server.jks
livy.keystore.password = password
livy.server.port = 8998
livy.spark.master = yarn
livy.spark.deploy-mode = cluster
livy.impersonation.enabled = true
livy.repl.enable-hive-context = true
livy.server.access-control.enabled = false
livy.server.auth.type = kerberos
livy.server.auth.kerberos.keytab=/path/to/http/keytab
livy.server.auth.kerberos.principal=HTTP/<principal>
livy.server.launch.kerberos.keytab= /path/to/my_proxy_user/keytab
livy.server.launch.kerberos.principal=my_proxy_user/<principal>
livy.superusers = my_proxy_user

Whenever I try to create a batch, the batch is successfully executing.
However, yarn diagnostics is throwing the above YarnException.
When I try to create a session, the session is finally ending up in a dead
state. with the above-mentioned Yarn Exception in the Yarn Diagnostics.

Thanks,
Vamsi

On Tue, 21 Aug 2018 at 00:49, Harsch, Tim <Tim.Harsch@teradata.com> wrote:

> Vamsi,
>
> It may be helpful if you read the thread on user impersonation from
> Tuesday, July 10, 2018.
>
>
>
> Do you have hive sql enabled?  What related configurations do you have set
> in livy.conf ?  Are you using kerberos as well?
>
>
>
>
>
> *From: *srungarapu vamsi <srungarapu1989@gmail.com>
> *Reply-To: *"user@livy.incubator.apache.org" <
> user@livy.incubator.apache.org>
> *Date: *Monday, August 20, 2018 at 3:36 AM
> *To: *"user@livy.incubator.apache.org" <user@livy.incubator.apache.org>
> *Subject: *Impersonation Error
>
>
>
> [External Email]
> ------------------------------
>
> Hi,
>
>
>
> We have a secure kerberized CDH cluster.
>
> I am trying to test impersonation through livy rest sever.
>
> I have created the proxy user which can impersonate other users in the
> while running the spark jobs.
>
> Here is my core-site.xml for the proxy user "my_proxy_user":
>
>
>
>  <property>
>
>     <name>hadoop.proxyuser.my_proxy_user.hosts</name>
>
>     <value>*</value>
>
>   </property>
>
>   <property>
>
>     <name>hadoop.proxyuser. my_proxy_user.groups</name>
>
>     <value> my_proxy_user_grp</value>
>
>   </property>
>
>   <property>
>
>     <name>hadoop.proxyuser. my_proxy_user.users</name>
>
>     <value>*</value>
>
>   </property>
>
> </configuration>
>
>
>
> With these configurations, whenever i am trying to create a session or
> batch, i am getting the following error.
>
>
>
> org.apache.hadoop.yarn.exceptions.YarnException: User my_proxy_user does not have privilage
to see this attempt appattempt_1533784624146_0323_-000001
>
>          at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationAttemptReport(ClientRMService.java:375)
>
>          at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationAttemptReport(ApplicationClientProtocolPBServiceImpl.java:355)
>
>          at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:425)
>
>          at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
>
>          at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
>
>          at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
>
>          at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
>
>          at java.security.AccessController.doPrivileged(Native Method)
>
>          at javax.security.auth.Subject.doAs(Subject.java:422)
>
>          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
>
>          at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2211)
>
>
>
> I am able to run the spark jobs by passsing the --proxyUser flag.
>
> Also, i am successfully able to run yarn jobs with HADOOP_PROXY_USER.
>
>
>
> But, on trying to create a session or batch, i am not getting in stderr of
> the session/batch logs. But "Yarn Diagnostics" show the above error.
>
>
>
> Can you please help me in solving this issue.
>
> --
>
> /Vamsi
>


-- 
/Vamsi

Mime
View raw message