livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@apache.org>
Subject Re: Yarn queues for spark jobs submitted via livy
Date Fri, 30 Nov 2018 08:18:35 GMT
Are you hitting https://issues.apache.org/jira/browse/KNOX-1098

Bikas
________________________________
From: Lian Jiang <jiangok2006@gmail.com>
Sent: Wednesday, November 28, 2018 2:21 PM
To: user@livy.incubator.apache.org
Subject: Yarn queues for spark jobs submitted via livy

We are using HDP3 and knox. Users can submit spark job using livy api like:

curl -k -u user1:"" -H "Content-Type: application/json" -H "X-Requested-By: admin" -d '{"className":"my.job<https://confluence.oci.oraclecorp.com/display/BDW/main.scala.com.oracle.ocidw.qea.spark.dwpoc.QeaDwPocRunner>",
"file":"/tmp/sparkApp-assembly-1.0.jar<https://confluence.oci.oraclecorp.com/display/BDW/sparkApp-assembly-1.0.jar>"}'
-X POST https://knox.com/gateway/ui/livy/v1/batches<https://confluence.oci.oraclecorp.com/display/BDW/dw-knox-prod1.us-phoenix-1.oracleiaas.com/gateway/ui/livy/v1/batches>

The jobs submitted by user1 and user2 both show livy as user on yarn UI. Now I setup a yarn
queue for user1 and another for user2. However can I let these two users use their corresponding
yarn queue? Appreciate any help!

Mime
View raw message