livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lian Jiang <jiangok2...@gmail.com>
Subject Yarn queues for spark jobs submitted via livy
Date Wed, 28 Nov 2018 22:21:20 GMT
We are using HDP3 and knox. Users can submit spark job using livy api like:

curl -k -u user1:"" -H "Content-Type: application/json" -H "X-Requested-By:
admin" -d '{"className":"my.job
<https://confluence.oci.oraclecorp.com/display/BDW/main.scala.com.oracle.ocidw.qea.spark.dwpoc.QeaDwPocRunner>",
"file":"/tmp/sparkApp-assembly-1.0.jar
<https://confluence.oci.oraclecorp.com/display/BDW/sparkApp-assembly-1.0.jar>
"}' -X POST https://knox.com/gateway/ui/livy/v1/batches
<https://confluence.oci.oraclecorp.com/display/BDW/dw-knox-prod1.us-phoenix-1.oracleiaas.com/gateway/ui/livy/v1/batches>

The jobs submitted by user1 and user2 both show livy as user on yarn UI.
Now I setup a yarn queue for user1 and another for user2. However can I let
these two users use their corresponding yarn queue? Appreciate any help!

Mime
View raw message