livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lian Jiang <jiangok2...@gmail.com>
Subject Re: Yarn queues for spark jobs submitted via livy
Date Wed, 28 Nov 2018 22:23:52 GMT
Meanwhile, user1 cannot user2's queue and vice versa. Thanks.

On Wed, Nov 28, 2018 at 2:21 PM Lian Jiang <jiangok2006@gmail.com> wrote:

> We are using HDP3 and knox. Users can submit spark job using livy api like:
>
> curl -k -u user1:"" -H "Content-Type: application/json" -H "X-Requested-By:
> admin" -d '{"className":"my.job
> <https://confluence.oci.oraclecorp.com/display/BDW/main.scala.com.oracle.ocidw.qea.spark.dwpoc.QeaDwPocRunner>",
> "file":"/tmp/sparkApp-assembly-1.0.jar
> <https://confluence.oci.oraclecorp.com/display/BDW/sparkApp-assembly-1.0.jar>
> "}' -X POST https://knox.com/gateway/ui/livy/v1/batches
> <https://confluence.oci.oraclecorp.com/display/BDW/dw-knox-prod1.us-phoenix-1.oracleiaas.com/gateway/ui/livy/v1/batches>
>
> The jobs submitted by user1 and user2 both show livy as user on yarn UI.
> Now I setup a yarn queue for user1 and another for user2. However can I let
> these two users use their corresponding yarn queue? Appreciate any help!
>

Mime
View raw message