livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meisam Fathi <>
Subject Re: Use existing SparkSession in POST/batches request
Date Fri, 16 Nov 2018 14:32:09 GMT
Hi Shubham!

POST/sessions is to create a session and submit statements to it. Once the
session is up, it can accept as many statements as needed. Users can
interactively submit new statements to it.
POST/batches is to send one Spark application. Anything that the job needs
to run is submit with the POST request (jar files, or a python script, or).
Once the job is submitted, users cannot interactively send new statements
to the application to run.

The only interesting case is when you submit a streaming job with
POST/batches which runs forever (or until it fails/dies).


On Fri, Nov 16, 2018 at 4:43 AM Shubham Gupta <>

> Hi committers and contributors
> I have this query since long, which I think I know the answer to, but am
> still looking for confirmation.
> Can I make use of the SparkSession that I created using POST/sessions request
> for submitting my Spark job using POST/batches request?
> Here's the link <> to my complete
> (elaborated) question on StackOverflow.
> *Luqman Ghani'*s response
> <>
> already hints that this isn't possible, but
>    - Is it on the roadmap?
>    - Any workarounds?
> Thanks
> *Shubham Gupta*
> Software Engineer
>  zomato

View raw message