livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Malveeka Bhandari <malveeka.bhand...@gmail.com>
Subject Re: Livy session
Date Mon, 21 May 2018 07:43:03 GMT
Hi,

Thanks for the idea. However, my use case is a little bit complicated. The
job code remains constant, but the user code goes in as a dependent class.
For e.g.  the job might be to query data, filter for some constraints and
run a simulation (multiple transformations and actions) on the data. This
job is constant. One of the functions inside the simulation would need the
user code. So I would need to add the user code as a jar as opposed to a
job class.

Is that possible in any way ?

Thanks !

On Mon, 21 May 2018 at 12:46 PM, Ghosh, Parthapratim <
parthapratim.ghosh@citi.com> wrote:

> Hi Malveeka,
>
>
>
>                 I am another user in the forum. However, your requirement
> may be met without creating a new jar. You may put the user code in a
> defined job as an instance level variable and use it everytime you use the
> client to submit a job. To call this, create a LivyClient and submit the
> job everytime with a new code as required, instantiating the job with the
> client code in overloaded constructor.
>
>
>
> *Thanks and Regards*
>
> *Partha Pratim Ghosh*
>
>
>
> *From:* Malveeka Bhandari [mailto:malveeka.bhandari@gmail.com]
> *Sent:* Monday, May 21, 2018 2:58 AM
> *To:* user@livy.incubator.apache.org
> *Subject:* Fwd: Livy session
>
>
>
>
>
>
>
> Hi everyone.
>
>
>
> We are using Livy for our custom use case where a user can submit a batch
> request. We send the user code in spark.jars configuration. But we are
> having to make a new session everytime. Is there any way that there could
> be one running session for a user and we could keep submitting new jobs but
> with a new dependent jar each time.
>
>
>
>
>
> Thanks
>

Mime
View raw message