livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ghosh, Parthapratim " <parthapratim.gh...@citi.com>
Subject RE: Livy session
Date Mon, 21 May 2018 07:54:36 GMT
Hi,

                Not sure of your exact use case. I was also mentioning different user code
in the same job’s codebase. My use case is different input while calling the same job multiple
times.

Thanks and Regards
Partha Pratim Ghosh

From: Malveeka Bhandari [mailto:malveeka.bhandari@gmail.com]
Sent: Monday, May 21, 2018 3:43 AM
To: user@livy.incubator.apache.org
Subject: Re: Livy session

Hi,

Thanks for the idea. However, my use case is a little bit complicated. The job code remains
constant, but the user code goes in as a dependent class. For e.g.  the job might be to query
data, filter for some constraints and run a simulation (multiple transformations and actions)
on the data. This job is constant. One of the functions inside the simulation would need the
user code. So I would need to add the user code as a jar as opposed to a job class.

Is that possible in any way ?

Thanks !

On Mon, 21 May 2018 at 12:46 PM, Ghosh, Parthapratim <parthapratim.ghosh@citi.com<mailto:parthapratim.ghosh@citi.com>>
wrote:
Hi Malveeka,

                I am another user in the forum. However, your requirement may be met without
creating a new jar. You may put the user code in a defined job as an instance level variable
and use it everytime you use the client to submit a job. To call this, create a LivyClient
and submit the job everytime with a new code as required, instantiating the job with the client
code in overloaded constructor.

Thanks and Regards
Partha Pratim Ghosh

From: Malveeka Bhandari [mailto:malveeka.bhandari@gmail.com<mailto:malveeka.bhandari@gmail.com>]
Sent: Monday, May 21, 2018 2:58 AM
To: user@livy.incubator.apache.org<mailto:user@livy.incubator.apache.org>
Subject: Fwd: Livy session



Hi everyone.

We are using Livy for our custom use case where a user can submit a batch request. We send
the user code in spark.jars configuration. But we are having to make a new session everytime.
Is there any way that there could be one running session for a user and we could keep submitting
new jobs but with a new dependent jar each time.


Thanks
Mime
View raw message