livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: What happens if Livy server crashes ? All the spark jobs are gone?
Date Mon, 12 Mar 2018 08:58:27 GMT
Sorry I see there is a recovery mode and also I can set state store to
zookeeper but looks like I need YARN? because I get the error message below

"requirement failed: Session recovery requires YARN"


I am using spark standalone and I don't use YARN anywhere in my cluster. is
there any other option for recovery in this case?


On Sun, Mar 11, 2018 at 11:57 AM, kant kodali <kanth909@gmail.com> wrote:

> Hi All,
>
> When my live server crashes it looks like all my spark jobs are gone. I am
> trying to see how I can make it more resilient? other words, I would like
> spark jobs that were spawned by Livy to be running even if my Livy server
> crashes because in theory Livy server can crash anytime and Spark Jobs
> should run for weeks or months in my case. How can I achieve this?
>
> Thanks!
>
>

Mime
View raw message