livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Durighetto <>
Subject Re: What happens if Livy server crashes ? All the spark jobs are gone?
Date Mon, 12 Mar 2018 10:41:16 GMT
2018-03-12 9:58 GMT+01:00 kant kodali <>:

> Sorry I see there is a recovery mode and also I can set state store to
> zookeeper but looks like I need YARN? because I get the error message below
> "requirement failed: Session recovery requires YARN"
> I am using spark standalone and I don't use YARN anywhere in my cluster.
> is there any other option for recovery in this case?
> On Sun, Mar 11, 2018 at 11:57 AM, kant kodali <> wrote:
>> Hi All,
>> When my live server crashes it looks like all my spark jobs are gone. I
>> am trying to see how I can make it more resilient? other words, I would
>> like spark jobs that were spawned by Livy to be running even if my Livy
>> server crashes because in theory Livy server can crash anytime and Spark
>> Jobs should run for weeks or months in my case. How can I achieve this?
>> Thanks!
> Hello,
             to enable recovery in Livy you need Spark on YARN

( )

Kind Regards

View raw message