livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lk_hadoop"<>
Subject Re: Re: about LIVY-424
Date Wed, 14 Nov 2018 07:12:15 GMT
I think it's really a mortal blow to livy about the repl scene . What I can do , I think is
to monitoring spark metrics, when the driver's memory was used to a high leve I will isolate
the session.



发件人:"Harsch, Tim" <>
发送时间:2018-11-14 05:52
主题:Re: about LIVY-424

While it's true LIVY-424 creates a session leak due to REPL leak in Scala it's not the only
thing that can.  I've run hundreds of simple scala commands and the leak is only mild/moderate.
  However, some scala commands can be really problematic.  For instance 

import org.apache.spark.sql._

run this import repeatedly and within only 10s of executions your sessions performance will
degrade and eventually run out of memory.

From: lk_hadoop <>
Sent: Sunday, November 11, 2018 5:37:34 PM
To: user
Subject: about LIVY-424 

[External Email] 

        I meet this issue  , anybody know how
to resolve it?

View raw message