livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsch, Tim" <Tim.Har...@Teradata.com>
Subject Re: Spark scala examples
Date Sun, 27 May 2018 20:28:01 GMT
Hi Nirav,
There are a few ways to use Livy: Shared sessions and Batch sessions are covered in the REST
API docs (https://livy.incubator.apache.org/docs/latest/rest-api.html) and GetHue.com wrote
some cool articles.  I created a little bash framework for testing out some example code I
found at GetHue here:  https://bitbucket.org/harschware/livy-shared-rdd/src/master/    Pointers
to the articles are in the README.  Livy also has a programmatic API you’ve probably seen
already ( https://livy.incubator.apache.org/docs/latest/programmatic-api.html )

With the shared session type “spark” the code that you execute with the Livy REST API
is pretty much exactly what you would execute in a spark-shell.  There are a lot of examples
via Google about running spark-shell and they translate well to running in Livy.

HTH,
Tim

From: Nirav Patel <npatel@xactlycorp.com>
Reply-To: "user@livy.incubator.apache.org" <user@livy.incubator.apache.org>
Date: Friday, May 25, 2018 at 11:25 PM
To: "user@livy.incubator.apache.org" <user@livy.incubator.apache.org>
Subject: Spark scala examples

Hi,

Can anyone point me to any spark-scala examples using livy as a spark jobserver? I am trying
to understand how much existing spark based work (rdd, dataframe based transformations) we
can leverage when using livy. seems like all but it'd be nice to see examples.

Thanks



[Image removed by sender. What's New with Xactly]<http://www.xactlycorp.com/email-click/>

[Image removed by sender.]<https://www.instagram.com/xactlycorp/>  [Image removed by
sender.] <https://www.linkedin.com/company/xactly-corporation>   [Image removed by sender.]
<https://twitter.com/Xactly>   [Image removed by sender.] <https://www.facebook.com/XactlyCorp>
  [Image removed by sender.] <http://www.youtube.com/xactlycorporation>
Mime
View raw message