livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Divya Arya <di...@cyberops.in>
Subject Re: Query: Local Path cannot read by Livy (Running with Docker)
Date Tue, 31 Jul 2018 04:36:41 GMT
I got a patch for that but it is *not a valid solution*. I installed
xampp on my windows system and shared my C drive, (few changes in the
configuration file) as a local server then access those paths.

On Mon, Jul 30, 2018 at 9:56 PM, Harsch, Tim <Tim.Harsch@teradata.com>
wrote:

> I created a Jira to start tracking the issue:
>
> https://issues.apache.org/jira/browse/LIVY-487
>
>
> When you see the issue are you using the files parameter of sessions
> endpoint?
>
>
> It would be helpful for the ticket if we could construct SSCCE (
> http://sscce.org/ ) to demonstrate the problem.  I don't have an
> environment going where I can do that easily at the moment.  If you do, can
> you please comment on the ticket?
>
>
> Thanks,
>
> Tim
>
> ------------------------------
> *From:* Divya Arya <divya@cyberops.in>
> *Sent:* Friday, July 27, 2018 11:17:30 PM
> *To:* user@livy.incubator.apache.org
> *Subject:* Re: Query: Local Path cannot read by Livy (Running with Docker)
>
> [External Email]
>
>
> Yes, that is the exact same scenario I have, just instead of jar files I
> am using CSV files stored in my client machine's file system and no
> protocol is supporting to access those files.
>
> On Sat, Jul 28, 2018 at 3:42 AM, Harsch, Tim <Tim.Harsch@teradata.com>
> wrote:
>
> When you say not able to use my files in Spark Session what do you mean?
>
>
>
> I think I’m seeing a similar problem.   If I submit jars using a client
> machine that is not the same as the server (so file system is not shared)
> and I submit the jar it’s looks like it gets transmitted to the server but
> then I can’t reference any of the classes.  If I look at the Spark UI that
> gets launched for the session then I can see references to paths in my
> client machine, which aren’t resolvable on the server.   Is this the
> scenario you have?   I think it might occur in a docker env if you are
> doing client stuff on the host os, and livy is running in a container.
>
>
>
>
>
>
>
> *From: *Divya Arya <divya@cyberops.in>
> *Reply-To: *"user@livy.incubator.apache.org" <
> user@livy.incubator.apache.org>
> *Date: *Thursday, July 26, 2018 at 11:45 PM
> *To: *"user@livy.incubator.apache.org" <user@livy.incubator.apache.org>
> *Subject: *Re: Query: Local Path cannot read by Livy (Running with Docker)
>
>
>
> *[External Email] *
>
>
>
> Please help me out. I am really looking forward.
>
>
>
> On Thu, Jul 26, 2018 at 5:49 PM, Divya Arya <divya@cyberops.in> wrote:
>
> Hello Team,
>
>
>
> I need a little support from you, I am trying to bundle up my application
> which is working on Apache Livy, Spark (SCALA), so it can run on a
> Windows-based system, for that, I am using Apache Docker.
>
>
>
> I have set up Livy and Spark in that, which is working perfectly fine but
> I am not able to use my Local path files (suppose files in C:/ or D:/) in
> SparkSession and SparkContext. Please help me out here.
>
>
>
>
>
> Thanks and Regards,
>
>
>
> Divya Arya
>
>
>
> divya@cyberops.in
>
> divya.arya69@gmail.com
>
>
>
>
>
>
>

Mime
View raw message