phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Mahonin <jmaho...@gmail.com>
Subject Re: Spark 2.1.1 (Scala 2.11.8) write to Phoenix 4.7 (HBase 1.1.2)
Date Wed, 31 Jan 2018 14:32:59 GMT
Hi,

As per https://phoenix.apache.org/phoenix_spark.html, Apache Phoenix is
compiled against Spark2 only in versions 4.10 and above. If you must use
Phoenix 4.7 against Spark 2.x, you may need to apply PHOENIX-3333 yourself:

https://github.com/apache/phoenix/commit/a0e5efcec5a1a732b2dce9794251242c3d66eea6#diff-600376dffeb79835ede4a0b285078036

Note that if you're using a vendor-provided Phoenix distribution, there may
already be support for the Spark version you're using. Please follow-up
with them if that's the case.

Best,

Josh




On Tue, Jan 30, 2018 at 10:06 AM, Margusja <margus@roo.ee> wrote:

> Also I see that Logging is moved to internal/Logging. But is there package
> for my environment I can use?
>
> Margus
>
>
> On 30 Jan 2018, at 17:00, Margusja <margus@roo.ee> wrote:
>
> Hi
>
> Followed page (https://phoenix.apache.org/phoenix_spark.html <https://
> phoenix.apache.org/phoenix_spark.html>) and trying to save to phoenix.
>
> Using spark-1.6.3 it is successful but using spark-2.1.1 it is not.
> First error I am getting using spark-2.1.1 is that:
>
> Error:scalac: missing or invalid dependency detected while loading class
> file 'ProductRDDFunctions.class'.
> Could not access type Logging in package org.apache.spark,
> because it (or its dependencies) are missing. Check your build definition
> for
> missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see
> the problematic classpath.)
> A full rebuild may help if 'ProductRDDFunctions.class' was compiled
> against an incompatible version of org.apache.spark.
>
> I can see that Logging is removed after 1.6.3 and does not exist in 2.1.1.
>
> What are my options?
>
> Br
> Margus
>
>
>

Mime
View raw message