sis-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Derrick Oswald (Jira)" <j...@apache.org>
Subject [jira] [Updated] (SIS-491) sis-embedded-data database fails to start on Apache Spark
Date Sun, 08 Mar 2020 15:31:00 GMT

     [ https://issues.apache.org/jira/browse/SIS-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Derrick Oswald updated SIS-491:
-------------------------------
    Description: 
Operation _CRS_.forCode fails when used within the Apache Spark environment. 

Steps to reproduce (see full log below):

    1) Start spark-shell with the --packages argument specifying the sis-embedded-data artifact

    2) Execute CRS.forCode ("EPSG:21781") {color:#808080}// Swiss coordinate system [https://epsg.io/21781]{color}

{color:#172b4d}Result:{color}

{color:#172b4d}    NoSuchAuthorityCodeException (with a prior warning message: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata'){color}

{color:#172b4d}Expected result:{color}

{color:#172b4d}    Returns a valid CoordinateReferenceSystem for "EPSG:21781".{color}

{color:#172b4d}Workaround:{color}

{color:#172b4d}    Use _CRS.fromWKT_ instead.{color}

{color:#172b4d}Note:{color}

{{{color:#172b4d}It works fine in a stand-alone Scala program with build.sbt:{color}}}

{{}}{{{color:#660e7a}name {color}:= {color:#008000}"ApacheSISTest"{color}}}

{{{color:#660e7a}version {color}:= {color:#008000}"0.1"{color}{color:#660e7a}scalaVersion {color}:= {color:#008000}"2.11.12"{color}}}

{{{color:#660e7a}libraryDependencies {color}+= {color:#008000}"org.apache.sis.core" {color}% {color:#008000}"sis-referencing" {color}% {color:#008000}"1.0"{color}+}}

{{+{color:#660e7a}libraryDependencies{color}+ = {color:#008000}"org.apache.sis.non-free" {color}% {color:#008000}"sis-embedded-data" {color}% {color:#008000}"1.0"{color}}}

and code

{color:#000080}import {color}org.apache.sis.geometry.DirectPosition2D
 {color:#000080}import {color}org.apache.sis.referencing.CRS

{color:#000080}object {color}Test {color:#000080}extends {color}App {

     {color:#000080}val {color}{color:#660e7a}position {color}= {color:#000080}new {color}DirectPosition2D ({color:#0000ff}678544.6095273666{color}, {color:#0000ff}248462.881516265{color})
     println ({color:#008000}"Source: " {color}+ {color:#660e7a}position{color})
      {color:#000080}val {color}{color:#660e7a}sourceCRS {color}= CRS.forCode ({color:#008000}"EPSG:21781"{color}) {color:#808080}// Swiss coordinate system [https://epsg.io/21781]{color}     {color:#000080}val {color}{color:#660e7a}targetCRS {color}= CRS.forCode ({color:#008000}"EPSG:4326"{color}) {color:#808080}// WGS 84 [https://epsg.io/4326]{color}     {color:#000080}val {color}{color:#660e7a}operation {color}= CRS.findOperation ({color:#660e7a}sourceCRS{color}, {color:#660e7a}targetCRS{color}, {color:#000080}null{color})

    {color:#000080}val {color}{color:#660e7a}transform {color}= {color:#660e7a}operation{color}.getMathTransform
     {color:#000080}val {color}{color:#660e7a}latlong {color}= {color:#660e7a}transform{color}.transform ({color:#660e7a}position{color}, {color:#000080}null{color}) {color:#808080}// °N, °E{color}     println ({color:#008000}"Target: " {color}+ {color:#660e7a}latlong{color})
 }

 

{color:#172b4d}Full Spark log:{color}

$ *spark-shell --master local[*] --verbose --packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0*
 Using properties file: null
 20/03/08 15:41:42 WARN Utils: Your hostname, <computer> resolves to a loopback address: 127.0.1.1; using 192.168.10.221 instead (on interface wlp4s0)
 20/03/08 15:41:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
 Parsed arguments:
 master local[*]
 deployMode null
 executorMemory null
 executorCores null
 totalExecutorCores null
 propertiesFile null
 driverMemory null
 driverCores null
 driverExtraClassPath null
 driverExtraLibraryPath null
 driverExtraJavaOptions null
 supervise false
 queue null
 numExecutors null
 files null
 pyFiles null
 archives null
 mainClass org.apache.spark.repl.Main
 primaryResource spark-shell
 name Spark shell
 childArgs []
 jars null
 packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0
 packagesExclusions null
 repositories null
 verbose true

Spark properties used, including those specified through
 --conf and those from the properties file null:

Ivy Default Cache set to: /home/<user>/.ivy2/cache
 The jars for the packages stored in: /home/<user>/.ivy2/jars
 :: loading settings :: url = jar:[file:/home/|file:///home/]<user>/spark/spark-2.4.4-bin-hadoop2.7/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
 org.apache.sis.core#sis-referencing added as a dependency
 org.apache.sis.non-free#sis-embedded-data added as a dependency
 :: resolving dependencies :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a;1.0
 confs: [default]
 found org.apache.sis.core#sis-referencing;1.0 in central
 found org.apache.sis.core#sis-utility;1.0 in central
 found javax.measure#unit-api;1.0 in central
 found org.opengis#geoapi;3.0.1 in central
 found org.apache.sis.core#sis-metadata;1.0 in central
 found org.apache.sis.non-free#sis-embedded-data;1.0 in central
 found org.apache.derby#derby;10.14.2.0 in central
 downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-referencing/1.0/sis-referencing-1.0.jar] ...
 [SUCCESSFUL ] org.apache.sis.core#sis-referencing;1.0!sis-referencing.jar (245ms)
 downloading [https://repo1.maven.org/maven2/org/apache/sis/non-free/sis-embedded-data/1.0/sis-embedded-data-1.0.jar] ...
 [SUCCESSFUL ] org.apache.sis.non-free#sis-embedded-data;1.0!sis-embedded-data.jar (489ms)
 downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-utility/1.0/sis-utility-1.0.jar] ...
 [SUCCESSFUL ] org.apache.sis.core#sis-utility;1.0!sis-utility.jar (92ms)
 downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-metadata/1.0/sis-metadata-1.0.jar] ...
 [SUCCESSFUL ] org.apache.sis.core#sis-metadata;1.0!sis-metadata.jar (134ms)
 :: resolution report :: resolve 7312ms :: artifacts dl 970ms
 :: modules in use:
 javax.measure#unit-api;1.0 from central in [default]
 org.apache.derby#derby;10.14.2.0 from central in [default]
 org.apache.sis.core#sis-metadata;1.0 from central in [default]
 org.apache.sis.core#sis-referencing;1.0 from central in [default]
 org.apache.sis.core#sis-utility;1.0 from central in [default]
 org.apache.sis.non-free#sis-embedded-data;1.0 from central in [default]
 org.opengis#geoapi;3.0.1 from central in [default]
 ---------------------------------------------------------------------
| |modules||artifacts||
|conf|number|search|dwnlded|evicted||number|dwnlded|

---------------------------------------------------------------------
|default|7|6|6|0||7|4|

---------------------------------------------------------------------

:: problems summary ::
 :::: ERRORS
 unknown resolver null

unknown resolver null

unknown resolver null

:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
 :: retrieving :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a
 confs: [default]
 7 artifacts copied, 0 already retrieved (10240kB/20ms)
 Main class:
 org.apache.spark.repl.Main
 Arguments:

Spark config:
 (spark.jars,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,[file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,[file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
 (spark.app.name,Spark shell)
 (spark.ui.showConsoleProgress,true)
 (spark.submit.deployMode,client)
 (spark.master,local[*])
 (spark.repl.local.jars,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,[file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,[file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
 Classpath elements:
 [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar
 [file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar
 [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar
 [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar
 [file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar
 [file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar
 [file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar

20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
 20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
 Spark context Web UI available at [http://192.168.10.221:4042|http://192.168.10.221:4042/]
 Spark context available as 'sc' (master = local[*], app id = local-1583678518329).
 Spark session available as 'spark'.
 Welcome to
 ____ __
 / __/__ ___ _____/ /__
 _\ \/ _ \/ _ `/ __/ '_/
 /___/ .__/_,_/_/ /_/_\ version 2.4.4
 /_/

Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_242)
 Type in expressions to have them evaluated.
 Type :help for more information.

scala> *import org.apache.sis.geometry.DirectPosition2D*
 import org.apache.sis.geometry.DirectPosition2D

scala> *import org.apache.sis.referencing.CRS*
 import org.apache.sis.referencing.CRS

scala> *val position = new DirectPosition2D (678544.6095273666, 248462.881516265)*
 position: org.apache.sis.geometry.DirectPosition2D = POINT(678544.6095273666 248462.881516265)

scala> *println ("Source: " + position)*
 Source: POINT(678544.6095273666 248462.881516265)

scala>

scala> *val sourceCRS = CRS.forCode ("EPSG:21781")*
 20/03/08 15:43:14 WARN factory: Can not use the EPSG geodetic parameters: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
 org.opengis.referencing.NoSuchAuthorityCodeException: No ‘CoordinateReferenceSystem’ object found for code “21781”. However only a subset of the EPSG geodetic dataset has been queried. See [http://sis.apache.org/epsg.html] for instruction about how to install the full EPSG database.
 at org.apache.sis.referencing.EPSGFactoryFallback.predefined(EPSGFactoryFallback.java:355)
 at org.apache.sis.referencing.EPSGFactoryFallback.createCoordinateReferenceSystem(EPSGFactoryFallback.java:231)
 at org.apache.sis.referencing.CRS.forCode(CRS.java:236)
 ... 49 elided

scala> *val targetCRS = CRS.forCode ("EPSG:4326")* 
 20/03/08 15:43:17 WARN system: Can not connect to “jdbc/SpatialMetadata”.
 java.sql.SQLException: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
 at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
 at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
 at org.apache.sis.metadata.sql.MetadataSource.connection(MetadataSource.java:490)
 at org.apache.sis.metadata.sql.MetadataSource.install(MetadataSource.java:457)
 at org.apache.sis.metadata.sql.MetadataSource.getProvided(MetadataSource.java:318)
 at org.apache.sis.internal.simple.CitationConstant.delegate(CitationConstant.java:177)
 at org.apache.sis.internal.simple.CitationConstant.getIdentifiers(CitationConstant.java:204)
 at org.apache.sis.internal.metadata.Identifiers.getIdentifier(Identifiers.java:117)
 at org.apache.sis.referencing.ImmutableIdentifier.formatTo(ImmutableIdentifier.java:517)
 at org.apache.sis.io.wkt.Formatter.append(Formatter.java:691)
 at org.apache.sis.io.wkt.Formatter.appendComplement(Formatter.java:849)
 at org.apache.sis.io.wkt.Formatter.append(Formatter.java:738)
 at org.apache.sis.io.wkt.FormattableObject.formatWKT(FormattableObject.java:200)
 at org.apache.sis.io.wkt.FormattableObject.toString(FormattableObject.java:126)
 at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:332)
 at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:337)
 at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:345)
 at $line21.$eval$.$print$lzycompute(<console>:10)
 at $line21.$eval$.$print(<console>:6)
 at $line21.$eval.$print(<console>)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
 at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644)
 at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
 at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
 at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
 at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
 at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:819)
 at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:691)
 at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404)
 at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425)
 at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285)
 at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
 at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
 at org.apache.spark.repl.Main$.doMain(Main.scala:78)
 at org.apache.spark.repl.Main$.main(Main.scala:58)
 at org.apache.spark.repl.Main.main(Main.scala)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
 at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
 at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
 at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
 at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
 at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
 at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
 at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
 Caused by: ERROR XJ040: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
 ... 65 more
 Caused by: ERROR XCL20: Catalogs at version level 'null' cannot be upgraded to version level '10.12'.
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DD_Version.upgradeIfNeeded(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.loadDictionaryTables(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
 at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
 at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
 at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
 at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
 ... 62 more
 targetCRS: org.opengis.referencing.crs.CoordinateReferenceSystem =
 GeodeticCRS["WGS 84",
 Datum["World Geodetic System 1984",
 Ellipsoid["WGS 84", 6378137.0, 298.257223563]],
 CS[ellipsoidal, 2],
 Axis["Geodetic latitude (φ)", north],
 Axis["Geodetic longitude (λ)", east],
 Unit["degree", 0.017453292519943295],
 Scope["Horizontal component of 3D system."],
 Area["World"],
 BBox[-90.00, -180.00, 90.00, 180.00],
 Id["EPSG", 4326, URI["urn:ogc:def:crs:EPSG::4326"]]]

scala>

  was:
Operation _CRS_.forCode fails when used within the Apache Spark environment. 

Steps to reproduce (see full log below):

    1) Start spark-shell with the --packages argument specifying the sis-embedded-data artifact

    2) Execute CRS.forCode ("EPSG:21781") {color:#808080}// Swiss coordinate system [https://epsg.io/21781]
{color}

{color:#172b4d}Result:{color}

{color:#172b4d}    NoSuchAuthorityCodeException (with a prior warning message: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata'){color}

{color:#172b4d}Expected result:{color}

{color:#172b4d}    Returns a valid CoordinateReferenceSystem for "EPSG:21781".
{color}

{color:#172b4d}Workaround:{color}

{color:#172b4d}    Use _CRS.fromWKT_ instead.
{color}

{color:#172b4d}Note:{color}

{{{color:#172b4d}It works fine in a stand-alone Scala program with build.sbt:{color}}}

{{{color:#172b4d}
{color}}}{{{color:#660e7a}name {color}:= {color:#008000}"ApacheSISTest"
{color}{color:#660e7a}version {color}:= {color:#008000}"0.1"
{color}{color:#660e7a}scalaVersion {color}:= {color:#008000}"2.11.12"
{color}{color:#660e7a}libraryDependencies {color}+= {color:#008000}"org.apache.sis.core" {color}% {color:#008000}"sis-referencing" {color}% {color:#008000}"1.0"
{color}{color:#660e7a}libraryDependencies {color}+= {color:#008000}"org.apache.sis.non-free" {color}% {color:#008000}"sis-embedded-data" {color}% {color:#008000}"1.0"{color}}}

and code

{color:#000080}import {color}org.apache.sis.geometry.DirectPosition2D
{color:#000080}import {color}org.apache.sis.referencing.CRS

{color:#000080}object {color}Test {color:#000080}extends {color}App {

     {color:#000080}val {color}{color:#660e7a}position {color}= {color:#000080}new {color}DirectPosition2D ({color:#0000ff}678544.6095273666{color}, {color:#0000ff}248462.881516265{color})
     println ({color:#008000}"Source: " {color}+ {color:#660e7a}position{color})
     {color:#000080}val {color}{color:#660e7a}sourceCRS {color}= CRS.forCode ({color:#008000}"EPSG:21781"{color}) {color:#808080}// Swiss coordinate system https://epsg.io/21781
{color}     {color:#000080}val {color}{color:#660e7a}targetCRS {color}= CRS.forCode ({color:#008000}"EPSG:4326"{color}) {color:#808080}// WGS 84 https://epsg.io/4326
{color}{color:#808080}
{color}     {color:#000080}val {color}{color:#660e7a}operation {color}= CRS.findOperation ({color:#660e7a}sourceCRS{color}, {color:#660e7a}targetCRS{color}, {color:#000080}null{color})

     {color:#000080}val {color}{color:#660e7a}transform {color}= {color:#660e7a}operation{color}.getMathTransform
     {color:#000080}val {color}{color:#660e7a}latlong {color}= {color:#660e7a}transform{color}.transform ({color:#660e7a}position{color}, {color:#000080}null{color}) {color:#808080}// °N, °E
{color}     println ({color:#008000}"Target: " {color}+ {color:#660e7a}latlong{color})
}

 

{color:#172b4d}Full Spark log:
{color}

$ *spark-shell --master local[*] --verbose --packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0*
Using properties file: null
20/03/08 15:41:42 WARN Utils: Your hostname, <computer> resolves to a loopback address: 127.0.1.1; using 192.168.10.221 instead (on interface wlp4s0)
20/03/08 15:41:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
Parsed arguments:
 master local[*]
 deployMode null
 executorMemory null
 executorCores null
 totalExecutorCores null
 propertiesFile null
 driverMemory null
 driverCores null
 driverExtraClassPath null
 driverExtraLibraryPath null
 driverExtraJavaOptions null
 supervise false
 queue null
 numExecutors null
 files null
 pyFiles null
 archives null
 mainClass org.apache.spark.repl.Main
 primaryResource spark-shell
 name Spark shell
 childArgs []
 jars null
 packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0
 packagesExclusions null
 repositories null
 verbose true

Spark properties used, including those specified through
 --conf and those from the properties file null:


Ivy Default Cache set to: /home/<user>/.ivy2/cache
The jars for the packages stored in: /home/<user>/.ivy2/jars
:: loading settings :: url = jar:file:/home/<user>/spark/spark-2.4.4-bin-hadoop2.7/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
org.apache.sis.core#sis-referencing added as a dependency
org.apache.sis.non-free#sis-embedded-data added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a;1.0
 confs: [default]
 found org.apache.sis.core#sis-referencing;1.0 in central
 found org.apache.sis.core#sis-utility;1.0 in central
 found javax.measure#unit-api;1.0 in central
 found org.opengis#geoapi;3.0.1 in central
 found org.apache.sis.core#sis-metadata;1.0 in central
 found org.apache.sis.non-free#sis-embedded-data;1.0 in central
 found org.apache.derby#derby;10.14.2.0 in central
downloading https://repo1.maven.org/maven2/org/apache/sis/core/sis-referencing/1.0/sis-referencing-1.0.jar ...
 [SUCCESSFUL ] org.apache.sis.core#sis-referencing;1.0!sis-referencing.jar (245ms)
downloading https://repo1.maven.org/maven2/org/apache/sis/non-free/sis-embedded-data/1.0/sis-embedded-data-1.0.jar ...
 [SUCCESSFUL ] org.apache.sis.non-free#sis-embedded-data;1.0!sis-embedded-data.jar (489ms)
downloading https://repo1.maven.org/maven2/org/apache/sis/core/sis-utility/1.0/sis-utility-1.0.jar ...
 [SUCCESSFUL ] org.apache.sis.core#sis-utility;1.0!sis-utility.jar (92ms)
downloading https://repo1.maven.org/maven2/org/apache/sis/core/sis-metadata/1.0/sis-metadata-1.0.jar ...
 [SUCCESSFUL ] org.apache.sis.core#sis-metadata;1.0!sis-metadata.jar (134ms)
:: resolution report :: resolve 7312ms :: artifacts dl 970ms
 :: modules in use:
 javax.measure#unit-api;1.0 from central in [default]
 org.apache.derby#derby;10.14.2.0 from central in [default]
 org.apache.sis.core#sis-metadata;1.0 from central in [default]
 org.apache.sis.core#sis-referencing;1.0 from central in [default]
 org.apache.sis.core#sis-utility;1.0 from central in [default]
 org.apache.sis.non-free#sis-embedded-data;1.0 from central in [default]
 org.opengis#geoapi;3.0.1 from central in [default]
 ---------------------------------------------------------------------
 | | modules || artifacts |
 | conf | number| search|dwnlded|evicted|| number|dwnlded|
 ---------------------------------------------------------------------
 | default | 7 | 6 | 6 | 0 || 7 | 4 |
 ---------------------------------------------------------------------

:: problems summary ::
:::: ERRORS
 unknown resolver null

unknown resolver null

unknown resolver null


:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:: retrieving :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a
 confs: [default]
 7 artifacts copied, 0 already retrieved (10240kB/20ms)
Main class:
org.apache.spark.repl.Main
Arguments:

Spark config:
(spark.jars,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,file:///home/<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,file:///home/<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,file:///home/<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
(spark.app.name,Spark shell)
(spark.ui.showConsoleProgress,true)
(spark.submit.deployMode,client)
(spark.master,local[*])
(spark.repl.local.jars,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,file:///home/<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,file:///home/<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,file:///home/<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
Classpath elements:
file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar
file:///home/<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar
file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar
file:///home/<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar
file:///home/<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar
file:///home/<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar
file:///home/<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar


20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
Spark context Web UI available at http://192.168.10.221:4042
Spark context available as 'sc' (master = local[*], app id = local-1583678518329).
Spark session available as 'spark'.
Welcome to
 ____ __
 / __/__ ___ _____/ /__
 _\ \/ _ \/ _ `/ __/ '_/
 /___/ .__/\_,_/_/ /_/\_\ version 2.4.4
 /_/
 
Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_242)
Type in expressions to have them evaluated.
Type :help for more information.

scala> *import org.apache.sis.geometry.DirectPosition2D*
import org.apache.sis.geometry.DirectPosition2D

scala> *import org.apache.sis.referencing.CRS*
import org.apache.sis.referencing.CRS

scala> *val position = new DirectPosition2D (678544.6095273666, 248462.881516265)*
position: org.apache.sis.geometry.DirectPosition2D = POINT(678544.6095273666 248462.881516265)

scala> *println ("Source: " + position)*
Source: POINT(678544.6095273666 248462.881516265)

scala>

scala> *val sourceCRS = CRS.forCode ("EPSG:21781")*
20/03/08 15:43:14 WARN factory: Can not use the EPSG geodetic parameters: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
org.opengis.referencing.NoSuchAuthorityCodeException: No ‘CoordinateReferenceSystem’ object found for code “21781”. However only a subset of the EPSG geodetic dataset has been queried. See http://sis.apache.org/epsg.html for instruction about how to install the full EPSG database.
 at org.apache.sis.referencing.EPSGFactoryFallback.predefined(EPSGFactoryFallback.java:355)
 at org.apache.sis.referencing.EPSGFactoryFallback.createCoordinateReferenceSystem(EPSGFactoryFallback.java:231)
 at org.apache.sis.referencing.CRS.forCode(CRS.java:236)
 ... 49 elided

scala> *val targetCRS = CRS.forCode ("EPSG:4326")* 
20/03/08 15:43:17 WARN system: Can not connect to “jdbc/SpatialMetadata”.
java.sql.SQLException: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
 at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
 at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
 at org.apache.sis.metadata.sql.MetadataSource.connection(MetadataSource.java:490)
 at org.apache.sis.metadata.sql.MetadataSource.install(MetadataSource.java:457)
 at org.apache.sis.metadata.sql.MetadataSource.getProvided(MetadataSource.java:318)
 at org.apache.sis.internal.simple.CitationConstant.delegate(CitationConstant.java:177)
 at org.apache.sis.internal.simple.CitationConstant.getIdentifiers(CitationConstant.java:204)
 at org.apache.sis.internal.metadata.Identifiers.getIdentifier(Identifiers.java:117)
 at org.apache.sis.referencing.ImmutableIdentifier.formatTo(ImmutableIdentifier.java:517)
 at org.apache.sis.io.wkt.Formatter.append(Formatter.java:691)
 at org.apache.sis.io.wkt.Formatter.appendComplement(Formatter.java:849)
 at org.apache.sis.io.wkt.Formatter.append(Formatter.java:738)
 at org.apache.sis.io.wkt.FormattableObject.formatWKT(FormattableObject.java:200)
 at org.apache.sis.io.wkt.FormattableObject.toString(FormattableObject.java:126)
 at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:332)
 at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:337)
 at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:345)
 at $line21.$eval$.$print$lzycompute(<console>:10)
 at $line21.$eval$.$print(<console>:6)
 at $line21.$eval.$print(<console>)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
 at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644)
 at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
 at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
 at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
 at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
 at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
 at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:819)
 at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:691)
 at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404)
 at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425)
 at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285)
 at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
 at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
 at org.apache.spark.repl.Main$.doMain(Main.scala:78)
 at org.apache.spark.repl.Main$.main(Main.scala:58)
 at org.apache.spark.repl.Main.main(Main.scala)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
 at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
 at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
 at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
 at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
 at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
 at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
 at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: ERROR XJ040: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
 ... 65 more
Caused by: ERROR XCL20: Catalogs at version level 'null' cannot be upgraded to version level '10.12'.
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DD_Version.upgradeIfNeeded(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.loadDictionaryTables(Unknown Source)
 at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
 at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
 at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
 at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
 at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
 at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
 at java.security.AccessController.doPrivileged(Native Method)
 at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
 ... 62 more
targetCRS: org.opengis.referencing.crs.CoordinateReferenceSystem =
GeodeticCRS["WGS 84",
 Datum["World Geodetic System 1984",
 Ellipsoid["WGS 84", 6378137.0, 298.257223563]],
 CS[ellipsoidal, 2],
 Axis["Geodetic latitude (φ)", north],
 Axis["Geodetic longitude (λ)", east],
 Unit["degree", 0.017453292519943295],
 Scope["Horizontal component of 3D system."],
 Area["World"],
 BBox[-90.00, -180.00, 90.00, 180.00],
 Id["EPSG", 4326, URI["urn:ogc:def:crs:EPSG::4326"]]]

scala>


> sis-embedded-data database fails to start on Apache Spark
> ---------------------------------------------------------
>
>                 Key: SIS-491
>                 URL: https://issues.apache.org/jira/browse/SIS-491
>             Project: Spatial Information Systems
>          Issue Type: Bug
>          Components: Metadata
>    Affects Versions: 0.8, 1.0
>         Environment: Spark version 2.4.4
> Hadoop version 2.7.6
> Scala version 2.11.12
> Java version 1.8.0_242
> OS: Linux <computer> 4.15.0-88-generic #88-Ubuntu SMP Tue Feb 11 20:11:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
>            Reporter: Derrick Oswald
>            Priority: Minor
>
> Operation _CRS_.forCode fails when used within the Apache Spark environment. 
> Steps to reproduce (see full log below):
>     1) Start spark-shell with the --packages argument specifying the sis-embedded-data artifact
>     2) Execute CRS.forCode ("EPSG:21781") {color:#808080}// Swiss coordinate system [https://epsg.io/21781]{color}
> {color:#172b4d}Result:{color}
> {color:#172b4d}    NoSuchAuthorityCodeException (with a prior warning message: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata'){color}
> {color:#172b4d}Expected result:{color}
> {color:#172b4d}    Returns a valid CoordinateReferenceSystem for "EPSG:21781".{color}
> {color:#172b4d}Workaround:{color}
> {color:#172b4d}    Use _CRS.fromWKT_ instead.{color}
> {color:#172b4d}Note:{color}
> {{{color:#172b4d}It works fine in a stand-alone Scala program with build.sbt:{color}}}
> {{}}{{{color:#660e7a}name {color}:= {color:#008000}"ApacheSISTest"{color}}}
> {{{color:#660e7a}version {color}:= {color:#008000}"0.1"{color}{color:#660e7a}scalaVersion {color}:= {color:#008000}"2.11.12"{color}}}
> {{{color:#660e7a}libraryDependencies {color}+= {color:#008000}"org.apache.sis.core" {color}% {color:#008000}"sis-referencing" {color}% {color:#008000}"1.0"{color}+}}
> {{+{color:#660e7a}libraryDependencies{color}+ = {color:#008000}"org.apache.sis.non-free" {color}% {color:#008000}"sis-embedded-data" {color}% {color:#008000}"1.0"{color}}}
> and code
> {color:#000080}import {color}org.apache.sis.geometry.DirectPosition2D
>  {color:#000080}import {color}org.apache.sis.referencing.CRS
> {color:#000080}object {color}Test {color:#000080}extends {color}App {
>      {color:#000080}val {color}{color:#660e7a}position {color}= {color:#000080}new {color}DirectPosition2D ({color:#0000ff}678544.6095273666{color}, {color:#0000ff}248462.881516265{color})
>      println ({color:#008000}"Source: " {color}+ {color:#660e7a}position{color})
>       {color:#000080}val {color}{color:#660e7a}sourceCRS {color}= CRS.forCode ({color:#008000}"EPSG:21781"{color}) {color:#808080}// Swiss coordinate system [https://epsg.io/21781]{color}     {color:#000080}val {color}{color:#660e7a}targetCRS {color}= CRS.forCode ({color:#008000}"EPSG:4326"{color}) {color:#808080}// WGS 84 [https://epsg.io/4326]{color}     {color:#000080}val {color}{color:#660e7a}operation {color}= CRS.findOperation ({color:#660e7a}sourceCRS{color}, {color:#660e7a}targetCRS{color}, {color:#000080}null{color})
>     {color:#000080}val {color}{color:#660e7a}transform {color}= {color:#660e7a}operation{color}.getMathTransform
>      {color:#000080}val {color}{color:#660e7a}latlong {color}= {color:#660e7a}transform{color}.transform ({color:#660e7a}position{color}, {color:#000080}null{color}) {color:#808080}// °N, °E{color}     println ({color:#008000}"Target: " {color}+ {color:#660e7a}latlong{color})
>  }
>  
> {color:#172b4d}Full Spark log:{color}
> $ *spark-shell --master local[*] --verbose --packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0*
>  Using properties file: null
>  20/03/08 15:41:42 WARN Utils: Your hostname, <computer> resolves to a loopback address: 127.0.1.1; using 192.168.10.221 instead (on interface wlp4s0)
>  20/03/08 15:41:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
>  Parsed arguments:
>  master local[*]
>  deployMode null
>  executorMemory null
>  executorCores null
>  totalExecutorCores null
>  propertiesFile null
>  driverMemory null
>  driverCores null
>  driverExtraClassPath null
>  driverExtraLibraryPath null
>  driverExtraJavaOptions null
>  supervise false
>  queue null
>  numExecutors null
>  files null
>  pyFiles null
>  archives null
>  mainClass org.apache.spark.repl.Main
>  primaryResource spark-shell
>  name Spark shell
>  childArgs []
>  jars null
>  packages org.apache.sis.core:sis-referencing:1.0,org.apache.sis.non-free:sis-embedded-data:1.0
>  packagesExclusions null
>  repositories null
>  verbose true
> Spark properties used, including those specified through
>  --conf and those from the properties file null:
> Ivy Default Cache set to: /home/<user>/.ivy2/cache
>  The jars for the packages stored in: /home/<user>/.ivy2/jars
>  :: loading settings :: url = jar:[file:/home/|file:///home/]<user>/spark/spark-2.4.4-bin-hadoop2.7/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
>  org.apache.sis.core#sis-referencing added as a dependency
>  org.apache.sis.non-free#sis-embedded-data added as a dependency
>  :: resolving dependencies :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a;1.0
>  confs: [default]
>  found org.apache.sis.core#sis-referencing;1.0 in central
>  found org.apache.sis.core#sis-utility;1.0 in central
>  found javax.measure#unit-api;1.0 in central
>  found org.opengis#geoapi;3.0.1 in central
>  found org.apache.sis.core#sis-metadata;1.0 in central
>  found org.apache.sis.non-free#sis-embedded-data;1.0 in central
>  found org.apache.derby#derby;10.14.2.0 in central
>  downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-referencing/1.0/sis-referencing-1.0.jar] ...
>  [SUCCESSFUL ] org.apache.sis.core#sis-referencing;1.0!sis-referencing.jar (245ms)
>  downloading [https://repo1.maven.org/maven2/org/apache/sis/non-free/sis-embedded-data/1.0/sis-embedded-data-1.0.jar] ...
>  [SUCCESSFUL ] org.apache.sis.non-free#sis-embedded-data;1.0!sis-embedded-data.jar (489ms)
>  downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-utility/1.0/sis-utility-1.0.jar] ...
>  [SUCCESSFUL ] org.apache.sis.core#sis-utility;1.0!sis-utility.jar (92ms)
>  downloading [https://repo1.maven.org/maven2/org/apache/sis/core/sis-metadata/1.0/sis-metadata-1.0.jar] ...
>  [SUCCESSFUL ] org.apache.sis.core#sis-metadata;1.0!sis-metadata.jar (134ms)
>  :: resolution report :: resolve 7312ms :: artifacts dl 970ms
>  :: modules in use:
>  javax.measure#unit-api;1.0 from central in [default]
>  org.apache.derby#derby;10.14.2.0 from central in [default]
>  org.apache.sis.core#sis-metadata;1.0 from central in [default]
>  org.apache.sis.core#sis-referencing;1.0 from central in [default]
>  org.apache.sis.core#sis-utility;1.0 from central in [default]
>  org.apache.sis.non-free#sis-embedded-data;1.0 from central in [default]
>  org.opengis#geoapi;3.0.1 from central in [default]
>  ---------------------------------------------------------------------
> | |modules||artifacts||
> |conf|number|search|dwnlded|evicted||number|dwnlded|
> ---------------------------------------------------------------------
> |default|7|6|6|0||7|4|
> ---------------------------------------------------------------------
> :: problems summary ::
>  :::: ERRORS
>  unknown resolver null
> unknown resolver null
> unknown resolver null
> :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>  :: retrieving :: org.apache.spark#spark-submit-parent-83046344-c80a-4a8a-9d41-e6145875a27a
>  confs: [default]
>  7 artifacts copied, 0 already retrieved (10240kB/20ms)
>  Main class:
>  org.apache.spark.repl.Main
>  Arguments:
> Spark config:
>  (spark.jars,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,[file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,[file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
>  (spark.app.name,Spark shell)
>  (spark.ui.showConsoleProgress,true)
>  (spark.submit.deployMode,client)
>  (spark.master,local[*])
>  (spark.repl.local.jars,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar,[file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar,[file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar,[file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar)
>  Classpath elements:
>  [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-referencing-1.0.jar
>  [file:///home/]<user>/.ivy2/jars/org.apache.sis.non-free_sis-embedded-data-1.0.jar
>  [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-utility-1.0.jar
>  [file:///home/]<user>/.ivy2/jars/org.apache.sis.core_sis-metadata-1.0.jar
>  [file:///home/]<user>/.ivy2/jars/javax.measure_unit-api-1.0.jar
>  [file:///home/]<user>/.ivy2/jars/org.opengis_geoapi-3.0.1.jar
>  [file:///home/]<user>/.ivy2/jars/org.apache.derby_derby-10.14.2.0.jar
> 20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
>  20/03/08 15:41:58 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
>  Spark context Web UI available at [http://192.168.10.221:4042|http://192.168.10.221:4042/]
>  Spark context available as 'sc' (master = local[*], app id = local-1583678518329).
>  Spark session available as 'spark'.
>  Welcome to
>  ____ __
>  / __/__ ___ _____/ /__
>  _\ \/ _ \/ _ `/ __/ '_/
>  /___/ .__/_,_/_/ /_/_\ version 2.4.4
>  /_/
> Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_242)
>  Type in expressions to have them evaluated.
>  Type :help for more information.
> scala> *import org.apache.sis.geometry.DirectPosition2D*
>  import org.apache.sis.geometry.DirectPosition2D
> scala> *import org.apache.sis.referencing.CRS*
>  import org.apache.sis.referencing.CRS
> scala> *val position = new DirectPosition2D (678544.6095273666, 248462.881516265)*
>  position: org.apache.sis.geometry.DirectPosition2D = POINT(678544.6095273666 248462.881516265)
> scala> *println ("Source: " + position)*
>  Source: POINT(678544.6095273666 248462.881516265)
> scala>
> scala> *val sourceCRS = CRS.forCode ("EPSG:21781")*
>  20/03/08 15:43:14 WARN factory: Can not use the EPSG geodetic parameters: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
>  org.opengis.referencing.NoSuchAuthorityCodeException: No ‘CoordinateReferenceSystem’ object found for code “21781”. However only a subset of the EPSG geodetic dataset has been queried. See [http://sis.apache.org/epsg.html] for instruction about how to install the full EPSG database.
>  at org.apache.sis.referencing.EPSGFactoryFallback.predefined(EPSGFactoryFallback.java:355)
>  at org.apache.sis.referencing.EPSGFactoryFallback.createCoordinateReferenceSystem(EPSGFactoryFallback.java:231)
>  at org.apache.sis.referencing.CRS.forCode(CRS.java:236)
>  ... 49 elided
> scala> *val targetCRS = CRS.forCode ("EPSG:4326")* 
>  20/03/08 15:43:17 WARN system: Can not connect to “jdbc/SpatialMetadata”.
>  java.sql.SQLException: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
>  at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>  at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>  at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
>  at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
>  at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
>  at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
>  at org.apache.derby.jdbc.InternalDriver$1.run(Unknown Source)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at org.apache.derby.jdbc.InternalDriver.getNewEmbedConnection(Unknown Source)
>  at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
>  at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
>  at org.apache.derby.jdbc.BasicEmbeddedDataSource40.getConnection(Unknown Source)
>  at org.apache.sis.metadata.sql.MetadataSource.connection(MetadataSource.java:490)
>  at org.apache.sis.metadata.sql.MetadataSource.install(MetadataSource.java:457)
>  at org.apache.sis.metadata.sql.MetadataSource.getProvided(MetadataSource.java:318)
>  at org.apache.sis.internal.simple.CitationConstant.delegate(CitationConstant.java:177)
>  at org.apache.sis.internal.simple.CitationConstant.getIdentifiers(CitationConstant.java:204)
>  at org.apache.sis.internal.metadata.Identifiers.getIdentifier(Identifiers.java:117)
>  at org.apache.sis.referencing.ImmutableIdentifier.formatTo(ImmutableIdentifier.java:517)
>  at org.apache.sis.io.wkt.Formatter.append(Formatter.java:691)
>  at org.apache.sis.io.wkt.Formatter.appendComplement(Formatter.java:849)
>  at org.apache.sis.io.wkt.Formatter.append(Formatter.java:738)
>  at org.apache.sis.io.wkt.FormattableObject.formatWKT(FormattableObject.java:200)
>  at org.apache.sis.io.wkt.FormattableObject.toString(FormattableObject.java:126)
>  at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:332)
>  at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:337)
>  at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:345)
>  at $line21.$eval$.$print$lzycompute(<console>:10)
>  at $line21.$eval$.$print(<console>:6)
>  at $line21.$eval.$print(<console>)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
>  at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
>  at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:645)
>  at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:644)
>  at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
>  at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
>  at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
>  at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
>  at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
>  at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:819)
>  at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:691)
>  at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:404)
>  at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:425)
>  at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:285)
>  at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
>  at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
>  at org.apache.spark.repl.Main$.doMain(Main.scala:78)
>  at org.apache.spark.repl.Main$.main(Main.scala:58)
>  at org.apache.spark.repl.Main.main(Main.scala)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
>  at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
>  at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
>  at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
>  at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
>  at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
>  at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
>  at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>  Caused by: ERROR XJ040: Failed to start database 'classpath:SIS_DATA/Databases/SpatialMetadata' with class loader sun.misc.Launcher$AppClassLoader@f2a0b8e, see the next exception for details.
>  at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
>  at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source)
>  ... 65 more
>  Caused by: ERROR XCL20: Catalogs at version level 'null' cannot be upgraded to version level '10.12'.
>  at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
>  at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
>  at org.apache.derby.impl.sql.catalog.DD_Version.upgradeIfNeeded(Unknown Source)
>  at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.loadDictionaryTables(Unknown Source)
>  at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.boot(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
>  at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
>  at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source)
>  at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
>  at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source)
>  at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
>  at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
>  at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
>  at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
>  at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
>  at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source)
>  ... 62 more
>  targetCRS: org.opengis.referencing.crs.CoordinateReferenceSystem =
>  GeodeticCRS["WGS 84",
>  Datum["World Geodetic System 1984",
>  Ellipsoid["WGS 84", 6378137.0, 298.257223563]],
>  CS[ellipsoidal, 2],
>  Axis["Geodetic latitude (φ)", north],
>  Axis["Geodetic longitude (λ)", east],
>  Unit["degree", 0.017453292519943295],
>  Scope["Horizontal component of 3D system."],
>  Area["World"],
>  BBox[-90.00, -180.00, 90.00, 180.00],
>  Id["EPSG", 4326, URI["urn:ogc:def:crs:EPSG::4326"]]]
> scala>



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message