phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Di Spaltro <dan.dispal...@gmail.com>
Subject Re: Phoenix client maven dependencies
Date Tue, 23 Sep 2014 07:15:06 GMT
As a side note, Ivy resolvers don't like pom-only releases.  So using Ivy
or an ivy backed tool (SBT) requires you specify the full list of
dependencies explicitly.

On Mon, Sep 22, 2014 at 11:36 AM, Andrew Purtell <apurtell@apache.org>
wrote:

> > HBase does some build magic so they can release against both hadoop1 and
> hadoop2 by changing the version to include -hadoopX in the name so they can
> different named artifacts for each when they release. So obviously, its
> possible for us to do it too :)
>
> It's an ugly hack not magic. We have a script in dev-support,
> generate-hadoopX-poms.sh, that will munge the versions found in a POM
> and write a new set of POMs:
>
>   # Now we do search and replace of explicit strings.  The best
>   # way of seeing what the below does is by doing a diff between
>   # the original pom and the generated pom (pom.xml.hadoop1 or
>   # pom.xml.hadoop2). We replace the compat.module variable with
>   # either hbase-hadoop1-compat or hbase-hadoop2-compat, we
>   # replace the version string in all poms, we change modules
>   # to include reference to the non-standard pom name, we
>   # adjust relative paths so child modules can find the parent pom,
>   # and we enable/disable hadoop 1 and hadoop 2 profiles as
>   # appropriate removing a comment string too.  We output the
>   # new pom beside the original.
>
> We then point Maven at the appropriate munged and version-suffixed
> root POM, once for hadoop1, once for hadoop2.
>
> This practice will thankfully be a thing of the past after HBase 1.0.
> I don't recommend anyone else adopt it.
>
>
> On Mon, Sep 22, 2014 at 10:19 AM, Jesse Yates <jesse.k.yates@gmail.com>
> wrote:
> > From @Flavio
> >
> >> I think the jars on maven central are built against hadoop2, so it would
> >> be REALLY nice to publish both flavours in the next release
> >
> >
> > At the very least, the jars in maven should be what we build by default
> > (which, AFAIK, is hadoop1). HBase does some build magic so they can
> release
> > against both hadoop1 and hadoop2 by changing the version to include
> -hadoopX
> > in the name so they can different named artifacts for each when they
> > release. So obviously, its possible for us to do it too :)
> >
> > The question to me is if we should even continue to support hadoop1 in
> main?
> > I don't think we should, though we could continue it in the 4.X line for
> > compatibility's sake.
> >
> > -------------------
> > Jesse Yates
> > @jesse_yates
> > jyates.github.com
> >
> > On Mon, Sep 22, 2014 at 9:31 AM, James Taylor <jamestaylor@apache.org>
> > wrote:
> >>
> >> +1 to doing the same for hbase-testing-util. Thanks for the analysis,
> >> Andrew!
> >>
> >>     James
> >>
> >> On Mon, Sep 22, 2014 at 9:18 AM, Andrew Purtell <apurtell@apache.org>
> >> wrote:
> >> > On Thu, Sep 18, 2014 at 3:01 PM, James Taylor <jamestaylor@apache.org
> >
> >> > wrote:
> >> >> I see. That makes sense, but it's more of an HBase request than a
> >> >> Phoenix request. If HBase had a "client-only" pom, then Phoenix could
> >> >> have a "client-only" pom as well.
> >> >>
> >> >> Thanks,
> >> >> James
> >> >
> >> > If you specify only the 'hbase-client' Maven module as the HBase
> >> > dependency then this pulls in client-only deps (hbase-common,
> >> > hbase-protocol, commons-codec, commons-collections, commons-io,
> >> > commons-lang, commons-logging, guava, protobuf-java, netty, zookeeper,
> >> > htrace-core, jackson-mapper-asl, log4j)
> >> >
> >> > I think Phoenix might be pulling in all of the other HBase modules
> >> > because phoenix-core specifies the 'hbase-testing-util' HBase module
> >> > as a dependency, and not at test scope:
> >> >
> >> >       <dependencies>
> >> >           <dependency>
> >> >             <groupId>org.apache.hbase</groupId>
> >> >             <artifactId>hbase-testing-util</artifactId>
> >> >             <exclusions>
> >> >               <exclusion>
> >> >                 <groupId>org.jruby</groupId>
> >> >                 <artifactId>jruby-complete</artifactId>
> >> >               </exclusion>
> >> >             </exclusions>
> >> >           </dependency>
> >> >         ...
> >> >
> >> > hbase-testing-util doesn't contain any code. It is a module you can
> >> > use that will pull in *everything* needed to start up mini cluster
> >> > tests, all of the HBase modules including hbase-server and the compat
> >> > modules, with compile scope. Maven doc says about compile scope: "This
> >> > is the default scope, used if none is specified. Compile dependencies
> >> > are available in all classpaths of a project. Furthermore, those
> >> > dependencies are propagated to dependent projects."
> >> >
> >> > Other test dependencies in the phoenix-core POM are included at test
> >> > scope and tagged as optional, e.g.
> >> >
> >> >           <dependency>
> >> >             <groupId>org.apache.hadoop</groupId>
> >> >             <artifactId>hadoop-test</artifactId>
> >> >             <optional>true</optional>
> >> >             <scope>test</scope>
> >> >           </dependency>
> >> >
> >> > Perhaps the same should be done for hbase-testing-util ?
> >> >
> >> >
> >> >> On Thu, Sep 18, 2014 at 1:52 PM, Flavio Pompermaier
> >> >> <pompermaier@okkam.it> wrote:
> >> >>> Because it is not clear which are the exact dependencies to add
to a
> >> >>> pom.xml
> >> >>> that uses the Phoenix client.
> >> >>> For example, why do I need to transitively hold all depency of
> >> >>> hbase-server
> >> >>> in the client side or both hadoop (1 and 2) compatibility jars?!
> >> >>> Wouldn't be
> >> >>> more easy to create a client jar that select only those needed
> classes
> >> >>> of
> >> >>> such jar?
> >> >>>
> >> >>> On Sep 18, 2014 9:12 PM, "Mujtaba Chohan" <mujtaba@apache.org>
> wrote:
> >> >>>>
> >> >>>> Falvio - Client jar is composed on multiple dependency jars
> extracted
> >> >>>> in
> >> >>>> one and is for convenience available in binary download only.
This
> >> >>>> type of
> >> >>>> bundled jars are not supposed to go in maven repo. as maven
> >> >>>> automatically
> >> >>>> resolves required dependencies. To use maven repo., use the
> >> >>>> phoenix-core as
> >> >>>> artifact id and maven will pull in all the required dependencies
> for
> >> >>>> your
> >> >>>> project. Then if you want a single jar for your project, you
can
> >> >>>> bundle your
> >> >>>> project jars in one.
> >> >>>>
> >> >>>>     <dependency>
> >> >>>>         <groupId>org.apache.phoenix</groupId>
> >> >>>>         <artifactId>phoenix-core</artifactId>
> >> >>>>         <version>4.1.0</version>
> >> >>>>     </dependency>
> >> >>>>
> >> >>>> //mujtaba
> >> >>>>
> >> >>>> On Thu, Sep 18, 2014 at 11:54 AM, Flavio Pompermaier
> >> >>>> <pompermaier@okkam.it> wrote:
> >> >>>>>
> >> >>>>> Is the client jar available on maven central?
> >> >>>>>
> >> >>>>> On Sep 18, 2014 8:46 PM, "anil gupta" <anilgupta84@gmail.com>
> wrote:
> >> >>>>>>
> >> >>>>>> Hi Flavio,
> >> >>>>>>
> >> >>>>>> Phoenix-core is for sever side. If this app is for
client side
> then
> >> >>>>>> you
> >> >>>>>> can use client jars. Maybe you can try to use
> >> >>>>>> phoenix-client-without-hbase.jar. In this case, i think
you will
> at
> >> >>>>>> least
> >> >>>>>> need hbase-client jar in your client app to be added
in your pom.
> >> >>>>>>
> >> >>>>>> Thanks,
> >> >>>>>> Anil Gupta
> >> >>>>>>
> >> >>>>>> On Thu, Sep 18, 2014 at 6:13 AM, Flavio Pompermaier
> >> >>>>>> <pompermaier@okkam.it> wrote:
> >> >>>>>>>
> >> >>>>>>> Is it really necessary the dependency to hbase-server..?
Then
> if I
> >> >>>>>>> try
> >> >>>>>>> to run my application from a webapp using resteasy
I get this
> >> >>>>>>> error:
> >> >>>>>>>
> >> >>>>>>> Caused by:
> >> >>>>>>>
> org.jboss.as.server.deployment.DeploymentUnitProcessingException:
> >> >>>>>>> JBAS011232: Only one JAX-RS Application Class allowed.
> >> >>>>>>> com.sun.jersey.api.core.ClasspathResourceConfig
> >> >>>>>>> com.sun.jersey.server.impl.application.DeferredResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.ScanningResourceConfig
> >> >>>>>>> org.apache.hadoop.hbase.rest.ResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.ClassNamesResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.ApplicationAdapter
> >> >>>>>>> com.sun.jersey.api.core.WebAppResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.PackagesResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.DefaultResourceConfig
> >> >>>>>>> com.sun.jersey.api.core.ResourceConfig
> >> >>>>>>> at
> >> >>>>>>>
> >> >>>>>>>
> org.jboss.as.jaxrs.deployment.JaxrsScanningProcessor.scan(JaxrsScanningProcessor.java:209)
> >> >>>>>>>
> >> >>>>>>> On Thu, Sep 18, 2014 at 11:40 AM, Flavio Pompermaier
> >> >>>>>>> <pompermaier@okkam.it> wrote:
> >> >>>>>>>>
> >> >>>>>>>> I was able to obtain a minimal set of dependencies
*for
> hadoop1)
> >> >>>>>>>> using:
> >> >>>>>>>>                 <dependency>
> >> >>>>>>>> <groupId>org.apache.phoenix</groupId>
> >> >>>>>>>> <artifactId>phoenix</artifactId>
> >> >>>>>>>> <version>4.1.0</version>
> >> >>>>>>>> </dependency>
> >> >>>>>>>> <dependency>
> >> >>>>>>>> <groupId>org.apache.hbase</groupId>
> >> >>>>>>>> <artifactId>hbase-client</artifactId>
> >> >>>>>>>> <version>0.98.4-hadoop1</version>
> >> >>>>>>>> </dependency>
> >> >>>>>>>>                 <!-- Need hbase server just
for
> >> >>>>>>>> ConstantSizeRegionSplitPolicy class.. -->
> >> >>>>>>>> <dependency>
> >> >>>>>>>> <groupId>org.apache.hbase</groupId>
> >> >>>>>>>> <artifactId>hbase-server</artifactId>
> >> >>>>>>>> <version>0.98.4-hadoop1</version>
> >> >>>>>>>> <exclusions>
> >> >>>>>>>> <exclusion>
> >> >>>>>>>> <artifactId>*</artifactId>
> >> >>>>>>>> <groupId>*</groupId>
> >> >>>>>>>> </exclusion>
> >> >>>>>>>> </exclusions>
> >> >>>>>>>> </dependency>
> >> >>>>>>>> <dependency>
> >> >>>>>>>> <groupId>org.apache.phoenix</groupId>
> >> >>>>>>>> <artifactId>phoenix-hadoop-compat</artifactId>
> >> >>>>>>>> <version>4.1.0</version>
> >> >>>>>>>> <exclusions>
> >> >>>>>>>>                                  <!-- conflict
with
> >> >>>>>>>> hbase-client-0.98.4-hadoop1 version -->
> >> >>>>>>>> <exclusion>
> >> >>>>>>>> <artifactId>hbase-common</artifactId>
> >> >>>>>>>> <groupId>org.apache.hbase</groupId>
> >> >>>>>>>> </exclusion>
> >> >>>>>>>> </exclusions>
> >> >>>>>>>> </dependency>
> >> >>>>>>>>
> >> >>>>>>>> but I still get this exception (that however
seems not causing
> >> >>>>>>>> any
> >> >>>>>>>> issue):
> >> >>>>>>>> java.lang.RuntimeException: Could not create
 interface
> >> >>>>>>>> org.apache.phoenix.trace.PhoenixSpanReceiver
Is the hadoop
> >> >>>>>>>> compatibility jar
> >> >>>>>>>> on the classpath?
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:60)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.trace.TracingCompat.newTraceMetricSource(TracingCompat.java:40)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.trace.util.Tracing.addTraceMetricsSource(Tracing.java:294)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.jdbc.PhoenixConnection.<clinit>(PhoenixConnection.java:125)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1516)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1489)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1489)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:129)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133)
> >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:571)
> >> >>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:233)
> >> >>>>>>>> Caused by: java.util.NoSuchElementException
> >> >>>>>>>> at
> >> >>>>>>>>
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:357)
> >> >>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
> >> >>>>>>>> at
> >> >>>>>>>>
> >> >>>>>>>>
> org.apache.hadoop.hbase.CompatibilityFactory.getInstance(CompatibilityFactory.java:46)
> >> >>>>>>>> ... 14 more
> >> >>>>>>>>
> >> >>>>>>>> How can I fix those problems?
> >> >>>>>>>>
> >> >>>>>>>> On Thu, Sep 18, 2014 at 10:34 AM, Flavio Pompermaier
> >> >>>>>>>> <pompermaier@okkam.it> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>> Hi to all,
> >> >>>>>>>>>
> >> >>>>>>>>> I'm trying to write a J2EE application
that uses the Phoenix
> >> >>>>>>>>> JDBC
> >> >>>>>>>>> driver for HBase 0.98 (not on yarn for
tests but it will be in
> >> >>>>>>>>> production)
> >> >>>>>>>>> through maven.
> >> >>>>>>>>> Do I really need all the jars of the hbase-server
side
> contained
> >> >>>>>>>>> in
> >> >>>>>>>>> the phoenix core jar..?
> >> >>>>>>>>> I can't see a phoenix-client jar on the
central maven
> repository
> >> >>>>>>>>> while in the documentation there's to use
that jar!
> >> >>>>>>>>>
> >> >>>>>>>>> Thanks in advance,
> >> >>>>>>>>> Flavio
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> --
> >> >>>>>> Thanks & Regards,
> >> >>>>>> Anil Gupta
> >> >>>>
> >> >>>>
> >> >>>
> >> >
> >> >
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> >    - Andy
> >> >
> >> > Problems worthy of attack prove their worth by hitting back. - Piet
> >> > Hein (via Tom White)
> >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein (via Tom White)
>



-- 
Dan Di Spaltro

Mime
View raw message