phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Heather <james.heat...@mendeley.com>
Subject Re: setting up community repo of Phoenix for CDH5?
Date Mon, 21 Sep 2015 10:25:38 GMT
@JM how did you get on with the parcel building?

Has anyone managed to get 4.5 working on CDH5 now? I was going to stick 
with 4.3 on our cluster until we had a parcel, but I'm now needing to 
use pherf, and that doesn't seem to exist in 4.3.

James

On 16/09/15 12:46, Jean-Marc Spaggiari wrote:
> @James: I'm working on the parcel building ;) If not me I will try to 
> find someone to do it. Stay tuned.
> @Andrewy: It works for me that way, cool! I just have a signature 
> issue where it says I have no signature. Will that be an issue?
>
> Thanks all,
>
> JM
>
> 2015-09-16 3:24 GMT-04:00 James Heather <james.heather@mendeley.com 
> <mailto:james.heather@mendeley.com>>:
>
>     Great! Thank you!
>
>     I'd wondered about parcel building. It did look as though a parcel
>     is just a .tgz, containing the classes and a few bits of meta, so
>     hopefully it's doable. It would be really nice if we could provide
>     a working 4.5 parcel.
>
>     James
>
>     On 16 Sep 2015 01:02, "Andrew Purtell" <apurtell@apache.org
>     <mailto:apurtell@apache.org>> wrote:
>
>         I used dev/make_rc.sh, built with Maven 3.2.2, Java 7u79.
>         Ubuntu build host.
>
>
>         On Tue, Sep 15, 2015 at 4:58 PM, Jean-Marc Spaggiari
>         <jean-marc@spaggiari.org <mailto:jean-marc@spaggiari.org>> wrote:
>
>             No, I don't know why. I will ask and see if I can get a
>             respons on that. I have also started the thread for the
>             Parcel. I will see if I find enough help to work on that.
>
>             Regarding the branch you made, I tried to build it but got
>             the error below. what's the command to build it?
>
>             Thanks,
>
>             JM
>
>             [INFO]
>             ------------------------------------------------------------------------
>             [ERROR] FATAL ERROR
>             [INFO]
>             ------------------------------------------------------------------------
>             [INFO] null
>             [INFO]
>             ------------------------------------------------------------------------
>             [INFO] Trace
>             java.lang.NullPointerException
>                 at
>             org.apache.maven.plugin.surefire.report.DefaultReporterFactory.mergeFromOtherFactories(DefaultReporterFactory.java:82)
>                 at
>             org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:182)
>                 at
>             org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1019)
>                 at
>             org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:853)
>                 at
>             org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:751)
>                 at
>             org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>                 at
>             org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>                 at
>             org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>                 at
>             org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>                 at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>                 at
>             org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>                 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>             Method)
>                 at
>             sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>                 at
>             sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>                 at java.lang.reflect.Method.invoke(Method.java:606)
>                 at
>             org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>                 at
>             org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>                 at
>             org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>                 at
>             org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>             [INFO]
>             ------------------------------------------------------------------------
>             [INFO] Total time: 2 minutes 17 seconds
>             [INFO] Finished at: Tue Sep 15 19:55:03 EDT 2015
>             [INFO] Final Memory: 134M/1648M
>             [INFO]
>             ------------------------------------------------------------------------
>
>
>             2015-09-15 19:55 GMT-04:00 Andrew Purtell
>             <apurtell@apache.org <mailto:apurtell@apache.org>>:
>
>                 Cool, thanks J-M.
>
>                 Do you know why support for query tracing was removed?
>                 If it's just a matter of porting it to the HTrace that
>                 ships with CDH, I can look at that.
>
>
>                 On Tue, Sep 15, 2015 at 4:49 PM, Jean-Marc Spaggiari
>                 <jean-marc@spaggiari.org
>                 <mailto:jean-marc@spaggiari.org>> wrote:
>
>                     Nice! I will see if there is a way to build a
>                     parcel from that the same way there is a parcel
>                     for Apache Phoenix 4.3 in Cloudera Labs... Will
>                     clone what you did and try to build it locally...
>
>                     2015-09-15 19:45 GMT-04:00 Andrew Purtell
>                     <andrew.purtell@gmail.com
>                     <mailto:andrew.purtell@gmail.com>>:
>
>                         I pushed updates to branch 4.5-HBase-1.0-cdh5
>                         and the tag v4.5.2-cdh5.4.5 (1fcb5cf). This is
>                         the pending Phoenix 4.5.2 release, currently
>                         at RC1, likely to pass, that will build
>                         against CDH 5.4.5. If you want release
>                         tarballs I built from this, get them here:
>                             Binary:
>                         http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.5.2-cdh5.4.5-bin.tar.gz
>                             Source:
>                         http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.5.2-cdh5.4.5-src.tar.gz
>
>
>                         ​The source and these binaries incorporate
>                         changes from the Cloudera Labs fork of Phoenix
>                         (https://github.com/cloudera-labs/phoenix),
>                         licensed under the ASL v2, Neither the source
>                         or binary artifacts are in any way "official"
>                         or supported by the Apache Phoenix project.
>                         The source and artifacts are provided by me in
>                         a personal capacity for the convenience of
>                         would-be Phoenix users that also use CDH
>                         5.4(.5). Please don't contact the Apache
>                         Phoenix project for any issues regarding this
>                         source and these binaries.
>                         ​
>
>                         On Mon, Sep 14, 2015 at 10:52 AM, James
>                         Heather <james.heather@mendeley.com
>                         <mailto:james.heather@mendeley.com>> wrote:
>
>                             Done! Thanks for helping!
>
>                             The branches in the repo mirror those in
>                             vanilla Phoenix. We shouldn't push any
>                             changes to the vanilla branches, but only
>                             to "*-cdh5" branches (or any temporary
>                             side branches we need to create).
>
>                             The issue tracker will be very useful, yes.
>
>                             James
>
>
>                             On 14/09/15 17:22, Andrew Purtell wrote:
>>                             This is great James.
>>
>>                             Since this is conveniently on Github,
>>                             maybe we use the issue tracker there?
>>                             Interested parties can set a watch. Would
>>                             you be willing to add 'apurtell' as a
>>                             collaborator on the repo? I will fork and
>>                             send over PRs of course, but you might
>>                             want help?
>>
>>
>>                             On Sep 14, 2015, at 6:21 AM, James
>>                             Heather <james.heather@mendeley.com
>>                             <mailto:james.heather@mendeley.com>> wrote:
>>
>>>                             I've set up a repo at
>>>
>>>                             https://github.com/chiastic-security/phoenix-for-cloudera
>>>
>>>                             It is a fork of the vanilla Phoenix
>>>                             github mirror. I've created a branch
>>>                             called "4.5-HBase-1.0-cdh5", which we
>>>                             can use for making a CDH5-compatible
>>>                             version. I've not made any of the
>>>                             necessary changes so far.
>>>
>>>                             I chose that branch, by the way, because
>>>                             it's the latest release, and is using
>>>                             the same version of HBase as CDH5.4. The
>>>                             master branch of the Phoenix repo is
>>>                             building a snapshot of (the forthcoming)
>>>                             Phoenix 4.6, against HBase 1.1...
>>>                             presumably there will also be a Phoenix
>>>                             4.6 for HBase 1.0?
>>>
>>>                             I'm not certain of the best way to
>>>                             manage this. Perhaps we need a new
>>>                             mailing list for those who want to help,
>>>                             to avoid cluttering this list up.
>>>
>>>                             James
>>>
>>>                             On 13/09/15 02:54, Jean-Marc Spaggiari
>>>                             wrote:
>>>>                             Exact. There is some some code change
>>>>                             because of what has been back ported
>>>>                             into CDH and what has not been. But
>>>>                             overall, it should not be rocket
>>>>                             science. Mostly method signatures...
>>>>
>>>>                             Let us know when the repo is available
>>>>                             so we can help...
>>>>
>>>>                             Thanks,
>>>>
>>>>                             JM
>>>>
>>>>                             2015-09-12 18:38 GMT-04:00 Krishna
>>>>                             <research800@gmail.com
>>>>                             <mailto:research800@gmail.com>>:
>>>>
>>>>                                 As explained here, there are some
>>>>                                 code changes too in addition to pom
>>>>                                 related changes.
>>>>
>>>>                                 http://stackoverflow.com/a/31934434/165130
>>>>
>>>>
>>>>
>>>>                                 On Friday, September 11, 2015,
>>>>                                 Andrew Purtell
>>>>                                 <andrew.purtell@gmail.com
>>>>                                 <mailto:andrew.purtell@gmail.com>>
>>>>                                 wrote:
>>>>
>>>>                                     Or once parameterized, add a
>>>>                                     default off profile that
>>>>                                     redefines them all in one shot
>>>>                                     after the builder activates the
>>>>                                     profile on the maven command
>>>>                                     line with -P ...
>>>>
>>>>
>>>>
>>>>                                     On Sep 11, 2015, at 7:05 AM,
>>>>                                     Andrew Purtell
>>>>                                     <andrew.purtell@gmail.com
>>>>                                     <mailto:andrew.purtell@gmail.com>>
>>>>                                     wrote:
>>>>
>>>>>                                     The group IDs and versions can
>>>>>                                     be parameterized in the POM so
>>>>>                                     they can be overridden on the
>>>>>                                     maven command line with -D.
>>>>>                                     That would be easy and
>>>>>                                     something I think we could get
>>>>>                                     committed without any
>>>>>                                     controversy.
>>>>>
>>>>>
>>>>>                                     On Sep 11, 2015, at 6:53 AM,
>>>>>                                     James Heather
>>>>>                                     <james.heather@mendeley.com>
>>>>>                                     wrote:
>>>>>
>>>>>>                                     Yes, my plan is to create a
>>>>>>                                     fork of the main repo, so
>>>>>>                                     that we can still merge new
>>>>>>                                     Phoenix code into the
>>>>>>                                     CDH-compatible version.
>>>>>>
>>>>>>                                     Before that, I do wonder
>>>>>>                                     whether it's possible to
>>>>>>                                     suggest a few changes to the
>>>>>>                                     main repo that would allow
>>>>>>                                     for compiling a
>>>>>>                                     CDH-compatible version,
>>>>>>                                     without needing to maintain a
>>>>>>                                     separate repo. The bulk of
>>>>>>                                     the changes are to
>>>>>>                                     dependencies in the pom,
>>>>>>                                     which suggests that it could
>>>>>>                                     be done to accept a switch to
>>>>>>                                     mvn build.
>>>>>>
>>>>>>                                     James
>>>>>>
>>>>>>                                     On 11/09/15 14:50, Andrew
>>>>>>                                     Purtell wrote:
>>>>>>>                                     The first step I think is
a
>>>>>>>                                     repo with code that
>>>>>>>                                     compiles. Please initialize
>>>>>>>                                     it by forking
>>>>>>>                                     github.com/apache/phoenix
>>>>>>>                                     <http://github.com/apache/phoenix>
>>>>>>>                                     so we have common ancestors.
>>>>>>>                                     Once we have a clear idea
>>>>>>>                                     (by diff) what is required
>>>>>>>                                     we can figure out if we can
>>>>>>>                                     support compatibility in
>>>>>>>                                     some way.
>>>>>>>
>>>>>>>
>>>>>>>                                     On Sep 9, 2015, at 11:00
PM,
>>>>>>>                                     Krishna
>>>>>>>                                     <research800@gmail.com
>>>>>>>                                     <mailto:research800@gmail.com>>
>>>>>>>                                     wrote:
>>>>>>>
>>>>>>>>                                     I can volunteer to spend
>>>>>>>>                                     some time on this.
>>>>>>>>
>>>>>>>>                                     CDH artifacts are available
>>>>>>>>                                     in Maven repo but
>>>>>>>>                                     from reading other threads
>>>>>>>>                                     on CDH-Phoenix
>>>>>>>>                                     compatibilty, it looks
like
>>>>>>>>                                     there are some code changes
>>>>>>>>                                     to be made in Phoenix
to
>>>>>>>>                                     successfully compile
>>>>>>>>                                     against CDH.
>>>>>>>>
>>>>>>>>                                     Here are questions to
address:
>>>>>>>>                                     1) How to maintain CDH
>>>>>>>>                                     compatible Phoenix code
base?
>>>>>>>>                                     2) Is having a CDH
>>>>>>>>                                     compatible branch even
an
>>>>>>>>                                     option?
>>>>>>>>
>>>>>>>>                                     Krishna
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                                     On Friday, August 28,
2015,
>>>>>>>>                                     Andrew Purtell
>>>>>>>>                                     <andrew.purtell@gmail.com
>>>>>>>>                                     <mailto:andrew.purtell@gmail.com>>
>>>>>>>>                                     wrote:
>>>>>>>>
>>>>>>>>                                         Yes I am interested.
>>>>>>>>                                         Assuming CDH artifacts
>>>>>>>>                                         are publicly available
>>>>>>>>                                         in a Maven repo
>>>>>>>>                                         somewhere, which
I
>>>>>>>>                                         believe is the case,
>>>>>>>>                                         perhaps we (the Phoenix
>>>>>>>>                                         project/community)
>>>>>>>>                                         could set up a Jenkins
>>>>>>>>                                         job that builds against
>>>>>>>>                                         them and makes the
>>>>>>>>                                         resulting build
>>>>>>>>                                         artifacts available.
>>>>>>>>                                         They would never
be an
>>>>>>>>                                         official release,
just
>>>>>>>>                                         a best effort
>>>>>>>>                                         convenience. Would
that
>>>>>>>>                                         work? I think little
>>>>>>>>                                         must be done besides
>>>>>>>>                                         compile against the
CDH
>>>>>>>>                                         artifacts for binary
>>>>>>>>                                         compatibility.
>>>>>>>>
>>>>>>>>
>>>>>>>>                                         > On Aug 28, 2015,
at
>>>>>>>>                                         11:19 AM, James Heather
>>>>>>>>                                         <james.heather@mendeley.com
>>>>>>>>                                         <mailto:james.heather@mendeley.com>>
>>>>>>>>                                         wrote:
>>>>>>>>                                         >
>>>>>>>>                                         > Is anyone interested
>>>>>>>>                                         in helping with getting
>>>>>>>>                                         an up-to-date
>>>>>>>>                                         CDH5-compatible build
>>>>>>>>                                         of Phoenix up and
running?
>>>>>>>>                                         >
>>>>>>>>                                         > Cloudera has
a build
>>>>>>>>                                         of Phoenix 4.3
>>>>>>>>                                         (https://github.com/cloudera-labs/phoenix),
>>>>>>>>                                         but this is now two
>>>>>>>>                                         versions behind,
and
>>>>>>>>                                         there seems little
>>>>>>>>                                         desire at Cloudera
to
>>>>>>>>                                         keep it updated.
>>>>>>>>                                         >
>>>>>>>>                                         > I imagine that
by
>>>>>>>>                                         looking at the
>>>>>>>>                                         differences between
>>>>>>>>                                         vanilla 4.3 and
>>>>>>>>                                         cloudera labs 4.3,
and
>>>>>>>>                                         with some guidance
from
>>>>>>>>                                         this list, we could
get
>>>>>>>>                                         a good idea of what
>>>>>>>>                                         would need to be
>>>>>>>>                                         modified in 4.5+
and
>>>>>>>>                                         keep a CDH5-compatible
>>>>>>>>                                         build up to date.
>>>>>>>>                                         >
>>>>>>>>                                         > Yes?
>>>>>>>>                                         >
>>>>>>>>                                         > James
>>>>>>>>
>>>>>>
>>>>
>>>
>
>
>
>
>
>
>                 -- 
>                 Best regards,
>
>                    - Andy
>
>                 Problems worthy of attack prove their worth by hitting
>                 back. - Piet Hein (via Tom White)
>
>
>
>
>
>         -- 
>         Best regards,
>
>            - Andy
>
>         Problems worthy of attack prove their worth by hitting back. -
>         Piet Hein (via Tom White)
>
>


Mime
View raw message