phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Error when using Pig Storage: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
Date Tue, 19 Aug 2014 22:26:45 GMT
I don't think an Apache project should spend precious bandwidth tracking
the various and sundry redistributors of Apache ecosystem projects. This is
putting the cart before the horse. The horse is the Apache upstream
projects. The cart is the commercial distributions leveraging the Apache
ecosystem for profit. Spark is not a good example, it is supported by a
commercial concern, Databricks. What commercial company supports Phoenix?
Why do you think it is appropriate to pick on volunteers because editing
POM files is too much work?


On Tue, Aug 19, 2014 at 3:09 PM, Russell Jurney <russell.jurney@gmail.com>
wrote:

> I also created https://issues.apache.org/jira/browse/PHOENIX-1185 because
> requiring users to hand-edit the pom.xml just to build against CDH and HDP
> is nuts.
> ᐧ
>
>
> On Tue, Aug 19, 2014 at 3:03 PM, Russell Jurney <russell.jurney@gmail.com>
> wrote:
>
>> I have created https://issues.apache.org/jira/browse/PHOENIX-1183 and
>> https://issues.apache.org/jira/browse/PHOENIX-1184
>>
>> I hope this can be resolved soon, we are pretty screwed right now.
>> ᐧ
>>
>>
>> On Tue, Aug 19, 2014 at 2:08 PM, Jesse Yates <jesse.k.yates@gmail.com>
>> wrote:
>>
>>> Yup, that looks like an issue to me :-/
>>>
>>> -------------------
>>> Jesse Yates
>>> @jesse_yates
>>> jyates.github.com
>>>
>>>
>>> On Tue, Aug 19, 2014 at 2:06 PM, Russell Jurney <
>>> russell.jurney@gmail.com> wrote:
>>>
>>>> Running against any version would be ok, but it does not work. I get
>>>> this error:
>>>>
>>>> 2014-08-19 14:03:46,904 FATAL org.apache.hadoop.mapred.Child: Error running
child : java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
>>>>
>>>> ᐧ
>>>> I went back and checked out the Phoenix commit before the pom.xml was
>>>> upgraded to 0.98.4
>>>> <https://github.com/apache/phoenix/commit/36a41c86a4cdb16ab91b95882dbdb02e6006831d>,
>>>> and I get the same error.
>>>>
>>>> Not sure what to do. Stuck. I imagine most users must be too, as CDH 5
>>>> is probably the most popular installation there is?
>>>>
>>>> On Tue, Aug 19, 2014 at 11:50 AM, James Taylor <jamestaylor@apache.org>
>>>> wrote:
>>>>
>>>>>  The dependencies on HBase 0.98.4 are *compile time* dependencies. Is
>>>>> it necessary for you to compile against CDH 5.1 or just run against it?
>>>>>
>>>>>
>>>>> On Tuesday, August 19, 2014, Russell Jurney <russell.jurney@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thats really bad. That means... CDH 5.x can't run Phoenix? How can
>>>>>> this be fixed? I'm not sure what to do. We're in limbo on our new
cluster
>>>>>> now.
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 18, 2014 at 11:57 PM, Ravi Kiran <
>>>>>> maghamravikiran@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Russel,
>>>>>>>     Apparently, Phoenix 4.0.0 leverages few API methods of HBase
>>>>>>> 0.98.4 v  which aren't present within 0.98.1 that comes with
CDH 5.1 .
>>>>>>> That's the primary cause for the build issues.
>>>>>>>
>>>>>>> Regards
>>>>>>> Ravi
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 18, 2014 at 5:56 PM, Russell Jurney <
>>>>>>> russell.jurney@gmail.com> wrote:
>>>>>>>
>>>>>>>> Talking to myself, but hopefully creating good docs. Replacing
the
>>>>>>>> previous hadoop version with one I found here:
>>>>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-core/,
2.3.0-mr1-cdh5.1.0,
>>>>>>>> makes things get a little further.
>>>>>>>>
>>>>>>>> I can't get past some build errors, however. Has anyone done
this
>>>>>>>> before me who can help?
>>>>>>>>
>>>>>>>> [ERROR]
>>>>>>>> /Users/rjurney/Software/phoenix4/phoenix-4.0.0-incubating-src/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java:[143,27]
>>>>>>>> cannot find symbol
>>>>>>>>
>>>>>>>>   symbol:   method valueOf(java.lang.String,int,int)
>>>>>>>>
>>>>>>>>   location: class org.apache.hadoop.hbase.ServerName
>>>>>>>>
>>>>>>>>
>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
>>>>>>>> (default-compile) on project phoenix-core: Compilation failure
>>>>>>>>
>>>>>>>> [ERROR]
>>>>>>>> /Users/rjurney/Software/phoenix4/phoenix-4.0.0-incubating-src/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java:[143,27]
>>>>>>>> cannot find symbol
>>>>>>>>
>>>>>>>> [ERROR] symbol:   method valueOf(java.lang.String,int,int)
>>>>>>>>
>>>>>>>> [ERROR] location: class org.apache.hadoop.hbase.ServerName
>>>>>>>>
>>>>>>>> [ERROR] -> [Help 1]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 18, 2014 at 5:41 PM, Russell Jurney <
>>>>>>>> russell.jurney@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Ok, so it is clear to me what I have to do. I have to
edit my
>>>>>>>>> pom.xml to point at CDH 5.1, which translates into:
>>>>>>>>>
>>>>>>>>> Add the cloudera repo:
>>>>>>>>>
>>>>>>>>>     <repository>
>>>>>>>>>       <id>cloudera</id>
>>>>>>>>>       <url>
>>>>>>>>> https://repository.cloudera.com/artifactory/cloudera-repos/</url>
>>>>>>>>>     </repository>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Then change the hadoop and hbase versions:
>>>>>>>>>
>>>>>>>>>      <!-- Hadoop Versions -->
>>>>>>>>>     <hbase-hadoop1.version>0.98.1-cdh5.1.0</hbase-hadoop1.version>
>>>>>>>>>     <hbase-hadoop2.version>0.98.1-cdh5.1.0</hbase-hadoop2.version>
>>>>>>>>>     <hadoop-one.version>2.3.0-cdh5.1.0</hadoop-one.version>
>>>>>>>>>     <hadoop-two.version>2.3.0-cdh5.1.0</hadoop-two.version>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> However, I get this error when I build, which tells me
there is
>>>>>>>>> more complex POM surgery required.
>>>>>>>>>
>>>>>>>>> [ERROR] Failed to execute goal on project phoenix-core:
Could not
>>>>>>>>> resolve dependencies for project
>>>>>>>>> org.apache.phoenix:phoenix-core:jar:4.0.0-incubating:
The following
>>>>>>>>> artifacts could not be resolved:
>>>>>>>>> org.apache.hadoop:hadoop-core:jar:2.3.0-cdh5.1.0,
>>>>>>>>> org.apache.hadoop:hadoop-test:jar:2.3.0-cdh5.1.0: Could
not find artifact
>>>>>>>>> org.apache.hadoop:hadoop-core:jar:2.3.0-cdh5.1.0 in apache
release (
>>>>>>>>> https://repository.apache.org/content/repositories/releases/)
->
>>>>>>>>> [Help 1]
>>>>>>>>>
>>>>>>>>> Beyond changing the versions, I do not know how to fix
this. Can
>>>>>>>>> anyone help?
>>>>>>>>>
>>>>>>>>> In general, is it possible to actually handle different
CDH
>>>>>>>>> versions in this project? One shouldn't have to do pom
surgery to build
>>>>>>>>> Phoenix for the most common platform.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Aug 18, 2014 at 5:15 PM, Russell Jurney <
>>>>>>>>> russell.jurney@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> When I try to store data into Phoenix from Pig, I
get this error.
>>>>>>>>>> I am on CDH 5.1, and Phoenix 4.0.
>>>>>>>>>>
>>>>>>>>>> Anyone know how to resolve this issue?
>>>>>>>>>>
>>>>>>>>>> 2014-08-18 17:11:25,165 INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader:
Current split being processed hdfs://cluster1-srv1.e8.com:8020/e8/prod/web_behavior/anomaly_profile.txt/2014/07/15/00/part-r-00000:0+845
>>>>>>>>>> 2014-08-18 <http://cluster1-srv1.e8.com:8020/e8/prod/web_behavior/anomaly_profile.txt/2014/07/15/00/part-r-00000:0+8452014-08-18>
17:11:25,173 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater
with mapRetainSize=-1 and reduceRetainSize=-1
>>>>>>>>>> 2014-08-18 17:11:25,175 FATAL org.apache.hadoop.mapred.Child:
Error running child : java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext,
but class was expected
>>>>>>>>>> 	at org.apache.phoenix.pig.hadoop.PhoenixOutputFormat.getRecordWriter(PhoenixOutputFormat.java:65)
>>>>>>>>>> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
>>>>>>>>>> 	at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:548)
>>>>>>>>>> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:653)
>>>>>>>>>> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>>>>>>>>>> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>>>>>>>>>> 	at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>>>>> 	at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>>>>>> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
>>>>>>>>>> 	at org.apache.hadoop.mapred.Child.main(Child.java:262)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>>>>>>>> datasyndrome.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>>>>>>> datasyndrome.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>>>>>> datasyndrome.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>>>> datasyndrome.com
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com
>>>> datasyndrome.com
>>>>
>>>
>>>
>>
>>
>> --
>> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.
>> com
>>
>
>
>
> --
> Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.
> com
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
View raw message