flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ac@hsk.hk" ...@hsk.hk>
Subject Re: Flume Compilation Error (1.5.0) for Hadoop 2.x and HBase 0.98
Date Sat, 26 Jul 2014 07:29:59 GMT
Hi Saurabh,

Thank you so much!!

I have managed to compile flume 1.5.0 successfully. However, it still returned warning about
"duplicate declaration of version (?) @ line 86, column 17"
May I know where to find the "duplicated declaration” so to remove it?


Regards
Arthur


==== below is major compilation log =====

$ mvn compile
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.flume.flume-ng-sinks:flume-ng-elasticsearch-sink:jar:1.5.0
[WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.apache.httpcomponents:httpclient:jar
-> duplicate declaration of version (?) @ line 86, column 17
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability
of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed
projects.
[WARNING] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Flume
[INFO] Flume NG SDK
[INFO] Flume NG Configuration
[INFO] Flume NG Core
[INFO] Flume NG Sinks
[INFO] Flume NG HDFS Sink
[INFO] Flume NG IRC Sink
[INFO] Flume NG Channels
[INFO] Flume NG JDBC channel
[INFO] Flume NG file-based channel
[INFO] Flume NG Spillable Memory channel
[INFO] Flume NG Node
[INFO] Flume NG Embedded Agent
[INFO] Flume NG HBase Sink
[INFO] Flume NG ElasticSearch Sink
[INFO] Flume NG Morphline Solr Sink
[INFO] Flume Sources
[INFO] Flume Scribe Source
[INFO] Flume JMS Source
[INFO] Flume Twitter Source
[INFO] Flume legacy Sources
[INFO] Flume legacy Avro source
[INFO] Flume legacy Thrift Source
[INFO] Flume NG Clients
[INFO] Flume NG Log4j Appender
[INFO] Flume NG Tools
[INFO] Flume NG distribution
[INFO] Flume NG Integration Tests
[INFO]                           
……

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Flume ...................................... SUCCESS [0.437s]
[INFO] Flume NG SDK ...................................... SUCCESS [1.653s]
[INFO] Flume NG Configuration ............................ SUCCESS [0.693s]
[INFO] Flume NG Core ..................................... SUCCESS [2.183s]
[INFO] Flume NG Sinks .................................... SUCCESS [0.006s]
[INFO] Flume NG HDFS Sink ................................ SUCCESS [0.948s]
[INFO] Flume NG IRC Sink ................................. SUCCESS [0.330s]
[INFO] Flume NG Channels ................................. SUCCESS [0.005s]
[INFO] Flume NG JDBC channel ............................. SUCCESS [0.632s]
[INFO] Flume NG file-based channel ....................... SUCCESS [1.519s]
[INFO] Flume NG Spillable Memory channel ................. SUCCESS [0.400s]
[INFO] Flume NG Node ..................................... SUCCESS [0.467s]
[INFO] Flume NG Embedded Agent ........................... SUCCESS [0.402s]
[INFO] Flume NG HBase Sink ............................... SUCCESS [11.263s]
[INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [0.725s]
[INFO] Flume NG Morphline Solr Sink ...................... SUCCESS [1.892s]
[INFO] Flume Sources ..................................... SUCCESS [0.005s]
[INFO] Flume Scribe Source ............................... SUCCESS [0.396s]
[INFO] Flume JMS Source .................................. SUCCESS [0.583s]
[INFO] Flume Twitter Source .............................. SUCCESS [0.323s]
[INFO] Flume legacy Sources .............................. SUCCESS [0.005s]
[INFO] Flume legacy Avro source .......................... SUCCESS [0.355s]
[INFO] Flume legacy Thrift Source ........................ SUCCESS [1.199s]
[INFO] Flume NG Clients .................................. SUCCESS [0.007s]
[INFO] Flume NG Log4j Appender ........................... SUCCESS [0.308s]
[INFO] Flume NG Tools .................................... SUCCESS [0.331s]
[INFO] Flume NG distribution ............................. SUCCESS [0.131s]
[INFO] Flume NG Integration Tests ........................ SUCCESS [0.276s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.879s
[INFO] Finished at: Sat Jul 26 15:20:20 HKT 2014
[INFO] Final Memory: 94M/2030M
[INFO] ------------------------------------------------------------------------






On 22 Jul, 2014, at 9:45 am, saurabh jain <sauravmanit@gmail.com> wrote:

> Hi Arthur,
> 
> I think in the future release Flume project pom will be updated to use the latest stable
release for Cloudera Kite SDK i.e 0.15.0. See the below email [1] from Joey.
> 
> Version prior to 0.15.0 are not available in maven central repository , to get those
artifacts you need to connect to the cloudera repository [2].
> 
> I am able to build the project using the versions as mentioned above.
> 
>  ----------------- ---------------------------------This is the release email for Cloudera
Kite SDK version 0.15.0. ---------------------------------------------
> 
> [1] We're pleased to announce the release of Kite SDK 0.15.0.
> 
> This release adds a new interface making it easier to open and work
> with Datasets, new introduction and concepts documentation[1.1],
> deprecates the partition API in favor of views, adds a CLI copy
> command, and is the first Kite release available in Maven Central[1.2].
> For more details see the release notes [1.3] and the documentation [1.4].
> 
> 
> [1.1] http://kitesdk.org/docs/current/guide/
> [1.2] http://central.maven.org/maven2/org/kitesdk/
> [1.3] http://kitesdk.org/docs/0.15.0/release_notes.html
> [1.4] http://kitesdk.org/docs/0.15.0
> 
> 
> [2] https://repository.cloudera.com/cloudera/cloudera-repos
> 
> 
> Thanks
> Saurabh
> 
> On Mon, Jul 21, 2014 at 7:37 PM, ac@hsk.hk <ac@hsk.hk> wrote:
> Hi,
> 
> I need to to compile Flume 1.5.0 for Hadoop 2.4.0 and HBase 0.98.3,  the compilation
returns warning and error, 
> 
> I am new to Flume, could anyone share the correct settings for pom.xml files or advise
the proper changes? 
> 
> 1) pom.xml
> 2) flume-ng-sinks/flume-hdfs-sink/pom.xm 
> 3) flume-ng-sinks/flume-ng-hbase-sink/pom.xml
> 4) flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml 
> 
> Many thanks!!
> Arthur
> 
> 
> 
> ===============
> mvn compile
> [INFO] Scanning for projects...
> Downloading: http://people.apache.org/~garyh/mvn/org/apache/apache/9/apache-9.pom
> Downloading: http://repo1.maven.org/maven2/org/apache/apache/9/apache-9.pom
> Downloaded: http://repo1.maven.org/maven2/org/apache/apache/9/apache-9.pom (15 KB at
15.9 KB/sec)
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model for org.apache.flume.flume-ng-sinks:flume-ng-elasticsearch-sink:jar:1.5.0
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique:
org.apache.httpcomponents:httpclient:jar -> duplicate declaration of version (?) @ line
86, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they threaten the stability
of your build.
> [WARNING] 
> [WARNING] For this reason, future Maven versions might no longer support building such
malformed projects.
> [WARNING] 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Build Order:
> 
> …
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Apache Flume ...................................... SUCCESS [1.023s]
> [INFO] Flume NG SDK ...................................... SUCCESS [25.901s]
> [INFO] Flume NG Configuration ............................ SUCCESS [0.688s]
> [INFO] Flume NG Core ..................................... SUCCESS [13.735s]
> [INFO] Flume NG Sinks .................................... SUCCESS [0.006s]
> [INFO] Flume NG HDFS Sink ................................ SUCCESS [33.551s]
> [INFO] Flume NG IRC Sink ................................. SUCCESS [1.325s]
> [INFO] Flume NG Channels ................................. SUCCESS [0.005s]
> [INFO] Flume NG JDBC channel ............................. SUCCESS [16.969s]
> [INFO] Flume NG file-based channel ....................... SUCCESS [4.056s]
> [INFO] Flume NG Spillable Memory channel ................. SUCCESS [0.375s]
> [INFO] Flume NG Node ..................................... SUCCESS [2.525s]
> [INFO] Flume NG Embedded Agent ........................... SUCCESS [0.407s]
> [INFO] Flume NG HBase Sink ............................... SUCCESS [31.639s]
> [INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [1:16.452s]
> [INFO] Flume NG Morphline Solr Sink ...................... FAILURE [28.769s]
> [INFO] Flume Sources ..................................... SKIPPED
> [INFO] Flume Scribe Source ............................... SKIPPED
> [INFO] Flume JMS Source .................................. SKIPPED
> [INFO] Flume Twitter Source .............................. SKIPPED
> [INFO] Flume legacy Sources .............................. SKIPPED
> [INFO] Flume legacy Avro source .......................... SKIPPED
> [INFO] Flume legacy Thrift Source ........................ SKIPPED
> [INFO] Flume NG Clients .................................. SKIPPED
> [INFO] Flume NG Log4j Appender ........................... SKIPPED
> [INFO] Flume NG Tools .................................... SKIPPED
> [INFO] Flume NG distribution ............................. SKIPPED
> [INFO] Flume NG Integration Tests ........................ SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:57.975s
> [INFO] Finished at: Mon Jul 21 21:38:58 HKT 2014
> [INFO] Final Memory: 83M/478M
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: Could not resolve
dependencies for project org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar:1.5.0:
Failure to find org.kitesdk:kite-morphlines-all:pom:0.12.0 in http://maven.oschina.net/content/groups/public/
was cached in the local repository, resolution will not be reattempted until the update interval
of nexus-osc has elapsed or updates are forced -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please read the
following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> [ERROR] 
> [ERROR] After correcting the problems, you can resume the build with the command
> [ERROR]   mvn <goals> -rf :flume-ng-morphline-solr-sink
> 
> 
> 
> 
>  
> 


Mime
View raw message