Hi

 

You re right. I also find some jira for that problem. (https://issues.apache.org/jira/browse/FLUME-2392) The problem is solved in 1.6.0 version.

 

Unfortunately, we are using oracle bda with cdh 5.4 ,therefore it is not easy to upgrade any service (flume, hive etc.) . We depended the oracle for upgrading.

 

How can we solve that problem without upgrading the flume? It must be some other solutions.

 

Thanks..

 

From: Gonzalo Herreros [mailto:gherreros@gmail.com]
Sent: Wednesday, May 18, 2016 10:48 AM
To: user
Subject: Re: Flume MorphlineSolrSink

 

Flume 1.5.0 is pretty old,

Why don't you use version 1.6.0 included in CDH, that will ensure the library compatibility.

 

On 18 May 2016 at 08:43, Baris Akgun (Garanti Teknoloji) <BarisAkgu@garanti.com.tr> wrote:

Hi,

 

I am trying to make real time indexing with using flume 1.5.0 and morphline solr sink. I am using packed base solr in my CDH.

 

 

tier1.sinks = solrSink

tier1.channels = channel1

#

tier1.channels.channel1.type=memory

tier1.channels.channel1.capacity=1000000

#

tier1.sinks.solrSink.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink

tier1.sinks.solrSink.batchSize = 100

tier1.sinks.solrSink.batchDurationMillis = 10000

tier1.sinks.solrSink.morphlineFile = morphlines.conf

tier1.sinks.solrSink.morphlineId = morphline1

#

tier1.sinks.solrSink.channel = channel1

 

 

The flume throw error in sink step.  The error is given below. What is your ideas to solve the problem?

 

Thanks.

 

 

ERROR

Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@748016d1 counterGroup:{ name:null counters:{} } } - Exception follows.

java.lang.NoSuchMethodError: org.apache.lucene.util.fst.FST$BytesReader.skipBytes(J)V

         at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1117)

         at org.apache.lucene.util.fst.FST.readFirstRealTargetArc(FST.java:1030)

         at org.apache.lucene.util.fst.FST.readRootArcs(FST.java:487)

         at org.apache.lucene.util.fst.FST.cacheRootArcs(FST.java:475)

         at org.apache.lucene.util.fst.FST.finish(FST.java:458)

         at org.apache.lucene.util.fst.Builder.finish(Builder.java:464)

         at org.apache.lucene.analysis.synonym.SynonymMap$Builder.build(SynonymMap.java:284)

         at org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.loadSynonyms(FSTSynonymFilterFactory.java:145)

         at org.apache.lucene.analysis.synonym.FSTSynonymFilterFactory.inform(FSTSynonymFilterFactory.java:112)

         at org.apache.lucene.analysis.synonym.SynonymFilterFactory.inform(SynonymFilterFactory.java:89)

         at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:675)

         at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:167)

         at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)

         at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)

         at org.kitesdk.morphline.solr.SolrLocator.getIndexSchema(SolrLocator.java:181)

         at org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.<init>(SanitizeUnknownSolrFieldsBuilder.java:70)

         at org.kitesdk.morphline.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:52)

         at org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)

         at org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)

         at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:46)

         at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

         at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:126)

         at org.kitesdk.morphline.base.Compiler.compile(Compiler.java:55)

         at org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)

         at org.apache.flume.sink.solr.morphline.MorphlineSink.start(MorphlineSink.java:97)

         at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)

         at org.apache.flume.SinkRunner.start(SinkRunner.java:79)

         at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)

         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

         at java.lang.Thread.run(Thread.java:745)

 

 

 

 

 

Barış Akgün
Analitik Veri Ambarı ve Büyük Veri Yönetimi
Uzman

Tel

:

Dahili

:

Faks

:

 

Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.

This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system.

 

Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.

This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system.