flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Ortiz <konstt2...@gmail.com>
Subject Reading with Spark from KafkaChannel
Date Fri, 06 Nov 2015 08:38:55 GMT

I'm using the KafkaChannel, I have a doubt about the documentation.

Expecting Avro datums with FlumeEvent schema in the channel. This
should be true if Flume source is writing to the channel And false if
other producers are writing into the topic that the channel is using
Flume source messages to Kafka can be parsed outside of Flume by using
org.apache.flume.source.avro.AvroFlumeEvent provided by the
flume-ng-sdk artifact

In my PoC, I'm writing just with Kafka to my topic but I want to read
from the KafkaChannel with Spark. I have configured parseAsFlumeEvent
as true because just Flume writes to this topic.
What should I do to read the "events" from Kafka? DO I have to use
org.apache.flume.source.avro.AvroFlumeEvent??. If I set the parameter
as false, how are the events inserted?

View raw message