flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ronald Van de Kuil" <ronald.van.de.k...@nl.ibm.com>
Subject Re: Twitter Source problem
Date Tue, 10 May 2016 06:10:37 GMT
You would have to create a twitter app (account) to get your key, token and so on.

> Op 10 mei 2016 om 05:04 heeft chandra sekar <thittaisekar@hotmail.com> het volgende
geschreven:
> 
> Hi  I am using Twitter source to collect  the data from twitter. Suddenly the process
throw an error and there is no data from twitter feed. 
> 
> Error: 
> 
> 404:The URI requested is invalid or the resource requested, such as a user, does not
exist.
> Unknown URL. See Twitter Streaming API documentation at http://dev.twitter.com/pages/streaming_api
> 
> Configuration:
> 
> TwitterAgent.sources = Twitter
> TwitterAgent.channels = MemChannel
> TwitterAgent.sinks = HDFS
> 
> TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
> # TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource
> TwitterAgent.sources.Twitter.channels = MemChannel
> TwitterAgent.sources.Twitter.consumerKey = 
> TwitterAgent.sources.Twitter.consumerSecret = 
> TwitterAgent.sources.Twitter.accessToken = 
> TwitterAgent.sources.Twitter.accessTokenSecret = 
> 
> TwitterAgent.sources.Twitter.keywords = data analytics, data science , hadoop , big data

> 
> TwitterAgent.sinks.HDFS.channel = MemChannel
> TwitterAgent.sinks.HDFS.type = hdfs
> TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:8020/data_analytics
> TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
> TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
> TwitterAgent.sinks.HDFS.hdfs.batchSize = 10000
> TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
> TwitterAgent.sinks.HDFS.hdfs.rollCount = 100000
> 
> TwitterAgent.channels.MemChannel.type = memory
> TwitterAgent.channels.MemChannel.capacity = 500000
> TwitterAgent.channels.MemChannel.transactionCapacity = 3000
> 
> 
> Thanks & Regards 
> Chandrasekar
Tenzij hierboven anders aangegeven: / Unless stated otherwise above:
IBM Nederland B.V.
Gevestigd te Amsterdam
Inschrijving Handelsregister Amsterdam Nr. 33054214

Mime
View raw message