flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Yates <sya...@stevendyates.com>
Subject Re: Analysis of Data
Date Thu, 07 Feb 2013 23:04:54 GMT
The only missing link within the Flume architecture I see in this
conversation is the actual channel's and brokers themselves which
orchestrate this lovely undertaking of data collection. One opportunity I do
see (and I may be wrong) is for the data to offloaded into a system such as
Apache Mahout  before being sent to the sink. Perhaps the concept of a
ChannelAdapter of sorts? I.e Mahout Adapter ? Just thinking out loud and it
may be well out of the question.


From:  Nitin Pawar <nitinpawar432@gmail.com>
Reply-To:  <user@flume.apache.org>
Date:  Thu, 7 Feb 2013 16:52:12 +0530
To:  <user@flume.apache.org>
Subject:  Re: Analysis of Data

1) Flume is isolated distributed system in the sense one agent does not idea
about any other agent
2) Flume in the sense when needs to collect data from multiple references
and work across different data sets, it may not have the entire data set
3) let us assume we have required data on agents for processing it in
batches, do we really want to pressurize a live production server for data
processing which can be done by systems like storm or hadoop or other

these are my ideas .. i can be totally wrong but just from systems point of
view it looks good option to keep data acquisition separate from data
processing and then storing the processed data for further data serving

On Thu, Feb 7, 2013 at 4:29 PM, Mike Percy <mpercy@apache.org> wrote:
> Let's take this conversation further. What is missing?
> On Thu, Feb 7, 2013 at 2:39 AM, Inder Pall <inder.pall@gmail.com> wrote:
>> flume is a platform to get events to the right sink (HDFS, local-file, ....)
>> analytics is not something which falls in it's territory
>> - Inder
>> On Thu, Feb 7, 2013 at 3:22 PM, Surindhar <surindhar@gmail.com> wrote:
>>> Hi,
>>> Does Flume supports Analysis of Data?
>>> Br,
>> -- 
>> - Inder
>> "You are average of the 5 people you spend the most time with"

Nitin Pawar

View raw message