flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharninder <sharnin...@gmail.com>
Subject Re: Flume interseptor and multiplexing
Date Tue, 16 Feb 2016 08:34:24 GMT
Not sure I understood the question correctly but you might be able to use
the regex interceptor to do what you want.

--
Sharninder


On Tue, Feb 16, 2016 at 1:56 PM, Gonzalo Herreros <gherreros@gmail.com>
wrote:

> I don't think that's possible without writing/reusing custom code.
> You would need an interceptor to add the header following the conditions
> you describe so the multiplexer can do the routing.
>
> Gonzalo
>
> On 15 February 2016 at 21:06, chandra koripella <chandsks@gmail.com>
> wrote:
>
>> Hi,
>>
>>
>> Is there a way implement multiplexing when there is no header information
>> attached to the event (based on first column)?
>>
>>
>> we have customer data coming in in a file. There is no header information
>> attached to the data. First column is even type(highlighted in red).
>>
>>
>>
>> ·        Each even type is bound to different metadata but it is not
>> attached to incoming record.
>>
>> ·        There are 10 different event types.
>>
>> ·        Data needs to route to 10 different hdfs sinks.
>>
>> I am wondering if there is way to accomplish this without writing custom
>> code.  I appreciate any help.
>>
>>
>>
>> 1|20160129010003|_abc|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com
>> <1%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com>
>>
>> 7|20160129010003|_aec|xxxx@aol.com|31-EA10ARI|31-EA10ARE|R|2|xxxx@aol.com
>> <7%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxx@aol.com%7C31-EA10ARI%7C31-EA10ARE%7CR%7C2%7Cxxxx@aol.com>
>> |test|test|test
>>
>> 1
>> 0|20160129010003|_asaasd|xxxx@aol.com|31-EA10ARI|31-EA10ARE|R|2|xxxx@aol.com|www.yahoo.com
>> <10%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxx@aol.com%7C31-EA10ARI%7C31-EA10ARE%7CR%7C2%7Cxxxx@aol.com%7Cwww.yahoo.com>
>> |test
>>
>> 1
>> |20160129010003|_rtert|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com
>> <1%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com>
>>
>> 4
>> |20160129010003|_w3423|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com|xx|xx|
>> <4%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com%7Cxx%7Cxx%7C>
>>
>>
>>
>> Thanks,
>>
>> Chandra
>>
>>
>>
>


-- 
--
Sharninder

Mime
View raw message