flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Flume/HDFS Encoding
Date Fri, 14 Dec 2012 21:02:43 GMT
Did you fix the parameter naming problem I described in the earlier message?

On Fri, Dec 14, 2012 at 2:59 PM, Cormier, Christopher
<christopher.cormier@teamaol.com> wrote:
> Thanks Brock,
> When I run it as a DataStream I still get some weird characters between records.
>
> [DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬                  ;)
>  [DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
>                                          ;)
>                                            Î[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
>                                                                                     
                                                                                         
                            ;0
>                                                                                     
                                                                                         
                              ½[DATA_HERE]ÿÿÿÿ×ùÎ0ÆÜ9Ig::¬
> ;0
> :[DATA_HERE]
>
> I was hoping to avoid the ÿÿÿÿ and spaces (I'm assuming they're characters that are
encoded such that -cat won't show them).
>
> Any thoughts?
>
> Thanks again,
>
> Chris
>
> -----Original Message-----
> From: Brock Noland [mailto:brock@cloudera.com]
> Sent: Friday, December 14, 2012 3:52 PM
> To: user@flume.apache.org
> Subject: Re: Flume/HDFS Encoding
>
> Hi,
>
> On Fri, Dec 14, 2012 at 2:48 PM, Cormier, Christopher <christopher.cormier@teamaol.com>
wrote:
>> SEQ!org.apache.hadoop.io.LongWritableorg.apache.hadoop.io.TextY]
>> õpµ^R÷ﳬÕ
>>
>
> This is a SequenceFile.
>
>>
>> requestToHDFS.sinks.HDFS.hdfs.file.Type = DataStream
>>
>> # also tried...
>>
>> #requestToHDFS.sinks.HDFS.hdfs.file.Type = SequenceFile
>>
>
> The parameter is hdfs.fileType. See here:
>
> http://flume.apache.org/FlumeUserGuide.html#hdfs-sink
>
> It sounds like you want a text file so you should use DataStream.
>
> Brock



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message