Did you try specifying encoding for your source, for example like this:
a1.sources.r1.inputCharset = ISO8859-1 << whatever charset you need>>
?

Marina


From: "Vishwakarma, Chhaya" <Chhaya.Vishwakarma@Teradata.com>
To: "user@flume.apache.org" <user@flume.apache.org>
Sent: Tuesday, May 12, 2015 4:20 AM
Subject: Unicode data handling with flume

Hi all,
I'm trying to put a CSV file into HDFS using flume, file contains some unicode characters also.
Once the file is there in HDFS I tried to view the content, but unable to see the records properly.
File content
Name    age  sal    msg
 
Abc     21  1200    Lukè éxample àpple
 
Xyz     23  1400    er stîget ûf mit grôzer
Output in console
I did hdfs dfs -get /flume/events/csv/events.1234567
Below is the output
Name,age,sal,msg
 
Abc,21,1200,Luk��xample��pple
 
Xyz,23,1400,er stget f mit grzer
Does flume supports Unicode characters? If not how it can be handled?
Flume version is 1.4.0