1) It's a sequence file, you can change it a text file if you want. See FileType here http://flume.apache.org/FlumeUserGuide.html#hdfs-sink

2) The "tailDir(" syntax is Flume 0.9 and is no long used. The first example has the correct syntax.


On Thu, Nov 29, 2012 at 11:14 AM, Emile Kao <emilekao@gmx.net> wrote:
Dear support,
I would like to ask you some questions about issues I am facing trying to implement Flume in a customer environment.

I am using following release of Flume: apache-flume-1.4.0-SNAPSHOT-bin

Here are my questions:

Question no.1
I have define the following command in flume.conf:
agent1.sources.tail.command = tail -F /opt/apache2/logs/access_log

Now the resulting files (FlumeData.xxxxxxxxxxxxx) are not readable. At least for human being. I guess they are in binary format. Now , my question is: is there a way to make or convert those files in an ascii / readable format for human being?

Question no.2
I am trying to use the tailDir command without success. Here is the setting in flume,conf:
agent1.sources.tail.command = tailDir("/opt/apache2/logs/")

--> here is what I am getting as result...Can you help?

2012-11-29 16:48:17,548 (pool-6-thread-1) [ERROR - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:284)] Failed while running command: tailDir("/opt/apache2/logs/")
java.io.IOException: Cannot run program "tailDir("/opt/apache2/logs/")": java.io.IOException: error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:259)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 7 more
2012-11-29 16:48:17,549 (pool-6-thread-1) [INFO - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:307)] Command [tailDir("/opt/apache2/logs/")] exited with -1073741824

Many Thank!

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/