flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emile Kao" <emile...@gmx.net>
Subject Flume and HDFS integration
Date Thu, 29 Nov 2012 17:14:09 GMT
Dear support,
I would like to ask you some questions about issues I am facing trying to implement Flume
in a customer environment.

I am using following release of Flume: apache-flume-1.4.0-SNAPSHOT-bin

Here are my questions:  

Question no.1
I have define the following command in flume.conf:
agent1.sources.tail.command = tail -F /opt/apache2/logs/access_log

Now the resulting files (FlumeData.xxxxxxxxxxxxx) are not readable. At least for human being.
I guess they are in binary format. Now , my question is: is there a way to make or convert
those files in an ascii / readable format for human being? 

Question no.2
I am trying to use the tailDir command without success. Here is the setting in flume,conf:
agent1.sources.tail.command = tailDir("/opt/apache2/logs/")

--> here is what I am getting as result...Can you help?

2012-11-29 16:48:17,548 (pool-6-thread-1) [ERROR - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:284)]
Failed while running command: tailDir("/opt/apache2/logs/")
java.io.IOException: Cannot run program "tailDir("/opt/apache2/logs/")": java.io.IOException:
error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
    at org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:259)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
    at java.lang.ProcessImpl.start(ProcessImpl.java:65)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
    ... 7 more
2012-11-29 16:48:17,549 (pool-6-thread-1) [INFO - org.apache.flume.source.ExecSource$ExecRunnable.run(ExecSource.java:307)]
Command [tailDir("/opt/apache2/logs/")] exited with -1073741824

Many Thank!

View raw message