flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zijad Purkovic <zijadpurko...@gmail.com>
Subject Re: Need help in writing to HDFS
Date Sun, 22 Jan 2012 10:12:06 GMT
Try looking at flume logs on agent and collector. Also, what is your
Hadoop version? Cause, if you use some other version than cdh3, you
will need to add hadoop-core-"version".jar to your flume lib directory
to be able to write to hdfs.

On Sun, Jan 22, 2012 at 4:26 AM, DIPESH KUMAR SINGH
<dipesh.tech@gmail.com> wrote:
> Thanks Zijad.
>
> I tried this :
>
> localhost: text("/etc/services") | agentSink("localhost",35853);
> localhost: collectorSource(35853) |
> collectorSink("hdfs://localhost:8020/home","dservices11.txt");
>
> But still, i cannot find my file written in HDFS and web interface shows the
> command state "SUCCEEDED".
> Since Web interface reflecting the state "SUCCEEDED" i am unable to figure
> out where things went wrong.
>
> --
> Dipesh
>
>
> On Sun, Jan 22, 2012 at 5:16 AM, Zijad Purkovic <zijadpurkovic@gmail.com>
> wrote:
>>
>> instead of dfs sink try using:
>> collectorSink("hdfs://localhost:8020/home","some_filename")
>>
>> On Sat, Jan 21, 2012 at 6:23 PM, DIPESH KUMAR SINGH
>> <dipesh.tech@gmail.com> wrote:
>> > Hi All,
>> >
>> > I am new to flume. I tried writing to HDFS through following command
>> > from
>> > command line and was able to successfully write in to HDFS:
>> >
>> >  flume node_nowatch -1 -s -n dump -c 'dump: text("/etc/services") |
>> > dfs("hdfs://localhost:8020/home/dservices.txt");'
>> >
>> > To try out the collector, I tried the following commands from
>> > multiconfig
>> > area of web interface :
>> >
>> > localhost: text("/etc/services") | agentSink("localhost",35853);
>> > localhost: collectorSource(35853) |
>> > dfs("hdfs://localhost:8020/home/dservices11.txt");
>> >
>> >
>> > I might be missing something basic in between.
>> > But, I am unable to write into HDFS, though the web interface displays
>> > the
>> > command has been "SUCCEEDED".
>> >
>> >
>> >
>> > Please guide, I wish to move data [ like plain text, PDF ] from a node
>> > to
>> > HDFS.
>> >
>> > Thanks!
>> >
>> > --
>> > Dipesh Kr. Singh
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Zijad Purković
>> Dobrovoljnih davalaca krvi 3/19, Zavidovići
>> 061/ 690 - 241
>
>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>



-- 
Zijad Purković
Dobrovoljnih davalaca krvi 3/19, Zavidovići
061/ 690 - 241

Mime
View raw message