flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Shreedharan" <hshreedha...@cloudera.com>
Subject Re: Writing file to HDFS from Java
Date Tue, 11 Nov 2014 18:43:36 GMT
Are you talking about the embedded agent? Embedded agent is meant to only write data to another
Flume agent and does not support the HDFS sink. Only Avro Sink is supported.


On Tue, Nov 11, 2014 at 10:10 AM, Tinte garcia, Miguel Angel
<miguel.tinte@atos.net> wrote:

> Hi,
> I am working on a Java program that will write a file into HDFS using Flume as event
manager. To do so, I have created a Flume agent from Java:
>         …
>         properties.put("sinks", "HDFSSink");
>         // HDFS Sink
>         properties.put("HDFSSink.hostname", "localhost");
>         properties.put("HDFSSink.port", port);
>         properties.put("HDFSSink.type", "hdfs");
>         properties.put("HDFSSink.channel", "MemChannel");
>         properties.put("HDFSSink.hdfs.path", "hdfs://localhost:9000/home/hdfs/files");
> …
> Besides this, I am thinking on writing a method to specifically send the file to HDFS.
How can I do this through Flume?
> Thanks in advance
> This e-mail and the documents attached are confidential and intended solely for the addressee;
it may also be privileged. If you receive this e-mail in error, please notify the sender immediately
and destroy it.
> As its integrity cannot be secured on the Internet, the Atos group liability cannot be
triggered for the message content. Although the sender endeavors to maintain a computer virus-free
network, the sender does not warrant that this transmission is virus-free and will not be
liable for any damages resulting from any virus transmitted.
> Este mensaje y los ficheros adjuntos pueden contener información confidencial destinada
solamente a la(s) persona(s) mencionadas anteriormente y pueden estar protegidos por secreto
> Si usted recibe este correo electrónico por error, gracias por informar inmediatamente
al remitente y destruir el mensaje.
> Al no estar asegurada la integridad de este mensaje sobre la red, Atos no se hace responsable
por su contenido. Su contenido no constituye ningún compromiso para el grupo Atos, salvo
ratificación escrita por ambas partes.
> Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor no puede
garantizar nada al respecto y no será responsable de cualesquiera daños que puedan resultar
de una transmisión de virus.
View raw message