flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: No columns to insert for #1 item
Date Wed, 30 Sep 2015 09:39:19 GMT
The column family is a required field for the sink. You should have an
error in the log explaining that (unless you are running a very old Flume
version or something).
You need something like this:

agent.sinks.hbaseSink.columnFamily=cf

Regards,
Gonzalo

On 30 September 2015 at 09:21, Tinte garcia, Miguel Angel <
miguel.tinte@atos.net> wrote:

> Hi Gonzalo,
>
> Thanks for the answer. My flume agent configuration is the following:
>
> agent.sources=avroSource
>
> agent.channels=channel1
>
> agent.sinks=hbaseSink
>
>
>
> agent.sources.avroSource.type=avro
>
> agent.sources.avroSource.channels=channel1
>
> agent.sources.avroSource.bind=0.0.0.0
>
> agent.sources.avroSource.port=61621
>
> agent.sources.avroSource.interceptors=i1
>
> agent.sources.avroSource.interceptors.i1.type=timestamp
>
> #agent.sources.avroSource.interceptors.i1.payloadColumn=
>
>
>
> agent.channels.channel1.type=memory
>
> agent.channels.channel1.capacity=1000000
>
> agent.channels.channel1.transactionCapacity=100000
>
> #agent.channels.channel1.byteCapacityBufferPercentage=20
>
> agent.channels.channel1.byteCapacity=134217728
>
>
>
> agent.sinks.hbaseSink.type=hbase
>
> agent.sinks.hbaseSink.channel=channel1
>
> agent.sinks.hbaseSink.channel.capacity=1000000
>
> agent.sinks.hbaseSink.channel.transactionCapacity=100000
>
> agent.sinks.hbaseSink.table=Test_Simulations_Data
>
>
>
>
>
> PayloadColumn is commented. Should I uncomment it and fill with any
> specific value?
>
>
>
> Thanks
>
>
>
> *From:* Gonzalo Herreros [mailto:gherreros@gmail.com]
> *Sent:* Wednesday, September 30, 2015 9:56 AM
> *To:* user@flume.apache.org
> *Subject:* Re: No columns to insert for #1 item
>
>
>
> My guess is that the HBase serializer is not filling the payload column
> correctly. i.e. an empty value in the property "payloadColumn"
>
> Can you share the sink configuration?
>
>
>
> Regards,
>
> Gonzalo
>
>
>
> On 29 September 2015 at 23:40, Tinte garcia, Miguel Angel <
> miguel.tinte@atos.net> wrote:
>
> Hi,
>
> I am trying write data into HBase sending it through Flume agents, but
> some error appears in the during transaction:
>
>
>
> 2015-09-30 00:34:55,745 (SinkRunner-PollingRunner-DefaultSinkProcessor)
> [ERROR - org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:353)]
> Failed to commit transaction.Transaction rolled back.
>
> java.lang.IllegalArgumentException: No columns to insert for #1 item
>
>         at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:468)
>
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:2332)
>
>         at
> org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:842)
>
>         at
> org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:857)
>
>         at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:832)
>
>         at org.apache.flume.sink.hbase.HBaseSink$3.run(HBaseSink.java:386)
>
>         at org.apache.flume.sink.hbase.HBaseSink$3.run(HBaseSink.java:374)
>
>         at
> org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
>
>         at
> org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:374)
>
>         at
> org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
>
>         at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>
>         at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>
>
>
> Does anybody know what it could be caused for?
>
>
>
> Thanks in advance
>
>
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive this
> e-mail in error, please notify the sender immediately and destroy it.
> As its integrity cannot be secured on the Internet, the Atos group
> liability cannot be triggered for the message content. Although the sender
> endeavors to maintain a computer virus-free network, the sender does not
> warrant that this transmission is virus-free and will not be liable for any
> damages resulting from any virus transmitted.
>
> Este mensaje y los ficheros adjuntos pueden contener información
> confidencial destinada solamente a la(s) persona(s) mencionadas
> anteriormente y pueden estar protegidos por secreto profesional.
> Si usted recibe este correo electrónico por error, gracias por informar
> inmediatamente al remitente y destruir el mensaje.
> Al no estar asegurada la integridad de este mensaje sobre la red, Atos no
> se hace responsable por su contenido. Su contenido no constituye ningún
> compromiso para el grupo Atos, salvo ratificación escrita por ambas partes.
> Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor
> no puede garantizar nada al respecto y no será responsable de cualesquiera
> daños que puedan resultar de una transmisión de virus.
>
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive this
> e-mail in error, please notify the sender immediately and destroy it.
> As its integrity cannot be secured on the Internet, the Atos group
> liability cannot be triggered for the message content. Although the sender
> endeavors to maintain a computer virus-free network, the sender does not
> warrant that this transmission is virus-free and will not be liable for any
> damages resulting from any virus transmitted.
>
> Este mensaje y los ficheros adjuntos pueden contener información
> confidencial destinada solamente a la(s) persona(s) mencionadas
> anteriormente y pueden estar protegidos por secreto profesional.
> Si usted recibe este correo electrónico por error, gracias por informar
> inmediatamente al remitente y destruir el mensaje.
> Al no estar asegurada la integridad de este mensaje sobre la red, Atos no
> se hace responsable por su contenido. Su contenido no constituye ningún
> compromiso para el grupo Atos, salvo ratificación escrita por ambas partes.
> Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor
> no puede garantizar nada al respecto y no será responsable de cualesquiera
> daños que puedan resultar de una transmisión de virus.
>

Mime
View raw message