phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: Line separator option in Bulk loader
Date Thu, 12 Feb 2015 07:40:51 GMT
Hi Siva,

Handling multi-line records with the Bulk CSV Loader (i.e.
MapReduce-based loader) definitely won't support records split over
multiple input lines. It could be that loading via PSQL (as described
on http://phoenix.apache.org/bulk_dataload.html) will allow multi-line
records, as this might be supported by the underlying CSV parsing
library (commons-csv), although I'm not sure. In any case, I can't
really give you any advice on how to make it work there if it isn't
working right now.

I assume this also won't work in HBase's ImportTsv.

- Gabriel


On Thu, Feb 5, 2015 at 10:28 PM, Siva <sbhavanari@gmail.com> wrote:
> We have table contains a NOTE column, this column contains lines of text
> separated by new lines. When I export the data from .csv through bulkloader,
> Phoenix is failing with error and Hbase terminates the text till encounters
> the new line and assumes rest of NOTE as new record.
>
>
>
> Is there a way to specify new line separator in Hbase or Phoenix bulk load?
>
>
>
> With phoenix:
>
>
>
> HADOOP_CLASSPATH=/usr/hdp/2.2.0.0-2041/hbase/lib/hbase-protocol.jar:/usr/hdp/2.2.0.0-2041/hbase/conf
> hadoop jar
> /usr/hdp/2.2.0.0-2041/phoenix/phoenix-4.2.0.2.2.0.0-2041-client.jar
> org.apache.phoenix.mapreduce.CsvBulkLoadTool --table test_leadwarehouse
> --input /user/sbhavanari/test_leadwarehouse.csv --zookeeper <zookeeper
> Ip>:2181:/hbase
>
>
>
> With hbase importtsv:
>
>
>
> base org.apache.hadoop.hbase.mapreduce.ImportTsv '-Dimporttsv.separator=,'
> -Dimporttsv.columns=<col_list> test_leadwarehouse
> /user/data/test_leadwarehouse.csv

Mime
View raw message