phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devin Pinkston <dpinks...@technicacorp.com>
Subject RE: Import Delimiter
Date Thu, 06 Feb 2014 13:41:27 GMT
James,

Interesting thanks for the info.  So if I were to import data containing pipe delimiters,
I would have to use the non map-reduce bulk loader.  Are you referencing that sqlline would
have to be used?

Sorry I am trying to figure out how I can import these large flat files this way.

Thank you.

From: James Taylor [mailto:jamestaylor@apache.org]
Sent: Wednesday, February 05, 2014 8:25 PM
To: user@phoenix.incubator.apache.org
Subject: Re: Import Delimiter

You're right. It was added to the non map-reduce bulk loader. This is the loader that loads
local CSV files through the bin/psql.sh script. There's a -d option that was added in this
pull request[1]. It would be nice to add this same functionality to our csv map-reduce bulk
loader too if anyone is interested.
Thanks,
James

[1] https://github.com/forcedotcom/phoenix/pull/514
On Wed, Feb 5, 2014 at 9:35 AM, Nick Dimiduk <ndimiduk@gmail.com<mailto:ndimiduk@gmail.com>>
wrote:
Hi James,

I'm looking through the bulkload job, and it looks to me light this isn't configurable at
the moment. Have a look at https://github.com/apache/incubator-phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/MapReduceJob.java#L136

Is there something I'm missing? Perhaps I'm looking in the wrong place?

Thanks,
Nick

On Wed, Feb 5, 2014 at 10:16 AM, Devin Pinkston <dpinkston@technicacorp.com<mailto:dpinkston@technicacorp.com>>
wrote:
James,

Thanks for the quick response.  Do you know what the argument or command is to pass in?

For instance ./csv-bulk-loader.sh -delimiter '|'

Thanks

From: James Taylor [mailto:jamestaylor@apache.org<mailto:jamestaylor@apache.org>]
Sent: Wednesday, February 05, 2014 11:51 AM
To: user@phoenix.incubator.apache.org<mailto:user@phoenix.incubator.apache.org>
Subject: Re: Import Delimiter

Hello,
The CSV map-reduce based bulk loader supports custom delimiters. Might need to be doc-ed,
though.
Thanks,
James

On Wednesday, February 5, 2014, Devin Pinkston <dpinkston@technicacorp.com<mailto:dpinkston@technicacorp.com>>
wrote:

Hello,



I am trying to import data into HBASE however I have '|' or pipe delimiters=  in my file instead
of commas.  I don't see a way to pass in a different separator/delimiter with the jar.  What
would be the best way to import data = like this?



Thanks


The information contained in this transmission may contain privileged and confidential information.
It is intended only for the use of the person(s) named above.
If you are not the intended recipient, you are hereby notified that any review, dissemination,
distribution or duplication of this communication is strictly prohibited.
If you are not the intended recipient, please contact the sender by reply e-mail and destroy
all copies of the original message.
Technica Corporation does not represent this e-mail to be free from any virus, fault or defect
and it is therefore the responsibility of the recipient to first scan it for viruses, faults
and defects.
To reply to our e-mail administrator directly, please send an e-mail to postmaster@technicacorp.com<mailto:postmaster@technicacorp.com>.
Thank you.

The information contained in this transmission may contain privileged and confidential information.
It is intended only for the use of the person(s) named above.
If you are not the intended recipient, you are hereby notified that any review, dissemination,
distribution or duplication of this communication is strictly prohibited.
If you are not the intended recipient, please contact the sender by reply e-mail and destroy
all copies of the original message.
Technica Corporation does not represent this e-mail to be free from any virus, fault or defect
and it is therefore the responsibility of the recipient to first scan it for viruses, faults
and defects.
To reply to our e-mail administrator directly, please send an e-mail to postmaster@technicacorp.com<mailto:postmaster@technicacorp.com>.
Thank you.


The information contained in this transmission may contain privileged and confidential information.

It is intended only for the use of the person(s) named above. 
If you are not the intended recipient, you are hereby notified that any review, dissemination,
distribution or duplication of this communication is strictly prohibited. 
If you are not the intended recipient, please contact the sender by reply e-mail and destroy
all copies of the original message. 
Technica Corporation does not represent this e-mail to be free from any virus, fault or defect
and it is therefore the responsibility of the recipient to first scan it for viruses, faults
and defects. 
To reply to our e-mail administrator directly, please send an e-mail to postmaster@technicacorp.com.
Thank you.

Mime
View raw message