phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: replace CsvToKeyValueMapper with my implementation
Date Thu, 29 Oct 2015 15:10:02 GMT
Hi Noam,

That specific piece of code in CsvBulkLoadTool that you referred to
allows packaging the CsvBulkLoadTool within a different job jar file,
but won't allow setting a different mapper class. The actual setting
of the mapper class is done further down in the submitJob method,
specifically the following piece:

   job.setMapperClass(CsvToKeyValueMapper.class);

There isn't currently a way to load a custom mapper in the
CsvBulkLoadTool, so the only (current) option is to create a fully new
custom implementation of the bulk load tool (probably copying or
reusing most of the existing tool). However, I can certainly imagine
this being a useful feature to have in some situations.

Could you log this request in jira? It would also be really good to
have some more detail on your specific use case. And even better is a
patch that implements it :-)

- Gabriel


On Thu, Oct 29, 2015 at 3:22 PM, Bulvik, Noam <Noam.Bulvik@teoco.com> wrote:
> Hi,
>
>
>
> We have private logic to be executed when parsing each line before it is
> uploaded to phoenix. I saw the following in the code of the CsvBulkLoadTool
>
> // Allow overriding the job jar setting by using a -D system property at
> startup
>
> if (job.getJar() == null)
>
>  {
>
>
> job.setJarByClass(CsvToKeyValueMapper.class);
>
>                  }
>
>
>
> Assuming I have the implementation for MyKeyValueMapper how can I make sure
> it will be loaded instead of standard one ?
>
>
>
> Also in CsvToKeyValueMapper class there are some private members like
>
> ·         private PhoenixConnection conn;
>
> ·         private byte[] tableName;
>
>
>
> can you add option to access these member or make them protected so we will
> be able to use them in the class we create that extends CsvToKeyValueMapper
> and not to duplicate them and the code that init them
>
>
>
> we are using  phoenix 4.5.2 over CDH
>
>
>
> thanks
>
> Noam
>
>
>
> Noam Bulvik
>
> R&D Manager
>
>
>
> TEOCO CORPORATION
>
> c: +972 54 5507984
>
> p: +972 3 9269145
>
> Noam.Bulvik@teoco.com
>
> www.teoco.com
>
>
>
>
> ________________________________
>
> PRIVILEGED AND CONFIDENTIAL
> PLEASE NOTE: The information contained in this message is privileged and
> confidential, and is intended only for the use of the individual to whom it
> is addressed and others who have been specifically authorized to receive it.
> If you are not the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication is strictly
> prohibited. If you have received this communication in error, or if any
> problems occur with transmission, please contact sender. Thank you.

Mime
View raw message