phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bulvik, Noam" <Noam.Bul...@teoco.com>
Subject RE: replace CsvToKeyValueMapper with my implementation
Date Thu, 29 Oct 2015 19:56:05 GMT
Thanks for the quick response ,  I have created https://issues.apache.org/jira/browse/PHOENIX-2360
for this request

can you also check the second part - to change the modifiers or add getters for conn and tablename
members of  CsvToKeyValueMapper  to be able to get their value rather to duplicate their init
code.

Thanks

-----Original Message-----
From: Gabriel Reid [mailto:gabriel.reid@gmail.com]
Sent: Thursday, October 29, 2015 5:10 PM
To: user@phoenix.apache.org
Subject: Re: replace CsvToKeyValueMapper with my implementation

Hi Noam,

That specific piece of code in CsvBulkLoadTool that you referred to allows packaging the CsvBulkLoadTool
within a different job jar file, but won't allow setting a different mapper class. The actual
setting of the mapper class is done further down in the submitJob method, specifically the
following piece:

   job.setMapperClass(CsvToKeyValueMapper.class);

There isn't currently a way to load a custom mapper in the CsvBulkLoadTool, so the only (current)
option is to create a fully new custom implementation of the bulk load tool (probably copying
or reusing most of the existing tool). However, I can certainly imagine this being a useful
feature to have in some situations.

Could you log this request in jira? It would also be really good to have some more detail
on your specific use case. And even better is a patch that implements it :-)

- Gabriel


On Thu, Oct 29, 2015 at 3:22 PM, Bulvik, Noam <Noam.Bulvik@teoco.com> wrote:
> Hi,
>
>
>
> We have private logic to be executed when parsing each line before it
> is uploaded to phoenix. I saw the following in the code of the
> CsvBulkLoadTool
>
> // Allow overriding the job jar setting by using a -D system property
> at startup
>
> if (job.getJar() == null)
>
>  {
>
>
> job.setJarByClass(CsvToKeyValueMapper.class);
>
>                  }
>
>
>
> Assuming I have the implementation for MyKeyValueMapper how can I make
> sure it will be loaded instead of standard one ?
>
>
>
> Also in CsvToKeyValueMapper class there are some private members like
>
> ·         private PhoenixConnection conn;
>
> ·         private byte[] tableName;
>
>
>
> can you add option to access these member or make them protected so we
> will be able to use them in the class we create that extends
> CsvToKeyValueMapper and not to duplicate them and the code that init
> them
>
>
>
> we are using  phoenix 4.5.2 over CDH
>
>
>
> thanks
>
> Noam
>
>
>
> Noam Bulvik
>
> R&D Manager
>
>
>
> TEOCO CORPORATION
>
> c: +972 54 5507984
>
> p: +972 3 9269145
>
> Noam.Bulvik@teoco.com
>
> www.teoco.com
>
>
>
>
> ________________________________
>
> PRIVILEGED AND CONFIDENTIAL
> PLEASE NOTE: The information contained in this message is privileged
> and confidential, and is intended only for the use of the individual
> to whom it is addressed and others who have been specifically authorized to receive it.
> If you are not the intended recipient, you are hereby notified that
> any dissemination, distribution or copying of this communication is
> strictly prohibited. If you have received this communication in error,
> or if any problems occur with transmission, please contact sender. Thank you.

________________________________

PRIVILEGED AND CONFIDENTIAL
PLEASE NOTE: The information contained in this message is privileged and confidential, and
is intended only for the use of the individual to whom it is addressed and others who have
been specifically authorized to receive it. If you are not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this communication is strictly
prohibited. If you have received this communication in error, or if any problems occur with
transmission, please contact sender. Thank you.
Mime
View raw message