phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Phoenix Query Server Avatica Upsert
Date Fri, 04 Mar 2016 16:12:42 GMT
Yeah, I don't think the inclusion of Python code should be viewed as a 
barrier to inclusion (maybe just a hurdle). I've seen other projects 
(Ambari, iirc) which have tons of Python code and lots of integration.

The serialization for PQS can be changed via a single configuration 
property in hbase-site.xml. If this is insufficient, we can explore CLI 
options to queryserver.py to make this even easier. Protobuf is going to 
inherently be a more stable wire format, but that doesn't mean you can't 
use JSON if you have a reason (that's why it wasn't removed).

James Taylor wrote:
> Sorry for the delayed response, Lukáš. I still think it would be a good
> addition and the JSON/Protobuf issue you brought up is all the more
> reason to keep the JSON binding too.
>
> We have precedence for non Java code with our Spark integration. Sure,
> it makes it a bit more work to add this new module, but it's definitely
> doable. And worst-case, the uploading to PyPI could be a manual, post
> release task.
>
> TLDR; if you're willing to package this as a new module (including
> regression tests), we'd be excited to have it.
>
> Any opinions, Josh?
>
> Thanks,
> James
>
> On Fri, Feb 5, 2016 at 11:37 PM, Lukáš Lalinský <lalinsky@gmail.com
> <mailto:lalinsky@gmail.com>> wrote:
>
>     On Sat, Feb 6, 2016 at 7:28 AM, James Taylor <jamestaylor@apache.org
>     <mailto:jamestaylor@apache.org>> wrote:
>
>         Lukás- your Python Query Server support would be a welcome
>         addition to Phoenix or Avatica. Send us a pull request for a new
>         module if you're interested.
>
>
>     I was considering that when I got the first working version, but I
>     was not sure how to best do it. Phoenix would probably be the better
>     place for it, since it's really specific to Phoenix (data types, etc.).
>
>     There is also the practical issue of having non-Java code in a
>     mainly Java project. For Python module to be easily installable, it
>     should be uploaded to PyPI (equivalent to Maven Central in the Java
>     world) and that would have to be a part of the release process. Or
>     it could just live in the repository, but have a separate release
>     schedule, which would be easier to manage.
>
>     Anyway, since the protobuf3 serialization is now the default in
>     Phoenix, I don't know if it makes sense anymore. That significantly
>     complicates packaging of the Python module, so I was planning to
>     stick with JSON.
>
>     Lukas
>
>
>

Mime
View raw message