mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Chung <cinchu...@gmail.com>
Subject Review Request 66649: Added pb2gen.sh for generating python protobuf bindings.
Date Tue, 17 Apr 2018 19:03:18 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66649/
-----------------------------------------------------------

Review request for mesos.


Repository: mesos


Description
-------

Added pb2gen.sh for generating python protobuf bindings.

This change addresses the issue of protobuf bindings for the mesos python client. Generating
python protobuf bindings is not as straightforward as it seems since vanilla `protoc` has
two limitiations in terms of python code generation:
1. it does not create the `__init__.py` files required for python packages
2. it does not allow arbitrary prefixes to the import paths, even though it allows a custo
`python_path` when generating code

To this end, this change adds the following:
1. A script (`pb2gen.sh`) for generating protobuf python bindings and post-processing the
generated python code so that it is usable by the rest of the code under `src/python/lib/mesos`.
2. Adds python bindings generated from `pb2gen.sh` using the current set of protobuf definitions.


Diffs
-----

  src/python/lib/mesos/pb2/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/agent/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/agent/agent_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/allocator/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/allocator/allocator_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/appc/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/appc/spec_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/authentication/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/authentication/authentication_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/authorizer/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/authorizer/acls_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/authorizer/authorizer_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/docker/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/docker/spec_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/docker/v1_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/docker/v2_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/executor/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/executor/executor_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/fetcher/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/fetcher/fetcher_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/maintenance/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/maintenance/maintenance_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/master/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/master/master_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/mesos_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/module/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/module/hook_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/module/module_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/oci/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/oci/spec_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/quota/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/quota/quota_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/resource_provider/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/resource_provider/resource_provider_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/scheduler/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/scheduler/scheduler_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/slave/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/slave/containerizer_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/slave/oversubscription_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/state/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/state/state_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/uri/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/uri/uri_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/agent/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/agent/agent_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/allocator/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/allocator/allocator_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/executor/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/executor/executor_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/maintenance/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/maintenance/maintenance_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/master/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/master/master_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/mesos_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/quota/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/quota/quota_pb2.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/resource_provider/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/resource_provider/resource_provider_pb2.py PRE-CREATION

  src/python/lib/mesos/pb2/mesos/v1/scheduler/__init__.py PRE-CREATION 
  src/python/lib/mesos/pb2/mesos/v1/scheduler/scheduler_pb2.py PRE-CREATION 
  src/python/lib/pb2gen.sh PRE-CREATION 
  src/python/lib/requirements.in 0742f3d846c99c1c4907d9628fb49845564563b2 


Diff: https://reviews.apache.org/r/66649/diff/1/


Testing
-------

1. under `src/python/lib`, run `pb2gen.sh`
2. initialize and activate virtualenv: `virtualenv env && . env/bin/activate`
3. install reqs: `pip install -r requirements.in`
4. try to import modules from generated python code: `python -c 'from mesos.pb2.mesos.v1.master
import master_pb2'`


Thanks,

Eric Chung


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message