mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vinodk...@gmail.com>
Subject Re: Review Request 56052: Added the 'Secret' protobuf message.
Date Sat, 11 Feb 2017 01:40:33 GMT


> On Feb. 9, 2017, 6:31 p.m., Jie Yu wrote:
> > include/mesos/mesos.proto, line 1994
> > <https://reviews.apache.org/r/56052/diff/3/?file=1628247#file1628247line1994>
> >
> >     Actually, one thought here. Given that environment varaible's source cannot
be a byte stream. Do we want to have explicit typing: REFERENCE, TEXT, BYTES for secret so
that the validation for environment source can be easier and more explicit?
> 
> Greg Mann wrote:
>     When a user specifies a REFERENCE-type Secret, would they need to be able to specify
text vs. bytes in that case? If so, then we might want two different sets of type information.
I don't know if a user's module implementation might need to know the encoding (or lack thereof)
of a REFERENCE Secret when fetching it?
>     
>     If we don't think the REFERENCE-type secret needs any encoding information, then
we could use {REFERENCE, TEXT} for the enum types in this patch, so that we could add a BYTES
type at the top level later. Another option would be to include type information inside the
Value message, but this adds a bit more complexity to the message.
> 
> Jie Yu wrote:
>     Yes, you're right that this also applies to REFERENCE type secret. What happen in
the future that we start to support REFERENCE type through modules. What the agent should
do for environment variable. Does the agent need to do base64 encoding (as you mentioned,
how does the agent know that the content received is a string or byte stream, and how does
the user know?), or leave it as a plain string?
>     
>     Looks like k8s secret always do base64 encoding? Maybe that's how they solve the
issue?
>     
>     Let's do our due diligence to evaluate all the options thoroughly because this is
a very important API.
> 
> Greg Mann wrote:
>     Yea it looks like k8s solves this by always base64-encoding the secret data. Since
k8s provides its own secret-store, they can establish this constraint easily. In our case,
if we want to support arbitrary back-ends, we can't necessarily make this assumption. For
example, see the API section of the docs for the Hashicorp Vault PKI backend: https://www.vaultproject.io/docs/secrets/pki/.
Some of these endpoints return raw binary data. If we require that a secret fetcher module
be used to populate REFERENCE type secrets, then we could still require base64-encoding, and
the secret fetcher could be responsible for encoding fetched secrets appropriately.
>     
>     Let's consider a concrete example: a cluster using a Hashicorp Vault secret store
which has two back-ends loaded: 'generic' (the default) and 'pki'. Given the current `Secret`
design in this patch, a fetcher module could infer the correct back-end to use via the `reference.name`.
For example, the module could map these names directly onto the Vault API: a name of '/pki/ca'
would grab the PKI back-end's CA, and a name of '/pki/cert/<serial#>' could grab a particular
certificate from the PKI back-end. A name of '/generic/my_secret' could fetch the secret named
"my_secret" in the generic back-end.
>     
>     In any case, if we wanted to mandate only base64-encoded secrets within Mesos, the
secret fetcher could be aware of the type of data received from the back-end's API and encode
it appropriately, if necessary. Or, we could allow binary secrets and include explicit type
information to distinguish between text and binary. If we do this, however, it's not clear
to me that the user launching a task should need to specify whether the secret is text or
bytes - perhaps the secret fetcher could determine this from a particular secret name, or
perhaps some secret stores may return both text and binary secrets from the same API call?
I'm not sure, need to do some more research.
>     
>     If we want to provide a more generic HTTP-based secret-fetching model in which a
user could specify an arbitrary endpoint that would return the desired secret, then it makes
sense for that user to specify text vs. bytes. However, in that case it's not clear to me
where the generic fetcher gets the credentials needed to fetch the secret itself.
>     
>     This brings me to another important issue regarding secret fetching: how does the
fetcher establish that the executor or task in question is authorized to fetch the specified
secret? With this proto, we're attempting to provide a first-class object permitting the secure
fetching of secrets; do we also want to provide a first-class identity associated with the
executor/task for the purpose of authorizing the secret-fetching?

Instead of first-classing bytes, I think it is simpler to only allow string in the API and
let the user do encoding/decoding while doing storing/using.


- Vinod


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56052/#review164960
-----------------------------------------------------------


On Feb. 9, 2017, 6:33 a.m., Greg Mann wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56052/
> -----------------------------------------------------------
> 
> (Updated Feb. 9, 2017, 6:33 a.m.)
> 
> 
> Review request for mesos, Adam B, Jie Yu, Kapil Arya, Jan Schlicht, and Vinod Kone.
> 
> 
> Bugs: MESOS-6996
>     https://issues.apache.org/jira/browse/MESOS-6996
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This patch adds a new `Secret` protobuf message which
> is designed to serve as a generic mechanism for passing
> priviledged information within Mesos.
> 
> 
> Diffs
> -----
> 
>   include/mesos/mesos.proto 34a288b6f5c2fca090a7aa7a61798e3255d6663a 
>   include/mesos/v1/mesos.proto 6638111d10f4a36cdf91dfce1019871e9839c306 
> 
> Diff: https://reviews.apache.org/r/56052/diff/
> 
> 
> Testing
> -------
> 
> `make check`
> 
> 
> Thanks,
> 
> Greg Mann
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message