mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jiang Yan Xu" <...@jxu.me>
Subject Re: Review Request 37382: Introduced provisioner Backend interface.
Date Wed, 19 Aug 2015 06:36:29 GMT


> On Aug. 14, 2015, 11:15 a.m., Jie Yu wrote:
> > src/slave/containerizer/provisioners/backend.hpp, lines 55-57
> > <https://reviews.apache.org/r/37382/diff/2/?file=1040433#file1040433line55>
> >
> >     Hum, what do yo mean here?
> >     
> >     I think the backend should be container aware. I.e., we should pass in containerid
in 'provision'  and destroy will take a containerid as well.
> >     
> >     There could be multiple rootfs for a container (because image in volumes). When
the container finishes, the provisioner will try to destroy all rootfs provisioned for that
container.
> >     
> >     Thoughts?
> 
> Jiang Yan Xu wrote:
>     This is intended for the image in volumes case but I was trying to not be too explicit
because it's not implemented yet (we should track it with a separate ticket).
>     
>     My comment about 'nested' was incorrect, let me clarify here.
>     
>     The 'caller' is the provisioner. What I had in mind was: each Backend call handles
one image, if `Backend::provision()`s one image and then it should `destroy()` the rootfs
provisioned by one image.
>     
>     Suppose we have such a rootfs dir and each container has multiple rootfses, some
from ContainerInfo::volumnes and one from ContainerInfo::mesos::image but they are stored
side-by-side under <container_id>/<image_type>.
>     
>     ```
>     <--rootfs_dir>
>     |--<container_id>
>        |--<appc>
>           |--<image_id>
>           |--<image_id>
>        |--<docker>
>     ```
>     
>     IMO since LinuxFilesystemIsolatorProcess::cleanup() calls each `Provisioner::destory(containerId)`
(appc|docker), the provisioner can look at the subdir it manages and scan it and call `Backend::destroy(rootfsPath)`
for each image under it.
>     
>     So each `Backend::destroy(rootfsPath)` call destroys one rootfs and there is no `nesting`
under it.
>     Passing `containerId` into Backend isn't as clean as it should be the provisioner
who converts the id into a path and Backends are this dumb thing which only knows paths.
>     
>     More clear?
> 
> Jie Yu wrote:
>     So the directory layout listed above is managed by the provisioner or backend? It
cannot be backend because otherwise that means provisioner needs to know about the layout
used by the backend.
>     
>     So if it's managed by provisioner, the layout should probably be:
>     ```
>     /var/lib/mesos/provisioners/
>         |-- appc/
>               |-- <container_id>
>                      |-- <image1>
>                      |-- <image2>
>         |-- docker/
>               |-- <container_id>
>                      |-- <image3>
>                      |-- <image4>
>     ```
>     
>     So we only allow one backend, or we allow multiple backends (e.g., what if we want
to use bind mount backend for appc but overlayfs backend for docker, or we want to gradually
roll out overlayfs backend so that two backends co-exists for a while)? If the later, how
can we figure out which backend to use when destroying root filesystems?
>     
>     I just want to get the full picture sorted out. We can chat next week.
> 
> Jiang Yan Xu wrote:
>     Note that the above rootfs I mentioned is the result of the Backend either collapsing
its layers or overlay them and in the provisioner's view it's just one entity.
>     
>     So the backend know about layout within a rootfs (e.g., its knows that a rootfs is
"ImageA(a script) -> ImageB(essential software) ->ImageC(base OS)"); the provisioner
knows about / manages the direcotry layout of 'rootfs'es but not inside 'rootfs'es (e.g. This
container wants three rootfses, two of which from volumes).
>     
>     Yeah I agree the interface design requires a full picture of multi-image-per-contain
design. I will jot down my thoughts in a doc and let's chat next week.
> 
> Timothy Chen wrote:
>     Hey we should talk about this as it affects all of our work, I think I didn't realize
we're ending up doing multiple provisioner images for a single executor.
>     I think my assumption was that the backend when given a provision call is for a image
as Yan mentioned, and returning the rootfs directory path which the provisioner can hold on
to know where the backend has provisioned it to.
>     
>     We don't really allow multiple backends with our current design, I don't think there
is anything stopping us but it then become a very interesting API choice to see where we allow
the user to specify this. My intuition is to just use one single backend and we record the
backend that is used to provisioner a rootfs, so at least on recovery we can say we can't
recover due to a backend change.

Jie and I chatted offline and I am going to submit another review regarding the file system
layout that addresses multiple provisioners / mutliple backends. The current Backend interface
design is compatible with it and not itself controversial so I am just going to make some
documentation adjustments here.

Regardign a few issues which I'll describe in details in another review:

Multiple provisioners: To support [MESOS-3004](https://issues.apache.org/jira/browse/MESOS-3004)
there are already multiple images within a container and there is no real obstacle to prevent
them from being different image types.

Multiple backends: I agree that we don't have a use case for allowing the users to choose
the backend (so we don't need to design the API for it) but the case for updating the Backend
for a running cluster is real. We don't have to turn away a rootfs known to one of the backends
during recovery / cleanup just because it's not the currently selected backend for new containers.
I'll explain in a new review.


- Jiang Yan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37382/#review95455
-----------------------------------------------------------


On Aug. 14, 2015, 10:51 a.m., Jiang Yan Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37382/
> -----------------------------------------------------------
> 
> (Updated Aug. 14, 2015, 10:51 a.m.)
> 
> 
> Review request for mesos, Lily Chen, Ian Downes, Jie Yu, and Timothy Chen.
> 
> 
> Bugs: MESOS-2968
>     https://issues.apache.org/jira/browse/MESOS-2968
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> The Backend interface is made generic so it can be used by different provisioners. See
[MESOS-2968](https://issues.apache.org/jira/browse/MESOS-2968?focusedCommentId=14652859&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14652859)
for details.
> 
> 
> Diffs
> -----
> 
>   src/Makefile.am b5dbdc316cad179cd265bd81900999fab35940b9 
>   src/slave/containerizer/provisioners/backend.hpp PRE-CREATION 
>   src/slave/containerizer/provisioners/backend.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37382/diff/
> 
> 
> Testing
> -------
> 
> make check.
> 
> 
> Thanks,
> 
> Jiang Yan Xu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message