mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilbert Song <songzihao1...@gmail.com>
Subject Re: Review Request 54215: Fixed duplicate image layer ids returned by docker store.
Date Thu, 26 Jan 2017 00:34:08 GMT


> On Jan. 25, 2017, 7:45 a.m., Jie Yu wrote:
> > src/slave/containerizer/mesos/provisioner/docker/store.cpp, lines 276-279
> > <https://reviews.apache.org/r/54215/diff/6/?file=1613732#file1613732line276>
> >
> >     Can you also validate the ordering with docker code? For instance, say the layer
paths are:
> >     
> >     [a, b, c, d, a, e, f]
> >     
> >     will the end result be:
> >     
> >     [b, c, d, a, e, f]
> >     
> >     or
> >     
> >     [a, b, c, d, e, f]
> >     
> >     Please note that the ordering we have here in our code might not be the same
as that in docker (i.e., whether base layer is to the left most or to the right most?)

Did some research on docker. Basically, as I mentioned above, the layer ids are read from
the disk, so they are `unique` and should be ordered `alphabetically`. Then, docker loads
each layer depending on the layer `child-parent` relationship. It is similar to what we currently
do in local puller and it would not have the case of duplicate layer paths since it is like
a linked list from bottom to the top. The order should be:

[Grandparent, parent, child]

In our registry puller, the vector of layer ids are collected differently (not relying on
finding next parent from the base layer). We just queue all the layer ids using the returned
manifest, and fortunately the `v1Compatibility::id` from the manifest are sorted before hand
depending on the `parent-child` relationship.

Only some self-built images may contain duplicate layers (e.g., [a, a, b, b, b, c] from the
manifest). We should still fix it for them though. It seems to me that our current fix in
docker store is just a workaround. I would propose to fix it in registry puller `pull()`:

1. Depending on the leaf layer id, collect all layer ids using `parent-child` relationship.
2. Filter the vector of layer ids to make sure ids are unique in registry puller.

I prefer (2). Thoughts?


- Gilbert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54215/#review162963
-----------------------------------------------------------


On Jan. 24, 2017, 12:35 p.m., Gilbert Song wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54215/
> -----------------------------------------------------------
> 
> (Updated Jan. 24, 2017, 12:35 p.m.)
> 
> 
> Review request for mesos, Avinash sridharan, Artem Harutyunyan, Jie Yu, Qian Zhang, and
Zhitao Li.
> 
> 
> Bugs: MESOS-6654
>     https://issues.apache.org/jira/browse/MESOS-6654
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This issue is exposed by pulling the 'mesosphere/inky' docker
> image using registry puller. Due to the duplicate layer id
> from the manifest, there are duplicate layer pathes passed
> to the backend. The aufs backend cannot handle this case and
> returns 'invalid arguments' error. Ideally, we should make
> sure that layer paths that are passed to the backend are
> unique.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/provisioner/docker/store.cpp 9dccd0673dbc0c61abfd4b88097f86e7d7131c46

> 
> Diff: https://reviews.apache.org/r/54215/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> Tested by the unit test `ROOT_CURL_INTERNET_DockerDefaultEntryptRegistryPuller`.
> 
> Manually tested using the `mesosphere/inky` image, which contains duplicate layer ids.
> 
> 
> Thanks,
> 
> Gilbert Song
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message