mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qian Zhang <zhq527...@gmail.com>
Subject Re: Review Request 54639: Implemented the 'create()' method of OCI store.
Date Thu, 09 Feb 2017 07:41:18 GMT


> On Feb. 9, 2017, 7:36 a.m., Jie Yu wrote:
> > src/slave/containerizer/mesos/provisioner/oci/paths.hpp, lines 36-39
> > <https://reviews.apache.org/r/54639/diff/3/?file=1621023#file1621023line36>
> >
> >     Can you elaborate a bit more the layout here?
> >     
> >     so `layer_id` is the digest? What is `config.json`? Does it store decompressed
content, or the original content? what about image configs?
> >     
> >     Given this https://github.com/opencontainers/image-spec/blob/master/image-layout.md,
my hunch is that we should probably cache the original blob, as well as decompressed content,
potentially in separate subdirectories.
> >     
> >     ```
> >     <oci_store_dir>
> >        |--- blobs/  # this is origional CAS blobs.
> >                |--- sha256/
> >                        |--- <digest>
> >                        |--- <digest>
> >                        |--- ...
> >        |--- layers/ # this is decompressed filesystem change sets
> >                |--- <id? digest?>
> >                |--- ...
> >     ```
> >     
> >     Looks like we probably do not need `configs` or `manifests` directory for now
because the only media type is json, and can be directly read from `blobs/`.
> >     
> >     Another thing to consider is the conversion to overlayfs opaque file from whiteout
files as you did before. So for `layers` directory, we probably need to have a `backend` in
the path (e.g., `layers/overlayfs/...`).
> 
> Jie Yu wrote:
>     Also, worth taking a look at how Docker (or other OCI system) organize its cache.

Please take a look at https://reviews.apache.org/r/56147/diff/1#4, I have changed it to:
```
     |--layers
        |--<layer_id>
                |-- rootfs
        |--<layer_id>
                |-- rootfs
        |--<config.json>
```
Here both `<layer_id>` and `<config.json>` are digest. `<layer_id>/rootfs/`
stores decompressed content(filesystem) extracted by `command::untar()` from the layer tar
ball. `<config.json>` stores the image configuration.

Can you please let me know why we need to cache the original blobs? In my current implementation,
the orignal blobs will only be fetched by fetcher into a temp directory under `staging` directory,
`layers` directory will only store the decompressed content, and that temp directory under
`staging` directory will be removed after fetching image is done. I do not think we need the
original blobs once the fetching image is done and we have cached the image in the store.

And actually under `layers/<layer_id>`, there can be two sub-directories: `rootfs` (for
aufs/bind/copy) and `rootfs.overlay` (for overlay), this is same with what I did for for Docker
store. Do you think we need to change it to `layers/non-overlay/<layer_id>` and `layers/overlay/<layer_id>`
for OCI store?


- Qian


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54639/#review164794
-----------------------------------------------------------


On Feb. 1, 2017, 9:17 a.m., Qian Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54639/
> -----------------------------------------------------------
> 
> (Updated Feb. 1, 2017, 9:17 a.m.)
> 
> 
> Review request for mesos, Gilbert Song and Jie Yu.
> 
> 
> Bugs: MESOS-6681
>     https://issues.apache.org/jira/browse/MESOS-6681
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Implemented the 'create()' method of OCI store.
> 
> 
> Diffs
> -----
> 
>   src/CMakeLists.txt 09ef1aee680c6b91476092dbf61d4ca3a8d256bb 
>   src/Makefile.am 6c9be54fe0ce3faaa26e2f090773d1b77bb6e430 
>   src/slave/containerizer/mesos/provisioner/oci/paths.hpp PRE-CREATION 
>   src/slave/containerizer/mesos/provisioner/oci/paths.cpp PRE-CREATION 
>   src/slave/containerizer/mesos/provisioner/oci/store.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/54639/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message