mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Peach <jpe...@apache.org>
Subject Re: Review Request 68401: Added persistent volume support to the `disk/xfs` isolator.
Date Fri, 24 Aug 2018 23:49:07 GMT


> On Aug. 24, 2018, 11:38 p.m., Chun-Hung Hsiao wrote:
> > src/slave/containerizer/mesos/isolators/xfs/disk.hpp
> > Lines 94 (patched)
> > <https://reviews.apache.org/r/68401/diff/1/?file=2074209#file2074209line94>
> >
> >     I was wondering if a `hashmap<std::string, PathInfo>` or a `vector<PathInfo>`
(with a path field in `PathInfo`. is better here. The only cane we use the key to find the
`PathInfo` is for sandbox, but I feel that we could probably treat all paths homogeneously,
> >     so instead of the following in all methods:
> >     
> >     1. Process the sandbox
> >     2. Process the volume for each volume in paths
> >     
> >     We could just do:
> >     
> >     1. Process the volume for each volume in paths (including the sandbox, whose
`disk` is none)
> >     
> >     Also maybe `QuotaInfo` is a better name?
> >     
> >     Thoughts?

I ended up doing roughly what `posix/disk.cpp` does. I think that your suggestion works well
for `usage` but will be awkward in `update`. I'll see whether I can find a way to make it
work.


> On Aug. 24, 2018, 11:38 p.m., Chun-Hung Hsiao wrote:
> > src/slave/containerizer/mesos/isolators/xfs/disk.hpp
> > Lines 128 (patched)
> > <https://reviews.apache.org/r/68401/diff/1/?file=2074209#file2074209line140>
> >
> >     I was wondering if we want to keep the pair here. From what I read, we store
the result of `getDeviceForPath` when adding a path into `scheduledProjects`, then use it
once when removing it. Instead of storing the device path in this pair, we could leave `scheduledProjects`
as is, and then call `getDeciveForPath` in `reclaimProjectIds`. This would also avoid extra
copyes when doing `utils::copy`. WDYT?

The problem is that when you reclaim the quota record in `reclaimProjectIds`, you only do
so when the directory has been deleted. Once this happens, you can no longer find the device
that the quota was stored on (i.e. `getDeviceForPath` will always fail).


- James


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68401/#review207915
-----------------------------------------------------------


On Aug. 24, 2018, 11:26 p.m., James Peach wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68401/
> -----------------------------------------------------------
> 
> (Updated Aug. 24, 2018, 11:26 p.m.)
> 
> 
> Review request for mesos, Chun-Hung Hsiao, Ilya Pronin, Jie Yu, Joseph Wu, and Jiang
Yan Xu.
> 
> 
> Bugs: MESOS-5158
>     https://issues.apache.org/jira/browse/MESOS-5158
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Added persistent volume support to the `disk/xfs` isolator. This
> implementation largely tracks the `disk/du` implementation in that
> we now keep a map of paths in each container info structure. We now
> defer quota clean up to project ID reclaimation time so that we can
> use the same mechanism for sandbox and persistent volume paths.
> 
> We explicitly exclude mount disks from XFS project quotas, but we still
> track them so that we can correctly publish their usage information in
> the container `DiskStatistics` message. This means that mount disks are
> not required to be XFS filesystems or have project quotas configured.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/isolators/xfs/disk.hpp 38c467b47cb7c04803b0709b8239458fb26abb61

>   src/slave/containerizer/mesos/isolators/xfs/disk.cpp 783da0407528c044035d18cc59a744353921d64c

>   src/tests/containerizer/xfs_quota_tests.cpp 59ec182c1c3af3978156044f03d9e3d784d51fce

> 
> 
> Diff: https://reviews.apache.org/r/68401/diff/2/
> 
> 
> Testing
> -------
> 
> sudo make check (Fedora 28)
> 
> 
> Thanks,
> 
> James Peach
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message