mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Pronin <ipro...@twopensource.com>
Subject Re: Review Request 67914: Updated XFS disk isolator to reclaim project IDs after disk GC.
Date Tue, 24 Jul 2018 18:19:50 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67914/
-----------------------------------------------------------

(Updated July 24, 2018, 11:19 a.m.)


Review request for mesos and James Peach.


Bugs: MESOS-9007
    https://issues.apache.org/jira/browse/MESOS-9007


Repository: mesos


Description
-------

Currently upon container destruction its project ID is unallocated by
the isolator and removed from the container work directory. However due
to API limitations we can't unset project IDs on symlinks that may exist
inside the directory. Because of that the project may still exist until
the container directory is garbage collected. If the project ID is
reused for a new container, any lingering symlinks that still have that
project ID will contribute to disk usage of the new container. Typically
symlinks don't take much space, but still this leads to inaccuracy in
disk space usage accounting.

This patch postpones project ID reclaiming until sandbox GC time. The
isolator periodically checks if sandboxes of terminated containers still
exist and deallocates project IDs of the ones that were removed. Checks
are performed every "disk_watch_interval". This mechanism can be
improved in the future if we introduce a way for the isolators to learn
about disk GCs.

Current number of available project IDs can be tracked with the new
"containerizer/mesos/disk/project_ids_free" and
"containerizer/mesos/disk/project_ids_total" metrics.


Diffs (updated)
-----

  src/slave/containerizer/mesos/isolators/xfs/disk.hpp 0891f7709aa4f98758a727856d58e6177d46adca

  src/slave/containerizer/mesos/isolators/xfs/disk.cpp 25f52a43b34b141bdaf7c448817423cf4264e22a

  src/tests/containerizer/xfs_quota_tests.cpp dc18a8a59d1eb7fae3592ef6ba8c046e4f46ee4a 


Diff: https://reviews.apache.org/r/67914/diff/5/

Changes: https://reviews.apache.org/r/67914/diff/4-5/


Testing
-------

Added `ROOT_XFS_QuotaTest.ProjectIDReclaiming` test that verifies that project ID is reclaimed
and reused after sandbox GC. Ran `sudo make check`.


Thanks,

Ilya Pronin


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message