mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Janco <jjanco....@gmail.com>
Subject Re: Review Request 53479: Perform agent GC asynchronously.
Date Tue, 11 Jul 2017 18:36:35 GMT


> On June 25, 2017, 7:33 a.m., Jiang Yan Xu wrote:
> > src/slave/gc.cpp
> > Line 153 (original), 149-178 (patched)
> > <https://reviews.apache.org/r/53479/diff/3/?file=1704584#file1704584line170>
> >
> >     Would this be simpler?
> >     
> >     ```
> >         list<PathInfo> infos = paths.get(removalTime);
> >     
> >         auto rmdirs = [infos]() {
> >           foreach (const PathInfo& info, infos) {
> >             LOG(INFO) << "Deleting " << info.path;
> >     
> >             // ...
> >             Try<Nothing> rmdir = os::rmdir(info.path, true, true, true);
> >     
> >             if (rmdir.isError()) {
> >               LOG(WARNING) << "Failed to delete '" << info.path <<
"': "
> >                            << rmdir.error();
> >               info.promise->fail(rmdir.error());
> >             } else {
> >               LOG(INFO) << "Deleted '" << info.path << "'";
> >               info.promise->set(rmdir.get());
> >             }
> >           }
> >         }
> >     
> >         async(rmdirs);
> >           .then(defer(self(),&Self::_remove, infos);
> >     ```
> >     
> >     Since our main objective is to move rmdir out of the `GarbageCollectorProcess`
so perhaps one async call to sequentially remove all of the paths as it was done before when
it can simply the code is worth it. In practice there likely won't be many paths sharing the
same scheduled gc time anyway.

This makes sense to me, testing it, as this would avoid creating a bunch of new processes
with an unknown bound.


- Jacob


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53479/#review178839
-----------------------------------------------------------


On July 11, 2017, 6:28 p.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53479/
> -----------------------------------------------------------
> 
> (Updated July 11, 2017, 6:28 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6549
>     https://issues.apache.org/jira/browse/MESOS-6549
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Previously, rmdir operations were serialized
>   through the garbage collector. Expensive removal
>   events had the potential to delay task launch.
> - MESOS-6549
> 
> 
> Diffs
> -----
> 
>   src/slave/gc.hpp 5ea82456cffa374203371f227b6f0ee00296553e 
>   src/slave/gc.cpp 961f547793984449ea113d9664b12b5033638c58 
> 
> 
> Diff: https://reviews.apache.org/r/53479/diff/4/
> 
> 
> Testing
> -------
> 
> `./bin/mesos-tests.sh --gtest_filter="GarbageCollector*" --gtest_repeat=100 --gtest_break_on_failure`
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message