mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 65106: Removed the misleading `isRemovable` helper in the master.
Date Tue, 23 Jan 2018 00:11:27 GMT


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> >

Thank you for the review! :)


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> > src/master/master.hpp
> > Lines 2208 (patched)
> > <https://reviews.apache.org/r/65106/diff/2/?file=1940801#file1940801line2224>
> >
> >     can you use `CHECK_NE` here so that `task->state()` is auto printed when
this fails?

Ditto cases below.


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> > src/master/master.hpp
> > Lines 2334 (patched)
> > <https://reviews.apache.org/r/65106/diff/2/?file=1940801#file1940801line2350>
> >
> >     ditto. `CHECK_NE`?

See my comment below, I avoided it because the CHECK_NE will print the state but we don't
really need that here. Also added logging of the task/framework id.


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> > src/master/master.cpp
> > Lines 10111-10127 (original), 10111-10119 (patched)
> > <https://reviews.apache.org/r/65106/diff/2/?file=1940802#file1940802line10111>
> >
> >     move this to #1061
> >     
> >     Also, consider moving this to `protobuf_utils.hpp`. AFAICT this is used in 4
different places in this diff.

This needs to be up here since we mutate the task state.

I looked into it originally, but when I was writing its documentation it seemed strange to
need a helper for an or condition and it seemed like it only had some meaning according to
the master's current semantics around resource recovery. So I held off.


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> > src/master/master.cpp
> > Lines 10247-10261 (original), 10245-10261 (patched)
> > <https://reviews.apache.org/r/65106/diff/2/?file=1940802#file1940802line10255>
> >
> >     Not yours, but I wish we can remove this if altogether and CHECK that we are
only removing a terminal or unreachable task period. AFAICT, the only place where we violate
this invariant is in `Master::finalize()`. Maybe add a TODO to fix?

Yeah I was looking to track this down and CHECK here instead, and ended up writing a TODO
that captures why I couldn't, somehow I had removed that prior to publishing the patch. Added
one.


> On Jan. 22, 2018, 8:02 p.m., Vinod Kone wrote:
> > src/master/master.cpp
> > Line 11423 (original), 11423 (patched)
> > <https://reviews.apache.org/r/65106/diff/2/?file=1940802#file1940802line11435>
> >
> >     CHECK_NE

I avoided it here, since there wasn't a need to print the task state, if this fails it's TASK_UNREACHABLE.
It seemed a little more readable to see the != operator.

Added the task id and framework id here for debuggability.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65106/#review195647
-----------------------------------------------------------


On Jan. 23, 2018, 12:11 a.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65106/
> -----------------------------------------------------------
> 
> (Updated Jan. 23, 2018, 12:11 a.m.)
> 
> 
> Review request for mesos, James Peach, Vinod Kone, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-8389
>     https://issues.apache.org/jira/browse/MESOS-8389
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> In the past, the notion of a "removable" task meant: the task is
> terminal and acknowledged. However, the `isRemovable` helper only
> defines removability by the task state (terminal or unreachable)
> but not whether the terminal update is acknowledged.
> 
> As a result, the code that is calling `isRemovable` is unintuitive.
> One example of a confusing piece of code is within updateTask. Here,
> we have logic which says, if the task is removable, recover the
> resources but don't remove it. This seems more intuitive if directly
> described as: "if the task is no longer consuming resources (e.g.
> transitioned to terminal or unreachable) then recover the resources".
> 
> If one looks up the documentation of `isRemovable`, it says "When a
> task becomes removable, it is erased from the master's primary task
> data structures", but that isn't accurate since this function doesn't
> say whether the terminal task has been acknowledged, which is
> required for a task to be removable.
> 
> To make an immediate improvement, this patch removes `isRemovable`
> (no pun intended) in favor of directly checking the states we care
> about. In the future, we may want to introduce some helpers like
> `isAllocatedResources` to describe whether the task's resources are
> considered allocated.
> 
> If we do introduce a notion of `isRemovable` in the future, it seems
> it should take the `Task*` so that it can say whether the task could
> be "removed" from the master, which includes checking that terminal
> tasks have been acknowledged. However, "remove" is somewhat of a
> confusing term, since what we're actually doing is "completing" the
> task.
> 
> 
> Diffs
> -----
> 
>   src/master/master.hpp 05136781b6b1539f37c283e8127e4bafb187a0d1 
>   src/master/master.cpp 3af96b1d2024ab1b951537ebc6bbc225cfa9cc88 
> 
> 
> Diff: https://reviews.apache.org/r/65106/diff/3/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message