mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Wu <jos...@mesosphere.io>
Subject Re: Review Request 72124: Added logging of tasks and operations during agent drain initiation.
Date Tue, 25 Feb 2020 18:42:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72124/#review219658
-----------------------------------------------------------


Fix it, then Ship it!





src/master/http.cpp
Lines 4083-4086 (patched)
<https://reviews.apache.org/r/72124/#comment307868>

    We should only be able to skip through this condition if the agent is removed between
calling DrainAgent and committing the change to the registrar.  So instead of this TODO, perhaps
we should add a comment saying so.  (Also, the DrainSlaveMessage would be sent when/if the
agent comes back).


- Joseph Wu


On Feb. 12, 2020, 6:05 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72124/
> -----------------------------------------------------------
> 
> (Updated Feb. 12, 2020, 6:05 p.m.)
> 
> 
> Review request for mesos, Greg Mann and Joseph Wu.
> 
> 
> Bugs: MESOS-10095
>     https://issues.apache.org/jira/browse/MESOS-10095
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> When draining an agent, it's hard to tell which tasks failed to
> terminate from the logs. The master prints a count of the tasks
> remaining (only as VLOG(1) however), but not the IDs.
> 
> This patch adds logging to the initiation of the drain on both the
> master and agent, that shows which tasks and operations are present.
> This makes it possible to then see which ones did not transition to
> a terminal state (with a bit of log analysis effort).
> 
> 
> Diffs
> -----
> 
>   src/master/http.cpp eeaac88c893b43170e655f8bff1d57dd0f7bbfb2 
>   src/slave/slave.cpp cce275a504effae7a6b71dd333ce8a300d1ce5be 
> 
> 
> Diff: https://reviews.apache.org/r/72124/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> Examples:
> 
> ```
> I0212 18:03:33.488853 14728 http.cpp:4102] Transitioning agent e1120790-a1cf-488d-8024-29cbaf1f63c4-S0
to the DRAINING state; agent has (pending tasks, tasks, operations) == ({  }, { e1120790-a1cf-488d-8024-29cbaf1f63c4-0000:
{ a3bcc29d-3f7c-4454-8363-4a5cccc39370 } }, {  })
> 
> I0212 18:03:32.496709 14720 slave.cpp:1097] Initiating drain with DrainConfig {"mark_gone":false};
agent has (pending tasks, queued tasks, launched tasks, operations) == ({  }, {  }, { 968682eb-95c3-4d1e-a2b6-d04cb1a16a94-0000:
{ 2d3e2ab5-b6ed-4a87-b390-9f88dda18b7a } }, {  })
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message