mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Rukletsov <ruklet...@gmail.com>
Subject Re: Review Request 68903: Avoid deadlock-prone blocking in master's parallel endpoint serving.
Date Thu, 04 Oct 2018 08:36:38 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68903/#review209216
-----------------------------------------------------------




src/master/http.cpp
Lines 2433-2440 (patched)
<https://reviews.apache.org/r/68903/#comment293537>

    The interesting scenario is when the only available working thread is the one currently
attached to the master actor. In this scenario, is the expectation here that each `process::async(worker)`
will be spawned and executed on this single worker thread before master actor proceeds to
launching the last worker?


- Alexander Rukletsov


On Oct. 3, 2018, 12:07 a.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68903/
> -----------------------------------------------------------
> 
> (Updated Oct. 3, 2018, 12:07 a.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Benno Evers.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> The approach taken to serve endpoints in parallel was to call
> `process::async` for each request and block until they finish.
> Blocking in this case assumes that there will be enough additional
> non-blocked worker threads to execute all the requests. When the
> number of blocking Processes is equal or greater than the number
> of worker threads, deadlock become possible. The parallel endpoint
> serving approach added another blocking Process.
> 
> While we could make blocking safe (see MESOS-8256) by, for example,
> detaching a worker thread prior to blocking (a la golang), such a
> change is more involved.
> 
> In lieu of making blocking safe, this patch updates the Master to
> block only when it's guaranteed to be safe. This is accomplished
> by:
> 
>   (1) using a "work queue" (just a shared atomic "next index") and
>       having workers pop items from this "queue",
>   (2) we execute one of the workers on the master Process directly,
>   (3) when this worker completes, we then know that all requests
>       are either processed or being processed (i.e. the `async`
>       `Process` is executing on a worker thread),
>   (4) the master can then block on a count down latch, knowing
>       that it will unblock because of (3).
> 
> 
> Diffs
> -----
> 
>   src/master/http.cpp d912789bcfdcf6eaefc29dba7b878fa4e02f9276 
> 
> 
> Diff: https://reviews.apache.org/r/68903/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message