mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam B" <a...@mesosphere.io>
Subject Re: Review Request 33490: MESOS-2633 Avoid adding duplicate tasks
Date Thu, 07 May 2015 08:37:41 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33490/#review82809
-----------------------------------------------------------


Thanks for attacking one of my TODOs. I've got a couple of questions, but it looks pretty
straightforward to me.

I'm confused about when `==` would be too strict. What fields would you expect to differ?
Maybe if you got a TASK_LOST followed by a TASK_FAILED for the same taskId, as is supposedly
possible in some weird race? https://github.com/apache/mesos/blob/master/src/master/master.cpp#L3738

Testing:
It's probably unfair to link to a private mesosphere-only build result in your Testing section.
Did the buildbot pass `make distcheck` as well as just `make check`?
Is there a unit test to verify your change? Do we need one?

I have yet to learn what our recommended coding style is for lambdas now, so I'll defer to
@jvanremoortere for his approval there.


src/master/framework.cpp
<https://reviews.apache.org/r/33490/#comment133637>

    Given that MAX_COMPLETED_TASKS_PER_FRAMEWORK = 1000, should we worry about the performance
hit of a linear scan through completedTasks each time a task completes? For very short-lived
tasks, and/or frameworks that launch thousands of simultaneous tasks, this could be happening
a lot.
    Maybe completedTasks should be a combination of a circular buffer and a set.
    
    Taking a step further back, we've discussed replacing this circular buffer with an "Archive"
abstraction, that might involve persisting the completed tasks/frameworks to disk/network,
so that we can keep a longer history. If/when this happens, the duplicate checking should
be pushed into the Archive::contains() implementation, rather than doing a find_if at the
call-site.
    I originally created this TODO because boost::circular_buffer doesn't have a contains().


- Adam B


On May 7, 2015, 1:26 a.m., Marco Massenzio wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33490/
> -----------------------------------------------------------
> 
> (Updated May 7, 2015, 1:26 a.m.)
> 
> 
> Review request for mesos, Adam B and Joris Van Remoortere.
> 
> 
> Bugs: MESOS-2633
>     https://issues.apache.org/jira/browse/MESOS-2633
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> In Framework::addCompletedTask(const Task& task) we did not check
> for duplicated tasks, so they could be added more than once.
> 
> A simple check has now been added (there still is the issue
> of whether the `operator ==` on `Task` is too strict, so as
> never to cause a match).
> 
> 
> Diffs
> -----
> 
>   src/master/framework.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/33490/diff/
> 
> 
> Testing
> -------
> 
> buildbot make check pass
> http://build.mesosphere.com:5555/builders/dev_test/builds/13
> 
> 
> Thanks,
> 
> Marco Massenzio
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message