mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Toenshoff <toensh...@me.com>
Subject Re: Review Request 54038: Added new hook for covering executor and task environment.
Date Fri, 25 Nov 2016 04:10:41 GMT


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > include/mesos/hook.hpp, line 111
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569509#file1569509line111>
> >
> >     "Deprecated"? Will we ever remove this hook?

I was discussing this with Kapil and we need to address this very soon. Definitely a tech-debt.
Created a ticket for tracking that job - https://issues.apache.org/jira/browse/MESOS-6641


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > include/mesos/hook.hpp, lines 146-147
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569509#file1569509line146>
> >
> >     Is `sandboxDirectory` the host or container path? Absolute or relative?
> >     Same question for `mappedDirectory`, also what is it? sandbox container path?

Super confusing indeed.

For a user, the sandbox-directory is the absolute, mapped path within the container. See https://github.com/apache/mesos/blob/8fd64fd5efb898baf99a74ea9104205969d9807b/src/slave/flags.cpp#L629-L633

For the superceded docker hooks, the sandbox-directory is the absolute container work directory
roof fs path on the agent host. See the callup of the old variant of this hook https://github.com/apache/mesos/blob/8fd64fd5efb898baf99a74ea9104205969d9807b/src/slave/containerizer/docker.cpp#L1135-L1136
as you noted below.

Will rename these towards `containerWorkDirectory, mappedSandboxDirectory` and also in a follow-up
patch on the deprecated implementations.


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > include/mesos/module/hook.proto, line 28
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569510#file1569510line28>
> >
> >     Why add this TODO instead of filing a JIRA? We can always add new protobuf fields
later, when we're ready.

True - filed a JIRA - https://issues.apache.org/jira/browse/MESOS-6642


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/hook/manager.cpp, line 221
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569513#file1569513line221>
> >
> >     Update this comment (and others) if you update the proto message name

Thanks for the reminder :) - appreciated!


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/slave/containerizer/docker.cpp, line 1132
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569514#file1569514line1132>
> >
> >     "as soon as possible" isn't very specific. How about "... remove this entire
block after Mesos 1.2's deprecation cycle ends."

Yup.


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/slave/containerizer/docker.cpp, lines 1193-1195
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569514#file1569514line1193>
> >
> >     Sounds like a great thing to Warn about in the agent log. Planning on doing
this in a subsequent patch?

Yes - however its a bit more involved and hence I am deferring that right now - hence the
TODO - also created a JIRA issue - https://issues.apache.org/jira/browse/MESOS-6643


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/slave/containerizer/docker.cpp, lines 1224-1225
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569514#file1569514line1224>
> >
> >     So, if executorEnvironment and taskEnvironment both have the same keys, then
the taskEnvironment will override?

Yes - will add a comment explaining this.


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > include/mesos/hook.hpp, line 128
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569509#file1569509line128>
> >
> >     Or TASK_ERROR? I thought FAILED meant that it ran and returned a non-zero exit
code.

No, the stated should be correct. When the hook returns a failed future, the (docker) containerizer
returns a failed future for `launch` which then gets processed at https://github.com/apache/mesos/blob/master/src/slave/slave.cpp#L4566,
turning the task state into `TASK_FAILED`.
That is what currently happens - now the question might be if this is really what we want
and that might be worth a discussion.


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/hook/manager.cpp, lines 245-249
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569513#file1569513line245>
> >
> >     Could save a couple lines if you invert the if statement and push the MergeFrom
into it.
> >     ```
> >     if (result.isSome()) {
> >       info.MergeFrom(result.get());
> >     }
> >     ```

Doh :)


> On Nov. 24, 2016, 3:38 a.m., Adam B wrote:
> > src/slave/containerizer/docker.cpp, lines 1178-1179
> > <https://reviews.apache.org/r/54038/diff/1/?file=1569514#file1569514line1178>
> >
> >     I still don't understand why `sandboxDirectory = container->directory` and
`mappedDirectory = flags.sandbox_directory`. Seems backwards as far as "sandbox directory"
goes.

s.o.


- Till


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54038/#review156809
-----------------------------------------------------------


On Nov. 25, 2016, 4:10 a.m., Till Toenshoff wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54038/
> -----------------------------------------------------------
> 
> (Updated Nov. 25, 2016, 4:10 a.m.)
> 
> 
> Review request for mesos, Adam B, Gastón Kleiman, Kapil Arya, and Joseph Wu.
> 
> 
> Bugs: MESOS-6396
>     https://issues.apache.org/jira/browse/MESOS-6396
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> For being able to supply environment additions for both, a task and
> its executor separately we need to introduce a new hook as the
> existing ones slavePreLaunchDockerHook (deprecated) as well as
> slavePreLaunchDockerEnvironmentDecorator do not allow for this.
> 
> This new hook will likely allow for further additions like e.g.
> adding volumes without having to adapt the signature but only the
> returned proto message TaskExecutorDecoratorInfo.
> 
> 
> Diffs
> -----
> 
>   include/mesos/hook.hpp f0606e3a68fa179cf7ea036f10563ef47c2aefa7 
>   include/mesos/module/hook.proto PRE-CREATION 
>   src/Makefile.am 5e0b8406f7f624bd8b03ff76b887f20e22fc66e0 
>   src/hook/manager.hpp 5ecfcab48da808c84d36f9bcfcb5a8e0ad2167e5 
>   src/hook/manager.cpp 24885226a788a7abd851e12b527f74fa972ec935 
>   src/slave/containerizer/docker.cpp ccabf99f305d7874e1c46bc618ea74341eb281ef 
> 
> Diff: https://reviews.apache.org/r/54038/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> *WIP - functional test pending - unit tests pending - WIP*
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message