mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Massenzio <m.massenzio+me...@gmail.com>
Subject Re: Review Request 41760: Add initialization method to Anonymous module
Date Sat, 13 Feb 2016 06:22:26 GMT


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > I am little concerned about `Flags` being passed to the module. Since there is no
visibility about the allowed master/agent flags from the module's perspective, how does it
cope with the changes to master/slave flags? Would we want to expose available master/slave
flags as well in include/mesos? It's not quite clear what the correct representation would
be.
> 
> Marco Massenzio wrote:
>     Hey Kapil,
>     
>     thanks for review!
>     I'm in the middle of something, but I'll get round to it this weekend - it all seems
very agreeable at a quick glance.
>     
>     I'll reply with more detail, but in the meantime, please note that teh idea was *not*
expose the master/agent flags as types - the module simply iterates over the map (and getting
the string values, not the actual typed objects); but I'm assuming that the flags' names will
be from *fairly* to *very* stable, so the risk is rather low?
>     
>     I think that exposing the flags in an include module would actually make matters
worse, as it would cause backward compatibility issues and would impact the ability of Mesos
to move forward wrt existing modules?
>     
>     In any event, the way to cope with change would be as usual: for the *required* ones
the module would have to fail (great suggestion to have a `Try` returned by the initialization)
and for the *optional* ones, it will just have to make do without (possibly emitting warnings).

So, as a practical example, this is how it would be used in a module:
```
  virtual void initialize(const flags::FlagsBase& flags)
  {
    string workDir;
    string sandboxDir;

    LOG(INFO) << "Executing init() for module; parsing runtime flags";
    for(auto flag : flags) {
      string name = flag.first;
      Option<string> value = flag.second.stringify(flags);
      if (name == "work_dir" && value.isSome()) {
        workDir = value.get();
      } else if (name == "sandbox_directory" && value.isSome()) {
        sandboxDir = value.get();
      }
    }
    LOG(INFO) << "Configured work dir to [" << workDir
              << "] and Sandbox dir to [" << sandboxDir << "]";
    process = new RemoteExecutionProcess(workDir, sandboxDir);
    spawn(process);
  }
```

This would only "break" if the names of the flags were to change - this would be, I believe,
extremely unlikely?


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > include/mesos/module/anonymous.hpp, line 59
> > <https://reviews.apache.org/r/41760/diff/2/?file=1180449#file1180449line59>
> >
> >     I am wondering if we should allow the module to indicate an error during initialization.
Should the return type be `Try<Nothing>` instead?

Absolutely!
done.


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > src/examples/test_anonymous_module.hpp, lines 25-48
> > <https://reviews.apache.org/r/41760/diff/2/?file=1180450#file1180450line25>
> >
> >     Let's move this to the cpp file as with the other modules.

Done - please note that we still need to leave the class definition in the header file, as
the test needs to cast the module object to verify it was correctly created.

moved ``initialize()`` to the .cpp file, though.


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > src/examples/test_anonymous_module.cpp, lines 56-60
> > <https://reviews.apache.org/r/41760/diff/2/?file=1180451#file1180451line56>
> >
> >     Do we also need a destructor just like in TestAnonymous?

Done; however TestAnonymous' destructor just logs, not a very useful destructor?
We don't allocate memory/resources that need freeing.

Also, in the current code, anon modules' destructors are never called (I was planning to implement
that - there is a TODO from benh - once we got this patch reviewed).


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > src/slave/main.cpp, lines 275-278
> > <https://reviews.apache.org/r/41760/diff/2/?file=1180452#file1180452line275>
> >
> >     Just add a CHECK here and remove the if-else. The module manager returns `Error`
for NULL.

Done.
I've also added a check on the Try returned by initialize() - I am also following the same
pattern as `create()`:
```
    if (create.isError()) {
      EXIT(EXIT_FAILURE)
        << "Failed to create anonymous module named '" << name << "'";
    }
```
however, I'm not entirely sure we should crash the Agent (or Master, for that matter) if an
anon module cannot be initialized?
Are you ok with that?


> On Feb. 8, 2016, 4:53 p.m., Kapil Arya wrote:
> > src/tests/anonymous_tests.cpp, line 106
> > <https://reviews.apache.org/r/41760/diff/2/?file=1180453#file1180453line106>
> >
> >     Is there a way to validate this typecast?

Not that I'm aware of (in Python you'd do `isinstance()` ;D )
I mean, we could use a `try / catch`, but those are not used in Mesos, right?

In any event, this is a test - if the cast fails, it means there's an error somewhere, the
test crashes and we catch the error: everyone is happy?


- Marco


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41760/#review118242
-----------------------------------------------------------


On Jan. 4, 2016, 8:16 p.m., Marco Massenzio wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41760/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2016, 8:16 p.m.)
> 
> 
> Review request for mesos, Anand Mazumdar and Kapil Arya.
> 
> 
> Bugs: MESOS-4253
>     https://issues.apache.org/jira/browse/MESOS-4253
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Jira: [MESOS-4253](https://issues.apache.org/jira/browse/MESOS-4253)
> 
> To provide a basic "context" to Anonymous modules, we pass in the `BaseFlags`.
> 
> This tries to be backward compatible with existing modules, so a no-op (`virtual`) implementation
of the method is provided in the base class.
> Currently the code is only implemented in the Agent's `main()` method, but if deemed
correct, adding it to Master is trivial.
> 
> 
> Diffs
> -----
> 
>   include/mesos/module/anonymous.hpp 629eb123b9d5fa9e07ddb1c704ee72e96e8fec5b 
>   src/examples/test_anonymous_module.hpp 3da33a42f04b7421cee8fbb85e29b66e352f5894 
>   src/examples/test_anonymous_module.cpp dd291cff3b5d47337e371cd2c1082fd6716af3fc 
>   src/slave/main.cpp 9d48a0823189ea6505073a2803f02d90dc382ab4 
>   src/tests/anonymous_tests.cpp bc29ff8be94af82dd97f43db4f9594036705e835 
>   src/tests/module.hpp 8e92774ddd51bc8a1368fb1cf6546300696b2d22 
>   src/tests/module.cpp 7968519996ca9f9d8895e73d5f173d26a7e794e0 
> 
> Diff: https://reviews.apache.org/r/41760/diff/
> 
> 
> Testing
> -------
> 
> Unit tests pass.
> 
> Also implemented in the [execute-module](http://github.com/massenz/execute-module) -
and it works there too:
> ```
> I0102 17:38:26.180788 19971 main.cpp:272] Initializing anonymous module 'com_alertavert_mesos_RemoteExecution'
> I0102 17:38:26.180800 19971 main.cpp:278] Sending flags to module 'com_alertavert_mesos_RemoteExecution'
> I0102 17:38:26.180810 19971 execute_module.hpp:129] Executing initialize() for module;
parsing runtime flags
> I0102 17:38:26.181658 19971 execute_module.hpp:139] Configured work dir to [/tmp/agent]
and Sandbox dir to [/mnt/mesos/sandbox]
> ```
> 
> 
> Thanks,
> 
> Marco Massenzio
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message