mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Wu <jos...@mesosphere.io>
Subject Re: Review Request 58900: Changed the default fetcher cache directory.
Date Mon, 08 May 2017 22:20:12 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58900/
-----------------------------------------------------------

(Updated May 8, 2017, 3:20 p.m.)


Review request for mesos and Jie Yu.


Changes
-------

Changed when the fetcher cache directory is generated (in the default case).  Instead of being
generated by the flags, it is instead generated by the fetcher process.


Bugs: MESOS-7304
    https://issues.apache.org/jira/browse/MESOS-7304


Repository: mesos


Description (updated)
-------

The fetcher cache directory was historically located (by default)
in `/tmp/mesos/fetch`.  The agent flag `--fetcher_cache_dir` could
be used to change this value.

The fetcher would create a subdirectory underneath `/tmp/mesos/fetch`
for each `SlaveID`.  This was done because multiple agents can run on
the same node.  If all the agents use the same default fetcher cache
directory, they will collide and cause unpredictable results.
As a result, the `SlaveID` needed to be passed into the fetcher
after the agent recovers and/or registers with the master, because
that is when the `SlaveID` is determined.

This changes the default fetcher cache directory to
`/tmp/mesos/fetch/XXXXXX`, where the 6 X's are replaced by `::mkdtemp`.
The `SlaveID` subdirectory has also been removed.

This change, while techically a breaking change, is safe because of
how the fetcher uses this directory.  Upon starting up, the fetcher
"recovers" by clearing this directory.  By using a temporary directory,
we similarly get an empty fetcher cache upon startup.

This change will only cause breakages if multiple agents are run
with the same `--fetcher_cache_dir`.  In this case, each agent
will delete the fetcher caches of all the other agents.

---

With the removal of the `SlaveID` field in the fetcher's methods,
it is no longer necessary to pass in the `SlaveID` or agent Flags
at agent recovery time.  Instead, the flags can be passed in during
the fetcher's construction.

Similarly, the fetcher's "recovery" (clearing the fetcher cache)
can be done immediately upon construction, which simplifies the
code slightly.


Diffs (updated)
-----

  src/local/local.cpp e47980929db2da1f31cf899a0e1fc452070e11f3 
  src/slave/containerizer/fetcher.hpp 9e3018dc087ed55c61b2824d0105bc5339b83043 
  src/slave/containerizer/fetcher.cpp a910fea5a5556afb376524c5bb2ff98d7d84e611 
  src/slave/flags.hpp e5784ef81ad0720c7ec061ee0b28b8fadae77afd 
  src/slave/flags.cpp ed99fadbf1aa91f7f0a57c9fb351c0247a40c6f4 
  src/slave/main.cpp 507d59996a90f51c7cd1e0f836a34c2e9c484791 
  src/slave/slave.cpp a37c6c888e7573209aadb07576cfb727fa1ec4ff 


Diff: https://reviews.apache.org/r/58900/diff/2/

Changes: https://reviews.apache.org/r/58900/diff/1-2/


Testing
-------

See last patch in chain.


Thanks,

Joseph Wu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message