mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jie Yu <yujie....@gmail.com>
Subject Re: Review Request 69009: Stout: Added a sync option for `write` and `rename`.
Date Mon, 15 Oct 2018 22:25:06 GMT


> On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote:
> > 3rdparty/stout/include/stout/os/posix/rename.hpp
> > Lines 41 (patched)
> > <https://reviews.apache.org/r/69009/diff/2/?file=2097010#file2097010line41>
> >
> >     While POSIX guarantees that `rename` e.g., does not see inconsistent state,
there is nothing preventing `to` from being deleted once we execute the conditional code here.
> >     
> >     Since it is hard to know what semantics users expect _in general_, it might
make more sense to not add the sync behavior to `rename`, but to e.g., ask users to perform
`fsync` themself.
> 
> Chun-Hung Hsiao wrote:
>     I considered this before. But `fsync`ing the directory is a POSIX-specific thing,
then the caller (e.g. `slave.cpp`) needs to write two snippets for POSIX and Windows. Adding
the `sync` option seems a better option for making the code cross-platform.
>     
>     Also I don't understand your concern about the deletion. Can you elaborate?
> 
> Benjamin Bannier wrote:
>     Re:deletion, imagine us working in a temp dir which is aggressively garbage-collected.
We `::rename` `/tmp/mesos/a` to `/tmp/mesos/b`, but before we `os::fsync` the directory `/tmp/mesos`
gets garbage-collected; this would make the `os::open` fail, even though the `::rename` finished
successfully.
>     
>     It is not immediately obvious to me when such a scenario is a failure when `sync=true`,
which lead me to suggest to let callers handle the `sync` (we could introduce a helper `sync(const&
Path)` in addition to `sync(int_fd)`.

I also prefer introducing sync behavior in rename, for two reasons:
1) consistent with windows API
2) force the caller to think if `sync` is needed or not. If we ask callers to do sync as a
second step, my worry is that most developers will forget.


> On Oct. 15, 2018, 2:21 p.m., Benjamin Bannier wrote:
> > 3rdparty/stout/include/stout/os/write.hpp
> > Lines 119 (patched)
> > <https://reviews.apache.org/r/69009/diff/2/?file=2097012#file2097012line119>
> >
> >     I personally find flag arguments hard to read (`write(.., .., false)` or `write(..,
.., true)` are not obvious). I'd suggest to either introduce a dedicated `write_sync` function
or let callers trigger the `fsync`.
> 
> Chun-Hung Hsiao wrote:
>     I agree that flag arguments sometimes are not easy to use but in this case it seems
fine to me. Let me also discuss this with Jie.

I think this is fine and is consistent with others. I hope C++ language can address this (similar
to python, groovy that allow .sync= true clause for function parameters)


- Jie


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69009/#review209538
-----------------------------------------------------------


On Oct. 12, 2018, 11:53 p.m., Chun-Hung Hsiao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69009/
> -----------------------------------------------------------
> 
> (Updated Oct. 12, 2018, 11:53 p.m.)
> 
> 
> Review request for mesos, Andrew Schwartzmeyer, Benjamin Bannier, Jie Yu, and Jan Schlicht.
> 
> 
> Bugs: MESOS-9281
>     https://issues.apache.org/jira/browse/MESOS-9281
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This patch adds an option for the caller to sync the file and directory
> contents to the disk after a write to prevent filesystem inconsistency
> against reboots.
> 
> 
> Diffs
> -----
> 
>   3rdparty/stout/include/stout/os/posix/rename.hpp 9cff6db16c8a4e5a79bf0082e18a1133bd287796

>   3rdparty/stout/include/stout/os/windows/rename.hpp 523912ac3bf315f70f542e8eab7d2d02249909b4

>   3rdparty/stout/include/stout/os/write.hpp cd35285a9595004bac627abf687588050aef8161

>   3rdparty/stout/include/stout/protobuf.hpp 1d03e1e3a8dd642f7239d777fb04759caf100a8b

> 
> 
> Diff: https://reviews.apache.org/r/69009/diff/2/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message