mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chun-Hung Hsiao <chhs...@apache.org>
Subject Re: Review Request 69085: Stout: Always `fsync` created directories in POSIX `mkdir`.
Date Mon, 22 Oct 2018 20:29:41 GMT


> On Oct. 22, 2018, 6:05 p.m., Benjamin Bannier wrote:
> > 3rdparty/stout/include/stout/os/posix/mkdir.hpp
> > Lines 45 (patched)
> > <https://reviews.apache.org/r/69085/diff/1/?file=2100406#file2100406line46>
> >
> >     I wonder if this has issues with symlinks.
> >     
> >     Imagine e.g., that we have a directory `/dir` which is symlinked as `/symlink`.
> >     
> >     * If we call `os::mkdir("/symlink/subdir")`, this code would just `fsync` `/symlink`,
but never `/dir`.
> >     * While above might be fixable, the situation becomes worse should a user `os::mkdir("/dir/subdir")`
where we might need to `fsync` `/symlink`. It looks like this might be hard, http://austingroupbugs.net/view.php?id=672.
Maybe there's a way to address this outside of POSIX, e.g., for Linux.

Actually, the `os::open` call in `os::fsync` will follow the symlink (since no `O_NOFOLLOW`
is specified), so the problem does not exist. See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html


> On Oct. 22, 2018, 6:05 p.m., Benjamin Bannier wrote:
> > 3rdparty/stout/include/stout/os/posix/mkdir.hpp
> > Lines 75 (patched)
> > <https://reviews.apache.org/r/69085/diff/1/?file=2100406#file2100406line76>
> >
> >     I am not sure we should abort on the first error here. Many documented failure
scenarios of `::fsync` are either very unlikley in this scope or retryable.
> >     
> >     To get to the error code we might need to invoke `::fsync` directly.

It seems to me that we should abort on any of the errors listed in http://pubs.opengroup.org/onlinepubs/9699919799/functions/fsync.html
except for `EBADF`, which cannot happen.


- Chun-Hung


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69085/#review209889
-----------------------------------------------------------


On Oct. 19, 2018, 5:54 p.m., Chun-Hung Hsiao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69085/
> -----------------------------------------------------------
> 
> (Updated Oct. 19, 2018, 5:54 p.m.)
> 
> 
> Review request for mesos, Benjamin Bannier, Jie Yu, and Jan Schlicht.
> 
> 
> Bugs: MESOS-9281
>     https://issues.apache.org/jira/browse/MESOS-9281
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> To ensure the directories created by `mkdir` are commited to their
> filesystems, an `fsync` will be called on the parent of each created
> directory.
> 
> 
> Diffs
> -----
> 
>   3rdparty/stout/include/stout/os/posix/mkdir.hpp 418db9af310ed763a5ae4735c2ebdd1ca38738ba

> 
> 
> Diff: https://reviews.apache.org/r/69085/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Chun-Hung Hsiao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message