From reviews-return-82973-apmail-mesos-reviews-archive=mesos.apache.org@mesos.apache.org Mon Oct 22 20:29:45 2018 Return-Path: X-Original-To: apmail-mesos-reviews-archive@minotaur.apache.org Delivered-To: apmail-mesos-reviews-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 08F9718875 for ; Mon, 22 Oct 2018 20:29:45 +0000 (UTC) Received: (qmail 50021 invoked by uid 500); 22 Oct 2018 20:29:44 -0000 Delivered-To: apmail-mesos-reviews-archive@mesos.apache.org Received: (qmail 49984 invoked by uid 500); 22 Oct 2018 20:29:44 -0000 Mailing-List: contact reviews-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: reviews@mesos.apache.org Delivered-To: mailing list reviews@mesos.apache.org Received: (qmail 49973 invoked by uid 99); 22 Oct 2018 20:29:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Oct 2018 20:29:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id D73C71A3BBC; Mon, 22 Oct 2018 20:29:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.7 X-Spam-Level: X-Spam-Status: No, score=0.7 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id mXzF3nFiA7r8; Mon, 22 Oct 2018 20:29:42 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 719775F230; Mon, 22 Oct 2018 20:29:42 +0000 (UTC) Received: from reviews.apache.org (unknown [10.41.0.12]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 2160EE00B4; Mon, 22 Oct 2018 20:29:42 +0000 (UTC) Received: from reviews-vm2.apache.org (localhost [IPv6:::1]) by reviews.apache.org (ASF Mail Server at reviews-vm2.apache.org) with ESMTP id E70C7C400DB; Mon, 22 Oct 2018 20:29:41 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============0861916152016457822==" MIME-Version: 1.0 Subject: Re: Review Request 69085: Stout: Always `fsync` created directories in POSIX `mkdir`. From: Chun-Hung Hsiao To: Benjamin Bannier , Jie Yu , Jan Schlicht Cc: Chun-Hung Hsiao , mesos Date: Mon, 22 Oct 2018 20:29:41 -0000 Message-ID: <20181022202941.48648.94193@reviews-vm2.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: Chun-Hung Hsiao X-ReviewGroup: mesos X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/69085/ X-Sender: Chun-Hung Hsiao References: <20181022180547.48649.59011@reviews-vm2.apache.org> In-Reply-To: <20181022180547.48649.59011@reviews-vm2.apache.org> Reply-To: Chun-Hung Hsiao X-ReviewRequest-Repository: mesos --===============0861916152016457822== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit > On Oct. 22, 2018, 6:05 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/posix/mkdir.hpp > > Lines 45 (patched) > > > > > > I wonder if this has issues with symlinks. > > > > Imagine e.g., that we have a directory `/dir` which is symlinked as `/symlink`. > > > > * If we call `os::mkdir("/symlink/subdir")`, this code would just `fsync` `/symlink`, but never `/dir`. > > * While above might be fixable, the situation becomes worse should a user `os::mkdir("/dir/subdir")` where we might need to `fsync` `/symlink`. It looks like this might be hard, http://austingroupbugs.net/view.php?id=672. Maybe there's a way to address this outside of POSIX, e.g., for Linux. Actually, the `os::open` call in `os::fsync` will follow the symlink (since no `O_NOFOLLOW` is specified), so the problem does not exist. See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html > On Oct. 22, 2018, 6:05 p.m., Benjamin Bannier wrote: > > 3rdparty/stout/include/stout/os/posix/mkdir.hpp > > Lines 75 (patched) > > > > > > I am not sure we should abort on the first error here. Many documented failure scenarios of `::fsync` are either very unlikley in this scope or retryable. > > > > To get to the error code we might need to invoke `::fsync` directly. It seems to me that we should abort on any of the errors listed in http://pubs.opengroup.org/onlinepubs/9699919799/functions/fsync.html except for `EBADF`, which cannot happen. - Chun-Hung ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69085/#review209889 ----------------------------------------------------------- On Oct. 19, 2018, 5:54 p.m., Chun-Hung Hsiao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69085/ > ----------------------------------------------------------- > > (Updated Oct. 19, 2018, 5:54 p.m.) > > > Review request for mesos, Benjamin Bannier, Jie Yu, and Jan Schlicht. > > > Bugs: MESOS-9281 > https://issues.apache.org/jira/browse/MESOS-9281 > > > Repository: mesos > > > Description > ------- > > To ensure the directories created by `mkdir` are commited to their > filesystems, an `fsync` will be called on the parent of each created > directory. > > > Diffs > ----- > > 3rdparty/stout/include/stout/os/posix/mkdir.hpp 418db9af310ed763a5ae4735c2ebdd1ca38738ba > > > Diff: https://reviews.apache.org/r/69085/diff/1/ > > > Testing > ------- > > make check > > > Thanks, > > Chun-Hung Hsiao > > --===============0861916152016457822==--