From: Stefano Brivio <sbrivio@redhat.com>
To: David Gibson <david@gibson.dropbear.id.au>
Cc: passt-dev@passt.top
Subject: Re: [PATCH v3 5/9] log: Don't use O_APPEND at all
Date: Wed, 30 Oct 2024 13:27:26 +0100 [thread overview]
Message-ID: <20241030132726.5d07c8ef@elisabeth> (raw)
In-Reply-To: <ZyGbBys9PD_UtxRW@zatzit>
On Wed, 30 Oct 2024 13:33:43 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:
> On Tue, Oct 29, 2024 at 11:23:29AM +0100, Stefano Brivio wrote:
> > On Tue, 29 Oct 2024 20:32:40 +1100
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >
> > > On Tue, Oct 29, 2024 at 09:48:50AM +0100, Stefano Brivio wrote:
> > > > On Tue, 29 Oct 2024 15:20:56 +1100
> > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > >
> > > > > On Mon, Oct 28, 2024 at 11:00:40AM +0100, Stefano Brivio wrote:
> > > > > > We open the log file with O_APPEND, but switch it off before seeking,
> > > > > > and turn it back on afterwards.
> > > > > >
> > > > > > We never seek when O_APPEND is on, so we don't actually need it, as
> > > > > > its only function is to override the offset for writes so that they
> > > > > > are always performed at the end regardless of the current offset
> > > > > > (which is at the end anyway, for us).
> > > > >
> > > > > Sorry, this sounded fishy to me on the call, but I figured I was just
> > > > > missing something. But looking at this the reasoning doesn't make
> > > > > sense to me.
> > > > >
> > > > > We don't seek with O_APPEND, but we do write(), which is exactly where
> > > > > it matters. AIUI the point of O_APPEND is that if you have multiple
> > > > > processes writing to the same file, they won't clobber each others
> > > > > writes because of a stale file pointer.
> > > >
> > > > That's not the reason why I originally added it though: it was there
> > > > because I thought I would lseek() to do the rotation and possibly end
> > > > up with the cursor somewhere before the end. Then restart writing, and
> > > > the write would happen in the middle of the file:
> > >
> > > I don't entirely follow. I see why you disable O_APPEND across the
> > > rotation, but I'm not clear on why it's opened with O_APPEND in the
> > > first place, if it's not for the typical logging reason.
> >
> > I initially opened it with O_APPEND because I _thought_ I would set the
> > offset to a possibly inconsistent value around the rotation.
> >
> > Then I dropped O_APPEND around the rotation, forgetting about the
> > initial reason why I added it at all. So it makes no sense to have
> > O_APPEND at all.
>
> Ok, that makes sense.
>
> Except that maybe there is a reason to use O_APPEND (the multiple
> writer thing), even if it's not the one you thought of initially.
>
> [snip]
> > > > > Of course the rotation process *can* clobber things (which is exactly
> > > > > why I was always a bit sceptical of this "in place" rotation, not that
> > > > > we really have other options).
> > > >
> > > > Why would it clobber things? logfile_rotate_fallocate() and
> > > > logfile_rotate_move() take care of cutting cleanly at a line boundary,
> > > > and tests check that.
> > >
> > > I mean that in the case that there are multiple writers, the rotation
> > > breaks that "no data loss, and probably readable-ish" property of
> > > O_APPEND.
> >
> > Ah, sure. But I think that supporting multiple writers would need more
> > work anyway (at least adding a prefix as you mentioned).
>
> That's fair. I wonder if it might make sense to flock() the logfile,
> to (somewhat) enforce that only one process uses it at a time.
...but if it kind of works for multiple writers, we shouldn't prevent
that usage, right?
On the other hand, I don't think we should try to make that usage all
nice and supported because we would need a prefix, which, in case of a
single writer, just adds noise and size. And I don't think we want to
detect if there are multiple writers...
So, all in all, I would choose to spend no effort and leave like it is,
until somebody comes up with a use case in one direction or the other.
--
Stefano
next prev parent reply other threads:[~2024-10-30 12:27 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-28 10:00 [PATCH v3 0/9] Take care of clang-tidy warnings with LLVM >= 16 Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 1/9] Makefile: Exclude qrap.c from clang-tidy checks Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 2/9] treewide: Comply with CERT C rule ERR33-C for snprintf() Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 3/9] treewide: Silence cert-err33-c clang-tidy warnings for fprintf() Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 4/9] Makefile: Disable readability-math-missing-parentheses clang-tidy check Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 5/9] log: Don't use O_APPEND at all Stefano Brivio
2024-10-29 4:20 ` David Gibson
2024-10-29 8:48 ` Stefano Brivio
2024-10-29 9:32 ` David Gibson
2024-10-29 10:23 ` Stefano Brivio
2024-10-30 2:33 ` David Gibson
2024-10-30 12:27 ` Stefano Brivio [this message]
2024-10-31 0:35 ` David Gibson
2024-10-28 10:00 ` [PATCH v3 6/9] treewide: Suppress clang-tidy warning if we already use O_CLOEXEC or if we can't Stefano Brivio
2024-10-29 4:24 ` David Gibson
2024-10-28 10:00 ` [PATCH v3 7/9] treewide: Address cert-err33-c clang-tidy warnings for clock and timer functions Stefano Brivio
2024-10-29 4:24 ` David Gibson
2024-10-28 10:00 ` [PATCH v3 8/9] udp: Take care of cert-int09-c clang-tidy warning for enum udp_iov_idx Stefano Brivio
2024-10-28 10:00 ` [PATCH v3 9/9] util: Don't use errno after a successful call in __daemon() Stefano Brivio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241030132726.5d07c8ef@elisabeth \
--to=sbrivio@redhat.com \
--cc=david@gibson.dropbear.id.au \
--cc=passt-dev@passt.top \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://passt.top/passt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).