From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202410 header.b=kQbEGAGA; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 92BA55A004E for ; Thu, 31 Oct 2024 01:38:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202410; t=1730335068; bh=zkGmfTnfYB5sfMY9KZIxg4/v4uI4tuN7rXKe43ddkcA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=kQbEGAGAu6yuH/QSzi3G9pdAtyMj11e3zfskQT7BJJ53Tu0c9ct0o8UzZB2otwITH kqFKFWm1zvpa2+rWjuDh0Iw8nd33g2huI+KX3yvVGrYyC85Ni5mYz9zfZO2LmYf7Tf vyMasUDz5V9FQWwj+Bi0Hrw8b/lSOaTWPAfP+QuaKYsCJ04Llxmb8pvlRn5hAKMO4q 2qiGkPoGLBMBW3hMp5CARlXAeYOBhAzBnTcNrDnOhv8gkuXXb7h9FGHybFqlQc64CL mnIaICbog3iOD9yzkiWZqGsW+h4Aex0rrP5f/0J5pGeKE/Wfw05laM0XhkVOjX/sJo 2vX2eedNI6WwA== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4Xf4pN3hxzz4xP1; Thu, 31 Oct 2024 11:37:48 +1100 (AEDT) Date: Thu, 31 Oct 2024 11:35:02 +1100 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v3 5/9] log: Don't use O_APPEND at all Message-ID: References: <20241028100044.939714-1-sbrivio@redhat.com> <20241028100044.939714-6-sbrivio@redhat.com> <20241029094850.206c06bc@elisabeth> <20241029112329.25f2503b@elisabeth> <20241030132726.5d07c8ef@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="e4pse+M51r5hlzg7" Content-Disposition: inline In-Reply-To: <20241030132726.5d07c8ef@elisabeth> Message-ID-Hash: 2IQ26DA6EIL7Z7IAT3XBDU632GLRA4C2 X-Message-ID-Hash: 2IQ26DA6EIL7Z7IAT3XBDU632GLRA4C2 X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --e4pse+M51r5hlzg7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 30, 2024 at 01:27:26PM +0100, Stefano Brivio wrote: > On Wed, 30 Oct 2024 13:33:43 +1100 > David Gibson wrote: >=20 > > On Tue, Oct 29, 2024 at 11:23:29AM +0100, Stefano Brivio wrote: > > > On Tue, 29 Oct 2024 20:32:40 +1100 > > > David Gibson wrote: > > > =20 > > > > On Tue, Oct 29, 2024 at 09:48:50AM +0100, Stefano Brivio wrote: =20 > > > > > On Tue, 29 Oct 2024 15:20:56 +1100 > > > > > David Gibson wrote: > > > > > =20 > > > > > > On Mon, Oct 28, 2024 at 11:00:40AM +0100, Stefano Brivio wrote:= =20 > > > > > > > We open the log file with O_APPEND, but switch it off before = seeking, > > > > > > > and turn it back on afterwards. > > > > > > >=20 > > > > > > > We never seek when O_APPEND is on, so we don't actually need = it, as > > > > > > > its only function is to override the offset for writes so tha= t they > > > > > > > are always performed at the end regardless of the current off= set > > > > > > > (which is at the end anyway, for us). =20 > > > > > >=20 > > > > > > Sorry, this sounded fishy to me on the call, but I figured I wa= s just > > > > > > missing something. But looking at this the reasoning doesn't m= ake > > > > > > sense to me. > > > > > >=20 > > > > > > We don't seek with O_APPEND, but we do write(), which is exactl= y where > > > > > > it matters. AIUI the point of O_APPEND is that if you have mult= iple > > > > > > processes writing to the same file, they won't clobber each oth= ers > > > > > > writes because of a stale file pointer. =20 > > > > >=20 > > > > > That's not the reason why I originally added it though: it was th= ere > > > > > because I thought I would lseek() to do the rotation and possibly= end > > > > > up with the cursor somewhere before the end. Then restart writing= , and > > > > > the write would happen in the middle of the file: =20 > > > >=20 > > > > I don't entirely follow. I see why you disable O_APPEND across the > > > > rotation, but I'm not clear on why it's opened with O_APPEND in the > > > > first place, if it's not for the typical logging reason. =20 > > >=20 > > > I initially opened it with O_APPEND because I _thought_ I would set t= he > > > offset to a possibly inconsistent value around the rotation. > > >=20 > > > Then I dropped O_APPEND around the rotation, forgetting about the > > > initial reason why I added it at all. So it makes no sense to have > > > O_APPEND at all. =20 > >=20 > > Ok, that makes sense. > >=20 > > Except that maybe there is a reason to use O_APPEND (the multiple > > writer thing), even if it's not the one you thought of initially. > >=20 > > [snip] > > > > > > Of course the rotation process *can* clobber things (which is e= xactly > > > > > > why I was always a bit sceptical of this "in place" rotation, n= ot that > > > > > > we really have other options). =20 > > > > >=20 > > > > > Why would it clobber things? logfile_rotate_fallocate() and > > > > > logfile_rotate_move() take care of cutting cleanly at a line boun= dary, > > > > > and tests check that. =20 > > > >=20 > > > > I mean that in the case that there are multiple writers, the rotati= on > > > > breaks that "no data loss, and probably readable-ish" property of > > > > O_APPEND. =20 > > >=20 > > > Ah, sure. But I think that supporting multiple writers would need more > > > work anyway (at least adding a prefix as you mentioned). =20 > >=20 > > That's fair. I wonder if it might make sense to flock() the logfile, > > to (somewhat) enforce that only one process uses it at a time. >=20 > ...but if it kind of works for multiple writers, we shouldn't prevent > that usage, right? >=20 > On the other hand, I don't think we should try to make that usage all > nice and supported because we would need a prefix, which, in case of a > single writer, just adds noise and size. And I don't think we want to > detect if there are multiple writers... >=20 > So, all in all, I would choose to spend no effort and leave like it is, > until somebody comes up with a use case in one direction or the other. Yeah, that's fair. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --e4pse+M51r5hlzg7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmci0LUACgkQzQJF27ox 2Gez9RAAmvae75pbGz7nhLgQLsgE7bH3XoSw6ipR2IjewRG3KHIVQ82LS368c1jE 0NKMNP2DN87AR6z9mT59XhJkHzXWY4gxA0AdNsXRaakYHlg04PI6s/Y9Yuv74tLw AUEjre5IRfmZtOU9tPohMRAN71nNmFBSrfOFrHEXJ9v3L3xr748MZEdGumSyjNEY InR8153urFQ2Js0E5AmEBUHpXyTvWVJl6b25m4+m570+Wnma1LBk9Yj8SWxBqfow WxNNaTMZMX+10vwYNh9+1WT4x2TIToBi0Dsr27hhJljkXVxuv0Oe2SwaeZ3TzCDJ 2FddPosXJNXbxWEGQR6aBTlnoYIzJrfpemenVuQRxt1wlkVUwnbwMhC52GlNOhSn o4AwQK898WgDpqmK5PKaMaN874V1HwJcCe10YJXl2gifduRDAarbCTV4PYIrVQOd skyKWBy2d3Y9RikYlhbf19unrIPOaP/lEOaMw2utJcquwqP4ktao0Pt4Q22OYDUm xuHiG+w1p3WsSoUc0VYUSzED+defyXg44Kt59igZuQtFJCp0Bse5Erx0euaIXpeI k6cqV0psHhL6T7bUalLc95l9gE1N/M9xggAVxZEUDL3mdAwJ2sydZfpQBYN1WTLM Khh0PJyQUypseGRqzTN/UCZIuOoY7YYAutgzDXNKAIDLoYL5oHM= =3gMJ -----END PGP SIGNATURE----- --e4pse+M51r5hlzg7--