From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bUr2nUQ/; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTP id 593AF5A004C for ; Tue, 29 Oct 2024 09:48:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730191736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zZscqR/7lQL4sks+ctYlUf/APnRWvHvEaCT5X22Epy4=; b=bUr2nUQ/pxzqXZywb/VQkvvP0Xa49fWQLzTOEP2b5YiHxHcHH2R0te+jIUyabkQbR3mkoX uGA0KtLYJIWTzeJ6eiWkvNENsqJ46UzHApQofd+mR7XiPsP59JgDDiVdzOgGwDNYOjziZ+ V+UoUaFCmYcFBvJ+7LI1EOJcvRPTGtM= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-31-s-WdItXZPJSrswfN96FH4A-1; Tue, 29 Oct 2024 04:48:54 -0400 X-MC-Unique: s-WdItXZPJSrswfN96FH4A-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-37d4922d8c7so2889693f8f.1 for ; Tue, 29 Oct 2024 01:48:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730191733; x=1730796533; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zZscqR/7lQL4sks+ctYlUf/APnRWvHvEaCT5X22Epy4=; b=IaZBDz8C0O9pkpcslCiKb6MU1gnpVphjS216VguVWIdBP5NGqpMailDtY1KNOd1u7C dzgNcLW8lT6uAq6O1mjrZ2Zx1YuYpbTaUqVj+ISxCNLPk8WgOlPA3ku2Q+DY2Dp/S109 VUF6FkDtzb/S4TWUhe9lC94IdFSXLpBoxP9SM48cIguSHWUDrdDpnVQehc+41SZCr96U jymZ/b+4wnfWsw6XfUGTeBSt6bh2lSqAo5kOcG3L9yH2F8FajG6zQrH5DdEGZxiT21Fy h3xKBNOXuD0j3VvgzLfWf7yXGr0CCO2NouSgcwirIczAZVNX4cELPIIhdqOoRMlCPlCX rf4Q== X-Gm-Message-State: AOJu0Yx758DRpWN1tcVMkYo9C1UEN8xgPnX1QlKm5fL4ltZmJhTzozff 3sKkELFQEeeMdukjn2fniRnYKQmKsmd2x9gQQj7U+FtJpT3COvo1nBBQqpbvR7z9GMpKuQNDSnh TN/Hd+X6h2+RgSoCPLHGnfOJ7L44EePM4AaE3CRNr1jczEzCsfg== X-Received: by 2002:a5d:42c9:0:b0:37c:cfbb:d356 with SMTP id ffacd0b85a97d-380611593c5mr7616986f8f.28.1730191733491; Tue, 29 Oct 2024 01:48:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGbMqocu10e4VY0doo5aUBXy2MltplPIs3XwqkqHbj1bluze+4yxrsQHrK2hj5JnnYqwns1XA== X-Received: by 2002:a5d:42c9:0:b0:37c:cfbb:d356 with SMTP id ffacd0b85a97d-380611593c5mr7616972f8f.28.1730191732869; Tue, 29 Oct 2024 01:48:52 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38058b3bb57sm11835194f8f.38.2024.10.29.01.48.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Oct 2024 01:48:52 -0700 (PDT) Date: Tue, 29 Oct 2024 09:48:50 +0100 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v3 5/9] log: Don't use O_APPEND at all Message-ID: <20241029094850.206c06bc@elisabeth> In-Reply-To: References: <20241028100044.939714-1-sbrivio@redhat.com> <20241028100044.939714-6-sbrivio@redhat.com> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: MNRZFOWGD6MXDDAS5ZAIZSRRAUFSBL4I X-Message-ID-Hash: MNRZFOWGD6MXDDAS5ZAIZSRRAUFSBL4I X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, 29 Oct 2024 15:20:56 +1100 David Gibson wrote: > On Mon, Oct 28, 2024 at 11:00:40AM +0100, Stefano Brivio wrote: > > We open the log file with O_APPEND, but switch it off before seeking, > > and turn it back on afterwards. > > > > We never seek when O_APPEND is on, so we don't actually need it, as > > its only function is to override the offset for writes so that they > > are always performed at the end regardless of the current offset > > (which is at the end anyway, for us). > > Sorry, this sounded fishy to me on the call, but I figured I was just > missing something. But looking at this the reasoning doesn't make > sense to me. > > We don't seek with O_APPEND, but we do write(), which is exactly where > it matters. AIUI the point of O_APPEND is that if you have multiple > processes writing to the same file, they won't clobber each others > writes because of a stale file pointer. That's not the reason why I originally added it though: it was there because I thought I would lseek() to do the rotation and possibly end up with the cursor somewhere before the end. Then restart writing, and the write would happen in the middle of the file: $ cat append.c #include #include #include #include int main(int argc, char **argv) { int flags = O_CREAT | O_TRUNC | O_WRONLY | ((argc == 3) ? O_APPEND : 0); int fd = open(argv[1], flags, S_IRUSR | S_IWUSR); char buf[BUFSIZ]; memset(buf, 'a', BUFSIZ); write(fd, buf, 10); lseek(fd, 1, SEEK_SET); memset(buf, 'b', BUFSIZ); write(fd, buf, 10); write(fd, (char *){ "\n" }, 1); return 0; } $ gcc -o append{,.c} $ ./append test append $ cat test aaaaaaaaaabbbbbbbbbb $ ./append test $ cat test abbbbbbbbbb > That's usually not > _necessary_ for us as such, but it's perhaps valuable since it reduces > the likelihood of data loss if somehow you do get two instances > logging to the same file. The result will be completely unreadable anyway, so I don't think it matters for us. > Of course the rotation process *can* clobber things (which is exactly > why I was always a bit sceptical of this "in place" rotation, not that > we really have other options). Why would it clobber things? logfile_rotate_fallocate() and logfile_rotate_move() take care of cutting cleanly at a line boundary, and tests check that. -- Stefano