From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=S0aRdG3C; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 3373D5A0619 for ; Tue, 07 Oct 2025 00:32:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1759789945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WRvSPTir1mNbYW8YlG8fXRUncInJsIwBNod+2f7Mg6s=; b=S0aRdG3Cq19iygQgL+hThUp9iJoNSkR4yJxjxW3CPIpx62Ir5KMqNvIoIb3wHNbymGvmMU qmjJstzBu7cXi1pcxmeVC/a7Y/JHMhd/78758YAdIN7KIQtDAqt8gfPfVqfcS+t3DyBACV bhv3RT3jKB2RHanN8ZDfF6U/woMvY4g= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-421-uh6sEaS8O4KXd4unTQ7x7Q-1; Mon, 06 Oct 2025 18:32:23 -0400 X-MC-Unique: uh6sEaS8O4KXd4unTQ7x7Q-1 X-Mimecast-MFC-AGG-ID: uh6sEaS8O4KXd4unTQ7x7Q_1759789943 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-3efe4fcc9ccso2938251f8f.3 for ; Mon, 06 Oct 2025 15:32:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759789942; x=1760394742; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=WRvSPTir1mNbYW8YlG8fXRUncInJsIwBNod+2f7Mg6s=; b=M/QuUtFh9CXEBmIYV3T06k31KSRJd1BQ+4a+LNsWArlszLcih4D/kq2zsnAhUDuzFK udKHldopsrR85XspKSaZ1EyqFtfxNcJKeGXopZY0J3NT4SzSEmMQpIxwQZxNM6ZbjjqP qZwPQThlWVFCzcc+MQdWc8trpu0/9aidMQ83Mz5WSDzo3waBpdAfD/xJQ09FdaQT2pPx Ge/Ug7l0mwFM6tk7t+DwDkKBCj3vm/fqulH+82gWtktaAZz5+Vy/1JuzgwPmxs8sAsn9 F06FJvl8WIn1dJfMIQTHfkGcsxWvoHh7FIzKrTRXVg1IMV8rSpzJSIhdu0AOdmzn7CDe lMNA== X-Gm-Message-State: AOJu0YyAd0ZmXMNFvzZS8eka1UHCVigEfEc5ZOCfqBgyElRXG0EAmunJ 5Bj1KDSVd37zQMnQ1ogJhhT2yueJsLv4B5d3jWtLi6OO3KVRuU4gToR+Ihqkr0bd1eX1zrmuBpe dMcgulDLF2g0r7js7XMgP/6AMrXxOc2asVr8hhSztAsxCN9YNbhhHaeCEl9IKiw== X-Gm-Gg: ASbGnctboU7aWm7Gn2bda6Uaq2RM13WmKiS1dHYi8ms0frK8rhfjFD6jCpKRN9aOdfV Yj8oFYi4zGdE7U/B6JUw/GoUEXLm7gO/ymDS28tq1ypP8e3aHN3PzBdVDwU92PNVk+qhGj6AC30 MoEfv82ex0np/aGLpBboD3AIuNF3x5NkQRXRv7hCGBLlL64hS8KPe+qKrLkAocGJ26ONTaqqznf wOftQ5OCuoQvKhXBu254JEUkPJLBXhQVKc2DDS93kkdbXUriQnFpl/j9Tt8kjP2uOgeZKJOlynT YC87rkPsLaaR5gob8YB3kpgckE5/KmK6niPn0sxf6b7try435ZvPtNL/7nkJjahShZYomUd7Kg= = X-Received: by 2002:a05:6000:2c11:b0:3ec:d7c4:25b6 with SMTP id ffacd0b85a97d-4256713ff18mr10494598f8f.27.1759789942097; Mon, 06 Oct 2025 15:32:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFdkIGDaagIn4fwSykKypUE3zY1xtSBH5DI3c5q+xFcfzDytyyq/YlMNHQ9EAn4py557I2RBA== X-Received: by 2002:a05:6000:2c11:b0:3ec:d7c4:25b6 with SMTP id ffacd0b85a97d-4256713ff18mr10494594f8f.27.1759789941678; Mon, 06 Oct 2025 15:32:21 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [176.103.220.4]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4255d8f027csm22897275f8f.40.2025.10.06.15.32.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Oct 2025 15:32:21 -0700 (PDT) Date: Tue, 7 Oct 2025 00:32:19 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH 1/4] tcp: Fix ACK sequence on FIN to tap Message-ID: <20251007003219.3f286b1d@elisabeth> In-Reply-To: References: <20251002000646.2136202-1-sbrivio@redhat.com> <20251002000646.2136202-2-sbrivio@redhat.com> <20251002135841.112eb4d3@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: iN3gnJVis1_SHW8pRYTiAPrjQawayM2xYoFhvynUTj8_1759789943 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: 4GDKLIOF75JVBYKFNOXRFAO3PNXVR6VW X-Message-ID-Hash: 4GDKLIOF75JVBYKFNOXRFAO3PNXVR6VW X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri, 3 Oct 2025 13:19:17 +1000 David Gibson wrote: > On Thu, Oct 02, 2025 at 01:58:41PM +0200, Stefano Brivio wrote: > > On Thu, 2 Oct 2025 12:41:08 +1000 > > David Gibson wrote: > > > > > On Thu, Oct 02, 2025 at 02:06:43AM +0200, Stefano Brivio wrote: > > > > If we reach end-of-file on a socket (or get EPOLLRDHUP / EPOLLHUP) and > > > > send a FIN segment to the guest / container acknowledging a sequence > > > > number that's behind what we received so far, we won't have any > > > > further trigger to send an updated ACK segment, as we are now > > > > switching the epoll socket monitoring to edge-triggered mode. > > > > > > > > To avoid this situation, in tcp_update_seqack_wnd(), we set the next > > > > acknowledgement sequence to the current observed sequence, regardless > > > > of what was acknowledged socket-side. > > > > > > To double check my understanding: things should work if we always > > > acknowledged everything we've received. Acknowledging only what the > > > peer has acked is a refinement to give the guest a view that's closer > > > to what it would be end-to-end with the peer (which might improve the > > > operation of flow control). > > > > Right. > > > > > We can't use that refined mechanism when the socket is closing > > > (amongst other cases), because while we can get the peer acked bytes > > > from TCP_INFO, we can't get events when that changes, so we have no > > > mechanism to provide updates to the guest at the right time. So we > > > fall back to the simpler method. > > > > > > Is that correct? > > > > Also correct, yes. If you have a better idea to summarise this in the > > comment in tcp_buf_data_from_sock() let me know. > > Hm, I might. Or actually a way to reorganise the code that I think > will be a bit clearer and probably allow a clearer comment too. I would keep reworks for a later moment. Right now, it's already taking me long enough to find a moment to investigate these issues, write these fixes, and test them. > > Maybe I could mention > > EPOLLET explicitly there? > > I don't think EPOLLET is actually relevant. Even if we had level > triggered events, a change in bytes_acked doesn't count as an event > (AFAIK). It doesn't count, but with level-triggered events, we would be busy polling bytes_acked, as you noted. I was mentioning EPOLLET because it could be taken, intuitively, as a "stop listening for events" (almost) step. I'll leave that out then. > So either some other event is on, in which case we'd > effectively be busy polling bytes_acked, or it's not in which case we > don't get updates, just like now. > > I principle we could implement some sort of timer based polling, but > that sounds like way more trouble than it's worth. We already have something similar, based on ACK_INTERVAL and ACK_TO_TAP_DUE, and it shouldn't be overly complicated to extend that to a new FIN_TO_TAP_DUE flag. But indeed beyond the scope of this series. > > > > However, we don't necessarily call tcp_update_seqack_wnd() before > > > > sending the FIN segment, which might potentially lead to a situation, > > > > not observed in practice, where we unnecessarily cause a > > > > retransmission at some point after our FIN segment. > > > > > > > > Avoid that by setting the ACK sequence to whatever we received from > > > > the container / guest, before sending a FIN segment and switching to > > > > EPOLLET. > > > > > > > > Signed-off-by: Stefano Brivio > > > > > > Based on my understanding above, this looks correct to me, so, > > > > > > Reviewed-by: David Gibson > > > > > > My only concern is whether we could instead insert an extra call to > > > tcp_update_seqack_wnd() to reduce duplicated logic. > > > > Hmm, maybe, but on the other hand we're closing the connection. Should > > we really spend time querying TCP_INFO to recalculate the window at > > this point? I wouldn't. > > Good point. I mean tcp_update_seqack_wnd() could skip the TCP_INFO in > that case, but that does look a bit fiddly. > > On the other hand, in favour of not duplicating logic... > > [snip] > > > > @@ -368,7 +368,19 @@ int tcp_buf_data_from_sock(const struct ctx *c, struct tcp_tap_conn *conn) > > > > conn_flag(c, conn, STALLED); > > > > } else if ((conn->events & (SOCK_FIN_RCVD | TAP_FIN_SENT)) == > > > > SOCK_FIN_RCVD) { > > > > - int ret = tcp_buf_send_flag(c, conn, FIN | ACK); > > > > + int ret; > > > > + > > > > + /* On TAP_FIN_SENT, we won't get further data events > > > > + * from the socket, and this might be the last ACK > > > > + * segment we send to the tap, so update its sequence to > > > > + * include everything we received until now. > > > > + * > > > > + * See also the special handling on CONN_IS_CLOSING() in > > > > + * tcp_update_seqack_wnd(). > > > > + */ > > > > + conn->seq_ack_to_tap = conn->seq_from_tap; > > ... the equivalent bits in tcp_update_seqack_wnd() have after them: > if (SEQ_LT(conn->seq_ack_to_tap, prev_ack_to_tap)) > conn->seq_ack_to_tap = prev_ack_to_tap; > > Don't we need that here as well, in case the guest is retransmitting > when we get the sock side FIN? Not really, because we don't rewind conn->seq_from_tap, so we don't risk jumping back here. In tcp_update_seqack_wnd(), we might jump back (that should be double checked eventually, I'm not sure it's still the case) if we happened to acknowledge more than acknowledged socket-side while handling some particular condition, and then we switch back to acknowledging only bytes_acked. It should happen if the destination is/was a low RTT one, but we run out of slots in low_rtt_dst in favour of other entries. I don't remember any other case. > > > > + > > > > + ret = tcp_buf_send_flag(c, conn, FIN | ACK); > > > > if (ret) { > > > > tcp_rst(c, conn); > > > > return ret; > > > > diff --git a/tcp_vu.c b/tcp_vu.c > > > > index ebd3a1e..3ec3538 100644 > > > > --- a/tcp_vu.c > > > > +++ b/tcp_vu.c > > > > @@ -410,7 +410,12 @@ int tcp_vu_data_from_sock(const struct ctx *c, struct tcp_tap_conn *conn) > > > > conn_flag(c, conn, STALLED); > > > > } else if ((conn->events & (SOCK_FIN_RCVD | TAP_FIN_SENT)) == > > > > SOCK_FIN_RCVD) { > > > > - int ret = tcp_vu_send_flag(c, conn, FIN | ACK); > > > > + int ret; > > > > + > > > > + /* See tcp_buf_data_from_sock() */ > > > > + conn->seq_ack_to_tap = conn->seq_from_tap; > > > > + > > > > + ret = tcp_vu_send_flag(c, conn, FIN | ACK); > > > > if (ret) { > > > > tcp_rst(c, conn); > > > > return ret; > > > > -- > > > > 2.43.0 > > > > -- Stefano