From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=G/4vzo9T; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id CE4D95A0619 for ; Wed, 29 Oct 2025 00:13:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761693221; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QajBn51yC58YZXiVpRh/zTpxRiQBWRXPLqjsnwDflHU=; b=G/4vzo9TeDllRxvluSa+9dCkBnS/xapaXb4rpfFfeiQ+i3/qVSSC0kfcnGw8bh0FUPcknt 1cTFzxrJ3rnJJ67zt5yFQfUZkDeOAyXQg/5mqqdwqp9lzA3VTjbUwkw9WkBsl1B+N/N/3g E5Pk/dd5c8sDnqMp5vfoRFpNe3QBQUU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-527-NbEXnDlANIaT92MROBjRFQ-1; Tue, 28 Oct 2025 19:13:39 -0400 X-MC-Unique: NbEXnDlANIaT92MROBjRFQ-1 X-Mimecast-MFC-AGG-ID: NbEXnDlANIaT92MROBjRFQ_1761693219 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-3f384f10762so3917836f8f.3 for ; Tue, 28 Oct 2025 16:13:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761693219; x=1762298019; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=19LKczvn3bsWncQ9gppirs/+tffOqU9OhAlQLHOqUXQ=; b=qxkp1f98wD/e7Q0df1HnKTA1VqwFg/r4S+cv653CSiUPDpl8klpxOZGQS6N4BXps1g Fe0WtyhIqPVeDKtAHkXCYoMxIIDMtx8ww0L54KZ93Bpv71TEiGCCq/EBoXwXDWgSGqXx pYGVY2f13Vm/6B7iyfDwBuQdROp5ezGQqGYpNsjG4gq/GgOppWt12hpBjMlFRbrwb65O ADPLWoR53htnF3SPgqJ1OTH/NleImXm/IBnPYmQ4jaKPrv0jjABxbs51Q/pyXbZpiZa8 E0BOCHyo1gG4RnjEYfAuUP8x1Ez4ZIakqH7f5TkIKnF0o+Q4AGwM8NXIrJcJTLO39Qp0 PR4g== X-Forwarded-Encrypted: i=1; AJvYcCW5FHt1c0vVVa7wGbKjeZ4MPTzkhV4Ur368Ot+A+ZlC6FrplrvEF4fxy8iA53xurlT27tpH/OmqVw4=@passt.top X-Gm-Message-State: AOJu0YwlEIo4mtpFhBtYyvWQlDdmH+0Ux0xhPrkwnm/lLI8puMeSv/bn bELpcLjZU9NQKAXyd5XWq0ItqlKgCpX4wsn+OB7Kbw+SBTvwQm3bSPz0yEcGJZOv4Tc5Krf35yW YQ1oc1ewSRDWNEXykXFFuuqWH4lmYury9PG/Holzv6HWEmn4y3n6PwQ== X-Gm-Gg: ASbGncvR8nJAFpcboesNru2qGQ4PVrp/X0KNOClJV6KpHD0ZqGrKvVhicKlGOi7NT/w 7AZa91kPwmPiLvUIlayNW6jXDhJ09l9KHKLH9LNG9Ngb+5Bn1x6BxTg96h+9UK+0VUugzbuWfSU uycGTDoyM/CjoYYo05AGdTezYYN/scSbc5cohkeKZQ9CPnSfQum163vZ3JVFcoOdbCSKCo/Wl/Z RbPEjNcCK/8r3sXXjM0Ra+98JdaONi4nxA9glitcYFCLD67ZVlf+PHpmytcFD3GQ9Je527g8mJM kYkzrcgWXUHXeTNL3X0EgAJwoAvJsgqmPG3yjAIExfkOVDUndzy3hYMB/9wlkgEH1fyBaPdcy8/ 0zq+4tQSi5A== X-Received: by 2002:a05:600c:3b07:b0:475:dba3:9ca with SMTP id 5b1f17b1804b1-4771e1f59ccmr7991865e9.39.1761693218540; Tue, 28 Oct 2025 16:13:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IELWNzhxRCuFSG3CJ+0u2ajusYpgVpmsM6avtmq9//g6X2PqGLCsqiNV1s2EWdRZcm13jo10A== X-Received: by 2002:a05:600c:3b07:b0:475:dba3:9ca with SMTP id 5b1f17b1804b1-4771e1f59ccmr7991725e9.39.1761693217990; Tue, 28 Oct 2025 16:13:37 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4771e196a9asm16316555e9.7.2025.10.28.16.13.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Oct 2025 16:13:37 -0700 (PDT) Date: Wed, 29 Oct 2025 00:13:30 +0100 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v3 4/4] tcp: Update data retransmission timeout Message-ID: <20251029001330.579cc85a@elisabeth> In-Reply-To: References: <20251014073836.18150-1-yuhuang@redhat.com> <20251014073836.18150-5-yuhuang@redhat.com> <20251017202812.173e9352@elisabeth> <20251020071107.42fd40e9@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qCCmC9W6KstSj-zF9T25jasaslC2e1KUyxUFGtLuuB0_1761693219 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: GF3I6OOOPHGZ53GYJ2YEXLYMJZGLXT37 X-Message-ID-Hash: GF3I6OOOPHGZ53GYJ2YEXLYMJZGLXT37 X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Yumei Huang , passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Mon, 20 Oct 2025 20:17:10 +1100 David Gibson wrote: > On Mon, Oct 20, 2025 at 07:11:07AM +0200, Stefano Brivio wrote: > > On Mon, 20 Oct 2025 11:20:19 +1100 > > David Gibson wrote: > > =20 > > > On Fri, Oct 17, 2025 at 08:28:12PM +0200, Stefano Brivio wrote: =20 > > > > On Thu, 16 Oct 2025 09:54:25 +1100 > > > > David Gibson wrote: > > > > =20 > > > > > On Wed, Oct 15, 2025 at 02:31:27PM +0800, Yumei Huang wrote: = =20 > > > > > > On Wed, Oct 15, 2025 at 8:05=E2=80=AFAM David Gibson > > > > > > wrote: =20 > > > > > > > > > > > > > > On Tue, Oct 14, 2025 at 03:38:36PM +0800, Yumei Huang wrote: = =20 > > > > > > > > According to RFC 2988 and RFC 6298, we should use an expone= ntial > > > > > > > > backoff timeout for data retransmission starting from one s= econd > > > > > > > > (see Appendix A in RFC 6298), and limit it to about 60 seco= nds > > > > > > > > as allowed by the same RFC: > > > > > > > > > > > > > > > > (2.5) A maximum value MAY be placed on RTO provided it i= s at > > > > > > > > least 60 seconds. =20 > > > > > > > > > > > > > > The interpretation of this isn't entirely clear to me. Does = it mean > > > > > > > if the total retransmit delay exceeds 60s we give up and RST = (what > > > > > > > this patch implements)? Or does it mean that if the retransm= it delay > > > > > > > reaches 60s we keep retransmitting, but don't increase the de= lay any > > > > > > > further? > > > > > > > > > > > > > > Looking at tcp_bound_rto() and related code in the kernel sug= gests the > > > > > > > second interpretation. > > > > > > > =20 > > > > > > > > Combine the macros defining the initial timeout for both SY= N and ACK. > > > > > > > > And add a macro ACK_RETRIES to limit the total timeout to a= bout 60s. > > > > > > > > > > > > > > > > Signed-off-by: Yumei Huang > > > > > > > > --- > > > > > > > > tcp.c | 32 ++++++++++++++++---------------- > > > > > > > > 1 file changed, 16 insertions(+), 16 deletions(-) > > > > > > > > > > > > > > > > diff --git a/tcp.c b/tcp.c > > > > > > > > index 3ce3991..84da069 100644 > > > > > > > > --- a/tcp.c > > > > > > > > +++ b/tcp.c > > > > > > > > @@ -179,16 +179,12 @@ > > > > > > > > * > > > > > > > > * Timeouts are implemented by means of timerfd timers, se= t based on flags: > > > > > > > > * > > > > > > > > - * - SYN_TIMEOUT_INIT: if no ACK is received from tap/gues= t during handshake > > > > > > > > - * (flag ACK_FROM_TAP_DUE without ESTABLISHED event) wit= hin this time, resend > > > > > > > > - * SYN. It's the starting timeout for the first SYN retr= y. If this persists > > > > > > > > - * for more than TCP_MAX_RETRIES or (tcp_syn_retries + > > > > > > > > - * tcp_syn_linear_timeouts) times in a row, reset the co= nnection > > > > > > > > - * > > > > > > > > - * - ACK_TIMEOUT: if no ACK segment was received from tap/= guest, after sending > > > > > > > > - * data (flag ACK_FROM_TAP_DUE with ESTABLISHED event), = re-send data from the > > > > > > > > - * socket and reset sequence to what was acknowledged. I= f this persists for > > > > > > > > - * more than TCP_MAX_RETRIES times in a row, reset the c= onnection > > > > > > > > + * - ACK_TIMEOUT_INIT: if no ACK segment was received from= tap/guest, eiher > > > > > > > > + * during handshake(flag ACK_FROM_TAP_DUE without ESTABL= ISHED event) or after > > > > > > > > + * sending data (flag ACK_FROM_TAP_DUE with ESTABLISHED = event), re-send data > > > > > > > > + * from the socket and reset sequence to what was acknow= ledged. It's the > > > > > > > > + * starting timeout for the first retry. If this persist= s for more than > > > > > > > > + * allowed times in a row, reset the connection > > > > > > > > * > > > > > > > > * - FIN_TIMEOUT: if a FIN segment was sent to tap/guest (= flag ACK_FROM_TAP_DUE > > > > > > > > * with TAP_FIN_SENT event), and no ACK is received with= in this time, reset > > > > > > > > @@ -342,8 +338,7 @@ enum { > > > > > > > > #define WINDOW_DEFAULT 14600 = /* RFC 6928 */ > > > > > > > > > > > > > > > > #define ACK_INTERVAL 10 /* ms= */ > > > > > > > > -#define SYN_TIMEOUT_INIT 1 /* s = */ > > > > > > > > -#define ACK_TIMEOUT 2 > > > > > > > > +#define ACK_TIMEOUT_INIT 1 /* s,= RFC 6298 */ =20 > > > > > > > > > > > > > > I'd suggest calling this RTO_INIT to match the terminology us= ed in the > > > > > > > RFCs. =20 > > > > > >=20 > > > > > > Sure. =20 > > > > > > > =20 > > > > > > > > #define FIN_TIMEOUT 60 > > > > > > > > #define ACT_TIMEOUT 7200 > > > > > > > > > > > > > > > > @@ -352,6 +347,11 @@ enum { > > > > > > > > > > > > > > > > #define ACK_IF_NEEDED 0 /* See tcp_se= nd_flag() */ > > > > > > > > > > > > > > > > +/* Number of retries calculated from the exponential backo= ff formula, limited > > > > > > > > + * by a total timeout of about 60 seconds. > > > > > > > > + */ > > > > > > > > +#define ACK_RETRIES 5 > > > > > > > > + =20 > > > > > > > > > > > > > > As noted above, I think this is based on a misunderstanding o= f what > > > > > > > the RFC is saying. TCP_MAX_RETRIES should be fine as it is, = I think. > > > > > > > We could implement the clamping of the RTO, but it's a "MAY" = in the > > > > > > > RFC, so we don't have to, and I don't really see a strong rea= son to do > > > > > > > so. =20 > > > > > >=20 > > > > > > If we use TCP_MAX_RETRIES and not clamping RTO, the total timeo= ut > > > > > > could be 255 seconds. > > > > > >=20 > > > > > > Stefano mentioned "Retransmitting data after 256 seconds doesn'= t make > > > > > > a lot of sense to me" in the previous comment. =20 > > > > >=20 > > > > > That's true, but it's pretty much true for 60s as well. For the = local > > > > > link we usually have between passt and guest, even 1s is an etern= ity. =20 > > > >=20 > > > > Rather than the local link I was thinking of whatever monitor or > > > > liveness probe in KubeVirt which might have a 60-second period, or = some > > > > firewall agent, or how long it typically takes for guests to stop a= nd > > > > resume again in KubeVirt. =20 > > >=20 > > > Right, I hadn't considered those. Although.. do those actually re-us= e > > > a single connection? I would have guessed they use a new connection > > > each time, making the timeouts here irrelevant. =20 > >=20 > > It depends on the definition of "each time", because we don't time out > > host-side connections immediately. =20 >=20 > Hm, ok. Is your concern that getting a negative answer from the probe > will take too long? More like getting a positive answer taking too long, because we retry so infrequently. > > Pretending passt isn't there, the timeout would come from the default > > values for TCP connections. It looks like there's no specific > > SO_SNDTIMEO value set for those probes, and you can't configure the > > timeout, at least according to: > >=20 > > https://kubernetes.io/docs/tasks/configure-pod-container/configure-li= veness-readiness-startup-probes/#define-a-tcp-liveness-probe =20 >=20 > My guess would be that the probe would probably time out at the > application level long before the TCP layer times out, but I don't > know for sure. I don't think so. What I was pointing out is that I couldn't find any place in the implementation of those probes where a particular *handshake timeout* (not probe interval) is set on top of Linux's defaults, so timeouts at TCP layer and application level should be the same (no additional timeout in application logic). > > and for tcp_syn_retries, tcp(7) says: > >=20 > > The default value is 6, which corresponds to retrying for up to > > approximately 127 seconds. > >=20 > > In this series, to make things transparent, we read out those values, > > so that part is fine. But does the Linux kernel clamp the RTO? > >=20 > > It turns out that yes, it does, TCP_RTO_MAX_SEC is 120 seconds (before > > 1280c26228bd ("tcp: add tcp_rto_max_ms sysctl") that was TCP_RTO_MAX, > > same value), and it's used by tcp_retransmit_timer() via tcp_rto_max(). > > That change makes it configurable. > >=20 > > I'm tempted to suggest that we should read out that value as well > > (with a 120-second fallback for older kernels) to make our behaviour > > as transparent as possible. > >=20 > > It's slightly more complicated and perhaps not strictly needed, but > > we've been bitten a few times by cases where applications and users > > expect us to behave like the Linux kernel, and we didn't... so maybe > > we could do this as well while at it? Given the rest of this series, > > it looks like a relatively small addition to it. =20 >=20 > I think that's a good idea. It's a bit more work, but it doesn't > greatly increase the conceptual complexity and will more closely match > the kernel's behaviour. --=20 Stefano