From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YMZLh5Gy; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id CE3D25A0619 for ; Tue, 28 Oct 2025 12:44:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1761651868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yJ8KNqhOl6OG4R7DbSPqbc5yZCytx/YNUneDI6IAOzY=; b=YMZLh5GyfDxoIm9Ke4iKLb0eYoT+i/jzU+8YWQKF/L3zrJigmYSCbzfoUlHaHwB+mGpwMG GaT97Ji1Q4WgEDPiN4t6GucqHc0Ct0Qiw46cdpEnlaqZlStQygMsnr1fzKJwXW3ePTD4PC rZc0kwPHZuCOgzfoRNBgTdefYUk54eg= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-345-1Ubo8SRgPLSuAf3MqNvfuw-1; Tue, 28 Oct 2025 07:44:27 -0400 X-MC-Unique: 1Ubo8SRgPLSuAf3MqNvfuw-1 X-Mimecast-MFC-AGG-ID: 1Ubo8SRgPLSuAf3MqNvfuw_1761651866 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4298955a6d9so4285225f8f.0 for ; Tue, 28 Oct 2025 04:44:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761651866; x=1762256666; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CKfUYdlSynjeC/G9ujJ7QdEV2wtI7t/LIFWBMYgbZLA=; b=ip5xA9pT/1o64fWU+5hNCyWE9hbOibanSdprp1I4SSpoRhYF61/PkQvsfxHRAHfGL+ vRA+ErccZi2nDhEE1IbuY3Fk8zY+oqk7aF6/ziC8NNRloi8L2aJnV9FUMvMGay2XZk1R RCGZ0uOxF+ExpTfoqYVMwRe+eETdAVsHLsJ6e3S2BYuBQuRMuh6giS+RMQ6Kp35qqVlv 9SjsRdWRxEIIl9HiRJh0gnLemfkPMFMykyMjxluOCEpke70m9UkzqU9h6ZqSEroFcK7z i7PVLT2+oZAoW4A2f0QcAAOLBhdG69NdUrOEZdUkSt6z3C+GihLUx6Pgpv/T6HqD/xIO Qe0g== X-Gm-Message-State: AOJu0YzsybFyVqxvjta1aSLfXbuoeYOJf1dPFvrDZ6ry0VKo3IZ8JelD QuQHsF55Sq8GY2jAvZiKT2JS/V6OFUJziaaMgBonoNNuO+JacSKlXBeJ+Ok/SItbOf3fTkhRvvO 9OHUHH85pcYvQ3p7z0KKxO4fX1JF5KTDhMVFgPcgCVDVubQXqx78cQQ== X-Gm-Gg: ASbGncsN0ohwP5RwYd3scTBEQ8TbSge+CxOR/DkJiDm6q6xpZT7UdQxcpR72MngW+7T YZUyE0ZdUZK4VOAMLIwZoRp9YsVLFVnyi6Os+LFLv8ui0bZL4avtDdKhZGt6r3pbFr5ga5x3wWB U6VWXmHzXk/GIbtxC0qgW7Iq+m17mTamnLISwSD1vLDTSuCvEmzD4XmsES2u/d9FxprzuSw6IHv DeQZAeP7EbYxqesc+cqK+GCGR4tNugJFm2Se2+VJHVjtMWA/hKKBYQxOVuRL503jX2ZKnghFyut 3szVN6U6WoQ6QLflE5PvGy2aNGLr92TlXfB5DJDByyUe1ly6VJODgUCE33pv/WgjgESpaUpShjY mZBQPqXv6cQ== X-Received: by 2002:a05:6000:647:b0:429:8bd7:774a with SMTP id ffacd0b85a97d-429a7e8204dmr2347789f8f.40.1761651865743; Tue, 28 Oct 2025 04:44:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFmo5mYh15NGklWAqBYWVBJdWXrJpSh2uXS6L1cHF6bB7DzAec0c8uYbreVg5y+HpMveZh36w== X-Received: by 2002:a05:6000:647:b0:429:8bd7:774a with SMTP id ffacd0b85a97d-429a7e8204dmr2347758f8f.40.1761651865122; Tue, 28 Oct 2025 04:44:25 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-429952cb7dcsm20668335f8f.11.2025.10.28.04.44.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Oct 2025 04:44:24 -0700 (PDT) Date: Tue, 28 Oct 2025 12:44:21 +0100 From: Stefano Brivio To: Yumei Huang Subject: Re: [PATCH v6 3/4] tcp: Resend SYN for inbound connections Message-ID: <20251028124421.5e49bfc8@elisabeth> In-Reply-To: References: <20251017062838.21041-1-yuhuang@redhat.com> <20251017062838.21041-4-yuhuang@redhat.com> <20251024010431.4329a843@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: lIYCJJXVwE2ilIdpb0abcnjWtdxi6WA5ue65-Lmf3W0_1761651866 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: ALLK3URVY2CUN77V4EEPRIBXXP44UM6K X-Message-ID-Hash: ALLK3URVY2CUN77V4EEPRIBXXP44UM6K X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, david@gibson.dropbear.id.au X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, 28 Oct 2025 15:43:25 +0800 Yumei Huang wrote: > On Fri, Oct 24, 2025 at 7:04=E2=80=AFAM Stefano Brivio wrote: > > > > On Fri, 17 Oct 2025 14:28:37 +0800 > > Yumei Huang wrote: > > =20 > > > If a client connects while guest is not connected or ready yet, > > > resend SYN instead of just resetting connection after 10 seconds. > > > > > > Use the same backoff calculation for the timeout as linux kernel. =20 > > > > Linux. > > =20 > > > > > > Link: https://bugs.passt.top/show_bug.cgi?id=3D153 > > > Signed-off-by: Yumei Huang > > > --- > > > tcp.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++-------- > > > tcp.h | 5 +++++ > > > 2 files changed, 52 insertions(+), 8 deletions(-) > > > > > > diff --git a/tcp.c b/tcp.c > > > index 2ec4b0c..9385132 100644 > > > --- a/tcp.c > > > +++ b/tcp.c > > > @@ -179,9 +179,11 @@ > > > * > > > * Timeouts are implemented by means of timerfd timers, set based on= flags: > > > * > > > - * - SYN_TIMEOUT: if no ACK is received from tap/guest during handsh= ake (flag > > > - * ACK_FROM_TAP_DUE without ESTABLISHED event) within this time, r= eset the > > > - * connection > > > + * - SYN_TIMEOUT_INIT: if no ACK is received from tap/guest during h= andshake > > > + * (flag ACK_FROM_TAP_DUE without ESTABLISHED event) within this t= ime, resend > > > + * SYN. It's the starting timeout for the first SYN retry. If this= persists =20 > > > > "If this persists" makes sense for the existing ACK_TIMEOUT > > description but not here, because it looks like it refers to "starting > > timeout". > > > > Coupled with the next patch, it becomes increasingly difficult to > > understand what "this" persisting thing is. > > > > Maybe directly say "Retry for ..., then reset the connection"? It's > > shorter and clearer. > > =20 > > > + * for more than TCP_MAX_RETRIES or (tcp_syn_retries + > > > + * tcp_syn_linear_timeouts) times in a row, reset the connection > > > * > > > * - ACK_TIMEOUT: if no ACK segment was received from tap/guest, aft= er sending > > > * data (flag ACK_FROM_TAP_DUE with ESTABLISHED event), re-send da= ta from the > > > @@ -340,7 +342,7 @@ enum { > > > #define WINDOW_DEFAULT 14600 /* RFC = 6928 */ > > > > > > #define ACK_INTERVAL 10 /* ms */ > > > -#define SYN_TIMEOUT 10 /* s */ > > > +#define SYN_TIMEOUT_INIT 1 /* s */ =20 > > > > Maybe mention RFC 6928 as done above? That's where this value comes > > from. > > > > I just noticed you do that in 4/4, so it's slightly nicer if you do > > that right away here for ease of future reference, but not really > > needed. > > =20 > > > #define ACK_TIMEOUT 2 > > > #define FIN_TIMEOUT 60 > > > #define ACT_TIMEOUT 7200 > > > @@ -365,6 +367,9 @@ uint8_t tcp_migrate_rcv_queue [TCP_MI= GRATE_RCV_QUEUE_MAX]; > > > > > > #define TCP_MIGRATE_RESTORE_CHUNK_MIN 1024 /* Try smaller whe= n above this */ > > > > > > +#define TCP_SYN_RETRIES "/proc/sys/net/ipv4/tcp_syn_ret= ries" > > > +#define TCP_SYN_LINEAR_TIMEOUTS "/proc/sys/net/ipv4/tcp_syn_lin= ear_timeouts" \ =20 > > > > This uses 121 columns. I'm not sure where all those tabs and \ come > > from. =20 >=20 > My bad. I renamed the macro and it fits in 80 columns but I forgot to > remove the '\' and the spaces before that. > > =20 > > > + > > > /* "Extended" data (not stored in the flow table) for TCP flow migra= tion */ > > > static struct tcp_tap_transfer_ext migrate_ext[FLOW_MAX]; > > > > > > @@ -581,8 +586,13 @@ static void tcp_timer_ctl(const struct ctx *c, s= truct tcp_tap_conn *conn) > > > if (conn->flags & ACK_TO_TAP_DUE) { > > > it.it_value.tv_nsec =3D (long)ACK_INTERVAL * 1000 * 100= 0; > > > } else if (conn->flags & ACK_FROM_TAP_DUE) { > > > - if (!(conn->events & ESTABLISHED)) > > > - it.it_value.tv_sec =3D SYN_TIMEOUT; > > > + if (!(conn->events & ESTABLISHED)) { > > > + if (conn->retries < c->tcp.syn_linear_timeouts) > > > + it.it_value.tv_sec =3D SYN_TIMEOUT_INIT= ; > > > + else > > > + it.it_value.tv_sec =3D SYN_TIMEOUT_INIT= << > > > + (conn->retries - c->tcp.syn_lin= ear_timeouts); =20 > > > > Probably more readable, but I haven't tried: always start from > > SYN_TIMEOUT_INIT, then multiply/shift if > > conn->retries >=3D c->tcp.syn_linear_timeouts. =20 >=20 > I guess you meant: >=20 > if (!(conn->events & ESTABLISHED)) { > it.it_value.tv_sec =3D SYN_TIMEOUT_INIT; > if (conn->retries >=3D c->tcp.syn_linear_timeouts) > it.it_value.tv_sec =3D SYN_TIMEOUT_INIT << > (conn->retries - c->tcp.syn_linear_timeouts); Well, yes, this, but without repeating the assignment, so <<=3D (conn->retries - c->tcp.syn_linear_timeouts). > other than something like: >=20 > it.it_value.tv_sec <<=3D 1 Hmm, no, why 1? > in above second if block, since the latter would cause > it.it_value.tv_sec=3D2 always when retries>=3Dsyn_linear_timeouts, as > it.it_value.tv_sec is set to SYN_TIMEOUT_INIT before the if condition. >=20 > If I'm correct, I'm not sure if it's more readable to change to that. > I feel the if/else clause quite matches the way the kernel handles it. > What do you think? Sorry, I took this for granted, but I rather meant: =09=09=09it.it_value.tv_sec =3D SYN_TIMEOUT_INIT; =09=09=09if (conn->retries >=3D c->tcp.syn_linear_timeouts) { =09=09=09=09it.it_value.tv_sec <<=3D =09=09=09=09=09=09(conn->retries - =09=09=09=09=09=09 c->tcp.syn_linear_timeouts); =09=09=09} because in both cases we're starting from SYN_TIMEOUT_INIT. And now that I wrote that, I'll pretend I actually meant this: =09=09if (!(conn->events & ESTABLISHED)) { =09=09=09int factor =3D conn->retries - c->tcp.syn_linear_timeouts; =09=09=09it.it_value.tv_sec =3D SYN_TIMEOUT_INIT << MAX(factor, 0); =09=09} else { =09=09=09... =09=09} ...where we make the conditional implicit (but I think intuitive) by adding 0 as a lower bound. Am I missing something...? > > > + } > > > else > > > it.it_value.tv_sec =3D ACK_TIMEOUT; > > > } else if (CONN_HAS(conn, SOCK_FIN_SENT | TAP_FIN_ACKED)) { > > > @@ -2409,8 +2419,17 @@ void tcp_timer_handler(const struct ctx *c, un= ion epoll_ref ref) > > > tcp_timer_ctl(c, conn); > > > } else if (conn->flags & ACK_FROM_TAP_DUE) { > > > if (!(conn->events & ESTABLISHED)) { > > > - flow_dbg(conn, "handshake timeout"); > > > - tcp_rst(c, conn); > > > + if (conn->retries >=3D TCP_MAX_RETRIES || > > > + conn->retries >=3D (c->tcp.tcp_syn_retries = + > > > + c->tcp.syn_linear_timeout= s)) { > > > + flow_dbg(conn, "handshake timeout"); > > > + tcp_rst(c, conn); > > > + } else { > > > + flow_trace(conn, "SYN timeout, retry"); > > > + tcp_send_flag(c, conn, SYN); > > > + conn->retries++; =20 > > > > I think I already raised this point on a previous revision: this needs > > to be zeroed as the connection is established, but I don't see that in > > the current version. > > =20 > > > + tcp_timer_ctl(c, conn); > > > + } > > > } else if (CONN_HAS(conn, SOCK_FIN_SENT | TAP_FIN_ACKED= )) { > > > flow_dbg(conn, "FIN timeout"); > > > tcp_rst(c, conn); > > > @@ -2766,6 +2785,24 @@ static socklen_t tcp_probe_tcp_info(void) > > > return sl; > > > } > > > > > > +/** > > > + * tcp_syn_params_init() - Get initial SYN parameters for inbound co= nnection =20 > > > > They're not initial, they'll be used for all the connections if I > > understand correctly. > > > > Maybe "Get SYN retries sysctl values"? > > I think the _init() in the > > function name is also somewhat misleading. =20 >=20 > Do you have a better idea for the function name? I named it just like > other functions called in tcp_init(), such as tcp_sock_iov_init(), > tcp_splice_init(). Given that David was suggesting "Get host kernel RTO parameters" as a comment to this function, maybe tcp_get_rto_params()? The only similar function we have is tcp_get_sndbuf(), so I would make it consistent with that. The most peculiar task of this function is that it gets parameters ("get"), I think, rather than assigning them to variables ("init"). > > > + * @c: Execution context > > > +*/ > > > +void tcp_syn_params_init(struct ctx *c) > > > +{ > > > + intmax_t tcp_syn_retries, syn_linear_timeouts; > > > + > > > + tcp_syn_retries =3D read_file_integer(TCP_SYN_RETRIES, 8); =20 > > > > Why 8? Perhaps a #define would help? =20 >=20 > The 8 actually comes from your suggestion in v1: > " > (optional, and might be in another series, but keeping it together > with the rest might be more convenient): read tcp_syn_retries, limit > to 8, and also read tcp_syn_linear_timeouts > " That was a *limit* of 8, coming from the fact that we have 3 bits to store the retry count. This is a default instead. > I guess it can be replaced with 6 as it's the default value according to > Documentation/networking/ip-sysctl.rst. I'd rather check the code because sometimes people forget to update that documentation but yes, it matches: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/i= nclude/net/tcp.h?id=3Da7556779745c047efb7b0ce8732889b0cdc80936#n125 > We already have TCP_SYN_RETRIES for the sysctl file, maybe have > another TCP_SYN_RETRIES_DEFAULT for the number? It sounds reasonable to me. > > > + syn_linear_timeouts =3D read_file_integer(TCP_SYN_LINEAR_TIMEOU= TS, 1); =20 >=20 > The default value is 4 as Documentation/networking/ip-sysctl.rst. I > don't quite remember where the 1 is from. Should we update it as well > and add the similar macro as above? I would, yes. By the way, it's hardcoded here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/n= et/ipv4/tcp_ipv4.c?id=3De97269257fe437910cddc7c642a636ca3cf9fb1d#n3669 but defined here for thin streams (connections where observed throughput is low: https://docs.kernel.org/networking/tcp-thin.html): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/i= nclude/net/tcp.h?id=3Da7556779745c047efb7b0ce8732889b0cdc80936#n266 I'd suggest to define a constant with value '4' and use it. > > > + c->tcp.tcp_syn_retries =3D MIN(tcp_syn_retries, UINT8_MAX); > > > + c->tcp.syn_linear_timeouts =3D MIN(syn_linear_timeouts, UINT8_M= AX); > > > + > > > + debug("TCP SYN parameters: retries=3D%"PRIu8", linear_timeouts= =3D%"PRIu8, =20 > > > > Similar to the comment above: these are not parameters of SYN segments > > (which would seem to imply TCP options, such as the MSS). > > > > We typically don't print C assignments, rather human-readable messages, > > so that could be "Read sysctl values tcp_syn_retries: ..., > > syn_linear_timeouts: ...". =20 >=20 > Got it. > > > > =20 > > > + c->tcp.tcp_syn_retries, c->tcp.syn_linear_timeouts); > > > +} > > > + > > > /** > > > * tcp_init() - Get initial sequence, hash secret, initialise per-so= cket data > > > * @c: Execution context > > > @@ -2776,6 +2813,8 @@ int tcp_init(struct ctx *c) > > > { > > > ASSERT(!c->no_tcp); > > > > > > + tcp_syn_params_init(c); > > > + > > > tcp_sock_iov_init(c); > > > > > > memset(init_sock_pool4, 0xff, sizeof(init_sock_pool4)= ); > > > diff --git a/tcp.h b/tcp.h > > > index 234a803..4369b52 100644 > > > --- a/tcp.h > > > +++ b/tcp.h > > > @@ -59,12 +59,17 @@ union tcp_listen_epoll_ref { > > > * @fwd_out: Port forwarding configuration for outbound pack= ets > > > * @timer_run: Timestamp of most recent timer run > > > * @pipe_size: Size of pipes for spliced connections > > > + * @tcp_syn_retries: Number of SYN retries during handshake > > > + * @syn_linear_timeouts: Number of SYN retries using linear backoff = timeout > > > + * before switching to exponential backoff timeo= ut =20 > > > > Maybe more compact: > > > > * @syn_linear_timeouts: SYN retries before using exponential ti= meout =20 >=20 > So we remove the '"Number of" as well? I guess we should make it > consistent for both tcp_syn_retries and syn_linear_timeouts. Ah, yes, we should. I mean, I guess it's clear that we're talking about the count / number of those and not their colour or taste... > > =20 > > > */ > > > struct tcp_ctx { > > > struct fwd_ports fwd_in; > > > struct fwd_ports fwd_out; > > > struct timespec timer_run; > > > size_t pipe_size; > > > + uint8_t tcp_syn_retries; > > > + uint8_t syn_linear_timeouts; > > > }; > > > --=20 Stefano