From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=maxchernoff.ca Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=maxchernoff.ca header.i=@maxchernoff.ca header.a=rsa-sha256 header.s=key1 header.b=LnZHEmGC; dkim-atps=neutral Received: from out-183.mta0.migadu.com (out-183.mta0.migadu.com [IPv6:2001:41d0:1004:224b::b7]) by passt.top (Postfix) with ESMTPS id 4A32A5A0271 for ; Mon, 08 Dec 2025 09:51:53 +0100 (CET) Message-ID: <6c87ae32518893269fa4a99c120f3990972242f9.camel@maxchernoff.ca> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=maxchernoff.ca; s=key1; t=1765183912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jdXFp7X2oWP6t0d2f0li3VPpOCl27l8vxd+9LmiYjLk=; b=LnZHEmGC8FbFu2sGEO5I46ZQBxeT5R1n1Uklyhrpnq2PdMQNO8YF9NjenPgyEJFe1fmMxQ TBBay1yGqv9fk/LfV+hHinv1GX68aLMRP984lZAFgrzWdy8jayNMeOU5Xok+0/OWCYiLaq zXWuUBYWl6B3PwRRPMbcn2zxvHk7ZaQRFfW1fvalMWZBLpFjynUHffn9dExc3D4mltH68B c0qHQhS1ymwLvJqjoCNHZ1PeOPyLtDq/XG9TMeBo2bmDkHxaGhUqALEiUoubfVB637NXEB rlTWOIbyyDwKGf8YZ1+EpUz0+QtIRLGgdNnNzomIKLI/yQSTdwsjZkoyo9YF8w== Subject: Re: [PATCH v3 00/10] tcp: Fix throughput issues with non-local peers X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Max Chernoff To: Stefano Brivio Date: Mon, 08 Dec 2025 01:51:48 -0700 In-Reply-To: <20251208092530.2638636a@elisabeth> References: <20251208072024.3884137-1-sbrivio@redhat.com> <20251208092530.2638636a@elisabeth> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Message-ID-Hash: OV2T5P6FLWNM7PAOBM5QLWQJLK2MAVLH X-Message-ID-Hash: OV2T5P6FLWNM7PAOBM5QLWQJLK2MAVLH X-MailFrom: git@maxchernoff.ca X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi Stefano, On Mon, 2025-12-08 at 09:25 +0100, Stefano Brivio wrote: > but, in general, I don't think things can work reliably > with the values you shared for tcp_notsent_lowat. Ok, that works for me. I know very little about TCP, so I just blindly copied that value for tcp_notsent_lowat from https://blog.cloudflare.com/http-2-prioritization-with-nginx/ but if that's incompatible with pasta, then I have no problem resetting tcp_notsent_lowat back to the kernel default. A random web search makes it look like changing tcp_notsent_lowat is somewhat common https://www.google.com/search?q=3Dtcp_notsent_lowat%3D131072 https://github.com/search?q=3Dtcp_notsent_lowat%3D131072+NOT+is%3Afork&= type=3Dcode so maybe it would be a good idea for pasta to either use setsockopt to override it, or to print a warning on startup if the sysctl is set too low? > Does this (upload now taking longer/timing out with 50 ms RTT) also > happen without "custom" values for tcp_notsent_lowat? > > I tested things quite extensively in that RTT region (without custom > sysctl values) and the improvement looks rather consistent to me. Ok, with tcp_notsent_lowat reset to the Fedora defaults, the upload speeds with large RTTs do indeed look *much* better $ sudo dnf install $ pasta --version pasta 0^20251208.g5943ea4-1.fc43.x86_64 $ ./pasta-upload-test.sh network ping_time wmem_max rmem_max tcp_notsent_low= at tcp_congestion_control default_qdisc download_time uploa= d_time host 50ms custom custom default custom custom 1.561761 = 2.045501 pasta 50ms custom custom default custom custom 1.575290 = 1.707500 host 170ms custom custom default custom custom 9.147689 = 3.220591 pasta 170ms custom custom default custom custom 13.351799 = 3.411078 $ sudo dnf downgrade pasta $ pasta --version # Back to the stock pasta in F43 pasta 0^20250919.g623dbf6-1.fc43.x86_64 $ ./pasta-upload-test.sh network ping_time wmem_max rmem_max tcp_notsent_low= at tcp_congestion_control default_qdisc download_time uploa= d_time host 50ms custom custom default custom custom 1.429540 = 1.674165 pasta 50ms custom custom default custom custom 1.503907 = 2.025471 host 170ms custom custom default custom custom 8.891267 = 3.039416 pasta 170ms custom custom default custom custom 11.056843 = 18.704653 Thanks again for all your help, -- Max