From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Zw7Jv2i1; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 920E45A0271 for ; Mon, 08 Dec 2025 10:01:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1765184461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eaJaP+v8SMNine009VBAsrO0pUDyk3JQZAYFA+JBCKY=; b=Zw7Jv2i1Yt0hAKKmLlp/rJ/pUA73XTNnRGDtSMGe1H3Ru8Z/nGMqVm23lbU8dgGir9p6Q8 38xlCIh3RKDGh+0/Yd+dn5bfUDM/iD6SF18ZuoCvHcQwKkHtJMPmCKSPYHpkiqhCUt0Eiv GOeHcjfNbL0eG8N9PhX1iqBtkfcj40Y= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-541-EaIIMQMyNGalLwaIuDwo4g-1; Mon, 08 Dec 2025 04:00:59 -0500 X-MC-Unique: EaIIMQMyNGalLwaIuDwo4g-1 X-Mimecast-MFC-AGG-ID: EaIIMQMyNGalLwaIuDwo4g_1765184459 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-477b8a667bcso54153065e9.2 for ; Mon, 08 Dec 2025 01:00:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765184458; x=1765789258; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eaJaP+v8SMNine009VBAsrO0pUDyk3JQZAYFA+JBCKY=; b=lPwuNpGd1e1aPKkmRls+TgeuLn1V1k046OOCTpy4vQfrgNCNDqDeN5X4tATafE661r uO2bGdeLpVj0+wgCXXssLV/Er2H5L2nOFqnz3b0iFSxWnHX3QLQEP4Lrww7yX+snBLDU AKPr9NDBS46MSEN5Hk70US1IJRNBGtZL+h7GMMyk8AKXtlTSTpm+5rEj9+Ihee++l5tH vsNVnWi9JYyu6dGs9Va9D0Fe1+mEEciShz17sP5ZrJArG1kaVJUFQYynCQuC42e4fRzz lOqRJZ5hR+B91o8JaNSAv1lMqPsBUuX6zVgv2vXys7nYepixJy0q2ktEbbnQDLTzuWyM 8Z5g== X-Gm-Message-State: AOJu0YzmH7w0oHR8epmZdZT9dIU6lkIUPL53W0jFcU1gdrd5/IAv+fET z0e2vTG0OFFp3PY5v62aABiYLdiPevv1jh6IJPk+mtLp/VUcVhV/m7jzXd9kSnCWmQxwgGVZgiR sTIWJIfZMnyJmjgMc1Cdigi16nNEvSsjOj6xN8cPyTOAh78Pm57YN7Q== X-Gm-Gg: ASbGncurZXjjq3JoVZ6ZdLx0SmVF16d8mB4/TVUJODjOfygBsUx/JKX/CgAv9vgffSi kTju9ETqESfEgcJYDB4LaTFBUJWYUL0rCLBFYgyONKxSyp5NWbLJLXz1i3FZ0SxXPFC7/tMC7IS LB74StIY+xkD3KQbaffIRG52NcLsUgBxS5vWphx2Y2cM3FpFtBLVi3LEGUu6sjFYAMd6E5LT/ie uiLqrK87cmI+6FXkM9Fopc+Q1dTFJPHQI9MZXOs/PzEyPmpYM9kIrdFZSnYUgwkHzEY+NngN+jw XA5safFSYLezmnGXUyK4HVZCrkGD2sRxNqHfDp/amAVa5hXVIC6A4LO7K9quW6VRIYoXhuhNCVK V/AjJfLs5S8EtbszQLaNY X-Received: by 2002:a05:600c:3b86:b0:477:5b0a:e616 with SMTP id 5b1f17b1804b1-47939deda5cmr70877695e9.5.1765184458510; Mon, 08 Dec 2025 01:00:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IFuo1gO3vOBtBLq2onvHHOa/g+4xfV9EELJlY1FLsuzmCI0z8GQWt1cS7vUjTq7//22cCndqg== X-Received: by 2002:a05:600c:3b86:b0:477:5b0a:e616 with SMTP id 5b1f17b1804b1-47939deda5cmr70877205e9.5.1765184457894; Mon, 08 Dec 2025 01:00:57 -0800 (PST) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47930ca15adsm236343735e9.13.2025.12.08.01.00.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Dec 2025 01:00:57 -0800 (PST) Date: Mon, 8 Dec 2025 10:00:56 +0100 From: Stefano Brivio To: Max Chernoff Subject: Re: [PATCH v3 00/10] tcp: Fix throughput issues with non-local peers Message-ID: <20251208100056.4d2b024d@elisabeth> In-Reply-To: <6c87ae32518893269fa4a99c120f3990972242f9.camel@maxchernoff.ca> References: <20251208072024.3884137-1-sbrivio@redhat.com> <20251208092530.2638636a@elisabeth> <6c87ae32518893269fa4a99c120f3990972242f9.camel@maxchernoff.ca> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: eDEur6267UCuUeo0u-3iI04s_oOFTkz6B9WONDb-GrA_1765184459 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: Z34CNCIOZ7JIK75YELGHWAQMYUPW4GIF X-Message-ID-Hash: Z34CNCIOZ7JIK75YELGHWAQMYUPW4GIF X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Mon, 08 Dec 2025 01:51:48 -0700 Max Chernoff wrote: > Hi Stefano, > > On Mon, 2025-12-08 at 09:25 +0100, Stefano Brivio wrote: > > but, in general, I don't think things can work reliably > > with the values you shared for tcp_notsent_lowat. > > Ok, that works for me. I know very little about TCP, so I just blindly > copied that value for tcp_notsent_lowat from > > https://blog.cloudflare.com/http-2-prioritization-with-nginx/ Right, I guess you mentioned it already. I still need to find some time to actually look into it in better detail and understand what's still true now (note that that article is from 2018 and there have been very relevant kernel changes against "bufferbloat" in between). At that point: > but if that's incompatible with pasta, then I have no problem resetting > tcp_notsent_lowat back to the kernel default. > > A random web search makes it look like changing tcp_notsent_lowat is > somewhat common > > https://www.google.com/search?q=tcp_notsent_lowat%3D131072 > > https://github.com/search?q=tcp_notsent_lowat%3D131072+NOT+is%3Afork&type=code > > so maybe it would be a good idea for pasta to either use setsockopt to > override it, or to print a warning on startup if the sysctl is set too > low? ...I would also get an opinion on this. I should be able to get back to you in a few days. > > Does this (upload now taking longer/timing out with 50 ms RTT) also > > happen without "custom" values for tcp_notsent_lowat? > > > > I tested things quite extensively in that RTT region (without custom > > sysctl values) and the improvement looks rather consistent to me. > > Ok, with tcp_notsent_lowat reset to the Fedora defaults, the upload > speeds with large RTTs do indeed look *much* better Ok, great, thanks for confirming. That's all I'm trying to fix for the moment. :) -- Stefano