Re: [PATCH v3 00/10] tcp: Fix throughput issues with non-local peers

[Max Chernoff <git@maxchernoff.ca>]

Hi Stefano,

On Mon, 2025-12-08 at 09:25 +0100, Stefano Brivio wrote:
> but, in general, I don't think things can work reliably
> with the values you shared for tcp_notsent_lowat.

Ok, that works for me. I know very little about TCP, so I just blindly
copied that value for tcp_notsent_lowat from

    https://blog.cloudflare.com/http-2-prioritization-with-nginx/

but if that's incompatible with pasta, then I have no problem resetting
tcp_notsent_lowat back to the kernel default.

A random web search makes it look like changing tcp_notsent_lowat is
somewhat common

    https://www.google.com/search?q=tcp_notsent_lowat%3D131072

    https://github.com/search?q=tcp_notsent_lowat%3D131072+NOT+is%3Afork&type=code

so maybe it would be a good idea for pasta to either use setsockopt to
override it, or to print a warning on startup if the sysctl is set too
low?

> Does this (upload now taking longer/timing out with 50 ms RTT) also
> happen without "custom" values for tcp_notsent_lowat?
>
> I tested things quite extensively in that RTT region (without custom
> sysctl values) and the improvement looks rather consistent to me.

Ok, with tcp_notsent_lowat reset to the Fedora defaults, the upload
speeds with large RTTs do indeed look *much* better

    $ sudo dnf install <freshly built pasta rpms>
    $ pasta --version
    pasta 0^20251208.g5943ea4-1.fc43.x86_64

    $ ./pasta-upload-test.sh
    network ping_time       wmem_max        rmem_max        tcp_notsent_lowat       tcp_congestion_control  default_qdisc        download_time   upload_time
    host    50ms    custom  custom  default  custom  custom  1.561761        2.045501
    pasta   50ms    custom  custom  default  custom  custom  1.575290        1.707500
    host    170ms   custom  custom  default  custom  custom  9.147689        3.220591
    pasta   170ms   custom  custom  default  custom  custom  13.351799       3.411078

    $ sudo dnf downgrade pasta
    $ pasta --version  # Back to the stock pasta in F43
    pasta 0^20250919.g623dbf6-1.fc43.x86_64

    $ ./pasta-upload-test.sh
    network ping_time       wmem_max        rmem_max        tcp_notsent_lowat       tcp_congestion_control  default_qdisc        download_time   upload_time
    host    50ms    custom  custom  default  custom  custom  1.429540        1.674165
    pasta   50ms    custom  custom  default  custom  custom  1.503907        2.025471
    host    170ms   custom  custom  default  custom  custom  8.891267        3.039416
    pasta   170ms   custom  custom  default  custom  custom  11.056843       18.704653

Thanks again for all your help,
-- Max