From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XYkmRgh9; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id D57575A0278 for ; Wed, 30 Jul 2025 08:12:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753855919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kJXZYLwY13+ZvoTw2FF7c+4L00kiG9+U3wAQiwoXbiI=; b=XYkmRgh9D4Qb83Xpfd5tDF3HNagqCspVl1kERcQsLFPaUBNT6N325tDu4mhA7U8dz+ANrX mPhwsjeswjKq55iTCdt4OQcM5T8yB3z1dl673qjRr1IDvk0Rhi0FtxfY41Sl6dgEB+/FxS cpx0ly047kFBGOotKicsuglqWzR/fhY= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-554-hpCJzNptNry8mfuwP4iHKw-1; Wed, 30 Jul 2025 02:11:57 -0400 X-MC-Unique: hpCJzNptNry8mfuwP4iHKw-1 X-Mimecast-MFC-AGG-ID: hpCJzNptNry8mfuwP4iHKw_1753855917 Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-31f729bf733so86202a91.1 for ; Tue, 29 Jul 2025 23:11:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753855917; x=1754460717; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kJXZYLwY13+ZvoTw2FF7c+4L00kiG9+U3wAQiwoXbiI=; b=a1OV2uBm5hGv0qWP6IdRDO6ZqFI4s1Ilx5c6XGx1Qq/s2GOLB0tBs9zFhPxFpXIBTS THfYyqAINSahl8N4cyxmQIk9PdrfyfasSmaR9h2hBtrIQTC7w1niJR2qs9Ax9lbhUTTY NNypM4yZEsYNlyI35v8NvXvpY5xbu8ZD+/ivKS04xphWJix0GWPQsxsRDCv9rDNL+fOL CTv6mezH4RD/LlbrElHqihAFwgPjBVvTzfal+nlec2KygYt9wkpkF+SpEFjacwtkqT3f 7EOJs7A0O6Vg1fgQROTtDsIF5T6V14t299a4HLV1ypaSfbx8uMesx4e/STxksA+LRaWA MHCw== X-Gm-Message-State: AOJu0YzirXGRA0/HXDWhWR+OdvRp/wjQK8nko+C18ZY36pm3k2TYK8kS EfjWBQ7CDvogHT7eQdes1mSQ+ocFXvKFD1ds50kMuRo0TBpQkso8kqo7if9NqKT0hluo2SC0FgF MUdCsmuetJSUtEwbrx8W6A3EY1zesP06jASEySSTr+tk3YiQGnnK5slsJwQ1Vw8mfbEG1fXzqZc hQKtACt/pS1UbQ5F12AwpYoQxjkZ0b X-Gm-Gg: ASbGncutvtRyJhzTTmgKHJ3wjMr8YerhK0zyGuCklyFTgT7TV7SEzmdTxJuua3NqxLj SbhV13/bTH6AbwzrBGjymBxM+nMvISR7X3XedGOqHVqzrBiaJ+vtftWbR4re/Xz4HCjLbyhnt+H k5f6qMM+6vP1wf9PbBIcSo X-Received: by 2002:a17:90b:4a49:b0:31f:b51:ef0a with SMTP id 98e67ed59e1d1-31f5de4199emr3153489a91.21.1753855916623; Tue, 29 Jul 2025 23:11:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHbErNF8Dm1/0DyrePcLHR+79FFXtYNIie6KXNY0ELGj7HM+NaDAwhDM+wQmAZUbfrbw4H1ssAFcYVOUL5jJEI= X-Received: by 2002:a17:90b:4a49:b0:31f:b51:ef0a with SMTP id 98e67ed59e1d1-31f5de4199emr3153451a91.21.1753855916105; Tue, 29 Jul 2025 23:11:56 -0700 (PDT) MIME-Version: 1.0 References: <20250709174748.3514693-1-eperezma@redhat.com> <20250709174748.3514693-11-eperezma@redhat.com> In-Reply-To: From: Eugenio Perez Martin Date: Wed, 30 Jul 2025 08:11:20 +0200 X-Gm-Features: Ac12FXwcK0zucMV_lemr8_lXOSsup0I2rf43sRdI4uz0UCKV1hOYWqcttENPIqw Message-ID: Subject: Re: [RFC v2 10/11] tap: add poll(2) to used_idx To: David Gibson X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: AyJj0sotoHttC5F1LhZsTiZk6SbWXXPuhjVPsHc__Vs_1753855917 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Message-ID-Hash: YBXEQFFS6S5WU6RAF5DPVBAS2ZTRKTPY X-Message-ID-Hash: YBXEQFFS6S5WU6RAF5DPVBAS2ZTRKTPY X-MailFrom: eperezma@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, jasowang@redhat.com X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, Jul 30, 2025 at 2:34=E2=80=AFAM David Gibson wrote: > > On Tue, Jul 29, 2025 at 09:04:19AM +0200, Eugenio Perez Martin wrote: > > On Tue, Jul 29, 2025 at 2:33=E2=80=AFAM David Gibson > > wrote: > > > > > > On Mon, Jul 28, 2025 at 07:03:12PM +0200, Eugenio Perez Martin wrote: > > > > On Thu, Jul 24, 2025 at 3:21=E2=80=AFAM David Gibson > > > > wrote: > > > > > > > > > > On Wed, Jul 09, 2025 at 07:47:47PM +0200, Eugenio P=C3=A9rez wrot= e: > > > > > > From ~13Gbit/s to ~11.5Gbit/s. > > > > > > > > > > Again, I really don't know what you're comparing to what here. > > > > > > > > > > > > > When the buffer is full I'm using poll() to wait until vhost free s= ome > > > > buffers, instead of actively checking the used index. This is the c= ost > > > > of the syscall. > > > > > > Ah, right. So.. I'm not sure if it's so much the cost of the syscall > > > itself, as the fact that you're actively waiting for free buffers, > > > rather than returning to the main epoll loop so you can maybe make > > > progress on something else before returning to the Tx path. > > > > > > > Previous patch also wait for free buffers, but it does it burning a > > CPU for that. > > Ah, ok. Hrm. I still find it hard to believe that it's the cost of > the syscall per se that's causing the slowdown. My guess is that the > cost is because having the poll() leads to a higher latency between > the buffer being released and us detecting it and re-using. > > > The next patch is the one that allows to continue progress as long as > > there are enough free buffers, instead of always wait until all the > > buffer has been sent. But there are situations where this conversion > > needs other code changes. In particular, all the calls to > > tcp_payload_flush after checking that we have enough buffers like: > > > > if (tcp_payload_sock_used > TCP_FRAMES_MEM - 2) { > > tcp_buf_free_old_tap_xmit(c, 2); > > tcp_payload_flush(c); > > ... > > } > > > > Seems like coroutines would be a good fix here, but maybe there are > > simpler ways to go back to the main loop while keeping the tcp socket > > "ready to read" by epoll POV. Out of curiosity, what do you think > > about setjmp()? :). > > I think it has its uses, but deciding to go with it is a big > architectural decision not to be entered into likely. > Got it, Another idea is to add the flows that are being processed but they had no space available into the virtqueue to a "pending" list. When the kernel tells pasta that new buffers are available, pasta checks that pending list. Maybe it can consist of only one element.