From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EbG/CeoW; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 34B1C5A027C for ; Wed, 21 May 2025 12:09:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1747822140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K9w4kpQlojzTuvCRogXP1MEu5nwrohK1PItJaU01vSY=; b=EbG/CeoW4cAS76VeXZ/ILZPjtMiScWT50P/BVW7CcuKwqiUQlye0PvtE7dRZtuK+QkNwDk Sapl1zbW/j6AfAHLIqmv7KF3qvpYtNxj4A4ZSH7tZnt2VlbTOW5M2Snp3DUBwWGtGz7R7w NRc0dk/hhdHDwR7p5XNYpjeXrjUmOho= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-91-TqACKbUcP-y0399By-8g5w-1; Wed, 21 May 2025 06:08:59 -0400 X-MC-Unique: TqACKbUcP-y0399By-8g5w-1 X-Mimecast-MFC-AGG-ID: TqACKbUcP-y0399By-8g5w_1747822138 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-43f251dc364so15156605e9.2 for ; Wed, 21 May 2025 03:08:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747822137; x=1748426937; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=K9w4kpQlojzTuvCRogXP1MEu5nwrohK1PItJaU01vSY=; b=kPbuOVCZQDnbXw1XAlnzoec2MT+psZknI21Ri56sgRVMXMQ+dNVmYjVqT4hkyNHuMK +5r4+cx8PtJUyqTlY3knsLVhe9bVc5jZfKG5QUNfVyhk3Uya3nJcBlT79nbZ7BaRrUyu lxCduHE4CXSSvES7yDdxMN508wy3tKMNnM0qg/+Ma8or26RvudACf0uY4+DdMZPXavzs CwSrLJXKXXSTxYF4lk8a4XsOulOCnYxLEwrabu98OBh4WV/VuznL9oYrKp4gWjc0YTwv iyo6OSYma9n8dtF7/3UwdslyE5aoFNeR6iBnvSXvFSRZlhdWRF6AhSx+x1gsVh+Rw4cO sc/Q== X-Gm-Message-State: AOJu0YyC2VekthbOTPT5nztBhRSdqy+/InbsZRnat6/c717qj5L9rAN9 DodrfZkxN5TDjkUGubP2Xx1UuszlqdtzN4sStXH4bBLAOq0qoiObnnS3owVtiy5rUxW2/GHkRnN RD/g4LwLd9PSeMqxnkwVrZB/1VxsfrGztQPfAKB1Y7aoraRPROnSLY+hFvCiUCsDcdKBjLV7BnI /WJLHA+cA6MnnBJ4ekL8Y/Fz6LPqvEzdBdQm9M X-Gm-Gg: ASbGncsy7kuPiHXaiJQjco6kx3X8rOBU7TGsXkPWB+LNxzZMkfJDVPCUOTI5Xzn5JTm XRXAWa7IYtk0/BAMBOCRUSNvxdiRDloNIEIYmkggAFZVbNXyqSg8c6QUNeOaH4RTl1BO2PJoX6p GBAg3/nGtCubOR2cOmK9kEWd28eIrzz/qyefgsr+xvoyjTojiiBrnczKGbQp7qXDZSgM30D/2Zt /mv17c9THk9TqZ0ygkh8O+kQ7tr+OhAqpJhVVtDv9nHXetCjvcrUjrvl3LGPO+92722ePGRJMQs AFLCz8bRxytYlVd0eIwUrNM= X-Received: by 2002:a05:600c:3e85:b0:442:ccfa:1461 with SMTP id 5b1f17b1804b1-442fd6206d8mr199614345e9.13.1747822137418; Wed, 21 May 2025 03:08:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEOZm5LYI272I6Bbk7ig+E//iV7hE1lSI9kOv/71C29M4+wg9ZLGLccHP/EYbOikqGCNL5lKQ== X-Received: by 2002:a05:600c:3e85:b0:442:ccfa:1461 with SMTP id 5b1f17b1804b1-442fd6206d8mr199613835e9.13.1747822136899; Wed, 21 May 2025 03:08:56 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-44804e89c42sm65387845e9.21.2025.05.21.03.08.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 May 2025 03:08:56 -0700 (PDT) Date: Wed, 21 May 2025 12:08:55 +0200 From: Stefano Brivio To: Eugenio Perez Martin Subject: Re: vhost-kernel net on pasta: from 26 to 37Gbit/s Message-ID: <20250521120855.5cdaeb04@elisabeth> In-Reply-To: References: Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: q4Vwt6rs6WO0bvAGyqRox-gtxgvizDRmr2qr9glpUpw_1747822138 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: 76YDZFPKCMS4F4RNMHS7W2ODSSKJG6XT X-Message-ID-Hash: 76YDZFPKCMS4F4RNMHS7W2ODSSKJG6XT X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, Jason Wang , Jeff Nelson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, 20 May 2025 17:09:44 +0200 Eugenio Perez Martin wrote: > [...] > > Now if I isolate the vhost kernel thread [1] I get way more > performance as expected: > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 43.1 GBytes 37.1 Gbits/sec 0 sender > [ 5] 0.00-10.04 sec 43.1 GBytes 36.9 Gbits/sec receiver > > After analyzing perf output, rep_movs_alternative is the most called > function in the three iperf3 (~20%Self), passt.avx2 (~15%Self) and > vhost (~15%Self) Interesting... s/most called function/function using the most cycles/, I suppose. So it looks somewhat similar to https://archives.passt.top/passt-dev/20241017021027.2ac9ea53@elisabeth/ now? > But I don't see any of them consuming 100% of CPU in > top: pasta consumes ~85% %CPU, both iperf3 client and server consumes > 60%, and vhost consumes ~53%. > > So... I have mixed feelings about this :). By "default" it seems to > have less performance, but my test is maybe too synthetic. Well, surely we can't ask Podman users to pin specific stuff to given CPU threads. :) > There is room for improvement with the mentioned optimizations so I'd > continue applying them, continuing with UDP and TCP zerocopy, and > developing zerocopy vhost rx. That definitely makes sense to me. > With these numbers I think the series should not be > merged at the moment. I could send it as RFC if you want but I've not > applied the comments the first one received, POC style :). I don't think it's really needed for you to spend time on semi-polishing something just to have an RFC if you're still working on it. I guess the implementation will change substantially anyway once you factor in further optimisations. -- Stefano