From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Vl0St3HS; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id D848B5A0265 for ; Wed, 20 May 2026 15:46:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779284812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=zhQ43m+q28YH6yTvWImBsStmJuFeq/0Qy+YKsoHR9g4=; b=Vl0St3HSuIPFziY9aAaf23TdZFgveF/c+wYT7dKT01b3bl0QVFgXkcYSnvomjjOvM/p+BA 1L4SnyM/3dBiLU3UJOJ+sACh/ioWhmd31t0HJaByoBy6b3QPDcETlF0aDYsH2IHWg6gST/ QDBUP9HABIcJXbcWSI1NwK6/7AA/Wjc= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-433-g__zyffeNWGdW85I_3KxCA-1; Wed, 20 May 2026 09:46:51 -0400 X-MC-Unique: g__zyffeNWGdW85I_3KxCA-1 X-Mimecast-MFC-AGG-ID: g__zyffeNWGdW85I_3KxCA_1779284810 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43d7b7bacddso3690280f8f.0 for ; Wed, 20 May 2026 06:46:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779284810; x=1779889610; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zhQ43m+q28YH6yTvWImBsStmJuFeq/0Qy+YKsoHR9g4=; b=KTWesq86C7+1467dpdTQDNEn/7nPO+KKQjSGU478XaJP5gvuodft62ZfIiLxU64UmV nrPZVdgCE5GQjVr5m7zwNQX1Nk0C2y2+EHG2FwhI9J27HmYVH0jhvNaQ/YQoERLfn1up +CQFpBUTIqayKpPVXv1aa8JdC4oufBc22LdDT/6YdeSSxbVbX7DECsYTc6fuGwGxnrRR gVPL7VLOHCJNYbxdm8TRxfopGlredr9zdQhgGqBDKoVbEV76weYhJ0odXazRakz4pWKF 3m4CwFIsj9+420xs7fuawNSLS4jkM9xd8bj4Rp5DP7H37C9Y02Vt7hJ9EXQ2XBj3ggx2 O4Og== X-Gm-Message-State: AOJu0Yx1JIa9gVn/DsAjPgxl9wP8f/Scz8hVZOXHbRueR51EWemzxWQN fWzllBhG8uOFaW4HuY0pP+n+zqOTbWN8/5/hgwykZccNoEEwJ0ddPmZllunZzaOA/TBvsKk55LF wkcVLkRmgd6NMy8/DhPkqGQm6P1/kdwyhYs0Y8dnC31ZLF0nxMroLXnjvyQuN8Q== X-Gm-Gg: Acq92OFX3HYr5WHSvCLSG1jgc3FyIB7Lt53SZmKLhLkr7OxY3rPhLyOjEAw3YiMoCyA XG0TeOHw1V4X4AB3mtzoY79O0MXPLKPXpElZPviUMko6W/y6x4KXC/ggC26ZSs0QMqobm8Mi4iv blIz4CxenduKLFl3t2drT+UX2i20XPo2sP0+hVfVxbxxAy8nkWdEDiMDpLT8mwu6gyiQz6Hs0rF fUIT5L6NgW4/NBsUAKSQOjmGwrmotTZJzALTNbtw+B9/xxxc0btF8UCowbz4oC3FsKWCJBN5m1+ wUqrvPEK9s58wncAnJ/iz3eGUjw4143xZOH6Kza+oWNnf/OGqHN7hgflMHPu0Nz3Zllqjwh45dQ u1nK1MmxsHuYCYtdrXNxtqTG+ScjeqkFGdzBtCsyMB5EjZoXfYuG+jQI= X-Received: by 2002:a05:600c:4e13:b0:490:31e:d6fd with SMTP id 5b1f17b1804b1-490031ed899mr272415115e9.25.1779284809855; Wed, 20 May 2026 06:46:49 -0700 (PDT) X-Received: by 2002:a05:600c:4e13:b0:490:31e:d6fd with SMTP id 5b1f17b1804b1-490031ed899mr272414555e9.25.1779284809378; Wed, 20 May 2026 06:46:49 -0700 (PDT) Received: from [192.168.100.100] (82-64-211-94.subs.proxad.net. [82.64.211.94]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48febe6faeasm132850655e9.26.2026.05.20.06.46.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 20 May 2026 06:46:49 -0700 (PDT) Message-ID: Date: Wed, 20 May 2026 15:46:48 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 3/4] tcp_vu: Support multibuffer frames in tcp_vu_sock_recv() To: David Gibson References: <20260416161618.3826904-1-lvivier@redhat.com> <20260416161618.3826904-4-lvivier@redhat.com> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 1fFXv1ABBJ5hZujHkhoNAvVE8Fsu3BiuTJPU4alD6P0_1779284810 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: XL76KY6TFKWK7LOEIY2AXHLCFXATWZF6 X-Message-ID-Hash: XL76KY6TFKWK7LOEIY2AXHLCFXATWZF6 X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 5/11/26 10:24, David Gibson wrote: > On Thu, Apr 16, 2026 at 06:16:17PM +0200, Laurent Vivier wrote: >> Previously, tcp_vu_sock_recv() assumed a 1:1 mapping between virtqueue >> elements and iovecs (one iovec per element), enforced by an ASSERT. >> This prevented the use of virtqueue elements with multiple buffers >> (e.g. when mergeable rx buffers are not negotiated and headers are >> provided in a separate buffer). >> >> Introduce a struct vu_frame to track per-frame metadata: the range of >> elements and iovecs that make up each frame, and the frame's total size. >> This replaces the head[] array which only tracked element indices. >> >> A separate iov_msg[] array is built for recvmsg() by cloning the data >> portions (after stripping headers) using iov_tail helpers. >> >> Then a frame truncation after recvmsg() properly walks the frame and >> element arrays to adjust iovec counts and element counts. >> >> Signed-off-by: Laurent Vivier > > LGTM apart from a couple of minor points noted below. > >> --- >> tcp_vu.c | 174 ++++++++++++++++++++++++++++++++++++------------------- >> 1 file changed, 113 insertions(+), 61 deletions(-) >> >> diff --git a/tcp_vu.c b/tcp_vu.c >> index 2017aec90342..96b16007701d 100644 >> --- a/tcp_vu.c >> +++ b/tcp_vu.c >> @@ -35,9 +35,24 @@ >> #include "vu_common.h" >> #include >> >> -static struct iovec iov_vu[VIRTQUEUE_MAX_SIZE + DISCARD_IOV_NUM]; >> +static struct iovec iov_vu[VIRTQUEUE_MAX_SIZE]; >> static struct vu_virtq_element elem[VIRTQUEUE_MAX_SIZE]; >> -static int head[VIRTQUEUE_MAX_SIZE + 1]; >> + >> +/** >> + * struct vu_frame - Descriptor for a TCP frame mapped to virtqueue elements >> + * @idx_element: Index of first element in elem[] for this frame >> + * @num_element: Number of virtqueue elements used by this frame >> + * @idx_iovec: Index of first iovec in iov_vu[] for this frame >> + * @num_iovec: Number of iovecs covering this frame's buffers >> + * @size: Total frame size including all headers >> + */ >> +static struct vu_frame { >> + int idx_element; >> + int num_element; >> + int idx_iovec; >> + int num_iovec; >> + size_t size; >> +} frame[VIRTQUEUE_MAX_SIZE]; >> >> /** >> * tcp_vu_hdrlen() - Sum size of all headers, from TCP to virtio-net >> @@ -174,8 +189,8 @@ int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) >> * @v6: Set for IPv6 connections >> * @already_sent: Number of bytes already sent >> * @fillsize: Maximum bytes to fill in guest-side receiving window >> - * @iov_cnt: number of iov (output) >> - * @head_cnt: Pointer to store the count of head iov entries (output) >> + * @elem_used: number of element (output) >> + * @frame_cnt: Pointer to store the number of frames (output) >> * >> * Return: number of bytes received from the socket, or a negative error code >> * on failure. >> @@ -183,57 +198,77 @@ int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) >> static ssize_t tcp_vu_sock_recv(const struct ctx *c, struct vu_virtq *vq, >> const struct tcp_tap_conn *conn, bool v6, >> uint32_t already_sent, size_t fillsize, >> - int *iov_cnt, int *head_cnt) >> + int *elem_used, int *frame_cnt) >> { >> + static struct iovec iov_msg[VIRTQUEUE_MAX_SIZE + DISCARD_IOV_NUM]; >> const struct vu_dev *vdev = c->vdev; >> struct msghdr mh_sock = { 0 }; >> uint16_t mss = MSS_GET(conn); >> size_t hdrlen, iov_used; >> int s = conn->sock; >> + ssize_t ret, dlen; >> int elem_cnt; >> - ssize_t ret; >> - int i; >> - >> - *iov_cnt = 0; >> + int i, j; >> >> hdrlen = tcp_vu_hdrlen(v6); >> >> + *elem_used = 0; >> + >> iov_used = 0; >> elem_cnt = 0; >> - *head_cnt = 0; >> + *frame_cnt = 0; >> while (fillsize > 0 && elem_cnt < ARRAY_SIZE(elem) && >> - iov_used < VIRTQUEUE_MAX_SIZE) { >> - size_t frame_size, dlen, in_total; >> - struct iovec *iov; >> + iov_used < ARRAY_SIZE(iov_vu) && >> + *frame_cnt < ARRAY_SIZE(frame)) { >> + size_t frame_size, in_total; >> int cnt; >> >> cnt = vu_collect(vdev, vq, &elem[elem_cnt], >> ARRAY_SIZE(elem) - elem_cnt, >> - &iov_vu[DISCARD_IOV_NUM + iov_used], >> - VIRTQUEUE_MAX_SIZE - iov_used, &in_total, >> + &iov_vu[iov_used], >> + ARRAY_SIZE(iov_vu) - iov_used, &in_total, >> MIN(mss, fillsize) + hdrlen, >> &frame_size); >> if (cnt == 0) >> break; >> - assert((size_t)cnt == in_total); /* one iovec per element */ >> + >> + frame[*frame_cnt].idx_element = elem_cnt; >> + frame[*frame_cnt].num_element = cnt; >> + frame[*frame_cnt].idx_iovec = iov_used; >> + frame[*frame_cnt].num_iovec = in_total; >> + frame[*frame_cnt].size = frame_size; >> + (*frame_cnt)++; >> >> iov_used += in_total; >> - dlen = frame_size - hdrlen; >> + elem_cnt += cnt; >> >> - /* reserve space for headers in iov */ >> - iov = &elem[elem_cnt].in_sg[0]; >> - assert(iov->iov_len >= hdrlen); >> - iov->iov_base = (char *)iov->iov_base + hdrlen; >> - iov->iov_len -= hdrlen; >> - head[(*head_cnt)++] = elem_cnt; >> + fillsize -= frame_size - hdrlen; >> + } >> >> - fillsize -= dlen; >> - elem_cnt += cnt; >> + /* build an iov array without headers */ >> + for (i = 0, j = DISCARD_IOV_NUM; i < *frame_cnt && >> + j < ARRAY_SIZE(iov_msg); i++) { >> + struct iov_tail data; >> + ssize_t cnt; >> + >> + data = IOV_TAIL(&iov_vu[frame[i].idx_iovec], >> + frame[i].num_iovec, 0); >> + iov_drop_header(&data, hdrlen); >> + >> + cnt = iov_tail_clone(&iov_msg[j], ARRAY_SIZE(iov_msg) - j, >> + &data); >> + if (cnt == -1) >> + die("Missing entries in iov_msg"); > > Is a fatal error really what we want here? As we are copying iov_vu into iov_msg and the sizes match (if we ignore the discard part), there is always enough room. An assert() would be cleaner but coverity doesn't manage it correctly. Thanks, Laurent