From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MrTuTAAb; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 5C17D5A0265 for ; Sun, 10 May 2026 04:03:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778378604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UIQERNMZt9grKbp7ZmlrMntUbIpdk9l5Ey2Xabe09ek=; b=MrTuTAAb/Hd5kZBheqh/j28jw15zBh13MdPDPtVMvnnCy5yX9qHtQEan2Ayuejqf0JNfvv GUhu0lG/ObTy56nv7aBNtFSeR9tJg7QsQ3LbL9jpNaXgUpwEJn5J0YVo0dLbvS1HowJP4v CDU9bIwY8M+DkEQw1vitckUVkx5DunI= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-355-n81jMS8gOl-_3JF3SD8O4A-1; Sat, 09 May 2026 22:03:22 -0400 X-MC-Unique: n81jMS8gOl-_3JF3SD8O4A-1 X-Mimecast-MFC-AGG-ID: n81jMS8gOl-_3JF3SD8O4A_1778378602 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-8ba9baef44cso52436156d6.0 for ; Sat, 09 May 2026 19:03:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778378602; x=1778983402; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=UIQERNMZt9grKbp7ZmlrMntUbIpdk9l5Ey2Xabe09ek=; b=poMv+SXn9O6Ke309FwavX4n3gP+AsahI0M4g/VvgxF3KWqW87JXoSiY+H8i52iYmUJ VG0WmO5V/Dw6cJnF8+MEsIAmBbL0k3JYX2zy/BVfLbKcvtkpxHPjZicR785FvmVBf4du SCqjGBl+4vAgOpGqlU/TP9oOQe+Hb9Gz8Bdg8eIEOKeJK+FcyCMN9pdB7oatDhnMKDUp Kjn403k7P/6rsfX7mLAkRjQZiNCJ5Q8bY5Bf0p49OnGoperreiNW4n7I3F3XES2uAvGd QZLe04QWrtleSGlkYvB3953uBMw5UpQFiv5gKjYLgUuw9n5fuGYm7hwphmvzK5ALTijx 9p2A== X-Forwarded-Encrypted: i=1; AFNElJ/U9nISKvwYw7TZIc0ju7ETXqKRunWM4ZEkYEdoSXRfeqJI8vgqUN3txtdjGdyY16bTIK0G772WqOg=@passt.top X-Gm-Message-State: AOJu0Yy7DEnATI9U0e+sJLBWEy0A33tTqOJUgLx9shDmPJYPmOr8uLUT NIg816ACSQdpYLXh2jVYSQrSp++xAdd7NNTMCFKi/xM8nAGUFZvbzHfN7WMpAo0N/FPi55Y0E4C B2jYBHaPzGMZxxhed3/Wm27yh4//taoZYDid3XNb6Xw6Q/KL/toZ6Bg== X-Gm-Gg: Acq92OE5Ni0Mg0KQLnPKyiM3JBcW6bRpOqcbcHFQgg5euqqcVok9SieMCGQgcnoXFo3 azbmNQ2KzGp2P/0/mU9fSTitVedhK8c12DDPpUBs9qLbzez6iK22wsIQN00IcRwejWrDCfUESzo bAgXatKqsvbzK60GgehZHCvckSHfLXDcLBX2iimfnMZSPo/4rLHK0CvvMEf340B9rsDH6igK5va g27rMzR1VEXh2Lk9+kBc5W49YR+WLCwB48//fPwIWOHLTCJd5ta/6r6CZhkHePzexJNIlG28s9C IMggV1bQUjMCeZ82segl93hg02VZmoRiXCPvBNnaGQn5mm009N1IN1PqdMhK+PJnux53YA7CXGa jNLaR5cjjEpB1baBaz6xQp0UAYV25sl/ozYY0x2zNjxyfJUuMLyFKINMeZIGLUCWSlauloLoGR4 30aGsXHoMeTRxK X-Received: by 2002:ad4:5ba9:0:b0:8b6:6d70:aaac with SMTP id 6a1803df08f44-8bc449aff20mr275902806d6.30.1778378601706; Sat, 09 May 2026 19:03:21 -0700 (PDT) X-Received: by 2002:ad4:5ba9:0:b0:8b6:6d70:aaac with SMTP id 6a1803df08f44-8bc449aff20mr275901656d6.30.1778378600109; Sat, 09 May 2026 19:03:20 -0700 (PDT) Received: from [192.168.2.15] (lnsm4-toronto63-142-116-28-118.internet.virginmobile.ca. [142.116.28.118]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-8bf39c7e641sm57666046d6.27.2026.05.09.19.03.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 09 May 2026 19:03:19 -0700 (PDT) Message-ID: Date: Sat, 9 May 2026 22:03:19 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 4/4] tcp_vu: Support multibuffer frames in tcp_vu_send_flag() To: Laurent Vivier , passt-dev@passt.top References: <20260416161618.3826904-1-lvivier@redhat.com> <20260416161618.3826904-5-lvivier@redhat.com> From: Jon Maloy In-Reply-To: <20260416161618.3826904-5-lvivier@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: q1CZZ0ntumSfkLSnqnII6mNl63PrzzqHdVe32DHoh0A_1778378602 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: 2Z5CWIMME6WSEN6A4AOWILKB7R6GIQKF X-Message-ID-Hash: 2Z5CWIMME6WSEN6A4AOWILKB7R6GIQKF X-MailFrom: jmaloy@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 2026-04-16 12:16, Laurent Vivier wrote: > Build the Ethernet, IP, and TCP headers on the stack instead of > directly in the buffer via pointer casts, then write them into the > iovec with IOV_PUSH_HEADER(). This mirrors the approach already used > in tcp_vu_prepare() and udp_vu_prepare(). > > Remove the vu_eth(), vu_ip(), vu_payloadv4() and vu_payloadv6() helpers > from vu_common.h, as they are no longer used anywhere. > > Introduce tcp_vu_send_dup() to handle DUP_ACK duplication using > vu_collect() and iov_memcopy() instead of a plain memcpy(), so that > the duplicated frame is also properly scattered across multiple iovecs. > > Signed-off-by: Laurent Vivier Reviewed-by: Jon Maloy But please look at my comment below. > --- > iov.c | 1 - > tcp_vu.c | 153 ++++++++++++++++++++++++++++++---------------------- > vu_common.h | 20 ------- > 3 files changed, 90 insertions(+), 84 deletions(-) > > diff --git a/iov.c b/iov.c > index c0d9c6d21322..31fc3e479572 100644 > --- a/iov.c > +++ b/iov.c > @@ -208,7 +208,6 @@ void iov_memset(const struct iovec *iov, size_t iov_cnt, size_t offset, int c, > * > * Return: total number of bytes copied > */ > -/* cppcheck-suppress unusedFunction */ > size_t iov_memcpy(struct iovec *dst_iov, size_t dst_iov_cnt, size_t dst_offset, > const struct iovec *src_iov, size_t src_iov_cnt, > size_t src_offset, size_t length) > diff --git a/tcp_vu.c b/tcp_vu.c > index 96b16007701d..7f7e43860b10 100644 > --- a/tcp_vu.c > +++ b/tcp_vu.c > @@ -74,6 +74,42 @@ static size_t tcp_vu_hdrlen(bool v6) > return hdrlen; > } > > +/** > + * tcp_vu_send_dup() - Duplicate a frame into a new virtqueue element > + * @c: Execution context > + * @vq: Receive virtqueue > + * @dest_elem: Destination virtqueue element to collect > + * @dest_iov: Destination iovec array for collected buffers > + * @max_dest_iov: Maximum number of entries in @dest_iov > + * @src_iov: Source iovec array containing the frame to duplicate > + * @src_cnt: Number of entries in @src_iov > + * > + * Return: number of virtqueue elements collected (0 if none available) > + */ > +static int tcp_vu_send_dup(const struct ctx *c, struct vu_virtq *vq, > + struct vu_virtq_element *dest_elem, > + struct iovec *dest_iov, size_t max_dest_iov, > + const struct iovec *src_iov, size_t src_cnt, > + size_t vnlen) You missed 'vnlen' in the comment above. /jon > +{ > + const struct vu_dev *vdev = c->vdev; > + size_t dest_cnt; > + int elem_cnt; > + > + elem_cnt = vu_collect(vdev, vq, dest_elem, 1, dest_iov, max_dest_iov, > + &dest_cnt, vnlen, NULL); > + if (elem_cnt == 0) > + return 0; > + > + iov_memcpy(dest_iov, dest_cnt, 0, src_iov, src_cnt, 0, > + MAX(VNET_HLEN + ETH_ZLEN, vnlen)); > + > + if (*c->pcap) > + pcap_iov(dest_iov, dest_cnt, VNET_HLEN, vnlen - VNET_HLEN); > + > + return elem_cnt; > +} > + > /** > * tcp_vu_send_flag() - Send segment with flags to vhost-user (no payload) > * @c: Execution context > @@ -86,97 +122,88 @@ int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) > { > struct vu_dev *vdev = c->vdev; > struct vu_virtq *vq = &vdev->vq[VHOST_USER_RX_QUEUE]; > + size_t optlen, hdrlen, iov_cnt, iov_used; > struct vu_virtq_element flags_elem[2]; > - size_t optlen, hdrlen, l2len; > - struct ipv6hdr *ip6h = NULL; > - struct iphdr *ip4h = NULL; > - struct iovec flags_iov[2]; > - struct tcp_syn_opts *opts; > + struct iovec flags_iov[64]; > + int elem_cnt, dup_elem_cnt = 0; > + struct tcp_syn_opts opts; > struct iov_tail payload; > - struct tcphdr *th; > - struct ethhdr *eh; > + struct ipv6hdr ip6h; > + struct iphdr ip4h; > + struct tcphdr th; > + struct ethhdr eh; > uint32_t seq; > - int elem_cnt; > int ret; > > hdrlen = tcp_vu_hdrlen(CONN_V6(conn)); > > elem_cnt = vu_collect(vdev, vq, &flags_elem[0], 1, > - &flags_iov[0], 1, NULL, > - hdrlen + sizeof(*opts), NULL); > - if (elem_cnt != 1) > + flags_iov, ARRAY_SIZE(flags_iov), &iov_cnt, > + hdrlen + sizeof(opts), NULL); > + if (elem_cnt == 0) > return -1; > > - assert(flags_elem[0].in_num == 1); > - assert(flags_elem[0].in_sg[0].iov_len >= > - MAX(hdrlen + sizeof(*opts), ETH_ZLEN + VNET_HLEN)); > - > - eh = vu_eth(flags_elem[0].in_sg[0].iov_base); > - > - memcpy(eh->h_dest, c->guest_mac, sizeof(eh->h_dest)); > - memcpy(eh->h_source, c->our_tap_mac, sizeof(eh->h_source)); > - > - if (CONN_V4(conn)) { > - eh->h_proto = htons(ETH_P_IP); > - > - ip4h = vu_ip(flags_elem[0].in_sg[0].iov_base); > - *ip4h = (struct iphdr)L2_BUF_IP4_INIT(IPPROTO_TCP); > - > - th = vu_payloadv4(flags_elem[0].in_sg[0].iov_base); > - } else { > - eh->h_proto = htons(ETH_P_IPV6); > + memcpy(eh.h_dest, c->guest_mac, sizeof(eh.h_dest)); > > - ip6h = vu_ip(flags_elem[0].in_sg[0].iov_base); > - *ip6h = (struct ipv6hdr)L2_BUF_IP6_INIT(IPPROTO_TCP); > - th = vu_payloadv6(flags_elem[0].in_sg[0].iov_base); > - } > + if (CONN_V4(conn)) > + ip4h = (struct iphdr)L2_BUF_IP4_INIT(IPPROTO_TCP); > + else > + ip6h = (struct ipv6hdr)L2_BUF_IP6_INIT(IPPROTO_TCP); > > - memset(th, 0, sizeof(*th)); > - th->doff = sizeof(*th) / 4; > - th->ack = 1; > + memset(&th, 0, sizeof(th)); > > seq = conn->seq_to_tap; > - opts = (struct tcp_syn_opts *)(th + 1); > - ret = tcp_prepare_flags(c, conn, flags, th, opts, &optlen); > + ret = tcp_prepare_flags(c, conn, flags, &th, &opts, &optlen); > if (ret <= 0) { > - vu_queue_rewind(vq, 1); > + vu_queue_rewind(vq, elem_cnt); > return ret; > } > > - payload = IOV_TAIL(flags_elem[0].in_sg, 1, hdrlen); > - > if (flags & KEEPALIVE) > seq--; > > - tcp_fill_headers(c, conn, eh, ip4h, ip6h, th, &payload, > + iov_used = iov_skip_bytes(flags_iov, iov_cnt, > + MAX(optlen + hdrlen, VNET_HLEN + ETH_ZLEN), > + NULL); > + if (iov_used < iov_cnt) > + iov_used++; > + iov_cnt = iov_used; > + > + payload = IOV_TAIL(flags_elem[0].in_sg, iov_cnt, hdrlen); > + iov_from_buf(payload.iov, payload.cnt, payload.off, &opts, optlen); > + tcp_fill_headers(c, conn, &eh, CONN_V4(conn) ? &ip4h : NULL, > + CONN_V6(conn) ? &ip6h : NULL, &th, &payload, > optlen, IP4_CSUM | (*c->pcap ? TCP_CSUM : 0), seq); > > - vu_pad(flags_elem[0].in_sg, 1, hdrlen + optlen); > - vu_flush(vdev, vq, flags_elem, 1, hdrlen + optlen); > + vu_pad(flags_elem[0].in_sg, iov_cnt, hdrlen + optlen); > + > + /* write headers */ > + payload = IOV_TAIL(flags_elem[0].in_sg, iov_cnt, VNET_HLEN); > + > + IOV_PUSH_HEADER(&payload, eh); > + if (CONN_V4(conn)) > + IOV_PUSH_HEADER(&payload, ip4h); > + else > + IOV_PUSH_HEADER(&payload, ip6h); > + IOV_PUSH_HEADER(&payload, th); > > - l2len = hdrlen + optlen - VNET_HLEN; > if (*c->pcap) > - pcap_iov(&flags_elem[0].in_sg[0], 1, VNET_HLEN, l2len); > + pcap_iov(flags_elem[0].in_sg, iov_cnt, VNET_HLEN, > + hdrlen + optlen - VNET_HLEN); > > if (flags & DUP_ACK) { > - elem_cnt = vu_collect(vdev, vq, &flags_elem[1], 1, > - &flags_iov[1], 1, NULL, > - hdrlen + optlen, NULL); > - if (elem_cnt == 1 && > - flags_elem[1].in_sg[0].iov_len >= > - flags_elem[0].in_sg[0].iov_len) { > - memcpy(flags_elem[1].in_sg[0].iov_base, > - flags_elem[0].in_sg[0].iov_base, > - flags_elem[0].in_sg[0].iov_len); > - > - vu_flush(vdev, vq, &flags_elem[1], 1, hdrlen + optlen); > - > - if (*c->pcap) { > - pcap_iov(&flags_elem[1].in_sg[0], 1, VNET_HLEN, > - l2len); > - } > - } > + dup_elem_cnt = tcp_vu_send_dup(c, vq, &flags_elem[elem_cnt], > + &flags_iov[iov_cnt], > + ARRAY_SIZE(flags_iov) - iov_cnt, > + flags_elem[0].in_sg, iov_cnt, > + hdrlen + optlen); > } > + vu_flush(vdev, vq, flags_elem, elem_cnt, hdrlen + optlen); > + if (dup_elem_cnt) { > + vu_flush(vdev, vq, &flags_elem[elem_cnt], dup_elem_cnt, > + hdrlen + optlen); > + } > + > vu_queue_notify(vdev, vq); > > return 0; > diff --git a/vu_common.h b/vu_common.h > index 51f70084a7cb..817384175a1d 100644 > --- a/vu_common.h > +++ b/vu_common.h > @@ -15,26 +15,6 @@ > #include "ip.h" > #include "virtio.h" > > -static inline void *vu_eth(void *base) > -{ > - return ((char *)base + VNET_HLEN); > -} > - > -static inline void *vu_ip(void *base) > -{ > - return (struct ethhdr *)vu_eth(base) + 1; > -} > - > -static inline void *vu_payloadv4(void *base) > -{ > - return (struct iphdr *)vu_ip(base) + 1; > -} > - > -static inline void *vu_payloadv6(void *base) > -{ > - return (struct ipv6hdr *)vu_ip(base) + 1; > -} > - > int vu_collect(const struct vu_dev *vdev, struct vu_virtq *vq, > struct vu_virtq_element *elem, int max_elem, > struct iovec *in_sg, size_t max_in_sg, size_t *in_total,