From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id BB4DA5A0275 for ; Fri, 8 Mar 2024 09:18:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709885933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0qx1Gzl2oX237399l3rO7ZtEw5FI6PSXzl6bhvR/4Ws=; b=KNwywAGNIdIkCoF6g+BRffV5QhlwguH+sVwCp1iKeNNkFgZYOaZQggN3RA74firCUy3/0p zzURTG7kjT5fMJrHmV4PBjWaRg19cMI9yOVovj5NWQVNdX1HFs4fgXyeKruNdqLLynTKoY ShDz9PCpWuUlN6edPV/m2nX/d+a4SSY= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-18-XFpMR-LLMDC7uRg0eL11xQ-1; Fri, 08 Mar 2024 03:18:52 -0500 X-MC-Unique: XFpMR-LLMDC7uRg0eL11xQ-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-690b5bbf149so3932866d6.3 for ; Fri, 08 Mar 2024 00:18:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709885932; x=1710490732; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0qx1Gzl2oX237399l3rO7ZtEw5FI6PSXzl6bhvR/4Ws=; b=LiS2XM0p2tmPE1cmqJSXnnmlsBl5g/EsjZliXC1yInWCCMJz1ftwtKEvyQPpY+mFKp bXIyXOTq8Rnz3uD2jxJj2SgdNnor9XBa1GP8aQNwOqX0WkdjAMNYtRLFTi3Cy185X0ng FfkIdTmUPb4dYfUITnHzGZcx7z+toRES3dzL4lY/mJ2XGED6G435t1YCbaAYQ5QSp1Hh kCYPBJ13rDtY7cxBbSTclyH4vVgO83cIGsTncatUTUe/uHW4V0I0AJbqyNRykzeQlTa9 7q8honMVnvpn0Q03CruO253o9S9ddCFvWCG6BzGLl633z9xvbXoQjEQTMP+2vmyyH14X 3nNA== X-Forwarded-Encrypted: i=1; AJvYcCVt944pJ+kPBuhs/6IlM6m4oEMFQu8S+MIFgiigAbg7L8dr/Cq2ZSVpccOCUYuJlZUbtsSMvXXjM3dsE4pVPJQUBixY X-Gm-Message-State: AOJu0Yz/dvp89yv44OmVYm25OyG3baKA4AKYJdwEUbErBD2XAwWloqAt vyqhsZB+o4otxbJPNWIBFlXbk5dcdO86CMIX0TVhXx4rfbBy7RrgL13Ju4XoHF1c/3EdZ8Tt+ER 3/Rw+OqcBNLkpViMwWLMd5RNjPhsomtydtf1MAslUzh1vzNg03g== X-Received: by 2002:ad4:4eea:0:b0:690:b35d:e8a3 with SMTP id dv10-20020ad44eea000000b00690b35de8a3mr1560325qvb.4.1709885931834; Fri, 08 Mar 2024 00:18:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IGXo3vlONOdcuTsJGWT7y4JXtVnD/E1cEc5QSFf19jXXdOtYlnQNmf4o1wMI17pqcQ37r5kXQ== X-Received: by 2002:ad4:4eea:0:b0:690:b35d:e8a3 with SMTP id dv10-20020ad44eea000000b00690b35de8a3mr1560311qvb.4.1709885931505; Fri, 08 Mar 2024 00:18:51 -0800 (PST) Received: from [192.168.100.30] ([82.142.8.70]) by smtp.gmail.com with ESMTPSA id lz4-20020a0562145c4400b0068f4520e42dsm9579795qvb.16.2024.03.08.00.18.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Mar 2024 00:18:51 -0800 (PST) Message-ID: <348a52c9-7c07-4150-b594-4743c7775be6@redhat.com> Date: Fri, 8 Mar 2024 09:18:48 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/4] Some improvements to the tap send path To: David Gibson , Stefano Brivio , passt-dev@passt.top References: <20240308065325.2181322-1-david@gibson.dropbear.id.au> From: Laurent Vivier In-Reply-To: <20240308065325.2181322-1-david@gibson.dropbear.id.au> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: OT6JR2XXA7DGKIWWFBXFU4YEPHWY4YU5 X-Message-ID-Hash: OT6JR2XXA7DGKIWWFBXFU4YEPHWY4YU5 X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 3/8/24 07:53, David Gibson wrote: > This series has a handful of small improvements to the tap send path. > See individual commit messages for the details. > > I expect this will conflict with Laurent's upcoming work. I hope the > conflicts won't be too bad, and indeed will set us up for less > duplication there in the end. I'm working on patch that devides TCP buffers in several buffers pointed out by an IOV arrays and then provided to tap_send_frames(). I'm going to base my patch on this series. The idea is: A frame is made with 4 iovecs: #define TCP_IOV_VNET 0 #define TCP_IOV_ETH 1 #define TCP_IOV_IP 2 #define TCP_IOV_PAYLOAD 3 #define TCP_IOV_NUM 4 typedef struct iovec tap_iovec_t[TCP_IOV_NUM]; The payload can be TCP + data or TCP + flags: struct tcp_l2_flags_t { struct tcphdr th; char opts[OPT_MSS_LEN + OPT_WS_LEN + 1]; }; struct tcp_l2_payload_t { struct tcphdr th; /* 20 bytes */ uint8_t data[MSS]; /* 65516 bytes */ #ifdef __AVX2__ } __attribute__ ((packed, aligned(32))); #else } __attribute__ ((packed, aligned(__alignof__(unsigned int)))); #endif static struct ethhdr tcp4_eth_src; static struct iphdr tcp4_l2_ip[TCP_FRAMES_MEM]; static struct tcp_l2_payload_t tcp4_l2_payload[TCP_FRAMES_MEM]; static struct tcp_buf_seq_update tcp4_l2_buf_seq_update[TCP_FRAMES_MEM]; static unsigned int tcp4_l2_buf_used; static tap_iovec_t tcp4_l2_iov [TCP_FRAMES_MEM]; Initialization looks like: tcp4_eth_src.h_proto = htons_constant(ETH_P_IP); for (i = 0; i < TCP_FRAMES_MEM; i++) { ... /* headers */ tcp4_l2_ip[i] = iph; tcp4_l2_payload[i].th = (struct tcphdr){ .doff = sizeof(struct tcphdr) / 4, .ack = 1 }; ... /* iovecs */ iov = tcp4_l2_iov[i]; iov[TCP_IOV_ETH].iov_base = &tcp4_eth_src; iov[TCP_IOV_ETH].iov_len = sizeof(struct ethhdr); iov[TCP_IOV_IP].iov_base = &tcp4_l2_ip[i]; iov[TCP_IOV_IP].iov_len = sizeof(struct iphdr); iov[TCP_IOV_PAYLOAD].iov_base = &tcp4_l2_payload[i]; ... } Then to fill the payload header (data are received by tcp_data_from_sock()): iov = tcp4_l2_iov[tcp4_l2_buf_used++]; iov[TCP_IOV_PAYLOAD].iov_len = tcp_l2_buf_fill_headers(c, conn, iov, plen, check, seq); And the frame is sent using: m = tap_send_iov(c, tcp4_l2_iov, tcp4_l2_buf_used); For the moment (I'll rebase on your series), tap_send_iov_passt() looks like: static size_t tap_send_iov_passt(const struct ctx *c, tap_iovec_t *tap_iov, size_t n) { unsigned int i; for (i = 0; i < n; i++) { uint32_t vnet_len; int j; vnet_len = 0; for (j = TCP_IOV_ETH; j < TCP_IOV_NUM; j++) vnet_len += tap_iov[i][j].iov_len; tap_iov[i][TCP_IOV_VNET].iov_base = &vnet_len; tap_iov[i][TCP_IOV_VNET].iov_len = sizeof(vnet_len); if (!tap_send_frames_passt(c, tap_iov[i], TCP_IOV_NUM)) break; } return i; } Thanks, Laurent > > This is based on Laurent's patch fixing pcap_multiple() not to capture > frames we failed to send. > > David Gibson (4): > tap: Extend tap_send_frames() to allow multi-buffer frames > tap: Simplify some casts in the tap "slow path" functions > tap: Implement tap_send() "slow path" in terms of fast path > tap: Rename tap_iov_{base,len} > > arp.c | 4 +- > tap.c | 158 +++++++++++++++++++++++++++++++--------------------------- > tap.h | 19 +++---- > tcp.c | 20 ++++---- > udp.c | 10 ++-- > 5 files changed, 111 insertions(+), 100 deletions(-) >