From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id F3FDE5A026D for ; Wed, 4 Jan 2023 06:44:37 +0100 (CET) Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4Nmz7d3G3vz4y0g; Wed, 4 Jan 2023 16:44:29 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=201602; t=1672811069; bh=9/jM2B2BP5We0n2Xu+G0PznzmtwzhiQ1/joILPDU3Fk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kPT9B3sJWyL/1buDCGsXejU8DKQjUxw1Bf87UejngwTO9ufrmznZeQvYa4iIEUL3M UO3+JPt8nQG62b/gfU8XigOqRuRBxfcpVwTj1HZ3caXIcoSbu1y51r1FTWlwAjL5ZL EEMPmVasi2LLCA1ga/xNCp+UNjmDNYzSMoLlDii8= From: David Gibson To: passt-dev@passt.top, Stefano Brivio Subject: [PATCH v3 6/8] udp: Unify udp_sock_handler_splice() with udp_sock_handler() Date: Wed, 4 Jan 2023 16:44:24 +1100 Message-Id: <20230104054426.120668-7-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230104054426.120668-1-david@gibson.dropbear.id.au> References: <20230104054426.120668-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: JAXQM4XPPSHYLMFGQVTRGZC72Z7NB4QT X-Message-ID-Hash: JAXQM4XPPSHYLMFGQVTRGZC72Z7NB4QT X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Gibson X-Mailman-Version: 3.3.3 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: These two functions now have a very similar structure, and their first part (calling recvmmsg()) is functionally identical. So, merge the two functions into one. This does have the side effect of meaning we no longer receive multiple packets at once for splice (we already didn't for tap). This does hurt throughput for small spliced packets, but improves it for large spliced packets and tap packets. Signed-off-by: David Gibson --- udp.c | 95 ++++++++++++++++++++++------------------------------------- 1 file changed, 35 insertions(+), 60 deletions(-) diff --git a/udp.c b/udp.c index f6c07a5..32ca83e 100644 --- a/udp.c +++ b/udp.c @@ -590,52 +590,6 @@ static void udp_splice_sendfrom(const struct ctx *c, unsigned start, unsigned n, sendmmsg(s, mmh_send + start, n, MSG_NOSIGNAL); } -/** - * udp_sock_handler_splice() - Handler for socket mapped to "spliced" connection - * @c: Execution context - * @ref: epoll reference - * @events: epoll events bitmap - * @now: Current timestamp - */ -static void udp_sock_handler_splice(const struct ctx *c, union epoll_ref ref, - uint32_t events, const struct timespec *now) -{ - in_port_t dst = ref.r.p.udp.udp.port; - int v6 = ref.r.p.udp.udp.v6, n, i, m; - struct mmsghdr *mmh_recv; - - if (!(events & EPOLLIN)) - return; - - if (v6) - mmh_recv = udp6_l2_mh_sock; - else - mmh_recv = udp4_l2_mh_sock; - - n = recvmmsg(ref.r.s, mmh_recv, UDP_MAX_FRAMES, 0, NULL); - - if (n <= 0) - return; - - if (v6) - udp6_localname.sin6_port = htons(dst); - else - udp4_localname.sin_port = htons(dst); - - for (i = 0; i < n; i += m) { - in_port_t src = sa_port(v6, mmh_recv[i].msg_hdr.msg_name); - - for (m = 1; i + m < n; m++) { - void *mname = mmh_recv[i + m].msg_hdr.msg_name; - if (sa_port(v6, mname) != src) - break; - } - - udp_splice_sendfrom(c, i, m, src, dst, v6, ref.r.p.udp.udp.ns, - ref.r.p.udp.udp.orig, now); - } -} - /** * udp_update_hdr4() - Update headers for one IPv4 datagram * @c: Execution context @@ -944,32 +898,53 @@ void udp_sock_handler(const struct ctx *c, union epoll_ref ref, uint32_t events, const struct timespec *now) { /* For not entirely clear reasons (data locality?) pasta gets - * better throughput if we receive the datagrams one at a - * time. + * better throughput if we receive tap datagrams one at a + * atime. For small splice datagrams throughput is slightly + * better if we do batch, but it's slightly worse for large + * splice datagrams. Since we don't know before we receive + * whether we'll use tap or splice, always go one at a time + * for pasta mode. */ ssize_t n = (c->mode == MODE_PASST ? UDP_MAX_FRAMES : 1); in_port_t dstport = ref.r.p.udp.udp.port; bool v6 = ref.r.p.udp.udp.v6; - struct mmsghdr *sock_mmh; + struct mmsghdr *mmh_recv; + unsigned int i, m; + ssize_t n; - if (events == EPOLLERR) + if (!(events & EPOLLIN)) return; - if (ref.r.p.udp.udp.splice) { - udp_sock_handler_splice(c, ref, events, now); - return; + if (v6) { + mmh_recv = udp6_l2_mh_sock; + udp6_localname.sin6_port = htons(dstport); + } else { + mmh_recv = udp4_l2_mh_sock; + udp4_localname.sin_port = htons(dstport); } - if (ref.r.p.udp.udp.v6) - sock_mmh = udp6_l2_mh_sock; - else - sock_mmh = udp4_l2_mh_sock; - - n = recvmmsg(ref.r.s, sock_mmh, n, 0, NULL); + n = recvmmsg(ref.r.s, mmh_recv, n, 0, NULL); if (n <= 0) return; - udp_tap_send(c, 0, n, dstport, v6, now); + if (!ref.r.p.udp.udp.splice) { + udp_tap_send(c, 0, n, dstport, v6, now); + return; + } + + for (i = 0; i < n; i += m) { + in_port_t src = sa_port(v6, mmh_recv[i].msg_hdr.msg_name); + + for (m = 1; i + m < n; m++) { + void *mname = mmh_recv[i + m].msg_hdr.msg_name; + if (sa_port(v6, mname) != src) + break; + } + + udp_splice_sendfrom(c, i, m, src, dstport, v6, + ref.r.p.udp.udp.ns, ref.r.p.udp.udp.orig, + now); + } } /** -- 2.39.0