From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 787ED5A026F for ; Mon, 5 Dec 2022 09:14:35 +0100 (CET) Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4NQbtY2gdsz4xP9; Mon, 5 Dec 2022 19:14:29 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=201602; t=1670228069; bh=lsvMvmRoc3ZR2IPsbL4fQwER/upyuieI6PDQAQrCi3Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OqruVvd54GCcYOwPSTs3+Q7oIW0yoGrLYaxtv/CvWWO795cl0sn2+qRI1Gyj4D5ul IGqLqkgiM4fZLQCx4grx6aeT/GKjZhq3DeTkNmrfqjZnsvrc2GAnClkfY56vwu1dJF H3Y2zZ6PlmOT/vEu/CXpg6U3X8N5CKvJWKrvCAG8= From: David Gibson To: Stefano Brivio , passt-dev@passt.top Subject: [PATCH 2/8] udp: Split sending to passt tap interface into separate function Date: Mon, 5 Dec 2022 19:14:19 +1100 Message-Id: <20221205081425.2614425-3-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221205081425.2614425-1-david@gibson.dropbear.id.au> References: <20221205081425.2614425-1-david@gibson.dropbear.id.au> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: 5OZHGVAKWLSQSM3PPEMRTLQ5JLG7KT2N X-Message-ID-Hash: 5OZHGVAKWLSQSM3PPEMRTLQ5JLG7KT2N X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Gibson X-Mailman-Version: 3.3.3 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The last part of udp_sock_handler() does the actual on-sending of frames to the tap interface. For pasta that's just a call to udp_tap_send_pasta() but for passt, it's a moderately complex and open coded. For symmetry, move the passt send path into its own function, udp_tap_send_passt(). This will make it easier to abstract the tap interface in future (e.g. when we want to add vhost-user). Signed-off-by: David Gibson --- udp.c | 130 ++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 72 insertions(+), 58 deletions(-) diff --git a/udp.c b/udp.c index a27fe88..7281bc3 100644 --- a/udp.c +++ b/udp.c @@ -812,6 +812,73 @@ static void udp_tap_send_pasta(const struct ctx *c, struct mmsghdr *mmh, } } +/** + * udp_tap_send_passt() - Send datagrams to the passt tap interface + * @c: Execution context + * @mmh: Array of message headers to send + * @n: Number of message headers to send + * + * #syscalls:passt sendmmsg sendmsg + */ +static void udp_tap_send_passt(const struct ctx *c, struct mmsghdr *mmh, int n) +{ + struct msghdr *last_mh; + ssize_t missing = 0; + size_t msg_len = 0; + unsigned int i; + int ret; + + ret = sendmmsg(c->fd_tap, mmh, n, MSG_NOSIGNAL | MSG_DONTWAIT); + if (ret <= 0) + return; + + /* If we lose some messages to sendmmsg() here, fine, it's UDP. However, + * the last message needs to be delivered completely, otherwise qemu + * will fail to reassemble the next message and close the connection. Go + * through headers from the last sent message, counting bytes, and, if + * and as soon as we see more bytes than sendmmsg() sent, re-send the + * rest with a blocking call. + * + * In pictures, given this example: + * + * iov #0 iov #1 iov #2 iov #3 + * tap_mmh[ret - 1].msg_hdr: .... ...... ..... ...... + * tap_mmh[ret - 1].msg_len: 7 .... ... + * + * when 'msglen' reaches: 10 ^ + * and 'missing' below is: 3 --- + * + * re-send everything from here: ^-- ----- ------ + */ + last_mh = &mmh[ret - 1].msg_hdr; + for (i = 0; i < last_mh->msg_iovlen; i++) { + if (missing <= 0) { + msg_len += last_mh->msg_iov[i].iov_len; + missing = msg_len - mmh[ret - 1].msg_len; + } + + if (missing > 0) { + uint8_t **iov_base; + int first_offset; + + iov_base = (uint8_t **)&last_mh->msg_iov[i].iov_base; + first_offset = last_mh->msg_iov[i].iov_len - missing; + *iov_base += first_offset; + last_mh->msg_iov[i].iov_len = missing; + + last_mh->msg_iov = &last_mh->msg_iov[i]; + + if (sendmsg(c->fd_tap, last_mh, MSG_NOSIGNAL) < 0) + debug("UDP: %li bytes to tap missing", missing); + + *iov_base -= first_offset; + break; + } + } + + pcapmm(mmh, ret); +} + /** * udp_sock_handler() - Handle new data from socket * @c: Execution context @@ -820,16 +887,14 @@ static void udp_tap_send_pasta(const struct ctx *c, struct mmsghdr *mmh, * @now: Current timestamp * * #syscalls recvmmsg - * #syscalls:passt sendmmsg sendmsg */ void udp_sock_handler(const struct ctx *c, union epoll_ref ref, uint32_t events, const struct timespec *now) { in_port_t dstport = ref.r.p.udp.udp.port; - ssize_t n, msg_len = 0, missing = 0; struct mmsghdr *tap_mmh, *sock_mmh; - int msg_bufs = 0, msg_i = 0, ret; - struct msghdr *last_mh; + int msg_bufs = 0, msg_i = 0; + ssize_t n, msg_len = 0; struct iovec *tap_iov; unsigned int i; @@ -879,61 +944,10 @@ void udp_sock_handler(const struct ctx *c, union epoll_ref ref, uint32_t events, } tap_mmh[msg_i].msg_hdr.msg_iovlen = msg_bufs; - if (c->mode == MODE_PASTA) { + if (c->mode == MODE_PASTA) udp_tap_send_pasta(c, tap_mmh, msg_i + 1); - return; - } - - ret = sendmmsg(c->fd_tap, tap_mmh, msg_i + 1, - MSG_NOSIGNAL | MSG_DONTWAIT); - if (ret <= 0) - return; - - /* If we lose some messages to sendmmsg() here, fine, it's UDP. However, - * the last message needs to be delivered completely, otherwise qemu - * will fail to reassemble the next message and close the connection. Go - * through headers from the last sent message, counting bytes, and, if - * and as soon as we see more bytes than sendmmsg() sent, re-send the - * rest with a blocking call. - * - * In pictures, given this example: - * - * iov #0 iov #1 iov #2 iov #3 - * tap_mmh[ret - 1].msg_hdr: .... ...... ..... ...... - * tap_mmh[ret - 1].msg_len: 7 .... ... - * - * when 'msglen' reaches: 10 ^ - * and 'missing' below is: 3 --- - * - * re-send everything from here: ^-- ----- ------ - */ - last_mh = &tap_mmh[ret - 1].msg_hdr; - for (i = 0, msg_len = 0; i < last_mh->msg_iovlen; i++) { - if (missing <= 0) { - msg_len += last_mh->msg_iov[i].iov_len; - missing = msg_len - tap_mmh[ret - 1].msg_len; - } - - if (missing > 0) { - uint8_t **iov_base; - int first_offset; - - iov_base = (uint8_t **)&last_mh->msg_iov[i].iov_base; - first_offset = last_mh->msg_iov[i].iov_len - missing; - *iov_base += first_offset; - last_mh->msg_iov[i].iov_len = missing; - - last_mh->msg_iov = &last_mh->msg_iov[i]; - - if (sendmsg(c->fd_tap, last_mh, MSG_NOSIGNAL) < 0) - debug("UDP: %li bytes to tap missing", missing); - - *iov_base -= first_offset; - break; - } - } - - pcapmm(tap_mmh, ret); + else + udp_tap_send_passt(c, tap_mmh, msg_i + 1); } /** -- 2.38.1