From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Lgmp1NQO; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id 0751F5A004E for ; Thu, 12 Sep 2024 14:41:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726144918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=xOC0YLC4Z10IBRB7DCbZ/9mtfd0bTul1ULmkpgkned4=; b=Lgmp1NQOzeRgYzOdEwrq67ngIpc+cATsYv9I7vvfP2f9OqhKQvW79IaiYzWXwcABTls1Q5 qJInmabL2bchpNgIzoSna60CX+eF5/RyB49kQre0pihT0ixeb0SJTF3SYvS7ugiQC4UPgf 0wK/fWZc25WkEznEEwKlFb4YKR33iag= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-684-0dz5V6spOW2rlRRGQfEZGA-1; Thu, 12 Sep 2024 08:41:57 -0400 X-MC-Unique: 0dz5V6spOW2rlRRGQfEZGA-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-42caf073db8so7068505e9.3 for ; Thu, 12 Sep 2024 05:41:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726144916; x=1726749716; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xOC0YLC4Z10IBRB7DCbZ/9mtfd0bTul1ULmkpgkned4=; b=omjjzFKLfZ6zBQimiqC3KKrriq7j7Vf50NN4Hh1XcPbuX4h5IYoCWAmokffflgd6Rj pT1QoeObag0lDH1JG1KKeQQvXZp+2PHI9/OYGHWxYaZTARgCWggGCXrHMpBhjv6J9Ha2 qxYUg7omMFVu359vrZ3Aq+gB0nzfnNOQ/KR7U2uHLdMOirLNgH0cqihecSskFvQIcz2P B4vplx3/0okYfpZzhAavbgKfkbe94PWBMePJUyFrtvCLEQSxFLUXBvZLzHlqLDhRMGmu gMKAp0ghoEKfJi2c9LbmZ0dFq3kZGMm4ybiG9qC69cBp9YfItgZ0MOcQUizwzhkY9xta ibzQ== X-Gm-Message-State: AOJu0Yz8J0JM8Pc0lmpb3uLlTTfVa7BCOQNA2S9r8RSbUUpShvmjqpaZ k1ZwcWzZvnvhAa9xIasooJXfAfuuopRndmUJOstUMWN/sUWrlfmLznGTQoNtOuMX+BHgckAdp1Z mJjrBzt1dtL25w6zwTh14oYWj6lRVqp4cZ6SpJVIV8hqGWV4MvQ== X-Received: by 2002:a05:600c:4f03:b0:42c:b97a:5f7d with SMTP id 5b1f17b1804b1-42cdb509fc7mr23770255e9.7.1726144916108; Thu, 12 Sep 2024 05:41:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHAuVxjcFrsd2kLwLFsBeOZU/9gvl+az2rl7XkbsVwDX56+j5N+EQ/XnMOHW+LyHJtnwnPRug== X-Received: by 2002:a05:600c:4f03:b0:42c:b97a:5f7d with SMTP id 5b1f17b1804b1-42cdb509fc7mr23769845e9.7.1726144915052; Thu, 12 Sep 2024 05:41:55 -0700 (PDT) Received: from ?IPV6:2a01:e0a:e10:ef90:4c84:58cb:a1ef:8b78? ([2a01:e0a:e10:ef90:4c84:58cb:a1ef:8b78]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cb2ca95a6sm157756395e9.21.2024.09.12.05.41.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Sep 2024 05:41:54 -0700 (PDT) Message-ID: <2a0d56e9-9c7d-4f22-bc61-4177d69032ae@redhat.com> Date: Thu, 12 Sep 2024 14:41:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 3/4] vhost-user: introduce vhost-user API To: Stefano Brivio References: <20240906160455.2088854-1-lvivier@redhat.com> <20240906160455.2088854-4-lvivier@redhat.com> <20240910174735.1e80713c@elisabeth> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: <20240910174735.1e80713c@elisabeth> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: KBFPOBTN5JSIDFGESGFY2R4DVC4S7NKN X-Message-ID-Hash: KBFPOBTN5JSIDFGESGFY2R4DVC4S7NKN X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 10/09/2024 17:47, Stefano Brivio wrote: > Nits and a couple of questions only: > > On Fri, 6 Sep 2024 18:04:48 +0200 > Laurent Vivier wrote: > >> Add vhost_user.c and vhost_user.h that define the functions needed >> to implement vhost-user backend. >> >> Signed-off-by: Laurent Vivier >> --- >> Makefile | 4 +- >> iov.c | 1 - >> vhost_user.c | 1265 ++++++++++++++++++++++++++++++++++++++++++++++++++ >> vhost_user.h | 203 ++++++++ >> virtio.c | 5 - >> virtio.h | 2 +- >> 6 files changed, 1471 insertions(+), 9 deletions(-) >> create mode 100644 vhost_user.c >> create mode 100644 vhost_user.h ... >> diff --git a/vhost_user.c b/vhost_user.c >> new file mode 100644 >> index 000000000000..6008a8adc967 >> --- /dev/null >> +++ b/vhost_user.c ... >> +/** >> + * vu_wait_queue() - wait for new free entries in the virtqueue >> + * @vq: virtqueue to wait on >> + */ >> +static int vu_wait_queue(const struct vu_virtq *vq) >> +{ >> + eventfd_t kick_data; >> + ssize_t rc; >> + int status; >> + >> + /* wait for the kernel to put new entries in the queue */ >> + status = fcntl(vq->kick_fd, F_GETFL); >> + if (status == -1) >> + return -1; > > Same as on v3 (I see you changed this below, but not here): if you > don't use status later, you can omit storing it. We need status with F_SETFL below: > >> + >> + if (fcntl(vq->kick_fd, F_SETFL, status & ~O_NONBLOCK)) >> + return -1; >> + >> + rc = eventfd_read(vq->kick_fd, &kick_data); >> + >> + if (fcntl(vq->kick_fd, F_SETFL, status)) >> + return -1; >> + >> + if (rc == -1) >> + return -1; >> + >> + return 0; >> +} ... >> +/** >> + * vu_handle_tx() - Receive data from the TX virtqueue >> + * @vdev: vhost-user device >> + * @index: index of the virtqueue >> + * @now: Current timestamp >> + */ >> +static void vu_handle_tx(struct vu_dev *vdev, int index, >> + const struct timespec *now) >> +{ >> + struct vu_virtq_element elem[VIRTQUEUE_MAX_SIZE]; >> + struct iovec out_sg[VIRTQUEUE_MAX_SIZE]; >> + struct vu_virtq *vq = &vdev->vq[index]; >> + int hdrlen = vdev->hdrlen; >> + int out_sg_count; >> + int count; >> + > > Excess newline (same as v3). Done. > >> + >> + if (!VHOST_USER_IS_QUEUE_TX(index)) { >> + debug("vhost-user: index %d is not a TX queue", index); >> + return; >> + } >> + >> + tap_flush_pools(); >> + >> + count = 0; >> + out_sg_count = 0; >> + while (count < VIRTQUEUE_MAX_SIZE) { > > So, I see that this is limited to 1024 iterations now (it was limited > also earlier, but I didn't realise that). > > If we loop at most VIRTQUEUE_MAX_SIZE times, that means, I guess, that > while we're popping elements, the queue can't be written to, correct? No, I think the queue can be read and write at the same time. > > Or it can be written to, but we'll get an additional kick after > vu_queue_notify() if that happens? I could check the protocol and the code, but I think it should work like that. > >> + int ret; >> + >> + elem[count].out_num = 1; >> + elem[count].out_sg = &out_sg[out_sg_count]; >> + elem[count].in_num = 0; >> + elem[count].in_sg = NULL; >> + ret = vu_queue_pop(vdev, vq, &elem[count]); >> + if (ret < 0) >> + break; >> + out_sg_count += elem[count].out_num; >> + >> + if (elem[count].out_num < 1) { >> + debug("virtio-net header not in first element"); >> + break; >> + } >> + ASSERT(elem[count].out_num == 1); >> + >> + tap_add_packet(vdev->context, >> + elem[count].out_sg[0].iov_len - hdrlen, >> + (char *)elem[count].out_sg[0].iov_base + hdrlen); >> + count++; >> + } >> + tap_handler(vdev->context, now); >> + >> + if (count) { >> + int i; >> + >> + for (i = 0; i < count; i++) >> + vu_queue_fill(vq, &elem[i], 0, i); >> + vu_queue_flush(vq, count); >> + vu_queue_notify(vdev, vq); >> + } >> +} >> + ... >> +/** >> + * vu_set_vring_err_exec() - Set the event file descriptor to signal when >> + * error occurs >> + * @vdev: vhost-user device >> + * @vmsg: vhost-user message >> + * >> + * Return: False as no reply is requested >> + */ >> +static bool vu_set_vring_err_exec(struct vu_dev *vdev, >> + struct vhost_user_msg *msg) >> +{ >> + bool nofd = msg->payload.u64 & VHOST_USER_VRING_NOFD_MASK; >> + int idx = msg->payload.u64 & VHOST_USER_VRING_IDX_MASK; >> + >> + debug("u64: 0x%016"PRIx64, msg->payload.u64); >> + >> + vu_check_queue_msg_file(msg); >> + >> + if (vdev->vq[idx].err_fd != -1) { >> + close(vdev->vq[idx].err_fd); >> + vdev->vq[idx].err_fd = -1; >> + } >> + >> + /* cppcheck-suppress redundantAssignment */ >> + vdev->vq[idx].err_fd = nofd ? -1 : msg->fds[0]; > > Maybe you missed this comment to v3: > > -- > Wouldn't it be easier (and not require a suppression) to say: > > if (!nofd) > vdev->vq[idx].err_fd = msg->fds[0]; > Yes, you're right. I thought I fixed that but I think I have overwritten my changes... (I also changed in this way call_fd and kick_fd). ... >> +/** >> + * vu_set_protocol_features_exec() - Enable protocol (vhost-user) features >> + * @vdev: vhost-user device >> + * @vmsg: vhost-user message >> + * >> + * Return: False as no reply is requested >> + */ >> +static bool vu_set_protocol_features_exec(struct vu_dev *vdev, >> + struct vhost_user_msg *msg) >> +{ >> + uint64_t features = msg->payload.u64; >> + >> + debug("u64: 0x%016"PRIx64, features); >> + >> + vdev->protocol_features = msg->payload.u64; >> + >> + if (vu_has_protocol_feature(vdev, >> + VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS) && >> + (!vu_has_protocol_feature(vdev, VHOST_USER_PROTOCOL_F_BACKEND_REQ) || >> + !vu_has_protocol_feature(vdev, VHOST_USER_PROTOCOL_F_REPLY_ACK))) { > > Same as v3: > > -- > Do we actually care about VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS at > all, I wonder? This whole part (coming from ff1320050a3a "libvhost-user: > implement in-band notifications") is rather hard to read/understand, so > it would be great if we could just get rid of it altogether. > > But if not, sure, let's leave it like the original, I'd say. I remove it. > -- > >> + /* >> + * The use case for using messages for kick/call is simulation, to make >> + * the kick and call synchronous. To actually get that behaviour, both >> + * of the other features are required. >> + * Theoretically, one could use only kick messages, or do them without >> + * having F_REPLY_ACK, but too many (possibly pending) messages on the >> + * socket will eventually cause the master to hang, to avoid this in >> + * scenarios where not desired enforce that the settings are in a way >> + * that actually enables the simulation case. >> + */ >> + die("F_IN_BAND_NOTIFICATIONS requires F_BACKEND_REQ && F_REPLY_ACK"); >> + } >> + >> + return false; >> +} Thanks, Laurent