From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=H0H4qjnA; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id 9373A5A004E for ; Wed, 27 Nov 2024 11:11:39 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1732702298; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=EsNKVIOkbfU56+e5plNz6riDXtpMuhSWkd4j4tI3B8k=; b=H0H4qjnAXpS/2XDMKAVb+gY00dmqCOmZC+OOkXp+IQ7Wc4RszlPj9gDKARFMNxsElkkOXR DeAkicbL41ARnBkBePsPFAd2S4tqTMoiag7TFnOJhodPYymCxmpQuW1ORHVDakpaBmkuZF Sp7r6781XEVd/MAVo72zVoluqSla8T0= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-74-yGZZTEM9Nj6sI3Jucb5IsA-1; Wed, 27 Nov 2024 05:11:36 -0500 X-MC-Unique: yGZZTEM9Nj6sI3Jucb5IsA-1 X-Mimecast-MFC-AGG-ID: yGZZTEM9Nj6sI3Jucb5IsA Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4349df2d87dso32710445e9.2 for ; Wed, 27 Nov 2024 02:11:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732702295; x=1733307095; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=EsNKVIOkbfU56+e5plNz6riDXtpMuhSWkd4j4tI3B8k=; b=E1J1Ihjq76GlPcP5GJ/wh9iXosHKGVR+0t7dZTmLAZN98soc2Ta1AfV5cmp94Sri9P PEIH+9LDOHpRInnhSkxBTubeLU7PAawARZgKvE2ZdQRY9LrVdnUWGfljzoIVqOj3j0or whhMzMiIWUNgj7e9ja+syWYDVcgMuBF/JYYN/V25yj4bss0A5BpUFAXag4ibM+039uVs bL++gNH3TPDZDN0q/sziXZUx5JRwMclfTkBY89MQB3Z3M1CkQNA1AzZrEwujWqh6xWSv mygfU4vU5PN3mzL/Wl0ExazyzNyJ7FOE1UZXpNWaAMOdgjjWpzATPAIxOXdmpe3FUp0n rGQA== X-Gm-Message-State: AOJu0YygKcXeElI39sUhLsWiKsvMoAMFcJZRo6xp7NQNxEk80RVHDB62 h3zO+hdrbj283kCgrMCZRqhfN936k2fQ+1n7igiLPBG2CzFBBwZT5kDD145S7M1iYTFGWs+myd/ V2PL81A4Jc5Xk8wK5wM9+jrAUX5nINbaSKbnFAz3FEoHPpGgiPuTj4XjBJQ== X-Gm-Gg: ASbGncuQe+bhooIf315ROGMAY7FTqrm54huUSOIZ0kJFQH7HYe45HdFtWtE1EmAmaAZ I8hzs8V7E5U8C24eq6V2QFfpAM8ACfbjnHEJoELKc67czNW0w749mcniwVbAKC0+f1UXxCsq5Zn dOeH3jYLg3j0QyoP2bjCHJ+HaQxlGSREIwG95THzq27JSW4cPL0T37RzeHuCnqvVnR/tU1nO/YY 2TuruS+Fhd2BCf7qnhhe8JCVaNCYFqSKT0sHDlQuUFRKMndSGdeP2VthDd9uqlfJLEk+gIYA3BS syssS+tnzWyI X-Received: by 2002:a05:600c:3ba5:b0:430:57e8:3c7e with SMTP id 5b1f17b1804b1-434a9df7c63mr18393725e9.28.1732702294936; Wed, 27 Nov 2024 02:11:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFJD2Wm66W2PR5yorMg3VjYxCOb0dKQVSyVpoL14Hw4wL+Mdvo4sfCejB105rOpV3PboRH73A== X-Received: by 2002:a05:600c:3ba5:b0:430:57e8:3c7e with SMTP id 5b1f17b1804b1-434a9df7c63mr18393495e9.28.1732702294450; Wed, 27 Nov 2024 02:11:34 -0800 (PST) Received: from ?IPV6:2a01:e0a:e10:ef90:343a:68f:2e91:95c? ([2a01:e0a:e10:ef90:343a:68f:2e91:95c]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3825fb25d74sm16106464f8f.47.2024.11.27.02.11.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 Nov 2024 02:11:33 -0800 (PST) Message-ID: Date: Wed, 27 Nov 2024 11:11:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v14 7/9] vhost-user: add vhost-user To: Stefano Brivio References: <20241122164337.3377854-1-lvivier@redhat.com> <20241122164337.3377854-8-lvivier@redhat.com> <20241127054749.7f1cfb25@elisabeth> <20241127104514.5a09c0d0@elisabeth> <83566556-2d9b-42ae-8876-588fe6b02b17@redhat.com> <20241127110355.402b1dbe@elisabeth> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: <20241127110355.402b1dbe@elisabeth> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 04hKaykssJeDdc8V5VETjbT8NsOM9vsRCzkZin6kgM4_1732702295 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: HMAYHO7UUULK6ZZ5UJMHXRIWDYUZGLKZ X-Message-ID-Hash: HMAYHO7UUULK6ZZ5UJMHXRIWDYUZGLKZ X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 27/11/2024 11:03, Stefano Brivio wrote: > On Wed, 27 Nov 2024 10:48:41 +0100 > Laurent Vivier wrote: > >> On 27/11/2024 10:45, Stefano Brivio wrote: >>> On Wed, 27 Nov 2024 10:09:53 +0100 >>> Laurent Vivier wrote: >>> >>>> On 27/11/2024 05:47, Stefano Brivio wrote: >>>>> On Fri, 22 Nov 2024 17:43:34 +0100 >>>>> Laurent Vivier wrote: >>>>> >>>>>> +/** >>>>>> + * tcp_vu_send_flag() - Send segment with flags to vhost-user (no payload) >>>>>> + * @c: Execution context >>>>>> + * @conn: Connection pointer >>>>>> + * @flags: TCP flags: if not set, send segment only if ACK is due >>>>>> + * >>>>>> + * Return: negative error code on connection reset, 0 otherwise >>>>>> + */ >>>>>> +int tcp_vu_send_flag(const struct ctx *c, struct tcp_tap_conn *conn, int flags) >>>>>> +{ >>>>>> + struct vu_dev *vdev = c->vdev; >>>>>> + struct vu_virtq *vq = &vdev->vq[VHOST_USER_RX_QUEUE]; >>>>>> + const struct flowside *tapside = TAPFLOW(conn); >>>>>> + size_t l2len, l4len, optlen, hdrlen; >>>>>> + struct vu_virtq_element flags_elem[2]; >>>>>> + struct tcp_payload_t *payload; >>>>>> + struct ipv6hdr *ip6h = NULL; >>>>>> + struct iovec flags_iov[2]; >>>>>> + struct iphdr *iph = NULL; >>>>>> + struct ethhdr *eh; >>>>>> + uint32_t seq; >>>>>> + int elem_cnt; >>>>>> + int nb_ack; >>>>>> + int ret; >>>>>> + >>>>>> + hdrlen = tcp_vu_hdrlen(CONN_V6(conn)); >>>>>> + >>>>>> + vu_set_element(&flags_elem[0], NULL, &flags_iov[0]); >>>>>> + >>>>>> + elem_cnt = vu_collect(vdev, vq, &flags_elem[0], 1, >>>>>> + hdrlen + sizeof(struct tcp_syn_opts), NULL); >>>>> >>>>> Oops, I made this crash, by starting a number of iperf3 client threads >>>>> on the host: >>>>> >>>>> $ iperf3 -c localhost -p 6001 -Z -l 500 -w 256M -t 600 -P20 >>>>> >>>>> with matching server in the guest, then terminating QEMU while the test >>>>> is running. >>>>> >>>>> Details (I saw it first, then I reproduced it under gdb): >>>>> >>>>> accepted connection from PID 3115463 >>>>> NDP: received RS, sending RA >>>>> DHCP: offer to discover >>>>> from 52:54:00:12:34:56 >>>>> DHCP: ack to request >>>>> from 52:54:00:12:34:56 >>>>> NDP: sending unsolicited RA, next in 212s >>>>> Client connection closed >>>>> >>>>> Program received signal SIGSEGV, Segmentation fault. >>>>> 0x00005555555884f5 in vring_avail_idx (vq=0x555559343f10 ) at virtio.c:138 >>>>> 138 vq->shadow_avail_idx = le16toh(vq->vring.avail->idx); >>>>> (gdb) list >>>>> 133 * >>>>> 134 * Return: the available ring index of the given virtqueue >>>>> 135 */ >>>>> 136 static inline uint16_t vring_avail_idx(struct vu_virtq *vq) >>>>> 137 { >>>>> 138 vq->shadow_avail_idx = le16toh(vq->vring.avail->idx); >>>>> 139 >>>>> 140 return vq->shadow_avail_idx; >>>>> 141 } >>>>> 142 >>>>> (gdb) bt >>>>> #0 0x00005555555884f5 in vring_avail_idx (vq=0x555559343f10 ) at virtio.c:138 >>>>> #1 vu_queue_empty (vq=vq@entry=0x555559343f10 ) at virtio.c:290 >>>>> #2 vu_queue_pop (dev=dev@entry=0x555559343a00 , vq=vq@entry=0x555559343f10 , elem=elem@entry=0x7ffffff6f510) at virtio.c:505 >>>>> #3 0x0000555555588c8c in vu_collect (vdev=vdev@entry=0x555559343a00 , vq=vq@entry=0x555559343f10 , elem=elem@entry=0x7ffffff6f510, max_elem=max_elem@entry=1, >>>>> size=size@entry=74, frame_size=frame_size@entry=0x0) at vu_common.c:86 >>>>> #4 0x000055555557e00e in tcp_vu_send_flag (c=0x7ffffff6f7a0, conn=0x5555555bd2d0 , flags=4) at tcp_vu.c:116 >>>>> #5 0x0000555555578125 in tcp_send_flag (flags=4, conn=0x5555555bd2d0 , c=0x7ffffff6f7a0) at tcp.c:1278 >>>>> #6 tcp_rst_do (conn=, c=) at tcp.c:1293 >>>>> #7 tcp_timer_handler (c=c@entry=0x7ffffff6f7a0, ref=..., ref@entry=...) at tcp.c:2266 >>>>> #8 0x0000555555558f26 in main (argc=, argv=) at passt.c:342 >>>>> (gdb) p *vq >>>>> $1 = {vring = {num = 256, desc = 0x0, avail = 0x0, used = 0x0, log_guest_addr = 4338774592, flags = 0}, last_avail_idx = 35133, shadow_avail_idx = 35133, used_idx = 35133, signalled_used = 0, >>>>> signalled_used_valid = false, notification = true, inuse = 0, call_fd = -1, kick_fd = -1, err_fd = -1, enable = 1, started = false, vra = {index = 0, flags = 0, desc_user_addr = 139660501995520, >>>>> used_user_addr = 139660502000192, avail_user_addr = 139660501999616, log_guest_addr = 4338774592}} >>>>> (gdb) p *vq->vring.avail >>>>> Cannot access memory at address 0x0 >>>>> >>>>> ...so we're sending a RST segment to the guest, but the ring doesn't >>>>> exist anymore. >>>>> >>>>> By the way, I still have the gdb session running, if you need something >>>>> else out of it. >>>>> >>>>> Now, I guess we should eventually introduce a more comprehensive >>>>> handling of the case where the guest suddenly terminates (not specific >>>>> to vhost-user), but given that we have vu_cleanup() working as expected >>>>> in this case, I wonder if we shouldn't simply avoid calling >>>>> vring_avail_idx() (it has a single caller) by checking for !vring.avail >>>>> in the caller, or something like that. >>>>> >>>> >>>> Yes, I think it's the lines I removed during the reviews: >>>> >>>> if (!vq->vring.avail) >>>> return true; >>> >>> Ah, right: >>> >>> https://archives.passt.top/passt-dev/20241114163859.7eeafa38@elisabeth/ >>> >>> ...so, at least in our case, it's more than "sanity checks" after all. >>> :) Well, I guess it depends on the definition. >>> >>>> Could you try to checkout virtio.c from v11? >>> >>> That would take a rather lengthy rebase, but I tried to reintroduce all >>> the checks you had: >>> >>> -- >>> diff --git a/virtio.c b/virtio.c >>> index 6a97435..0598ff4 100644 >>> --- a/virtio.c >>> +++ b/virtio.c >>> @@ -284,6 +284,9 @@ static int virtqueue_read_next_desc(const struct vring_desc *desc, >>> */ >>> bool vu_queue_empty(struct vu_virtq *vq) >>> { >>> + if (!vq->vring.avail) >>> + return true; >>> + >>> if (vq->shadow_avail_idx != vq->last_avail_idx) >>> return false; >>> >>> @@ -327,6 +330,9 @@ static bool vring_can_notify(const struct vu_dev *dev, struct vu_virtq *vq) >>> */ >>> void vu_queue_notify(const struct vu_dev *dev, struct vu_virtq *vq) >>> { >>> + if (!vq->vring.avail) >>> + return; >>> + >>> if (!vring_can_notify(dev, vq)) { >>> debug("vhost-user: virtqueue can skip notify..."); >>> return; >>> @@ -502,6 +508,9 @@ int vu_queue_pop(struct vu_dev *dev, struct vu_virtq *vq, struct vu_virtq_elemen >>> unsigned int head; >>> int ret; >>> >>> + if (!vq->vring.avail) >>> + return -1; >>> + >>> if (vu_queue_empty(vq)) >>> return -1; >>> >>> @@ -591,6 +600,9 @@ void vu_queue_fill_by_index(struct vu_virtq *vq, unsigned int index, >>> { >>> struct vring_used_elem uelem; >>> >>> + if (!vq->vring.avail) >>> + return; >>> + >>> idx = (idx + vq->used_idx) % vq->vring.num; >>> >>> uelem.id = htole32(index); >>> @@ -633,6 +645,9 @@ void vu_queue_flush(struct vu_virtq *vq, unsigned int count) >>> { >>> uint16_t old, new; >>> >>> + if (!vq->vring.avail) >>> + return; >>> + >>> /* Make sure buffer is written before we update index. */ >>> smp_wmb(); >>> >>> -- >>> >>> and it's all fine with those, I tried doing a few nasty things and >>> didn't observe any issue. >>> >>> Any check I missed? Do you want to submit it as follow-up patch? I can >>> also do that. I'd rather (still) avoid a re-post of v14 if possible. >> >> As you prefer. Let me know. > > It would save me some time if you could... it should be based on v14 as > it is. I can. > > I didn't have time to take care of gcc warnings on 32-bit and of the > build failure on musl, yet. > I will do too. Do you want them before to merge? Thanks, Laurent