From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=X4Wa/npr; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTP id 4ADDB5A061D for ; Thu, 14 Nov 2024 11:20:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1731579614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=U/JFWA5o3VzqCEJnrt75od1VX9LWJbUndpGrQx/rvYg=; b=X4Wa/nprlrsKagO9lFACGACjtheVZjp7gGfm4ajrSb6YkOhHww1+ReCVkn7erbJXpGuodb ruROACso91hrdaMsB+PTlVZP6BNTuSvuJT1jq7/0rxJNSnmBEicDZ9T2zj+2eA3WI1fgtc d5w7R3hqqTsGz6JAZI9J4TTWrixVi7o= Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-679-Q009BwgOMe6t1WQILCKJnA-1; Thu, 14 Nov 2024 05:20:12 -0500 X-MC-Unique: Q009BwgOMe6t1WQILCKJnA-1 X-Mimecast-MFC-AGG-ID: Q009BwgOMe6t1WQILCKJnA Received: by mail-lf1-f71.google.com with SMTP id 2adb3069b0e04-539fbbadca7so481047e87.3 for ; Thu, 14 Nov 2024 02:20:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731579611; x=1732184411; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=U/JFWA5o3VzqCEJnrt75od1VX9LWJbUndpGrQx/rvYg=; b=O2pMCIcf3PSEvMiYYuGEXROw7aA2v1G6RKBUy6n24xmhVU/Oefo7hmWK7uDVHCOe4s KRXJNa5jy5Ab1IIPVTfwB5mTQX49SUATCzqBpS3Dtk9ILu+MB25umpZ3S6mx4nv7wzUy Q6bRA9h4KwECeYXTHeDNWZOHbTdYwTvSnO+Mn0xjmeXqDebhyQmvjOoyEhwtAs5wOLU3 oXwtxYNjniqilujVqViIdiHMQGlrdTsK8NUz+y4fnxd+RclfgpJNwREwgfps4pNEkP0B mmkjEZBx2QHfmxMepXWCq99X2cxRh9Sc8DRborzD1RPcXtZSk4vjDRpBmLBGlfaKGe/L 50Cg== X-Gm-Message-State: AOJu0YymPzPGp3sZSxl8X9oix38T3IyoFdpI2U8UzS9mFLF4hz2hzKv7 l/xJzyxretLPMwYNRXXoN50Wxu6bJnVfw0z6eZ8LOxV/KwTFBi/So7wu4y+lAmct2Hw6wUJ7YgJ ZMpammOcRbbNr+s9hgihtGJntZ0R7S2uXy8kQcMy/6PHVtc2zEA== X-Received: by 2002:a05:6512:3043:b0:53b:163a:f295 with SMTP id 2adb3069b0e04-53d9fed2060mr3461166e87.38.1731579611028; Thu, 14 Nov 2024 02:20:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IH6y/Qg4MlnlymiMmKylgjbEpAGPZX+Q20jTib1oGMxd57AF9UFLLijk5cx2wj0AvtIZaJ3Iw== X-Received: by 2002:a05:6512:3043:b0:53b:163a:f295 with SMTP id 2adb3069b0e04-53d9fed2060mr3461143e87.38.1731579610428; Thu, 14 Nov 2024 02:20:10 -0800 (PST) Received: from ?IPV6:2a01:e0a:e10:ef90:343a:68f:2e91:95c? ([2a01:e0a:e10:ef90:343a:68f:2e91:95c]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3821ada2a8bsm1057367f8f.17.2024.11.14.02.20.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 14 Nov 2024 02:20:09 -0800 (PST) Message-ID: <880fe7a3-4925-49a0-8253-e130f976039e@redhat.com> Date: Thu, 14 Nov 2024 11:20:09 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v8 7/8] vhost-user: add vhost-user To: Stefano Brivio References: <20241010122903.1188992-1-lvivier@redhat.com> <20241010122903.1188992-8-lvivier@redhat.com> <20241017021034.437f3757@elisabeth> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: <20241017021034.437f3757@elisabeth> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: cC07nXeW5fj0Tq_aMN-y3W8BbrAHVu16C85efzI5ziA_1731579611 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: KS2PMTLUF424NCI3SP3KVOEUY7QRWVU6 X-Message-ID-Hash: KS2PMTLUF424NCI3SP3KVOEUY7QRWVU6 X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 17/10/2024 02:10, Stefano Brivio wrote: >> + if (frame_size == 0) >> + first = &iov_vu[i + 1]; >> + >> + if (iov_vu[i + 1].iov_len > (size_t)len) >> + iov_vu[i + 1].iov_len = len; >> + >> + len -= iov_vu[i + 1].iov_len; >> + iov_used++; >> + >> + frame_size += iov_vu[i + 1].iov_len; >> + num_buffers++; >> + >> + if (frame_size >= mss || len == 0 || >> + i + 1 == iov_cnt || !vu_has_feature(vdev, VIRTIO_NET_F_MRG_RXBUF)) { >> + if (i + 1 == iov_cnt) >> + check = NULL; >> + >> + /* restore first iovec base: point to vnet header */ >> + vu_set_vnethdr(vdev, first, num_buffers, l2_hdrlen); >> + >> + tcp_vu_prepare(c, conn, first, frame_size, &check); >> + if (*c->pcap) { >> + tcp_vu_update_check(tapside, first, num_buffers); >> + pcap_iov(first, num_buffers, >> + sizeof(struct virtio_net_hdr_mrg_rxbuf)); >> + } >> + >> + conn->seq_to_tap += frame_size; > We always increase this, even if, later... > >> + >> + frame_size = 0; >> + num_buffers = 0; >> + } >> + } >> + >> + /* release unused buffers */ >> + vu_queue_rewind(vq, iov_cnt - iov_used); >> + >> + /* send packets */ >> + vu_flush(vdev, vq, elem, iov_used); > we fail to send packets, that is, even if vu_queue_fill_by_index() > returns early because (!vq->vring.avail). vring.avail is a pointer to a structure. vring.avail is NULL if there is something wrong during the initialization. It's imported code, I think it's only a sanity check. So in theory vu_flush() cannot fail. > > We had this same issue on the non-vhost-user path until commit > a469fc393fa1 ("tcp, tap: Don't increase tap-side sequence counter for > dropped frames") (completely reworked with time). There, it was pretty > bad with small (default) values for wmem_max and rmem_max. > > Now, I_guess_ with vhost-user it won't be so easy to hit that, because > virtqueue buffers are (altogether) bigger, so we can probably fix this > later, but if it's not exceedingly complicated, we should consider > fixing it now. If we hit something like that, the behaviour is pretty > bad, with constant retransmissions and stalls. > > The mapping between queued frames and connections is done in > tcp_data_to_tap(), where tcp4_frame_conns[] and tcp6_frame_conns[] > items are set to the current (highest) value of tcp4_payload_used and > tcp6_payload_used. > > Then, if we fail to transmit some frames, tcp_revert_seq() uses those > arrays to revert the seq_to_tap values. > > I guess you could make vu_queue_fill_by_index() return an error, > propagate it, and make vu_flush() call something like tcp_revert_seq() > in case. Thanks, Laurent