From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=A3D+dVuy; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 1E29A5A061C for ; Fri, 17 Jan 2025 14:27:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737120464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=uAk74CBbqYg+VudxYryPz04JZiGH3YpsDu9TWYr8e8Y=; b=A3D+dVuyYfDJzLYy63UtQtnwz/5gvBWnXonxu4jc/fRgQ5cQfPcJzYWiAqogtuJehlwi6t lmcjeDhoY/KuRs9gNkSuELD1U/9qhkio9GcoLO5pbOly2YYnnxT1CYqhx/2yA/e/BOZAoX d4losFBm5M6YfZ19TKJEGMHVfHHIW0w= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-373-V4G7vWnuMtmSUY0g-mwHmw-1; Fri, 17 Jan 2025 08:27:42 -0500 X-MC-Unique: V4G7vWnuMtmSUY0g-mwHmw-1 X-Mimecast-MFC-AGG-ID: V4G7vWnuMtmSUY0g-mwHmw Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-6d8edc021f9so37702536d6.3 for ; Fri, 17 Jan 2025 05:27:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737120461; x=1737725261; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=uAk74CBbqYg+VudxYryPz04JZiGH3YpsDu9TWYr8e8Y=; b=pXFyLZYUA1jp215JCi8D6sv4H6fHNWctqZwvQW0E1GaIHFY8ltRNltP393tLoc4tSa lZ+3aZ9WTRALbkUtqNlPxzrblMXuJdouTZtChvpioYUhvCuLdHXimXYrz2NsrHlkcx5U 7I0Q8leX6lVIVHeEHwFB4he/l96+SvtaU29LQszXMyJZ5YeHID1s/YTYioKlvpaS/m0W bCYICBD5+gFiAbYFMHCajaeYcWGj9/79whws77YZFDFniRtWCngjZZ9FrEv51m7P3rMr O+o2h/Z90FXNuKGZlxbC192WJ6zqytZSkuiN3EVy5hfg7jiJgXILG/KmJnijXXBGiA1T GKQQ== X-Gm-Message-State: AOJu0YzdzTBC9nWJhviS6u86jM0t/W70DClIspSKAVilkdrS+P0qkmBK Hmgsli2XYyOr2I8DvRE7Lbg6lIRHNOm7IoWapHzH46uWAtcUWajt8Vo1t9wRnm31lhTNLm7dqJo XqXQ/UNuCgs+QB6/6R0uNt76Q3tDNSwnjLibwgkaeU3qd3JvT5Q== X-Gm-Gg: ASbGnctGzJ8Uo2aafahr/0zgFggpjGdi04e6CWC7gqoTaiZX/gF18rg05HGQ3itwm+x FVACanP3JHWdHAW/WzJ/mbTjKRyar1Yh94iOM0kmcR+8pBhYc+xqURIDPA18qTga+jk8vxSDC6T /uBBuNWrzM5RLCI8PVVCP+rP5WLYYqcp5M4szcNM3kENWrFBMGGUHGE1F6w7XyNSG9CDRJPbl5c uhS2FJDnQP+EE1EpFX8ziEkrtrrj0YQffnuOiLC4+pP1+cP+sls1uGxdfZDIVvJujmMPpkWKYcK u8NKRGyXajXFd/1BJSc= X-Received: by 2002:a05:6214:3197:b0:6d8:8a01:64e2 with SMTP id 6a1803df08f44-6e1b21e2e0emr41138336d6.43.1737120461098; Fri, 17 Jan 2025 05:27:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IGp8iETtn6IYfOfQVC0QIsiNDeJFN4KAB3ZtprQrEvNtllZiSHDSWbYOUfZjyb882aIMuHaVg== X-Received: by 2002:a05:6214:3197:b0:6d8:8a01:64e2 with SMTP id 6a1803df08f44-6e1b21e2e0emr41137916d6.43.1737120460607; Fri, 17 Jan 2025 05:27:40 -0800 (PST) Received: from ?IPV6:2a01:e0a:e10:ef90:343a:68f:2e91:95c? ([2a01:e0a:e10:ef90:343a:68f:2e91:95c]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7be6147fa95sm112535485a.39.2025.01.17.05.27.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Jan 2025 05:27:40 -0800 (PST) Message-ID: Date: Fri, 17 Jan 2025 14:27:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/9] vhost-user: Migration support To: Stefano Brivio References: <20241219111400.2352110-1-lvivier@redhat.com> <329e349d-f86c-479b-97ee-61397aa28c60@redhat.com> <20250117134445.5b341bdd@elisabeth> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: <20250117134445.5b341bdd@elisabeth> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: aDWNG4mdohsCwNeB2yAHww97dOBT7zKrfzruBGlR2SE_1737120461 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: XPNAJPGZMASSQ35ISCOIU7VTLYA7A4IU X-Message-ID-Hash: XPNAJPGZMASSQ35ISCOIU7VTLYA7A4IU X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 17/01/2025 13:44, Stefano Brivio wrote: > On Fri, 17 Jan 2025 13:13:36 +0100 > Laurent Vivier wrote: > >> On 19/12/2024 12:13, Laurent Vivier wrote: >>> This series allows a QEMU guest to be migrated while it is connected >>> to Passt using vhost-user interface. >>> >>> There are two parts: >>> >>> - first part enables the migration of QEMU without transferring the >>> internal state of Passt. All connections are lost. >>> >>> This is done by implementing VHOST_USER_SET_LOG_FD and >>> VHOST_USER_SET_LOG_BASE commands (and enabling >>> VHOST_USER_PROTOCOL_F_LOG_SHMFD feature) >>> >>> "vhost-user: add VHOST_USER_SET_LOG_FD command" >>> "vhost-user: add VHOST_USER_SET_LOG_BASE command" >>> "vhost-user: Report to front-end we support VHOST_USER_PROTOCOL_F_LOG_SHMFD" >>> >>> - second part allows source Passt instance to send its internal >>> state using QEMU migration channel to destination Passt instance. >>> >>> This is done implementing VHOST_USER_SET_DEVICE_STATE_FD and >>> VHOST_USER_CHECK_DEVICE_STATE (and enabling >>> VHOST_USER_PROTOCOL_F_DEVICE_STATE feature). >>> >>> "vhost-user: add VHOST_USER_CHECK_DEVICE_STATE command" >>> "vhost-user: add VHOST_USER_SET_DEVICE_STATE_FD command" >>> "vhost-user: Report to front-end we support VHOST_USER_PROTOCOL_F_DEVICE_STATE" >>> >>> For now, it only implements the function needed to transfer the >>> state but no state is transferred. >>> >>> VHOST_USER_PROTOCOL_F_DEVICE_STATE is implemented in QEMU >>> only for vhost-user-fs, to be able to use it with virtio-net >>> I have proposed a patch upstream: >>> >>> https://patchew.org/QEMU/20241218143453.1573185-1-lvivier@redhat.com/ >>> >>> Laurent Vivier (9): >>> virtio: Use const pointer for vu_dev >>> vhost-user: update protocol features and commands list >>> vhost-user: add VHOST_USER_SET_LOG_FD command >>> vhost-user: Pass vu_dev to more virtio functions >>> vhost-user: add VHOST_USER_SET_LOG_BASE command >>> vhost-user: Report to front-end we support >>> VHOST_USER_PROTOCOL_F_LOG_SHMFD >>> vhost-user: add VHOST_USER_CHECK_DEVICE_STATE command >>> vhost-user: add VHOST_USER_SET_DEVICE_STATE_FD command >>> vhost-user: Report to front-end we support >>> VHOST_USER_PROTOCOL_F_DEVICE_STATE >>> >>> epoll_type.h | 2 + >>> passt.c | 4 + >>> util.h | 3 + >>> vhost_user.c | 251 ++++++++++++++++++++++++++++++++++++++++++++++++++- >>> vhost_user.h | 50 +++++++++- >>> virtio.c | 116 +++++++++++++++++++++--- >>> virtio.h | 32 +++++-- >>> vu_common.c | 59 +++++++++++- >>> vu_common.h | 3 +- >>> 9 files changed, 484 insertions(+), 36 deletions(-) >>> >> >> Another point: >> >> vhost-user can ask backend (passt) to send a notification at the end of the migration on >> the destination side to the network to update the network topology. Do we need this? >> >> This is VHOST_USER_SEND_RARP: >> "Ask vhost user back-end to broadcast a fake RARP to notify the migration is terminated >> for guest that does not support GUEST_ANNOUNCE. >> >> Only legal if feature bit VHOST_USER_F_PROTOCOL_FEATURES is present in >> VHOST_USER_GET_FEATURES and protocol feature bit VHOST_USER_PROTOCOL_F_RARP is present in >> VHOST_USER_GET_PROTOCOL_FEATURES. The first 6 bytes of the payload contain the mac address >> of the guest to allow the vhost user back-end to construct and broadcast the fake RARP." > > This isn't really clear :( ...what does "fake RARP" even mean? RARP is > a protocol, and an obsolete one, despite: > > https://en.wikipedia.org/wiki/Reverse_Address_Resolution_Protocol#Modern_Day_Uses > > Anyway, it comes from 3e866365e1eb ("vhost user: add rarp sending after > live migration for legacy guest"). I think legacy is for guest that doesn't support VIRTIO_NET_F_GUEST_ANNOUNCE: See "5.1.6.5.4 Gratuitous Packet Sending" https://docs.oasis-open.org/virtio/virtio/v1.2/cs01/virtio-v1.2-cs01.html#x1-2560004 With VIRTIO_NET_F_GUEST_ANNOUNCE, QEMU notifies the guest kernel to send a packet to the network to update the configuration. In the guest kernel, it seems to be done by (it sends an ARP packet): /** * __netdev_notify_peers - notify network peers about existence of @dev, * to be called when rtnl lock is already held. * @dev: network device * * Generate traffic such that interested network peers are aware of * @dev, such as by generating a gratuitous ARP. This may be used when * a device wants to inform the rest of the network about some sort of * reconfiguration such as a failover event or virtual machine * migration. */ void __netdev_notify_peers(struct net_device *dev) { ASSERT_RTNL(); call_netdevice_notifiers(NETDEV_NOTIFY_PEERS, dev); call_netdevice_notifiers(NETDEV_RESEND_IGMP, dev); } On its side, QEMU always send a RARP packet (see qemu_announce_self_iter()) that is catched in vhost_user_receive() to be changed into a VHOST_USER_SEND_RARP by vhost_user_migration_done(). > > I have no idea what "legacy" means here, but I'm under the impression that > somebody just needed a way to notify the guest of the finished migration > (nothing related to networking) and, correct me if I'm wrong, the French > word for "fake" is rather similar to the one for "dummy". Not the guest, the network. The guest is notified with the GUEST_ANNOUNCE stuff (if supported) > > So my understanding of it is that they would just send a dummy packet, > and a RARP broadcast would be a good candidate because it's otherwise > unused, but wouldn't raise any firewall alert. > > Or maybe "legacy" has something to do with VMware vSphere, and there was > some guest previously running on VMware vSphere that needs a RARP packet > to "proceed" with something after the migration. > > If that's the case, and if there's still something like that around, I > guess that ideally we want it for compatibility, but it's *very* unlikely > to ever bite us. > > So all in all I would say that if the implementation is very small and > unproblematic we can hack up something in arp.c to build a RARP packet > (example here: https://wiki.wireshark.org/RARP but that's a request, not > a broadcast). Otherwise I wouldn't really care until somebody asks. > It only depends on how passt will inform its neighbors about its new location in the network after migration. But my feeling is passt doesn't need to do that as it shares the IP address of the host and the guest MAC address is not shown beyond passt. So we can implement VHOST_USER_SEND_RARP with an empty function (at least to silence the "Vhost user backend fails to broadcast fake RARP" message). Thanks, Laurent