From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202502 header.b=OTAVa4ZW; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 251935A061F for ; Wed, 05 Feb 2025 03:32:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202502; t=1738722703; bh=4b7UAa3Qwhaq7HL//SFe8pivbR7/cb1dA1QDZPxQ++U=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=OTAVa4ZWHaqwgWSAa8dQmO2pbGsGaJn+sl4qVnH5MsR7zAnG2x+KVC3ktC2aUagMX syrkdNGhV8vN5MaKdkbISiGHkkmfL3jTz5QIcnNV99AxQqPwxFmCW4PgKheClJFf8z EpqPEfhmIqpKWkpBDsH6yKqNHy1uhIWiFn0aacSVeOhW8eJxgEOmE1zHvyKMd1dP3Q mg369p2+QuJVMxJX95bSrhiSthiWS86xCuRG2McFke08PYnNvsvrfJ3opRLd1NrgIo 1Vxb4oGqDRZPJMoK9jPriKTPpe/kkTad5KfeQLoWrnE533II+nxqK9hQzC8l+SUR2d t9N/m71S4ug8A== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4Ynkl31YwYz4wcn; Wed, 5 Feb 2025 13:31:43 +1100 (AEDT) Date: Wed, 5 Feb 2025 13:09:42 +1100 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v5 5/6] vhost_user: Make source quit after reporting migration state Message-ID: References: <20250205003904.2797491-1-sbrivio@redhat.com> <20250205003904.2797491-6-sbrivio@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="75WPzdbfxCdGE1TW" Content-Disposition: inline In-Reply-To: <20250205003904.2797491-6-sbrivio@redhat.com> Message-ID-Hash: D4N4ZNMHPN6VJICQAVGQIOB4DJQMRH5W X-Message-ID-Hash: D4N4ZNMHPN6VJICQAVGQIOB4DJQMRH5W X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, Laurent Vivier X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --75WPzdbfxCdGE1TW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 05, 2025 at 01:39:03AM +0100, Stefano Brivio wrote: > On migration, the source process asks passt-helper to set TCP sockets > in repair mode, dumps the information we need to migrate connections, > and closes them. >=20 > At this point, we can't pass them back to passt-helper using > SCM_RIGHTS, because they are closed, from that perspective, and > sendmsg() will give us EBADF. But if we don't clear repair mode, the > port they are bound to will not be available for binding in the > target. >=20 > Terminate once we're done with the migration and we reported the > state. This is equivalent to clearing repair mode on the sockets we > just closed. As we've discussed, quitting still makes sense, but the description above is not really accurate. Perhaps, =3D=3D=3D Once we've passed the migration's "point of no return", there's no way to resume the guest on the source side, because we no longer own the connections. There's not really anything we can do except exit. =3D=3D=3D Except.. thinking about it, I'm not sure that's technically true. After migration, the source qemu enters a kind of limbo state. I suppose for the case of to-disk migration (savevm) the guest can actually be resumed. Which for us is not really compatible with completing at least a local migration properly. Not really sure what to do about that. I think it's also technically possible to use monitor commands to boot up essentially an entirely new guest instance in the original qemu, in which case for us it would make sense to basically reset ourselves (flush the low table). Hrm.. we really need to know the sequence of events in a bit more detail to get this right (not that this stops improving the guts of the logic in the meantime). I'm asking around to see if I can find who did the migration stuff or virtiofsd, so we can compare notes. > Signed-off-by: Stefano Brivio > --- > vhost_user.c | 11 +++++++++++ > 1 file changed, 11 insertions(+) >=20 > diff --git a/vhost_user.c b/vhost_user.c > index b107d0f..70773d6 100644 > --- a/vhost_user.c > +++ b/vhost_user.c > @@ -997,6 +997,8 @@ static bool vu_send_rarp_exec(struct vu_dev *vdev, > return false; > } > =20 > +static bool quit_on_device_state =3D false; > + > /** > * vu_set_device_state_fd_exec() - Set the device state migration channel > * @vdev: vhost-user device > @@ -1024,6 +1026,9 @@ static bool vu_set_device_state_fd_exec(struct vu_d= ev *vdev, > migrate_request(vdev->context, msg->fds[0], > direction =3D=3D VHOST_USER_TRANSFER_STATE_DIRECTION_LOAD); > =20 > + if (direction =3D=3D VHOST_USER_TRANSFER_STATE_DIRECTION_SAVE) > + quit_on_device_state =3D true; > + > /* We don't provide a new fd for the data transfer */ > vmsg_set_reply_u64(msg, VHOST_USER_VRING_NOFD_MASK); > =20 > @@ -1201,4 +1206,10 @@ void vu_control_handler(struct vu_dev *vdev, int f= d, uint32_t events) > =20 > if (reply_requested) > vu_send_reply(fd, &msg); > + > + if (quit_on_device_state && > + msg.hdr.request =3D=3D VHOST_USER_CHECK_DEVICE_STATE) { > + info("Migration complete, exiting"); > + exit(EXIT_SUCCESS); > + } > } --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --75WPzdbfxCdGE1TW Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmeiyGUACgkQzQJF27ox 2GdiOg/9E5WP2aKOy218ukBkH5H0dAkYVy+oh8R7D7AakfOE/5xXoAC13eYNR76j cIc1aQZUFXiqxrlRxUUKtMrn1V5lKsX/RboAaqbgpEziNsXGy1k/7jkYB//j30eN pTmx4hSkL7FxoB3858IgFEgbgl+jONXaUZLknR52nNFPH83szpMdFfHAEyUeI2bg FyREJxJS4Cv8LEkpT63Rp9PIY9v86k6sD/5R9Si4dotG5490SJPKmNOkdiQlYZI6 EZIpve4QK6uqET7U2WQkzLyYGwAQ6zuFYd0Yf5xn4ZJpN8UF3r29ZqNl7GonAxvD pf2dbUJ2oh32k3LMNikTpXjj1SMHTN6lmmwHorX4g9RSISWTCELDk29i0QgWt05T X6qyJMFbFTpMjqMicmiDPSvuL39ToJpye86j44ZwClD99sIrmfYnuGHEDRBO7JAH u3q+oDMBP/u4ULxsW6VkieIazfx44heaXaCZOTEJJSaK6WKsyF6yhfLL5UamcLXU 3qz3gzL5+GXy383bpOZunz8c9dM6KGP6zmESVoLKt8DmAt1OiO5KV/z+sCcDH5oX 1xQ9eU0tb0/pRFM2xLTx6BXmT8pII+xgT8fiyh3GXDL7pY04+wqA+BDDzyTr5SDb fldMEEiWhrZU7X8k3wmoYciR/ESaEqsdhDg2HadVudvqbW6vT1s= =pQdy -----END PGP SIGNATURE----- --75WPzdbfxCdGE1TW--