From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202502 header.b=H7uK/pcj; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 2E7E15A0272 for ; Mon, 03 Feb 2025 03:17:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202502; t=1738549000; bh=VP5gnAgJiH55uSzX8UnJ762rfshaYsoVuZv3os9XEJY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=H7uK/pcjuHwLNZFv7nDI+G+JlHY1kEa7DeCrRaKFVUrfwl+cGP5+tYnRJhfzKQ0MJ bcVRXCSIOAEpzy57rGxfn2IvskzW1E3ptYi/Hyb/RmEXhlXJe04xeracdfC7gW9mCb 68UnGc1d0ZIkHjpbYdXG0kaIfKaaIXKGWN5oWRd0DifKtr9Huq9jQ7VxwBt6tE6GLl /sUpv0ztG8t8bV0TXxBGyK99caHt3BzSbx8hfsXP2CTYhMn0nvIY4E3W9OKh40alMq InDdJqZ2sH/WX5Dp5UbU1e52SIvY4LO8iWOeVkke5MfXOBriCOnOALzB/BjcQwe4ze SvsLdHkN2YYbw== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4YmVVc4dvpz4wyl; Mon, 3 Feb 2025 13:16:40 +1100 (AEDT) Date: Mon, 3 Feb 2025 13:16:41 +1100 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v3 14/20] migrate: Don't handle the migration channel through epoll Message-ID: References: <20250131193953.3034031-1-sbrivio@redhat.com> <20250131193953.3034031-15-sbrivio@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QQikgV3Xz7kRzXa3" Content-Disposition: inline In-Reply-To: <20250131193953.3034031-15-sbrivio@redhat.com> Message-ID-Hash: YVTBBRFZ5N2RZBEUO2MAVYWPIW3UIAIF X-Message-ID-Hash: YVTBBRFZ5N2RZBEUO2MAVYWPIW3UIAIF X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, Laurent Vivier X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --QQikgV3Xz7kRzXa3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 31, 2025 at 08:39:47PM +0100, Stefano Brivio wrote: > From: David Gibson >=20 > Currently, once a migration device state fd is assigned, we wait for > EPOLLIN or EPOLLOUT events on it to actually perform the migration. Chan= ge > it so that once a migration is requested it we complete it synchronously > at the end of the current epoll cycle. This has several advantages: >=20 > 1. It makes it clear that everything about the migration must be dealt > with at once, not split between multiple epoll events on the channel > 2. It ensures the migration always takes place between epoll cycles, > rather than, for example, between handling TCP events and their > deferred handling in post_handler(). > 3. It reduces code setting up the epoll watch on the fd. >=20 > Signed-off-by: David Gibson > --- > epoll_type.h | 2 -- > migrate.c | 44 +++++++++++--------------------------------- > migrate.h | 2 +- > passt.c | 6 ++---- > passt.h | 2 ++ > vu_common.c | 27 +++++++++++++++++++++++++++ > 6 files changed, 43 insertions(+), 40 deletions(-) >=20 > diff --git a/epoll_type.h b/epoll_type.h > index b981d30..7f2a121 100644 > --- a/epoll_type.h > +++ b/epoll_type.h > @@ -40,8 +40,6 @@ enum epoll_type { > EPOLL_TYPE_VHOST_CMD, > /* vhost-user kick event socket */ > EPOLL_TYPE_VHOST_KICK, > - /* migration device state channel */ > - EPOLL_TYPE_DEVICE_STATE, > /* TCP_REPAIR helper listening socket */ > EPOLL_TYPE_REPAIR_LISTEN, > /* TCP_REPAIR helper socket */ > diff --git a/migrate.c b/migrate.c > index fc6a043..faa7841 100644 > --- a/migrate.c > +++ b/migrate.c > @@ -50,7 +50,6 @@ static union migrate_header header =3D { > =20 > /* Data sections for version 1 */ > static struct iovec sections_v1[] =3D { > - { &header, sizeof(header) }, > }; > =20 > /* Set of data versions */ > @@ -333,26 +332,6 @@ static int migrate_target(struct ctx *c, int fd) > return rc; > } > =20 > -/** > - * set_migration_watch() - Add the migration file descriptor to epoll > - * @c: Execution context > - * @fd: File descriptor to add > - * @target: Are we the target of the migration? > - */ > -static void set_migration_watch(const struct ctx *c, int fd, bool target) > -{ > - union epoll_ref ref =3D { > - .type =3D EPOLL_TYPE_DEVICE_STATE, > - .fd =3D fd, > - }; > - struct epoll_event ev =3D { 0 }; > - > - ev.data.u64 =3D ref.u64; > - ev.events =3D target ? EPOLLIN : EPOLLOUT; > - > - epoll_ctl(c->epollfd, EPOLL_CTL_ADD, ref.fd, &ev); > -} > - > /** > * migrate_init() - Set up things necessary for migration > * @c: Execution context > @@ -372,7 +351,6 @@ void migrate_close(struct ctx *c) > { > if (c->device_state_fd !=3D -1) { > debug("Closing migration channel, fd: %d", c->device_state_fd); > - epoll_del(c, c->device_state_fd); > close(c->device_state_fd); > c->device_state_fd =3D -1; > c->device_state_result =3D -1; > @@ -393,27 +371,27 @@ void migrate_request(struct ctx *c, int fd, bool ta= rget) > migrate_close(c); > =20 > c->device_state_fd =3D fd; > - set_migration_watch(c, c->device_state_fd, target); > - > + c->migrate_target =3D target; > } > =20 > /** > * migrate_handler() - Send/receive passt internal state to/from QEMU > * @c: Execution context > - * @events: epoll events > */ > -void migrate_handler(struct ctx *c, uint32_t events) > +void migrate_handler(struct ctx *c) > { > - int rc =3D EIO; > + int rc; > =20 > - debug("migrate_handler fd %d events %x", c->device_state_fd, events); > + if (c->device_state_fd < 0) > + return; > =20 > - if (events & EPOLLOUT) > - rc =3D migrate_source(c, c->device_state_fd); > - else if (events & EPOLLIN) > - rc =3D migrate_target(c, c->device_state_fd); > + debug("migrate_handler fd %d target %d", > + c->device_state_fd, c->migrate_target); > =20 > - /* EPOLLHUP without EPOLLIN/EPOLLOUT, or EPOLLERR? Migration failed */ > + if (c->migrate_target) > + rc =3D migrate_target(c, c->device_state_fd); > + else > + rc =3D migrate_source(c, c->device_state_fd); > =20 > migrate_close(c); > =20 > diff --git a/migrate.h b/migrate.h > index a222c48..158241f 100644 > --- a/migrate.h > +++ b/migrate.h > @@ -79,6 +79,6 @@ struct migrate_target_handlers { > void migrate_init(struct ctx *c); > void migrate_close(struct ctx *c); > void migrate_request(struct ctx *c, int fd, bool target); > -void migrate_handler(struct ctx *c, uint32_t events); > +void migrate_handler(struct ctx *c); > =20 > #endif /* MIGRATE_H */ > diff --git a/passt.c b/passt.c > index 3c3a331..1938290 100644 > --- a/passt.c > +++ b/passt.c > @@ -76,7 +76,6 @@ char *epoll_type_str[] =3D { > [EPOLL_TYPE_TAP_LISTEN] =3D "listening qemu socket", > [EPOLL_TYPE_VHOST_CMD] =3D "vhost-user command socket", > [EPOLL_TYPE_VHOST_KICK] =3D "vhost-user kick socket", > - [EPOLL_TYPE_DEVICE_STATE] =3D "migration device state channel", > [EPOLL_TYPE_REPAIR_LISTEN] =3D "TCP_REPAIR helper listening socket", > [EPOLL_TYPE_REPAIR] =3D "TCP_REPAIR helper socket", > }; > @@ -360,9 +359,6 @@ loop: > case EPOLL_TYPE_VHOST_KICK: > vu_kick_cb(c.vdev, ref, &now); > break; > - case EPOLL_TYPE_DEVICE_STATE: > - migrate_handler(&c, eventmask); > - break; > case EPOLL_TYPE_REPAIR_LISTEN: > repair_listen_handler(&c, eventmask); > break; > @@ -377,5 +373,7 @@ loop: > =20 > post_handler(&c, &now); > =20 > + migrate_handler(&c); > + > goto loop; > } > diff --git a/passt.h b/passt.h > index 5992cbe..4189a4a 100644 > --- a/passt.h > +++ b/passt.h > @@ -241,6 +241,7 @@ struct ip6_ctx { > * @vdev: vhost-user device > * @device_state_fd: Device state migration channel > * @device_state_result: Device state migration result > + * @migrate_target: Is this the target for next migration? > */ > struct ctx { > enum passt_modes mode; > @@ -313,6 +314,7 @@ struct ctx { > /* Migration */ > int device_state_fd; > int device_state_result; > + bool migrate_target; > }; > =20 > void proto_update_l2_buf(const unsigned char *eth_d, > diff --git a/vu_common.c b/vu_common.c > index 78d1c1b..4797ef9 100644 > --- a/vu_common.c > +++ b/vu_common.c > @@ -305,3 +305,30 @@ err: > =20 > return -1; > } > + > +/** > + * vu_migrate() - Send/receive passt internal state to/from QEMU > + * @c: Execution context > + * @events: epoll events > + */ > +void vu_migrate(struct ctx *c, uint32_t events) > +{ > + struct vu_dev *vdev =3D c->vdev; > + int rc =3D EIO; > + > + debug("vu_migrate fd %d events %x", vdev->device_state_fd, events); > + > + if (events & EPOLLOUT) > + rc =3D migrate_source(c, vdev->device_state_fd); > + else if (events & EPOLLIN) > + rc =3D migrate_target(c, vdev->device_state_fd); > + > + /* EPOLLHUP without EPOLLIN/EPOLLOUT, or EPOLLERR? Migration failed */ > + > + vdev->device_state_result =3D rc; > + > + epoll_ctl(c->epollfd, EPOLL_CTL_DEL, vdev->device_state_fd, NULL); > + debug("Closing migration channel"); > + close(vdev->device_state_fd); > + vdev->device_state_fd =3D -1; > +} Also, this hunk, restoring the function removed in an earlier patch seems to have crept in here, which also breaks build. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --QQikgV3Xz7kRzXa3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmegJwgACgkQzQJF27ox 2GcMfQ/9F1FXU1vZA1v/TEFeXW+SX669E/UTDlcLgVU7YBo7XEeqMliB7+2CHsCz TEJ3HNgIznit5hCzARlrww1tco3qnK9MNlVZffSg2W+ZiYGiHeMwg3sg6W2u3Lx/ V58wiPcgudzAGH/U3K/w60AFZ3gfTkk+N+0SD8cG0caO7o3DtV4AKo6HYD9aj73V kZP0WFR2TM+iHrwWj366h0jfO0RDcQg3PXNQGpJicS+0N0sLfIYpNFeJp79pbKcF pKgLcuoy71s9rKMXfZcHft9cXDowT7e/XFgWPW8QK5/uuAr50zUId9ILHiyg6ql3 kKEOAt0P1JxXUrNIxwUKnIydvLXinmLEI9zX60y2Yxt40hXeWoPa4TtGFsx41wE4 PXl9gAm+3E1PoLqpVsB/p/C2Vozw6yWu0BnsivbhWJxazSka3lg0ookFEWKImxXb MJtWjdsB/L8G4dWAmfKBX4yBGTNL7VGKPEOT8odbLKlA63YIEobdiNex5NCmYsOJ YMuRNGL9FZ3PPhYjGKCbZojnlQoflfnK8fuBRyzscjU8h+1gx93wq1DA6jR++RIc 1DnjocmVDTrDOkkqP2NlbtDq0VUxiuXwi9P0LDIwG4y/zndRyVy693Xt1xO8752/ P7tsnGyLh97lYLqJIbSiiH8JlR9ZiO8H6okwxnJZzF04SyRXnEc= =KY+p -----END PGP SIGNATURE----- --QQikgV3Xz7kRzXa3--