From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202508 header.b=bY66uWhj; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id C4DD85A026F for ; Fri, 03 Oct 2025 07:33:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202508; t=1759469623; bh=IVpw5GvgTzG0HkU7zTeY38AC04jk3Zc8cSQXVAeW5UQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bY66uWhjJuvSeSVb6wgtXSfmmdZgYhlBC0jxIJz9pgg7ANW1R7mXr3oSC0y53F8ry xIeIQ8dg0idDBXZA76fc60zZReQsDFoUphU/9SCNaKZ8TgIoToagjfnvwY9N+a4mdO 0q9nTCYqz/+5s7TB4DC8an7rPZ/Z2feq0XR0vlIlN2cM6BTw0wEbd4O/ZgJLJ/2sbg 0RPFlj82VltyQq/oDlejw5LtTCWUrYl9n2BE7yHzWcIoG4mzprNyNliZzaIlHApZfe 6imnrUR78TNdtwrczGL0pHXOHL04MuQWRh/x61quP6061/Ld3K5BaNhyebEgWPALat 3udc6rbhAoYAg== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4cdHQH5mpsz4wBC; Fri, 3 Oct 2025 15:33:43 +1000 (AEST) Date: Fri, 3 Oct 2025 14:01:39 +1000 From: David Gibson To: Jon Maloy Subject: Re: [PATCH v12 1/9] netlink: add subsciption on changes in NDP/ARP table Message-ID: References: <20251003003412.588801-1-jmaloy@redhat.com> <20251003003412.588801-2-jmaloy@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="coBjXxTHe+9IYiH1" Content-Disposition: inline In-Reply-To: <20251003003412.588801-2-jmaloy@redhat.com> Message-ID-Hash: G6M34WO5DMFTGJPZRBLNEWQMXLV4TWVZ X-Message-ID-Hash: G6M34WO5DMFTGJPZRBLNEWQMXLV4TWVZ X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: sbrivio@redhat.com, dgibson@redhat.com, passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --coBjXxTHe+9IYiH1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 02, 2025 at 08:34:04PM -0400, Jon Maloy wrote: > The solution to bug https://bugs.passt.top/show_bug.cgi?id=3D120 > requires the ability to translate from an IP address to its > corresponding MAC address in cases where those are present in > the ARP or NDP tables. >=20 > To keep track of the contents of these tables we add a netlink > based neighbour subscription feature. >=20 > Signed-off-by: Jon Maloy > Reviewed-by: David Gibson >=20 > --- > v3: - Added an attribute contianing NDA_DST to sent message, so > that we let the kernel do the filtering of the IP address > and return only one entry. > - Added interface index to the call signature. Since the only > interface we know is the template interface, this limits > the number of hosts that will be seen as 'network segment > local' from a PASST viewpoint. > v4: - Made loop independent of attribute order. > - Ignoring L2 addresses which are not of size ETH_ALEN. > v5: - Changed return value of new function, so caller can know if > a MAC address really was found. > v6: - Removed warning printout which had ended up in the wrong > commit. > v8: - Changed to neighbour event subscription model > - netlink: arp/ndp table subscription > v10:- Updated according to David's latest comments on v8 > - Added functionaly where we initially read current > state of ARP/NDP tables > v12:- Updates based on feedback from David and Stefano > --- > epoll_type.h | 2 + > netlink.c | 204 ++++++++++++++++++++++++++++++++++++++++++++++++++- > netlink.h | 4 + > passt.c | 7 ++ > 4 files changed, 214 insertions(+), 3 deletions(-) >=20 > diff --git a/epoll_type.h b/epoll_type.h > index 12ac59b..a90ffb6 100644 > --- a/epoll_type.h > +++ b/epoll_type.h > @@ -44,6 +44,8 @@ enum epoll_type { > EPOLL_TYPE_REPAIR_LISTEN, > /* TCP_REPAIR helper socket */ > EPOLL_TYPE_REPAIR, > + /* Netlink neighbour subscription socket */ > + EPOLL_TYPE_NL_NEIGH, > =20 > EPOLL_NUM_TYPES, > }; > diff --git a/netlink.c b/netlink.c > index c436780..3fe2fdd 100644 > --- a/netlink.c > +++ b/netlink.c > @@ -26,6 +26,7 @@ > #include > #include > #include > +#include > =20 > #include > #include > @@ -40,6 +41,10 @@ > #define RTNH_NEXT_AND_DEC(rtnh, attrlen) \ > ((attrlen) -=3D RTNH_ALIGN((rtnh)->rtnh_len), RTNH_NEXT(rtnh)) > =20 > +/* Convenience macro borrowed from kernel */ > +#define NUD_VALID \ > + (NUD_PERMANENT | NUD_NOARP | NUD_REACHABLE | NUD_PROBE | NUD_STALE) > + > /* Netlink expects a buffer of at least 8kiB or the system page size, > * whichever is larger. 32kiB is recommended for more efficient. > * Since the largest page size on any remotely common Linux setup is > @@ -50,9 +55,10 @@ > #define NLBUFSIZ 65536 > =20 > /* Socket in init, in target namespace, sequence (just needs to be monot= onic) */ > -int nl_sock =3D -1; > -int nl_sock_ns =3D -1; > -static int nl_seq =3D 1; > +int nl_sock =3D -1; > +int nl_sock_ns =3D -1; > +static int nl_sock_neigh =3D -1; > +static int nl_seq =3D 1; > =20 > /** > * nl_sock_init_do() - Set up netlink sockets in init or target namespace > @@ -1103,3 +1109,195 @@ int nl_link_set_flags(int s, unsigned int ifi, > =20 > return nl_do(s, &req, RTM_NEWLINK, 0, sizeof(req)); > } > + > +/** > + * nl_neigh_msg_read() - Interpret a neigbour state message from netlink > + * @c: Execution context > + * @nh: Message to be read > + * > + */ > +static void nl_neigh_msg_read(const struct ctx *c, struct nlmsghdr *nh) > +{ > + struct ndmsg *ndm =3D NLMSG_DATA(nh); > + struct rtattr *rta =3D (struct rtattr *)(ndm + 1); > + size_t na =3D NLMSG_PAYLOAD(nh, sizeof(*ndm)); > + char ip_str[INET6_ADDRSTRLEN]; > + char mac_str[ETH_ADDRSTRLEN]; > + const uint8_t *lladdr =3D NULL; > + const void *dst =3D NULL; > + size_t lladdr_len =3D 0; > + uint8_t mac[ETH_ALEN]; > + union inany_addr addr; > + size_t dstlen =3D 0; > + > + if (nh->nlmsg_type =3D=3D NLMSG_DONE) > + return; > + > + if (nh->nlmsg_type =3D=3D NLMSG_ERROR) { > + warn_perror("nlmsg_type error at msg read"); warn_perror() won't work here, because the error code isn't in global errno. See nl_status() for how to extract the error code from the message. > + return; > + } > + > + if (nh->nlmsg_type !=3D RTM_NEWNEIGH && > + nh->nlmsg_type !=3D RTM_DELNEIGH) > + return; > + > + for (; RTA_OK(rta, na); rta =3D RTA_NEXT(rta, na)) { > + switch (rta->rta_type) { > + case NDA_DST: > + dst =3D RTA_DATA(rta); > + dstlen =3D RTA_PAYLOAD(rta); > + break; > + case NDA_LLADDR: > + lladdr =3D RTA_DATA(rta); > + lladdr_len =3D RTA_PAYLOAD(rta); > + break; > + default: > + break; > + } > + } > + > + if (!dst) > + return; > + > + if (!lladdr || lladdr_len !=3D ETH_ALEN) > + return; Do we need lladdr for the DELNEIGH case? > + > + if (ndm->ndm_type !=3D ARPHRD_ETHER) > + return; > + > + memcpy(mac, lladdr, ETH_ALEN); > + eth_ntop(mac, mac_str, sizeof(mac_str)); > + > + if (ndm->ndm_family =3D=3D AF_INET && > + ndm->ndm_ifindex !=3D c->ifi4) > + return; > + > + if (ndm->ndm_family =3D=3D AF_INET6 && > + ndm->ndm_ifindex !=3D c->ifi6) > + return; > + > + if (ndm->ndm_family !=3D AF_INET && > + ndm->ndm_family !=3D AF_INET6) > + return; > + > + if (ndm->ndm_family =3D=3D AF_INET && > + dstlen !=3D sizeof(struct in_addr)) > + return; > + > + if (ndm->ndm_family =3D=3D AF_INET6 && > + dstlen !=3D sizeof(struct in6_addr)) > + return; I'd suggest a warn() for those last two cases, since it means the kernel is doing something really weird. > + inany_from_af(&addr, ndm->ndm_family, dst); > + inany_ntop(dst, ip_str, sizeof(ip_str)); > + > + if (nh->nlmsg_type =3D=3D RTM_NEWNEIGH && ndm->ndm_state & NUD_VALID) > + trace("neigh table update: %s / %s", ip_str, mac_str); > + else > + trace("neigh table delete: %s / %s", ip_str, mac_str); This will give a misleading message if you have RTM_NEWNEIGH and !NUD_VALID. > +} > + > +/** > + * nl_neigh_sync() - Read current contents ARP/NDP tables > + * @c: Execution context > + * @proto: Protocol, AF_INET or AF_INET6 > + * @ifi: Interface index > + * > + */ > +static void nl_neigh_sync(const struct ctx *c, int proto, int ifi) > +{ > + struct { > + struct nlmsghdr nlh; > + struct ndmsg ndm; > + } req =3D { > + .nlh =3D {0}, > + .ndm.ndm_family =3D proto, > + .ndm.ndm_ifindex =3D ifi, > + .ndm.ndm_state =3D 0, > + .ndm.ndm_flags =3D 0, > + .ndm.ndm_type =3D 0 > + }; > + struct nlmsghdr *nh; > + char buf[NLBUFSIZ]; > + ssize_t status; > + uint32_t seq; > + > + seq =3D nl_send(nl_sock_neigh, &req, RTM_GETNEIGH, NLM_F_DUMP, sizeof(r= eq)); > + nl_foreach_oftype(nh, status, nl_sock_neigh, buf, seq, RTM_NEWNEIGH) > + nl_neigh_msg_read(c, nh); > + if (status < 0) > + warn("Failed to read netlink message. status =3D=3D %li", status); > +} > + > +/** > + * nl_neigh_notify_handler() - Non-blocking drain of pending neighbour u= pdates > + * @c: Execution context > + */ > +void nl_neigh_notify_handler(const struct ctx *c) > +{ > + char buf[NLBUFSIZ]; > + > + for (;;) { > + ssize_t n =3D recv(nl_sock_neigh, buf, sizeof(buf), MSG_DONTWAIT); > + struct nlmsghdr *nh; > + > + if (n < 0) { > + if (errno !=3D EAGAIN) > + warn_perror("Failed to read from netlink sock"); > + return; > + } > + nh =3D (struct nlmsghdr *)buf; > + for (; NLMSG_OK(nh, n); nh =3D NLMSG_NEXT(nh, n)) > + nl_neigh_msg_read(c, nh); > + } > +} > + > +/** > + * nl_neigh_notify_init() - Open netlink sock and subscribe to neighbour= events > + * @c: Execution context > + * > + * Return: 0 on success, -1 on failure > + */ > +int nl_neigh_notify_init(const struct ctx *c) > +{ > + union epoll_ref ref =3D { > + .type =3D EPOLL_TYPE_NL_NEIGH > + }; > + struct epoll_event ev =3D { > + .events =3D EPOLLIN > + }; > + struct sockaddr_nl addr =3D { > + .nl_family =3D AF_NETLINK, > + .nl_groups =3D RTMGRP_NEIGH, > + }; > + > + if (nl_sock_neigh >=3D 0) { > + warn("RTMGRP_NEIGH already exists"); > + return 0; > + } > + nl_sock_neigh =3D socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, NETLINK_R= OUTE); > + if (nl_sock_neigh < 0) { > + warn("Failed create RTMGRP_NEIGH socket"); > + return -1; > + } > + if (bind(nl_sock_neigh, (struct sockaddr *)&addr, sizeof(addr)) < 0) { > + close(nl_sock_neigh); > + nl_sock_neigh =3D -1; > + warn("Failed bind RTMGRP_NEIGH socket"); > + return -1; > + } > + > + ev.data.u64 =3D ref.u64; > + if (epoll_ctl(c->epollfd, EPOLL_CTL_ADD, nl_sock_neigh, &ev) =3D=3D -1)= { > + close(nl_sock_neigh); > + nl_sock_neigh =3D -1; > + warn("Failed do epoll_ctrl() on RTMGRP_NEIGH socket"); > + return -1; > + } > + > + nl_neigh_sync(c, AF_INET, c->ifi4); > + nl_neigh_sync(c, AF_INET6, c->ifi6); > + > + return 0; > +} > diff --git a/netlink.h b/netlink.h > index b51e99c..8f1e9b9 100644 > --- a/netlink.h > +++ b/netlink.h > @@ -17,6 +17,8 @@ int nl_route_dup(int s_src, unsigned int ifi_src, > int s_dst, unsigned int ifi_dst, sa_family_t af); > int nl_addr_get(int s, unsigned int ifi, sa_family_t af, > void *addr, int *prefix_len, void *addr_l); > +bool nl_neigh_mac_get(int s, const union inany_addr *addr, int ifi, > + unsigned char *mac); > int nl_addr_set(int s, unsigned int ifi, sa_family_t af, > const void *addr, int prefix_len); > int nl_addr_get_ll(int s, unsigned int ifi, struct in6_addr *addr); > @@ -28,5 +30,7 @@ int nl_link_set_mac(int s, unsigned int ifi, const void= *mac); > int nl_link_set_mtu(int s, unsigned int ifi, int mtu); > int nl_link_set_flags(int s, unsigned int ifi, > unsigned int set, unsigned int change); > +int nl_neigh_notify_init(const struct ctx *c); > +void nl_neigh_notify_handler(const struct ctx *c); > =20 > #endif /* NETLINK_H */ > diff --git a/passt.c b/passt.c > index 31fbb75..e21d6ba 100644 > --- a/passt.c > +++ b/passt.c > @@ -53,6 +53,7 @@ > #include "vu_common.h" > #include "migrate.h" > #include "repair.h" > +#include "netlink.h" > =20 > #define EPOLL_EVENTS 8 > =20 > @@ -79,6 +80,7 @@ char *epoll_type_str[] =3D { > [EPOLL_TYPE_VHOST_KICK] =3D "vhost-user kick socket", > [EPOLL_TYPE_REPAIR_LISTEN] =3D "TCP_REPAIR helper listening socket", > [EPOLL_TYPE_REPAIR] =3D "TCP_REPAIR helper socket", > + [EPOLL_TYPE_NL_NEIGH] =3D "netlink neighbour subscription socket", > }; > static_assert(ARRAY_SIZE(epoll_type_str) =3D=3D EPOLL_NUM_TYPES, > "epoll_type_str[] doesn't match enum epoll_type"); > @@ -322,6 +324,8 @@ int main(int argc, char **argv) > =20 > pcap_init(&c); > =20 > + nl_neigh_notify_init(&c); > + > if (!c.foreground) { > if ((devnull_fd =3D open("/dev/null", O_RDWR | O_CLOEXEC)) < 0) > die_perror("Failed to open /dev/null"); > @@ -414,6 +418,9 @@ loop: > case EPOLL_TYPE_REPAIR: > repair_handler(&c, eventmask); > break; > + case EPOLL_TYPE_NL_NEIGH: > + nl_neigh_notify_handler(&c); > + break; > default: > /* Can't happen */ > ASSERT(0); > --=20 > 2.50.1 >=20 --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --coBjXxTHe+9IYiH1 Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmjfSqIACgkQzQJF27ox 2GeykQ//fyMyNV5RQiCJrqkNb7yxxljkBQLFj5rTXLEqlRpm/R33igRlc0FUrXKc dGZL5Yn6dw4IRecBbnq+1kx3C2HWVEjXDZK70fyooyd3yHYD16/HWWvxfSSKsovf M0jp44d4E0auXTObXV1LJD44FK75WPTklgDCFyb0m0iGpnKldq2Pev5WWerWC0EW mHfPIjukdIcWL6dua4NK3RuKeYpTx3RS+NxvxCqTlJcwIPXmIbfvnq91H1eDaCNn C3TP166jollu4rNYZgdGBCfCF0lFQvWVDOH5du9CfDLEVyxaXrt7pdzAvjpsxY+o tbdMqqTaUP6TXrOgonLdU1WhUpTFfvjjtfndk5zYezsuY1yTwOgi7QYbtpmXOcwv kwZnpc++2B6Z0UuBrzQXnYSkn9WzOBlyJtihg2g41yTYrHEfS9fpeZTVMKL4s3Cm 5aHpsAsqNLWBr6d+Hctb2oQ8Ged71O2pNgKrLs/u8vDYWeGMxjfwDiNc2gV+Hwvv 36cLNpjYsVaSfFtF1Y6KCS0m5FfmqxoY0hz8lU7abWLFRCSCPlvGpjXmCzNaIHQr d0zWWL4T1Pus5NTBcjlDDIsxYMi2xzn4ZSRnOdkDLV1kQ98tXCnRqpOcRwgyc4hK SYQfWxtJk5gpDmSrddYhRxxpykQhOMCH51phQCPOykdj5ZVFfTA= =7nlX -----END PGP SIGNATURE----- --coBjXxTHe+9IYiH1--