From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=M28ivM2R; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id 79A335A026F for ; Mon, 22 Sep 2025 00:09:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1758492556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4s/0babgEqt6txMI9yElUj+uMQGtHltIL2HyVyjpxG8=; b=M28ivM2RkQYd1P8kjbC5wdI8E+vk/oCxnvMO/sCJWhCgIK5gf3GMfgShb+rOPVa4D9YiRp 47lQDqn4Xgcc5QhM8+FTfnzN78mb895fE86/0ybR55MIXPBDCkfxIEzQ1b7/jZiJ39QSfz wZWSNLKnpKo1vRdNBgzltKg8vqNA3Bw= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-448-wf3DekTrPVWTRpTkNBZtTA-1; Sun, 21 Sep 2025 18:09:12 -0400 X-MC-Unique: wf3DekTrPVWTRpTkNBZtTA-1 X-Mimecast-MFC-AGG-ID: wf3DekTrPVWTRpTkNBZtTA_1758492552 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5956518004D4; Sun, 21 Sep 2025 22:09:11 +0000 (UTC) Received: from jmaloy-thinkpadp16vgen1.rmtcaqc.csb (unknown [10.22.88.62]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4358C3000198; Sun, 21 Sep 2025 22:09:10 +0000 (UTC) From: Jon Maloy To: sbrivio@redhat.com, dgibson@redhat.com, david@gibson.dropbear.id.au, jmaloy@redhat.com, passt-dev@passt.top Subject: [PATCH v8 2/8] fwd: Add cache table for ARP/NDP contents Date: Sun, 21 Sep 2025 18:08:59 -0400 Message-ID: <20250921220905.478621-3-jmaloy@redhat.com> In-Reply-To: <20250921220905.478621-1-jmaloy@redhat.com> References: <20250921220905.478621-1-jmaloy@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: I5P5gnmB7BMQTqKa75yWrwG8W6cwjz73O2mmJrz_2NQ_1758492552 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true Message-ID-Hash: WMG746RWRVOZXM5CBEKTSCHIEM7CTEIU X-Message-ID-Hash: WMG746RWRVOZXM5CBEKTSCHIEM7CTEIU X-MailFrom: jmaloy@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: We add a cache table to keep track of the contents of the kernel ARP and NDP tables. The table is fed from the just introduced netlink based neigbour subscription function. The new table eliminates the need for explicit netlink calls to find a host's MAC address. Signed-off-by: Jon Maloy --- v5: - Moved to earlier in series to reduce rebase conflicts v6: - Sqashed the hash list commit and the FIFO/LRU queue commit - Removed hash lookup. We now only use linear lookup in a linked list - Eliminated dynamic memory allocation. - Ensured there is only one call to clock_gettime() - Using MAC_ZERO instead of the previously dedicated definitions v7: - NOW using MAC_ZERO where needed - I am still using linear back-off for empty cache entries. Even an incoming, flow-creating packet from a local host gives no guarantee that its MAC address is in the ARP table, so we must allow for a few new attempts at first possible occasions. Only after several failed lookups can we conclude that we probably never will succeed. Hence the back-off. - Fixed a bug that David inadvertently made me aware of: I only intended to set the initial expiry value to MAC_CACHE_RENEWAL when an ARP/NDP table lookup was successful. - Improved struct and function description comments. v8: - Total re-design of table, adapting to the new, subscription based way of updating it. --- conf.c | 1 + fwd.c | 159 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fwd.h | 7 +++ netlink.c | 15 ++++-- 4 files changed, 177 insertions(+), 5 deletions(-) diff --git a/conf.c b/conf.c index 66b9e63..cc491c3 100644 --- a/conf.c +++ b/conf.c @@ -2133,6 +2133,7 @@ void conf(struct ctx *c, int argc, char **argv) c->udp.fwd_out.mode = fwd_default; fwd_scan_ports_init(c); + fwd_neigh_cache_init(); if (!c->quiet) conf_print(c); diff --git a/fwd.c b/fwd.c index 250cf56..139ad5c 100644 --- a/fwd.c +++ b/fwd.c @@ -32,6 +32,165 @@ static in_port_t fwd_ephemeral_min = (1 << 15) + (1 << 14); static in_port_t fwd_ephemeral_max = NUM_PORTS - 1; #define PORT_RANGE_SYSCTL "/proc/sys/net/ipv4/ip_local_port_range" +#define MAC_CACHE_SIZE 1024 + +/** + * mac_cache_entry - Entry in the ARP/NDP table cache + * @next: Next entry in slot or free list + * @addr: IP address of represented host + * @mac: MAC address of represented host, if known + */ +struct mac_cache_entry { + struct mac_cache_entry *next; + union inany_addr addr; + uint8_t mac[ETH_ALEN]; +}; + +/** + * mac_cache_table - Cache of ARP/NDP table contents + * @entries: Entries to be plugged into the hash slots when allocated + * @slots: Hash table slots + * @free: Linked list of unused entries + */ +struct mac_cache_table { + struct mac_cache_entry entries[MAC_CACHE_SIZE]; + struct mac_cache_entry *slots[MAC_CACHE_SIZE]; + struct mac_cache_entry *free; +}; + +static struct mac_cache_table mac_cache; + +/** + * fwd_mac_cache_slot_idx() - Hash key to a number within the table range + * @c: Execution context + * @key: The key to be used for the hash + * + * Return: The resulting hash value + */ +static inline size_t fwd_mac_cache_slot_idx(const struct ctx *c, + const union inany_addr *key) +{ + struct siphash_state st = SIPHASH_INIT(c->hash_secret); + uint32_t i; + + inany_siphash_feed(&st, key); + i = siphash_final(&st, sizeof(*key), 0); + + return ((size_t)i) & (MAC_CACHE_SIZE - 1); +} + +/** + * fwd_mac_cache_find() - Find a MAC cache table entry + * @c: Execution context + * @addr: Neighbour address to be used as key for the lookup + * + * Return: The found entry, if any. Otherwise NULL. + */ +static struct mac_cache_entry *fwd_mac_cache_find(const struct ctx *c, + const union inany_addr *addr) +{ + size_t idx = fwd_mac_cache_slot_idx(c, addr); + struct mac_cache_entry *e = mac_cache.slots[idx]; + + while (e && !inany_equals(&e->addr, addr)) + e = e->next; + + return e; +} + +/** + * fwd_mac_cache_alloc() - Allocate a mac cache table entry from the free list + * @c: Execution context + * @addr: Address used to determine insertion slot and store in entry + * @mac: The MAC address associated with the neighbour address + */ +void fwd_neigh_mac_cache_alloc(const struct ctx *c, + const union inany_addr *addr, uint8_t *mac) +{ + struct mac_cache_table *t = &mac_cache; + struct mac_cache_entry *e; + ssize_t idx; + + if (fwd_mac_cache_find(c, addr)) + return; + + e = t->free; + if (!e) + return; + + t->free = e->next; + + idx = fwd_mac_cache_slot_idx(c, addr); + e->next = t->slots[idx]; + t->slots[idx] = e; + + memcpy(&e->addr, addr, sizeof(*addr)); + memcpy(e->mac, mac, ETH_ALEN); +} + +/** + * fwd_mac_cache_free() - Remove an entry from a slot and add it to free list + * @c: Execution context + * @addr: Neighbour address used to find the slot for the entry + */ +void fwd_neigh_mac_cache_free(const struct ctx *c, const union inany_addr *addr) +{ + ssize_t idx = fwd_mac_cache_slot_idx(c, addr); + struct mac_cache_table *t = &mac_cache; + struct mac_cache_entry *e, **prev; + + prev = &t->slots[idx]; + e = t->slots[idx]; + while (e && !inany_equals(&e->addr, addr)) { + prev = &e->next; + e = e->next; + } + if (!e) + return; + + *prev = e->next; + e->next = t->free; + t->free = e; + memset(&e->addr, 0, sizeof(*addr)); + memset(e->mac, 0, ETH_ALEN); +} + +/** + * fwd_neigh_mac_get() - Lookup MAC address in the ARP/NDP cache table + * @c: Execution context + * @addr: Neighbour address used as lookup key + * @mac: Buffer for Ethernet MAC to return, found or default value. + * + * Return: true if real MAC found, false if not found or if failure + */ +bool fwd_neigh_mac_get(const struct ctx *c, const union inany_addr *addr, + uint8_t *mac) +{ + struct mac_cache_entry *e = fwd_mac_cache_find(c, addr); + + if (e) + memcpy(mac, e->mac, ETH_ALEN); + else + memcpy(mac, c->our_tap_mac, ETH_ALEN); + + return !!e; +} + +/** + * fwd_neigh_cache_init() - Initialize the neighbor ARP/NDP cache table + */ +void fwd_neigh_cache_init(void) +{ + struct mac_cache_table *t = &mac_cache; + struct mac_cache_entry *e; + + memset(t, 0, sizeof(*t)); + for (int i = 0; i < MAC_CACHE_SIZE; i++) { + e = &t->entries[i]; + e->next = t->free; + t->free = e; + } +} /** fwd_probe_ephemeral() - Determine what ports this host considers ephemeral * diff --git a/fwd.h b/fwd.h index 65c7c96..a0e8fbc 100644 --- a/fwd.h +++ b/fwd.h @@ -56,5 +56,12 @@ uint8_t fwd_nat_from_splice(const struct ctx *c, uint8_t proto, const struct flowside *ini, struct flowside *tgt); uint8_t fwd_nat_from_host(const struct ctx *c, uint8_t proto, const struct flowside *ini, struct flowside *tgt); +void fwd_neigh_mac_cache_alloc(const struct ctx *c, + const union inany_addr *addr, uint8_t *mac); +void fwd_neigh_mac_cache_free(const struct ctx *c, + const union inany_addr *addr); +bool fwd_neigh_mac_get(const struct ctx *c, const union inany_addr *addr, + uint8_t *mac); +void fwd_neigh_cache_init(void); #endif /* FWD_H */ diff --git a/netlink.c b/netlink.c index 1faf3da..1e1ec43 100644 --- a/netlink.c +++ b/netlink.c @@ -131,6 +131,8 @@ int nl_neigh_subscr_init(struct ctx *c) */ void nl_neigh_subscr_handler(struct ctx *c) { + union inany_addr addr; + uint8_t mac[ETH_ALEN]; struct nlmsghdr *nh; char buf[NLBUFSIZ]; ssize_t n; @@ -183,17 +185,20 @@ void nl_neigh_subscr_handler(struct ctx *c) dstlen != sizeof(struct in6_addr)) continue; - char abuf[INET6_ADDRSTRLEN]; + if (!lladdr || lladdr_len != ETH_ALEN) + continue; + + memcpy(mac, lladdr, ETH_ALEN); if (dstlen == sizeof(struct in_addr)) - inet_ntop(AF_INET, dst, abuf, sizeof(abuf)); + addr = inany_from_v4(*(struct in_addr *) dst); else - inet_ntop(AF_INET6, dst, abuf, sizeof(abuf)); + addr.a6 = *(struct in6_addr *) dst; if (nh->nlmsg_type == RTM_NEWNEIGH) - debug("neigh: NEW %s lladdr_len=%zu", abuf, lladdr_len); + fwd_neigh_mac_cache_alloc(c, &addr, mac); else - debug("neigh: DEL %s", abuf); + fwd_neigh_mac_cache_free(c, &addr); } } } -- 2.50.1