* [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address @ 2026-06-03 1:15 EJ Campbell 2026-06-05 11:57 ` Stefano Brivio 0 siblings, 1 reply; 4+ messages in thread From: EJ Campbell @ 2026-06-03 1:15 UTC (permalink / raw) To: passt-dev The latest source address seen on the tap interface (ip4.addr_seen) is taken to be the guest's and is used as the target for inbound port forwarding. The update accepts any source address. On a point-to-point tap that is always correct. But when the tap sits on a shared L2 segment -- for example when it is bridged together with other hosts -- pasta also sees frames from those other hosts. With a static configuration (-a / address set, DHCP disabled), an overheard frame from another host silently moves addr_seen to that host, and every subsequent inbound forwarded connection is sent there instead of to the guest, so connections are reset or hang. I hit this running pasta with its tap enslaved to a Linux bridge that also carries other traffic: inbound port forwarding to the guest broke as soon as another address was seen on the segment. Restrict the update so that, when DHCP is disabled (no_dhcp), only frames sourced from the configured guest address may move addr_seen. With DHCP enabled -- the default -- behaviour is unchanged, since the guest's address is whatever pasta leases and tracking it is still correct. The same reasoning applies to ip6.addr_seen on a shared segment; I left IPv6 alone for now since I only exercise IPv4 in this setup, and would be glad to extend it the same way if you prefer. Signed-off-by: EJ Campbell <ej.campbell@gmail.com> --- tap.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tap.c b/tap.c index 4cba4c7..514379e 100644 --- a/tap.c +++ b/tap.c @@ -770,7 +770,19 @@ resume: continue; } - if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr) + /* The latest source address seen on the tap is taken to be the + * guest's, and becomes the target for inbound forwarding. On a + * point-to-point tap that's always correct. But if the tap sits + * on a shared L2 segment (for example bridged together with + * other hosts) and we serve a fixed address (DHCP disabled), a + * frame overheard from another host on that segment must not + * move addr_seen and silently retarget forwarded connections + * away from the guest. When DHCP is disabled, only the + * configured guest address may update addr_seen; with DHCP + * enabled (the default) behaviour is unchanged. + */ + if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr && + (!c->no_dhcp || iph->saddr == c->ip4.addr.s_addr)) c->ip4.addr_seen.s_addr = iph->saddr; if (!iov_drop_header(&data, hlen)) base-commit: 4b2823784aab04a70dfc295b16fd6f0592955790 -- 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address 2026-06-03 1:15 [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address EJ Campbell @ 2026-06-05 11:57 ` Stefano Brivio 2026-06-09 1:54 ` David Gibson 0 siblings, 1 reply; 4+ messages in thread From: Stefano Brivio @ 2026-06-05 11:57 UTC (permalink / raw) To: EJ Campbell; +Cc: ej, passt-dev, Jon Maloy, David Gibson EJ, thanks for the detailed patch and commit message, and apologies for the delayed response. Some remarks: On Tue, 2 Jun 2026 18:15:24 -0700 EJ Campbell <ej.campbell@gmail.com> wrote: > The latest source address seen on the tap interface (ip4.addr_seen) is > taken to be the guest's and is used as the target for inbound port > forwarding. The update accepts any source address. > > On a point-to-point tap that is always correct. But when the tap sits on > a shared L2 segment -- for example when it is bridged together with other > hosts -- pasta also sees frames from those other hosts. That's a interesting use case, it's the first time I see pasta used like this. Thanks for sharing. > With a static > configuration (-a / address set, DHCP disabled), an overheard frame from > another host silently moves addr_seen to that host, and every subsequent > inbound forwarded connection is sent there instead of to the guest, so > connections are reset or hang. > > I hit this running pasta with its tap enslaved to a Linux bridge that also > carries other traffic: inbound port forwarding to the guest broke as soon > as another address was seen on the segment. Oops. We should definitely fix this, and maybe your patch is good enough for the moment (or better than the alternative), but I have a concern here: > Restrict the update so that, when DHCP is disabled (no_dhcp), only frames > sourced from the configured guest address may move addr_seen. With DHCP > enabled -- the default -- behaviour is unchanged, since the guest's > address is whatever pasta leases and tracking it is still correct. ...inferring possibly but not necessarily related information from whether DHCP is disabled or enabled looks a bit risky to me: - somebody might have DHCP enabled (just because it's the default option), but one single connected container / guest not using DHCP at all, and picking an address that we didn't expect. Your patch would break this (perhaps unlikely?) usage - somebody might have DHCP enabled but just the intended container / guest using it, and your same problem. Your patch wouldn't fix that (definitely less likely) usage So I've been thinking: shouldn't we rather use as a condition the fact that -a / --address was explicitly given? We don't track that from conf() at the moment but it would be a small change. If -a / --address is given, I'd say the user is very much expecting the container / guest to be using that address, and in that case I think it's safer to ignore other addresses. It would be even better if we could track our designated container / guest among a number of them, only if there are more than one. For context, Jon is working on a series adding support for multiple addresses (assigned, used, or observed): https://archives.passt.top/passt-dev/20260413005319.3295910-1-jmaloy@redhat.com/ and patch 8/13: https://archives.passt.top/passt-dev/20260413005319.3295910-9-jmaloy@redhat.com/ refactors the same exact part your patch is touching, here: https://archives.passt.top/passt-dev/7f04e12/s/?b=tap.c#n764 with no functional change. By the way, if you want to try things out on top of that series, you can apply it with: b4 shazam https://archives.passt.top/passt-dev/20260413005319.3295910-1-jmaloy@redhat.com/ ...you might want to check out a slightly older revision to apply cleanly without conflicts. The main purpose of this work is actually to implement a netlink monitor to track changes in host addresses: https://bugs.passt.top/show_bug.cgi?id=141 https://pad.passt.top/p/netlinkMonitor ...but it might also be convenient for this kind of tracking. For guests, we could establish some preference rules, for example: - if any guest made a DHCP / DHCPv6 request, select its address (I guess we should eventually implement guest-side MAC address tracking for a full solution but just preferring addresses we assigned might be quite good). I'm not sure how we would handle SLAAC - if we haven't seen any DHCP / DHCPv6 request, but we have seen a single address, default to that one as "guest" address and so on. For containers, we actually know which namespace we're associated to, so we could, on any new observed address, check the source MAC address of the frame and see if it corresponds to our namespace (querying via netlink). Thoughts? Jon? If this is all too complicated for the moment, would a condition based on explicit -a / --address option work for you? > The same reasoning applies to ip6.addr_seen on a shared segment; I left > IPv6 alone for now since I only exercise IPv4 in this setup, and would be > glad to extend it the same way if you prefer. I think we should definitely take care of IPv6 as well, I'm just not sure how. A couple of comments about the patch itself: > Signed-off-by: EJ Campbell <ej.campbell@gmail.com> > --- > tap.c | 14 +++++++++++++- > 1 file changed, 13 insertions(+), 1 deletion(-) > > diff --git a/tap.c b/tap.c > index 4cba4c7..514379e 100644 > --- a/tap.c > +++ b/tap.c > @@ -770,7 +770,19 @@ resume: > continue; It looks like the patch itself was copied and pasted from a terminal, because all tabs became spaces, and the patch doesn't apply. It's not a problem for small patches like this one as I tend to fix things up manually, but it would be good to have it directly from git send-email instead. > } > > - if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr) > + /* The latest source address seen on the tap is taken to be the > + * guest's, and becomes the target for inbound forwarding. On a > + * point-to-point tap that's always correct. But if the tap sits > + * on a shared L2 segment (for example bridged together with > + * other hosts) and we serve a fixed address (DHCP disabled), a > + * frame overheard from another host on that segment must not > + * move addr_seen and silently retarget forwarded connections > + * away from the guest. When DHCP is disabled, only the > + * configured guest address may update addr_seen; with DHCP > + * enabled (the default) behaviour is unchanged. The fact that the behaviour is _unchanged_ (by your patch) belongs in the commit message rather than the code, because if one reads the code they aren't explicitly reading your change and it's not clear to what previous behaviour you would be referring to. Other than that it's pretty well described, thanks. Let's see if we can find a better solution though. > + */ > + if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr && > + (!c->no_dhcp || iph->saddr == c->ip4.addr.s_addr)) > c->ip4.addr_seen.s_addr = iph->saddr; > > if (!iov_drop_header(&data, hlen)) > > base-commit: 4b2823784aab04a70dfc295b16fd6f0592955790 > -- > 2.53.0 > -- Stefano ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address 2026-06-05 11:57 ` Stefano Brivio @ 2026-06-09 1:54 ` David Gibson 2026-06-11 4:26 ` EJ Campbell 0 siblings, 1 reply; 4+ messages in thread From: David Gibson @ 2026-06-09 1:54 UTC (permalink / raw) To: Stefano Brivio; +Cc: EJ Campbell, ej, passt-dev, Jon Maloy [-- Attachment #1: Type: text/plain, Size: 9660 bytes --] On Fri, Jun 05, 2026 at 01:57:41PM +0200, Stefano Brivio wrote: > EJ, thanks for the detailed patch and commit message, and apologies for > the delayed response. > > Some remarks: > > On Tue, 2 Jun 2026 18:15:24 -0700 > EJ Campbell <ej.campbell@gmail.com> wrote: > > > The latest source address seen on the tap interface (ip4.addr_seen) is > > taken to be the guest's and is used as the target for inbound port > > forwarding. The update accepts any source address. > > > > On a point-to-point tap that is always correct. But when the tap sits on > > a shared L2 segment -- for example when it is bridged together with other > > hosts -- pasta also sees frames from those other hosts. > > That's a interesting use case, it's the first time I see pasta used > like this. Thanks for sharing. Likewise, it's the first I've heard of someone using it specifically like this, but it is a use model I've had in mind for a while. So this is a known limitation I've been thinking about for a while. It hasn't reached the top of my priority list because while your very specific case almost works already, the more general case of a bridge on the guest side requires a bunch more infrastructure. That's in progress or at least planned, but it's going to be a while. Fwiw, I really dislike the addr_seen mechanism even in cases where it "works": it allows poorly configured guests to work at the expense of substantial complexity which impacts all cases. However, it's established behaviour, so Stefano is understandably against simply removing it. > > With a static > > configuration (-a / address set, DHCP disabled), an overheard frame from > > another host silently moves addr_seen to that host, and every subsequent > > inbound forwarded connection is sent there instead of to the guest, so > > connections are reset or hang. > > > > I hit this running pasta with its tap enslaved to a Linux bridge that also > > carries other traffic: inbound port forwarding to the guest broke as soon > > as another address was seen on the segment. > > Oops. We should definitely fix this, and maybe your patch is good enough > for the moment (or better than the alternative), but I have a concern > here: > > > Restrict the update so that, when DHCP is disabled (no_dhcp), only frames > > sourced from the configured guest address may move addr_seen. With DHCP > > enabled -- the default -- behaviour is unchanged, since the guest's > > address is whatever pasta leases and tracking it is still correct. > > ...inferring possibly but not necessarily related information from > whether DHCP is disabled or enabled looks a bit risky to me: > > - somebody might have DHCP enabled (just because it's the default > option), but one single connected container / guest not using DHCP at > all, and picking an address that we didn't expect. Your patch would > break this (perhaps unlikely?) usage > > - somebody might have DHCP enabled but just the intended container / > guest using it, and your same problem. Your patch wouldn't fix that > (definitely less likely) usage Right. I agree with Stefano that basing this of --no-dhcp is not a good idea. It happens to work for your case, but it doesn't generalise well. In addition to the cases Stefano mentions, for the more general case of a guest-side bridge, we'd eventually like to have our DHCP implementation serve multiple address, in which case we wouldn't want addr_seen, even though DHCP is enabled. > So I've been thinking: shouldn't we rather use as a condition the fact > that -a / --address was explicitly given? We don't track that from > conf() at the moment but it would be a small change. > > If -a / --address is given, I'd say the user is very much expecting the > container / guest to be using that address, and in that case I think > it's safer to ignore other addresses. I think having an explcit -a is a sufficient condition to disable addr_seen, but maybe not a necessary one. I think requiring the guest to use the assigned address would be a reasonable request, even if that address is taken from the host. Stefano and others talked me out of an explicit --no-address-snooping option, on the grounds of option proliferation at our last call. However, thinking about it again, I'm still wondering if it might be useful: we can think of addr_seen as a (not very good) address assignment mechanism, along with DHCP, NDP, DHCPv6 and --config-net. All of those (better) mechanisms can be explicitly disabled, so why should we not be able to disable the worst one. > It would be even better if we could track our designated container / > guest among a number of them, only if there are more than one. > > For context, Jon is working on a series adding support for multiple > addresses (assigned, used, or observed): > > https://archives.passt.top/passt-dev/20260413005319.3295910-1-jmaloy@redhat.com/ > > and patch 8/13: > > https://archives.passt.top/passt-dev/20260413005319.3295910-9-jmaloy@redhat.com/ > > refactors the same exact part your patch is touching, here: > > https://archives.passt.top/passt-dev/7f04e12/s/?b=tap.c#n764 > > with no functional change. > > By the way, if you want to try things out on top of that series, you > can apply it with: > > b4 shazam > https://archives.passt.top/passt-dev/20260413005319.3295910-1-jmaloy@redhat.com/ > > ...you might want to check out a slightly older revision to apply > cleanly without conflicts. > > The main purpose of this work is actually to implement a netlink > monitor to track changes in host addresses: > > https://bugs.passt.top/show_bug.cgi?id=141 > https://pad.passt.top/p/netlinkMonitor > > ...but it might also be convenient for this kind of tracking. For > guests, we could establish some preference rules, for example: > > - if any guest made a DHCP / DHCPv6 request, select its address (I > guess we should eventually implement guest-side MAC address tracking > for a full solution but just preferring addresses we assigned might > be quite good). I'm not sure how we would handle SLAAC > > - if we haven't seen any DHCP / DHCPv6 request, but we have seen a > single address, default to that one as "guest" address > > and so on. > > For containers, we actually know which namespace we're associated to, > so we could, on any new observed address, check the source MAC address > of the frame and see if it corresponds to our namespace (querying via > netlink). > > Thoughts? Jon? > > If this is all too complicated for the moment, would a condition based > on explicit -a / --address option work for you? > > > The same reasoning applies to ip6.addr_seen on a shared segment; I left > > IPv6 alone for now since I only exercise IPv4 in this setup, and would be > > glad to extend it the same way if you prefer. > > I think we should definitely take care of IPv6 as well, I'm just not > sure how. > > A couple of comments about the patch itself: > > > Signed-off-by: EJ Campbell <ej.campbell@gmail.com> > > --- > > tap.c | 14 +++++++++++++- > > 1 file changed, 13 insertions(+), 1 deletion(-) > > > > diff --git a/tap.c b/tap.c > > index 4cba4c7..514379e 100644 > > --- a/tap.c > > +++ b/tap.c > > @@ -770,7 +770,19 @@ resume: > > continue; > > It looks like the patch itself was copied and pasted from a terminal, > because all tabs became spaces, and the patch doesn't apply. > > It's not a problem for small patches like this one as I tend to fix > things up manually, but it would be good to have it directly from git > send-email instead. > > > } > > > > - if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr) > > + /* The latest source address seen on the tap is taken to be the > > + * guest's, and becomes the target for inbound forwarding. On a > > + * point-to-point tap that's always correct. But if the tap sits > > + * on a shared L2 segment (for example bridged together with > > + * other hosts) and we serve a fixed address (DHCP disabled), a > > + * frame overheard from another host on that segment must not > > + * move addr_seen and silently retarget forwarded connections > > + * away from the guest. When DHCP is disabled, only the > > + * configured guest address may update addr_seen; with DHCP > > + * enabled (the default) behaviour is unchanged. > > The fact that the behaviour is _unchanged_ (by your patch) belongs in > the commit message rather than the code, because if one reads the code > they aren't explicitly reading your change and it's not clear to what > previous behaviour you would be referring to. > > Other than that it's pretty well described, thanks. Let's see if we can > find a better solution though. > > > + */ > > + if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr && > > + (!c->no_dhcp || iph->saddr == c->ip4.addr.s_addr)) > > c->ip4.addr_seen.s_addr = iph->saddr; > > > > if (!iov_drop_header(&data, hlen)) > > > > base-commit: 4b2823784aab04a70dfc295b16fd6f0592955790 > > -- > > 2.53.0 > > > > -- > Stefano > > -- David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address 2026-06-09 1:54 ` David Gibson @ 2026-06-11 4:26 ` EJ Campbell 0 siblings, 0 replies; 4+ messages in thread From: EJ Campbell @ 2026-06-11 4:26 UTC (permalink / raw) To: passt-dev; +Cc: sbrivio, david Hi Stefano, David, Thanks for the careful review. David, good to know this is a known gap and not just my unusual topology. You both made the same point: --no-dhcp was the wrong condition. Agreed. v2 below keys on -a / --address having been given explicitly: a new per-family addr_fixed flag, set where -a is parsed, and the tap handlers skip the addr_seen updates when it's set. Stefano, on your other points: - IPv6: the global-address updates now get the same guard. I left addr_ll_seen tracking alone, since a global -a says nothing about which link-local address the guest picked. - The "behaviour unchanged" code comments are gone; that reasoning lives in the commit message now. - Whitespace: sorry about that; my mail client flattened the tabs in v1. This one comes straight from git send-email and applies cleanly here. David, on --no-address-snooping: happy to do a v3 with an explicit knob if you'd prefer, but keying on -a covered my case and keeps the option count down. One more data point since v1: I caught the failure in the act with --trace. After overhearing the namespace kernel's broadcasts (ARP probes, IGMP joins, sourced from the bridge's address), pasta's inbound splice dials the bridge instead of the -a address: Flow 0 (TCP connection (spliced)): HOST [127.0.0.1]:57280 -> [127.0.0.2]:22844 => SPLICE [0.0.0.0]:0 -> [10.0.2.1]:80 <- bridge addr, not -a Flow 0 (TCP connection (spliced)): Error event on socket: Connection refused With the patch, a reproducer that failed one run in three (99 spliced connections per run) passed six consecutive runs. Thanks, EJ -- >8 -- Subject: [PATCH v2] tap: don't let overheard traffic move addr_seen when address is explicit When pasta's tap interface shares an L2 segment with other endpoints (e.g. bridged in a namespace with a VM behind a second tap), frames from those endpoints flood to pasta and their source addresses overwrite addr_seen — inbound flows are then spliced or forwarded to whichever address spoke last instead of the configured guest. In such a setup the namespace kernel's own broadcasts (ARP probes, IGMP joins, sourced from the bridge address) race the guest's traffic for addr_seen, and inbound port-forwarded connections intermittently get RST after pasta dials the bridge address, where nothing listens: Flow 0 (TCP connection (spliced)): HOST [127.0.0.1]:57280 -> [127.0.0.2]:22844 => SPLICE [0.0.0.0]:0 -> [10.0.2.1]:80 <- bridge addr, not -a Flow 0 (TCP connection (spliced)): Error event on socket: Connection refused When -a / --address is given, the user has explicitly said where the guest is: track that decision with a new addr_fixed flag (per address family) and skip the addr_seen updates in the tap handlers. Behaviour without -a is unchanged, and IPv6 link-local tracking (addr_ll_seen) is unaffected — a global -a says nothing about which link-local address the guest chose. With this change, a reproducer that previously failed one run in three (99 spliced connections per run) passed six consecutive runs. Signed-off-by: EJ Campbell <ej.campbell@gmail.com> --- conf.c | 2 ++ passt.h | 6 ++++++ tap.c | 9 ++++++--- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/conf.c b/conf.c index cd05adf..4755a9f 100644 --- a/conf.c +++ b/conf.c @@ -1583,10 +1583,12 @@ void conf(struct ctx *c, int argc, char **argv) if (inany_v4(&addr)) { c->ip4.addr = *inany_v4(&addr); c->ip4.prefix_len = prefix_len - 96; + c->ip4.addr_fixed = true; if (c->mode == MODE_PASTA) c->ip4.no_copy_addrs = true; } else { c->ip6.addr = addr.a6; + c->ip6.addr_fixed = true; if (c->mode == MODE_PASTA) c->ip6.no_copy_addrs = true; } diff --git a/passt.h b/passt.h index c5f51d1..19d8c59 100644 --- a/passt.h +++ b/passt.h @@ -82,6 +82,8 @@ enum passt_modes { * @ifname_out: Optional interface name to bind outbound sockets to * @no_copy_routes: Don't copy all routes when configuring target namespace * @no_copy_addrs: Don't copy all addresses when configuring namespace + * @addr_fixed: Address was given explicitly (-a): don't update + * addr_seen from traffic observed on tap */ struct ip4_ctx { /* PIF_TAP addresses */ @@ -103,6 +105,7 @@ struct ip4_ctx { bool no_copy_routes; bool no_copy_addrs; + bool addr_fixed; }; /** @@ -123,6 +126,8 @@ struct ip4_ctx { * @ifname_out: Optional interface name to bind outbound sockets to * @no_copy_routes: Don't copy all routes when configuring target namespace * @no_copy_addrs: Don't copy all addresses when configuring namespace + * @addr_fixed: Address was given explicitly (-a): don't update + * addr_seen from traffic observed on tap */ struct ip6_ctx { /* PIF_TAP addresses */ @@ -144,6 +149,7 @@ struct ip6_ctx { bool no_copy_routes; bool no_copy_addrs; + bool addr_fixed; }; #include <netinet/if_ether.h> diff --git a/tap.c b/tap.c index 4cba4c7..2b1699a 100644 --- a/tap.c +++ b/tap.c @@ -770,7 +770,8 @@ resume: continue; } - if (iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr) + if (!c->ip4.addr_fixed && + iph->saddr && c->ip4.addr_seen.s_addr != iph->saddr) c->ip4.addr_seen.s_addr = iph->saddr; if (!iov_drop_header(&data, hlen)) @@ -1003,13 +1004,15 @@ resume: if (IN6_IS_ADDR_LINKLOCAL(saddr)) { c->ip6.addr_ll_seen = *saddr; - if (IN6_IS_ADDR_UNSPECIFIED(&c->ip6.addr_seen)) { + if (!c->ip6.addr_fixed && + IN6_IS_ADDR_UNSPECIFIED(&c->ip6.addr_seen)) { c->ip6.addr_seen = *saddr; } if (IN6_IS_ADDR_UNSPECIFIED(&c->ip6.addr)) c->ip6.addr = *saddr; - } else if (!IN6_IS_ADDR_UNSPECIFIED(saddr)){ + } else if (!c->ip6.addr_fixed && + !IN6_IS_ADDR_UNSPECIFIED(saddr)) { c->ip6.addr_seen = *saddr; } -- 2.43.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-06-11 4:26 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-06-03 1:15 [PATCH] tap: don't let overheard traffic move addr_seen when serving a fixed address EJ Campbell 2026-06-05 11:57 ` Stefano Brivio 2026-06-09 1:54 ` David Gibson 2026-06-11 4:26 ` EJ Campbell
Code repositories for project(s) associated with this public inbox https://passt.top/passt This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for IMAP folder(s).