From: David Gibson <david@gibson.dropbear.id.au>
To: passt-dev@passt.top, Stefano Brivio <sbrivio@redhat.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Subject: [PATCH 00/32] Use dual stack sockets to listen for inbound TCP connections
Date: Wed, 16 Nov 2022 15:41:40 +1100 [thread overview]
Message-ID: <20221116044212.3876516-1-david@gibson.dropbear.id.au> (raw)
When forwarding many ports, passt can consume a lot of kernel memory
because of the many listening sockets it opens. There are not a lot
of ways we can reduce that, but here's one.
Currently we create separate listening sockets for each port for both
IPv4 and IPv6. However in Linux (and probably other platforms), it's
possible to listen for both IPv4 and IPv6 connections on an IPv6
socket. This series uses such dual stack sockets to halve the number
of listening sockets needed for TCP. When forwarding all TCP and UDP
ports, this reduces the kernel memory used from around 677 MiB to
around 487 MiB (kernel 6.0.8 on an x86_64 Fedora 37 machine).
This should also be possible for UDP, but that will require a mostly
separate implementation.
David Gibson (32):
clang-tidy: Suppress warning about assignments in if statements
style: Minor corrections to function comments
tcp_splice: #include tcp_splice.h in tcp_splice.c
tcp: Remove unused TCP_MAX_SOCKS constant
tcp: Better helpers for converting between connection pointer and
index
tcp_splice: Helpers for converting from index to/from tcp_splice_conn
tcp: Move connection state structures into a shared header
tcp: Add connection union type
tcp: Improved helpers to update connections after moving
tcp: Unify spliced and non-spliced connection tables
tcp: Unify tcp_defer_handler and tcp_splice_defer_handler()
tcp: Partially unify tcp_timer() and tcp_splice_timer()
tcp: Unify the IN_EPOLL flag
tcp: Separate helpers to create ns listening sockets
tcp: Unify part of spliced and non-spliced conn_from_sock path
tcp: Use the same sockets to listen for spliced and non-spliced
connections
tcp: Remove splice from tcp_epoll_ref
tcp: Don't store hash bucket in connection structures
inany: Helper functions for handling addresses which could be IPv4 or
IPv6
tcp: Hash IPv4 and IPv4-mapped-IPv6 addresses the same
tcp: Take tcp_hash_insert() address from struct tcp_conn
tcp: Simplify tcp_hash_match() to take an inany_addr
tcp: Unify initial sequence number calculation for IPv4 and IPv6
tcp: Have tcp_seq_init() take its parameters from struct tcp_conn
tcp: Fix small errors in tcp_seq_init() time handling
tcp: Remove v6 flag from tcp_epoll_ref
tcp: NAT IPv4-mapped IPv6 addresses like IPv4 addresses
tcp_splice: Allow splicing of connections from IPv4-mapped loopback
tcp: Consolidate tcp_sock_init[46]
util: Allow sock_l4() to open dual stack sockets
util: Always return -1 on error in sock_l4()
tcp: Use dual stack sockets for port forwarding when possible
Makefile | 11 +-
conf.c | 12 +-
inany.h | 94 +++++
siphash.c | 2 +
tap.c | 6 +-
tcp.c | 978 ++++++++++++++++++++++-----------------------------
tcp.h | 11 +-
tcp_conn.h | 192 ++++++++++
tcp_splice.c | 337 ++++++++----------
tcp_splice.h | 12 +-
util.c | 19 +-
11 files changed, 887 insertions(+), 787 deletions(-)
create mode 100644 inany.h
create mode 100644 tcp_conn.h
--
2.38.1
next reply other threads:[~2022-11-16 4:42 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-16 4:41 David Gibson [this message]
2022-11-16 4:41 ` [PATCH 01/32] clang-tidy: Suppress warning about assignments in if statements David Gibson
2022-11-16 23:10 ` Stefano Brivio
2022-11-17 1:20 ` David Gibson
2022-11-16 4:41 ` [PATCH 02/32] style: Minor corrections to function comments David Gibson
2022-11-16 23:11 ` Stefano Brivio
2022-11-17 1:21 ` David Gibson
2022-11-16 4:41 ` [PATCH 03/32] tcp_splice: #include tcp_splice.h in tcp_splice.c David Gibson
2022-11-16 4:41 ` [PATCH 04/32] tcp: Remove unused TCP_MAX_SOCKS constant David Gibson
2022-11-16 4:41 ` [PATCH 05/32] tcp: Better helpers for converting between connection pointer and index David Gibson
2022-11-16 23:11 ` Stefano Brivio
2022-11-17 1:24 ` David Gibson
2022-11-16 4:41 ` [PATCH 06/32] tcp_splice: Helpers for converting from index to/from tcp_splice_conn David Gibson
2022-11-16 4:41 ` [PATCH 07/32] tcp: Move connection state structures into a shared header David Gibson
2022-11-16 4:41 ` [PATCH 08/32] tcp: Add connection union type David Gibson
2022-11-16 4:41 ` [PATCH 09/32] tcp: Improved helpers to update connections after moving David Gibson
2022-11-16 4:41 ` [PATCH 10/32] tcp: Unify spliced and non-spliced connection tables David Gibson
2022-11-16 4:41 ` [PATCH 11/32] tcp: Unify tcp_defer_handler and tcp_splice_defer_handler() David Gibson
2022-11-16 4:41 ` [PATCH 12/32] tcp: Partially unify tcp_timer() and tcp_splice_timer() David Gibson
2022-11-16 4:41 ` [PATCH 13/32] tcp: Unify the IN_EPOLL flag David Gibson
2022-11-16 4:41 ` [PATCH 14/32] tcp: Separate helpers to create ns listening sockets David Gibson
2022-11-16 23:51 ` Stefano Brivio
2022-11-17 1:32 ` David Gibson
2022-11-16 4:41 ` [PATCH 15/32] tcp: Unify part of spliced and non-spliced conn_from_sock path David Gibson
2022-11-16 23:53 ` Stefano Brivio
2022-11-17 1:37 ` David Gibson
2022-11-17 7:30 ` Stefano Brivio
2022-11-17 8:58 ` David Gibson
2022-11-16 4:41 ` [PATCH 16/32] tcp: Use the same sockets to listen for spliced and non-spliced connections David Gibson
2022-11-16 23:54 ` Stefano Brivio
2022-11-17 1:43 ` David Gibson
2022-11-16 4:41 ` [PATCH 17/32] tcp: Remove splice from tcp_epoll_ref David Gibson
2022-11-16 4:41 ` [PATCH 18/32] tcp: Don't store hash bucket in connection structures David Gibson
2022-11-16 4:41 ` [PATCH 19/32] inany: Helper functions for handling addresses which could be IPv4 or IPv6 David Gibson
2022-11-16 23:54 ` Stefano Brivio
2022-11-17 1:48 ` David Gibson
2022-11-16 4:42 ` [PATCH 20/32] tcp: Hash IPv4 and IPv4-mapped-IPv6 addresses the same David Gibson
2022-11-16 4:42 ` [PATCH 21/32] tcp: Take tcp_hash_insert() address from struct tcp_conn David Gibson
2022-11-16 4:42 ` [PATCH 22/32] tcp: Simplify tcp_hash_match() to take an inany_addr David Gibson
2022-11-16 4:42 ` [PATCH 23/32] tcp: Unify initial sequence number calculation for IPv4 and IPv6 David Gibson
2022-11-16 4:42 ` [PATCH 24/32] tcp: Have tcp_seq_init() take its parameters from struct tcp_conn David Gibson
2022-11-16 4:42 ` [PATCH 25/32] tcp: Fix small errors in tcp_seq_init() time handling David Gibson
2022-11-16 4:42 ` [PATCH 26/32] tcp: Remove v6 flag from tcp_epoll_ref David Gibson
2022-11-17 0:15 ` Stefano Brivio
2022-11-17 1:50 ` David Gibson
2022-11-16 4:42 ` [PATCH 27/32] tcp: NAT IPv4-mapped IPv6 addresses like IPv4 addresses David Gibson
2022-11-17 0:15 ` Stefano Brivio
2022-11-17 2:00 ` David Gibson
2022-11-16 4:42 ` [PATCH 28/32] tcp_splice: Allow splicing of connections from IPv4-mapped loopback David Gibson
2022-11-17 0:15 ` Stefano Brivio
2022-11-17 2:05 ` David Gibson
2022-11-16 4:42 ` [PATCH 29/32] tcp: Consolidate tcp_sock_init[46] David Gibson
2022-11-16 4:42 ` [PATCH 30/32] util: Allow sock_l4() to open dual stack sockets David Gibson
2022-11-16 4:42 ` [PATCH 31/32] util: Always return -1 on error in sock_l4() David Gibson
2022-11-16 4:42 ` [PATCH 32/32] tcp: Use dual stack sockets for port forwarding when possible David Gibson
2022-11-17 0:15 ` Stefano Brivio
2022-11-17 2:08 ` David Gibson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221116044212.3876516-1-david@gibson.dropbear.id.au \
--to=david@gibson.dropbear.id.au \
--cc=passt-dev@passt.top \
--cc=sbrivio@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://passt.top/passt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).