From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 5DC3A5A0274 for ; Thu, 21 Dec 2023 08:02:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202312; t=1703142160; bh=mcqCe9BNNd5Ed4FE0N8F/jujaeREa13a8TOnrVL7hnk=; h=From:To:Cc:Subject:Date:From; b=HB8NZj02amL7bSUBiDHYg4e/8zGQcPTYmh0hX+BxnJUO0y1af9juh4OFtina/S8pv l73JEOUntRkIoqpq5U11t/aFGdD85PCyFc8mEAAy168YAZVt5WPQT8NJRtgVz0AQV+ 4ZZil1Uosav0C2Q7t6Q65M8Pk0YAF/3psPTqz2FD8G1RJTsiiHlQjy/KsWO2GBrClj fr+RDXpa3rvd4MnTmvbh5OFzA8P4zk84QJu918Mgn3EeDMrBCJc+ZhFgKFewNwl/WK TlvB4T0f3z8vZtpOEAuJ6ME7Yozku/boeTEP3tVi84yvcZR5Oe0W/6aaGWytyD8W/L gniScYJsnli5w== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4SwhFr6Y24z4xM2; Thu, 21 Dec 2023 18:02:40 +1100 (AEDT) From: David Gibson To: Stefano Brivio , passt-dev@passt.top Subject: [PATCH v3 00/15] RFC: Unified flow table Date: Thu, 21 Dec 2023 18:02:22 +1100 Message-ID: <20231221070237.1422557-1-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.43.0 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: 3EK5UXA5D6G3JWRHNTYK2AYLQVNXILVA X-Message-ID-Hash: 3EK5UXA5D6G3JWRHNTYK2AYLQVNXILVA X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This is a third draft of the first steps in implementing more general "connection" tracking, as described at: https://pad.passt.top/p/NewForwardingModel This series changes the TCP connection table and hash table into a more general flow table that can track other protocols as well. Each flow uniformly keeps track of all the relevant addresses and ports, which will allow for more robust control of NAT and port forwarding. ICMP is converted to use the new flow table. Caveats: * We significantly increase the size of a connection/flow entry - Can probably be mitigated, but I haven't investigated much yet * We perform a number of extra getsockname() calls to know some of the socket endpoints - Haven't yet measured how much performance impact that has - Can be mitigated in at least some cases, but again, haven't tried yet * UDP is not converted yet Changes since v2: * Cosmetic fixes based on review * Extra doc comments for enum flow_type * Rename flowside to flowaddrs which turns out to make more sense in light of future changes * Fix bug where the socket flowaddrs for tap initiated connections wasn't initialised to match the socket address we were using in the case of map-gw NAT * New flowaddrs_from_sock() helper used in most cases which is cleaner and should avoid bugs like the above * Using newer centralised workarounds for clang-tidy issue 58992 * Remove duplicate definition of FLOW_MAX as maximum flow type and maximum number of tracked flows * Rebased on newer versions of preliminary work (ICMP, flow based dispatch and allocation, bind/address cleanups) * Unified hash table as well as base flow table * Integrated ICMP Changes since v1: * Terminology changes - "Endpoint" address/port instead of "correspondent" address/port - "flowside" instead of "demiflow" * Actually move the connection table to a new flow table structure in new files * Significant rearrangement of earlier patchs on top of that new table, to reduce churn David Gibson (15): flow: Common data structures for tracking flow addresses tcp, flow: Maintain guest side flow information tcp, flow: Maintain host side flow information tcp_splice,flow: Maintain flow information for spliced connections flow, tcp, tcp_splice: Uniform debug helpers for new flows tcp, flow: Replace TCP specific hash function with general flow hash flow: Add helper to determine a flow's protocol flow, tcp: Generalise TCP hash table to general flow hash table tcp: Re-use flow hash for initial sequence number generation icmp: Store ping socket information in the flow table icmp: Populate guest side information for ping flows icmp: Populate and use host side flow information icmp: Use 'flowside' epoll references for ping sockets icmp: Merge EPOLL_TYPE_ICMP and EPOLL_TYPE_ICMPV6 icmp: Eliminate icmp_id_map Makefile | 6 +- flow.c | 260 ++++++++++++++++++++++++++++++++++++++++++ flow.h | 104 +++++++++++++++++ flow_table.h | 2 + icmp.c | 211 +++++++++++++++++++--------------- icmp.h | 15 +-- icmp_flow.h | 31 +++++ passt.c | 15 +-- passt.h | 9 +- tap.c | 11 -- tap.h | 1 - tcp.c | 313 +++++++++++++++------------------------------------ tcp_conn.h | 9 -- tcp_splice.c | 63 ++++++++--- tcp_splice.h | 3 +- util.c | 4 +- util.h | 18 +++ 17 files changed, 683 insertions(+), 392 deletions(-) create mode 100644 icmp_flow.h -- 2.43.0