From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 365475A027B for ; Thu, 30 Nov 2023 03:02:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=201602; t=1701309746; bh=cxkPGBYt90D9duAyBqlrKaRlAyhq0D+as3y+v/0pJIc=; h=From:To:Cc:Subject:Date:From; b=ghFjaRTczLtwaUt8B2LktWkR9HT6f0QZ1dFJa86FtEEJrn+Hj2sAqn3Ou+1AdToJX lm/HlIL7ktzNU/hSZzk65rghAHVwMS/EDFfD0VfOxyb4rnE5HHsSAgW0sKid/ckGah 7Dee7wUY3psK8WY4Rs1vipP+oZlsSR7l4AwH39q8= Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4Sgfb605BBz4xVg; Thu, 30 Nov 2023 13:02:26 +1100 (AEDT) From: David Gibson To: passt-dev@passt.top, Stefano Brivio Subject: [PATCH v3 00/16] Introduce unified flow table, first steps Date: Thu, 30 Nov 2023 13:02:06 +1100 Message-ID: <20231130020222.4056647-1-david@gibson.dropbear.id.au> X-Mailer: git-send-email 2.43.0 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID-Hash: R5LJQ32DWS4ZRASHE2K6UNPRV3AFDJM6 X-Message-ID-Hash: R5LJQ32DWS4ZRASHE2K6UNPRV3AFDJM6 X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Here's my latest revision of some of the basics of the flow table. So far it's basically just a renaming of the existing TCP connection table, along with some associated helpers. It's used for some new logging infrastructure, but otherwise doesn't really function any differently. However, this subset of the flow table work no longer bloats flow/connection entries over a single cache line. That removes the most prominent drawback of earlier revisions, meaning I think this series is ready for merge now. Doing so will mean the later series making more substantive changes to the flow behaviour are simpler. Tested on top of the patch updating shell prompt escape handling, but should be independent of it. Changes since v2: * Added a patch to only use C11 static_assert(), not C23 static_assert() (needed for next change) [1/16] * Better handling of the bounds on valid values of enum flow_type [2/16] * No longer introduce an additional C23 style static_assert() [8/16] * Add fix for overly long guestfish commands (needed for next change) [13/16] * Added a patch supporting names for pifs [14/16] * Added patches with some further TCP reworks in preparation for the general flow table [15-16/16] Changes since v1: * Removed a inaccurate stale comment * Added doc comment to FLOW() macro * Added new patches cleaning up signedness of 'side' variables * Added new patches introducing "sidx"s (flow+side indices) David Gibson (16): treewide: Add messages to static_assert() calls flow, tcp: Generalise connection types flow, tcp: Move TCP connection table to unified flow table flow, tcp: Consolidate flow pointer<->index helpers util: MAX_FROM_BITS() should be unsigned flow: Make unified version of flow table compaction flow, tcp: Add logging helpers for connection related messages flow: Introduce 'sidx' type to represent one side of one flow tcp: Remove unneccessary bounds check in tcp_timer_handler() flow,tcp: Generalise TCP epoll_ref to generic flows tcp_splice: Use unsigned to represent side flow,tcp: Use epoll_ref type including flow and side test: Avoid hitting guestfish command length limits pif: Add helpers to get the name of a pif tcp: "TCP" hash secret doesn't need to be TCP specific tcp: Don't defer hash table removal Makefile | 15 +- flow.c | 86 +++++++++++ flow.h | 74 ++++++++++ flow_table.h | 86 +++++++++++ inany.h | 6 +- passt.c | 40 ++++++ passt.h | 15 +- pif.c | 21 +++ pif.h | 19 +++ tcp.c | 288 ++++++++++++++++--------------------- tcp.h | 7 - tcp_conn.h | 46 ++---- tcp_splice.c | 128 +++++++---------- tcp_splice.h | 2 +- test/prepare-distro-img.sh | 2 +- util.h | 2 +- 16 files changed, 540 insertions(+), 297 deletions(-) create mode 100644 flow.c create mode 100644 flow.h create mode 100644 flow_table.h create mode 100644 pif.c -- 2.43.0