From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTP id D027E5A004E for ; Thu, 13 Jun 2024 10:24:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718267056; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=GVjjhYLFVPF/13xmtEiTQHMyRbvtswkscR/IJ3GbpS0=; b=UNtsSkyF66DhRHi5C37w4c9BxZiOFCE3zv56djCFNjjPxvybpj4ac7t60DKfSb9AmjYmHa dEvR9S1ugSrcnaeK4Ftt0mqATBV34Su+D3RlXZQHjhw4xbPvSqdm41SJWYAMBFw92BZKVq jAePqAlGC5BBlQhXRDkR4aVNFbd7axw= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-81-6rQ_-Cv3PfK6ltHf4_UmyQ-1; Thu, 13 Jun 2024 04:24:15 -0400 X-MC-Unique: 6rQ_-Cv3PfK6ltHf4_UmyQ-1 Received: by mail-qk1-f197.google.com with SMTP id af79cd13be357-795589ae41fso90886685a.0 for ; Thu, 13 Jun 2024 01:24:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718267054; x=1718871854; h=content-transfer-encoding:in-reply-to:autocrypt:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GVjjhYLFVPF/13xmtEiTQHMyRbvtswkscR/IJ3GbpS0=; b=NgnIoPAvXijvZEAND3cu6xVpC3naHuxAETeFNWFZR20wn7iBMyb1fDRcsiKWn8IYvJ YmK0Lg4TMlgR93nYKfIX2c3z41ay0SkWRNohXdZ5Q9sPjBKmO38pBUKWHIP7SmNY9H2j /7J28F90zSJnSyYiZAtfSfsTpLBo4xGOrOLwm8mZIxfdYo2ncygaDsSdn8d2IggjSrBW 6+ftGQlnOA3bpAW8ND2nAQfJxLZ+6ie4d/ze0sv/D01rJbkWmhv8ot/Yb5LTPtxNnkRl UwkNhctiEAwpMM2pH2/qozESTECODa4T4eHuZ7DQ/V6129+XONKRcL51JyqbtC9oY7Tq Zzaw== X-Gm-Message-State: AOJu0Yy9Q9+wABkGLg+CxogsMco9DG7SkN6RtTHfUXwvdh/1rrgN9SoU 5vsyp+RYvx/necNHy9nIIZDgG2LkgP0lna08GjvwxtZBi6TTsso3RYlZCTXSWT7w66tit32iLPY tyNf7U6W02Oxr2QH1jU7jJqxpxaN1lRXJAfgT8PxF5UfM68YjLQ== X-Received: by 2002:a05:620a:e96:b0:795:5ab3:ba5b with SMTP id af79cd13be357-797f60165aamr299967585a.32.1718267054303; Thu, 13 Jun 2024 01:24:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHgFIiWUNuCiGorEMXE50L1A3uKy2jJK7pZtGa8MOFYF46immRsrmH5+Ju9gbT9/h+zQRgYBw== X-Received: by 2002:a05:620a:e96:b0:795:5ab3:ba5b with SMTP id af79cd13be357-797f60165aamr299966185a.32.1718267053658; Thu, 13 Jun 2024 01:24:13 -0700 (PDT) Received: from ?IPV6:2a01:e0a:e10:ef90:8e19:3414:44f0:befd? ([2a01:e0a:e10:ef90:8e19:3414:44f0:befd]) by smtp.gmail.com with ESMTPSA id af79cd13be357-798aacb15efsm33352085a.14.2024.06.13.01.24.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Jun 2024 01:24:13 -0700 (PDT) Message-ID: Date: Thu, 13 Jun 2024 10:24:11 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 1/8] tcp: extract buffer management from tcp_send_flag() To: Stefano Brivio References: <20240612154734.1044883-1-lvivier@redhat.com> <20240612154734.1044883-2-lvivier@redhat.com> <20240612232210.3b1f975f@elisabeth> <20240613080719.67933d22@elisabeth> From: Laurent Vivier Autocrypt: addr=lvivier@redhat.com; keydata= xsFNBFYFJhkBEAC2me7w2+RizYOKZM+vZCx69GTewOwqzHrrHSG07MUAxJ6AY29/+HYf6EY2 WoeuLWDmXE7A3oJoIsRecD6BXHTb0OYS20lS608anr3B0xn5g0BX7es9Mw+hV/pL+63EOCVm SUVTEQwbGQN62guOKnJJJfphbbv82glIC/Ei4Ky8BwZkUuXd7d5NFJKC9/GDrbWdj75cDNQx UZ9XXbXEKY9MHX83Uy7JFoiFDMOVHn55HnncflUncO0zDzY7CxFeQFwYRbsCXOUL9yBtqLer Ky8/yjBskIlNrp0uQSt9LMoMsdSjYLYhvk1StsNPg74+s4u0Q6z45+l8RAsgLw5OLtTa+ePM JyS7OIGNYxAX6eZk1+91a6tnqfyPcMbduxyBaYXn94HUG162BeuyBkbNoIDkB7pCByed1A7q q9/FbuTDwgVGVLYthYSfTtN0Y60OgNkWCMtFwKxRaXt1WFA5ceqinN/XkgA+vf2Ch72zBkJL RBIhfOPFv5f2Hkkj0MvsUXpOWaOjatiu0fpPo6Hw14UEpywke1zN4NKubApQOlNKZZC4hu6/ 8pv2t4HRi7s0K88jQYBRPObjrN5+owtI51xMaYzvPitHQ2053LmgsOdN9EKOqZeHAYG2SmRW LOxYWKX14YkZI5j/TXfKlTpwSMvXho+efN4kgFvFmP6WT+tPnwARAQABzSNMYXVyZW50IFZp dmllciA8bHZpdmllckByZWRoYXQuY29tPsLBeAQTAQIAIgUCVgVQgAIbAwYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQ8ww4vT8vvjwpgg//fSGy0Rs/t8cPFuzoY1cex4limJQfReLr SJXCANg9NOWy/bFK5wunj+h/RCFxIFhZcyXveurkBwYikDPUrBoBRoOJY/BHK0iZo7/WQkur 6H5losVZtrotmKOGnP/lJYZ3H6OWvXzdz8LL5hb3TvGOP68K8Bn8UsIaZJoeiKhaNR0sOJyI YYbgFQPWMHfVwHD/U+/gqRhD7apVysxv5by/pKDln1I5v0cRRH6hd8M8oXgKhF2+rAOL7gvh jEHSSWKUlMjC7YwwjSZmUkL+TQyE18e2XBk85X8Da3FznrLiHZFHQ/NzETYxRjnOzD7/kOVy gKD/o7asyWQVU65mh/ECrtjfhtCBSYmIIVkopoLaVJ/kEbVJQegT2P6NgERC/31kmTF69vn8 uQyW11Hk8tyubicByL3/XVBrq4jZdJW3cePNJbTNaT0d/bjMg5zCWHbMErUib2Nellnbg6bc 2HLDe0NLVPuRZhHUHM9hO/JNnHfvgiRQDh6loNOUnm9Iw2YiVgZNnT4soUehMZ7au8PwSl4I KYE4ulJ8RRiydN7fES3IZWmOPlyskp1QMQBD/w16o+lEtY6HSFEzsK3o0vuBRBVp2WKnssVH qeeV01ZHw0bvWKjxVNOksP98eJfWLfV9l9e7s6TaAeySKRRubtJ+21PRuYAxKsaueBfUE7ZT 7zfOwU0EVgUmGQEQALxSQRbl/QOnmssVDxWhHM5TGxl7oLNJms2zmBpcmlrIsn8nNz0rRyxT 460k2niaTwowSRK8KWVDeAW6ZAaWiYjLlTunoKwvF8vP3JyWpBz0diTxL5o+xpvy/Q6YU3BN efdq8Vy3rFsxgW7mMSrI/CxJ667y8ot5DVugeS2NyHfmZlPGE0Nsy7hlebS4liisXOrN3jFz asKyUws3VXek4V65lHwB23BVzsnFMn/bw/rPliqXGcwl8CoJu8dSyrCcd1Ibs0/Inq9S9+t0 VmWiQWfQkz4rvEeTQkp/VfgZ6z98JRW7S6l6eophoWs0/ZyRfOm+QVSqRfFZdxdP2PlGeIFM C3fXJgygXJkFPyWkVElr76JTbtSHsGWbt6xUlYHKXWo+xf9WgtLeby3cfSkEchACrxDrQpj+ Jt/JFP+q997dybkyZ5IoHWuPkn7uZGBrKIHmBunTco1+cKSuRiSCYpBIXZMHCzPgVDjk4viP brV9NwRkmaOxVvye0vctJeWvJ6KA7NoAURplIGCqkCRwg0MmLrfoZnK/gRqVJ/f6adhU1oo6 z4p2/z3PemA0C0ANatgHgBb90cd16AUxpdEQmOCmdNnNJF/3Zt3inzF+NFzHoM5Vwq6rc1JP jfC3oqRLJzqAEHBDjQFlqNR3IFCIAo4SYQRBdAHBCzkM4rWyRhuVABEBAAHCwV8EGAECAAkF AlYFJhkCGwwACgkQ8ww4vT8vvjwg9w//VQrcnVg3TsjEybxDEUBm8dBmnKqcnTBFmxN5FFtI WlEuY8+YMiWRykd8Ln9RJ/98/ghABHz9TN8TRo2b6WimV64FmlVn17Ri6FgFU3xNt9TTEChq AcNg88eYryKsYpFwegGpwUlaUaaGh1m9OrTzcQy+klVfZWaVJ9Nw0keoGRGb8j4XjVpL8+2x OhXKrM1fzzb8JtAuSbuzZSQPDwQEI5CKKxp7zf76J21YeRrEW4WDznPyVcDTa+tz++q2S/Bp P4W98bXCBIuQgs2m+OflERv5c3Ojldp04/S4NEjXEYRWdiCxN7ca5iPml5gLtuvhJMSy36gl U6IW9kn30IWuSoBpTkgV7rLUEhh9Ms82VWW/h2TxL8enfx40PrfbDtWwqRID3WY8jLrjKfTd R3LW8BnUDNkG+c4FzvvGUs8AvuqxxyHbXAfDx9o/jXfPHVRmJVhSmd+hC3mcQ+4iX5bBPBPM oDqSoLt5w9GoQQ6gDVP2ZjTWqwSRMLzNr37rJjZ1pt0DCMMTbiYIUcrhX8eveCJtY7NGWNyx FCRkhxRuGcpwPmRVDwOl39MB3iTsRighiMnijkbLXiKoJ5CDVvX5yicNqYJPKh5MFXN1bvsB kmYiStMRbrD0HoY1kx5/VozBtc70OU0EB8Wrv9hZD+Ofp0T3KOr1RUHvCZoLURfFhSQ= In-Reply-To: <20240613080719.67933d22@elisabeth> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Message-ID-Hash: QPNWNMXRWEPQTQFY4ZGWGQMRSOBS4D7J X-Message-ID-Hash: QPNWNMXRWEPQTQFY4ZGWGQMRSOBS4D7J X-MailFrom: lvivier@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, David Gibson X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi, I think the problem can be because tcp_l2_buf_fill_headers() has been moved out of tcp_prepare_flags() and so moved after: if (th->fin || th->syn) conn->seq_to_tap++; and con->seq_to_tap is also a parameter of tcp_l2_buf_fill_headers(). So it is increased before and not after. Could you try: diff --git a/tcp.c b/tcp.c index 6800209d4122..647f42291fcf 100644 --- a/tcp.c +++ b/tcp.c @@ -1607,6 +1607,7 @@ static int tcp_prepare_flags(struct ctx *c, struct tcp_tap_conn *conn, if (!tcp_update_seqack_wnd(c, conn, flags, &tinfo) && !flags) return 0; + *optlen = 0; if (flags & SYN) { int mss; @@ -1643,7 +1644,6 @@ static int tcp_prepare_flags(struct ctx *c, struct tcp_tap_conn *conn, *data++ = conn->ws_to_tap; } else if (!(flags & RST)) { flags |= ACK; - *optlen = 0; } th->doff = (sizeof(*th) + *optlen) / 4; @@ -1663,10 +1663,6 @@ static int tcp_prepare_flags(struct ctx *c, struct tcp_tap_conn *conn, if (th->fin) conn_flag(c, conn, ACK_FROM_TAP_DUE); - /* RFC 793, 3.1: "[...] and the first data octet is ISN+1." */ - if (th->fin || th->syn) - conn->seq_to_tap++; - return 1; } @@ -1702,6 +1698,10 @@ static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) conn->seq_to_tap); iov[TCP_IOV_PAYLOAD].iov_len = l4len; + /* RFC 793, 3.1: "[...] and the first data octet is ISN+1." */ + if (th->fin || th->syn) + conn->seq_to_tap++; + if (flags & DUP_ACK) { struct iovec *dup_iov; int i; Thanks, Laurent On 13/06/2024 08:07, Stefano Brivio wrote: > On Wed, 12 Jun 2024 23:22:10 +0200 > Stefano Brivio wrote: > >> I couldn't find out why this patch breaks the the pasta_podman/bats >> test, yet, that is: >> >> not ok 19 [505] Single TCP port forwarding, IPv4, tap >> # (from function `bail-now' in file test/podman/test/system/helpers.bash, line 235, >> # from function `assert' in file test/podman/test/system/helpers.bash, line 929, >> # from function `pasta_test_do' in file test/podman/test/system/505-networking-pasta.bats, line 239, >> # in test file test/podman/test/system/505-networking-pasta.bats, line 472) >> # `pasta_test_do' failed >> # >> # [22:54:18.306131353] $ test/podman/bin/podman rm -t 0 --all --force --ignore >> # >> # [22:54:18.367462243] $ test/podman/bin/podman ps --all --external --format {{.ID}} {{.Names}} >> # >> # [22:54:18.394935392] $ test/podman/bin/podman images --all --format {{.Repository}}:{{.Tag}} {{.ID}} >> # [22:54:18.419773379] quay.io/libpod/testimage:20240123 1f6acd4c4a1d >> # >> # [22:54:19.246631856] $ test/podman/bin/podman info --format {{.Host.Pasta.Executable}} >> # [22:54:20.084392405] /home/sbrivio/passt/pasta >> # >> # [22:54:20.167980222] $ test/podman/bin/podman run --net=pasta -p [88.198.0.164]:5727:5727/tcp quay.io/libpod/testimage:20240123 sh -c for port in $(seq 5727 5727); do socat -u TCP4-LISTEN:${port},bind=[88.198.0.164] STDOUT & done; wait >> # [22:54:37.256040883] x2024/06/12 20:54:37 socat[3] E read(6, 0x7fe675cd6000, 8192): Connection reset by peer >> # #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv >> # #| FAIL: Mismatch between data sent and received >> # #| expected: = x >> # #| actual: x2024/06/12 20:54:37 socat\[3\] E read\(6\, 0x7fe675cd6000\, 8192\): Connection reset by peer >> # #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> meaning that the data transfer is actually fine, but we reset the >> connection instead of an orderly shutdown. >> >> I found a few issues while looking for that: >> >> On Wed, 12 Jun 2024 17:47:27 +0200 >> Laurent Vivier wrote: >> >>> This commit isolates the internal data structure management used for storing >>> data (e.g., tcp4_l2_flags_iov[], tcp6_l2_flags_iov[], tcp4_flags_ip[], >>> tcp4_flags[], ...) from the tcp_send_flag() function. The extracted >>> functionality is relocated to a new function named tcp_fill_flag_header(). >>> >>> tcp_fill_flag_header() is now a generic function that accepts parameters such >>> as struct tcphdr and a data pointer. tcp_send_flag() utilizes this parameter to >>> pass memory pointers from tcp4_l2_flags_iov[] and tcp6_l2_flags_iov[]. >>> >>> This separation sets the stage for utilizing tcp_fill_flag_header() to >>> set the memory provided by the guest via vhost-user in future developments. >>> >>> Signed-off-by: Laurent Vivier >>> --- >>> >>> Notes: >>> v6: >>> - rename tcp_fill_flag_header() to tcp_prepare_flags() >>> - set optlen to 0 in tcp_prepare_flags() >>> >>> v5: >>> - use tcp_fill_flag_header() rather than tcp_fill_headers4() and >>> tcp_fill_headers6(). >>> >>> tcp.c | 72 +++++++++++++++++++++++++++++++++++++++-------------------- >>> 1 file changed, 48 insertions(+), 24 deletions(-) >>> >>> diff --git a/tcp.c b/tcp.c >>> index dd8d46e08628..6800209d4122 100644 >>> --- a/tcp.c >>> +++ b/tcp.c >>> @@ -1567,24 +1567,25 @@ static void tcp_update_seqack_from_tap(const struct ctx *c, >>> } >>> >>> /** >>> - * tcp_send_flag() - Send segment with flags to tap (no payload) >>> + * tcp_prepare_flags() - Prepare header for flags-only segment (no payload) >>> * @c: Execution context >>> * @conn: Connection pointer >>> * @flags: TCP flags: if not set, send segment only if ACK is due >>> + * @th: TCP header to update >>> + * @data: buffer to store TCP option >>> + * @optlen: size of the TCP option buffer (output parameter) >>> * >>> - * Return: negative error code on connection reset, 0 otherwise >>> + * Return: < 0 error code on connection reset, >>> + * 0 if there is no flag to send >>> + * 1 otherwise >> >> This is often called with if (tcp_send_flag(...)) or >> if (!tcp_send_flag(...)). Those need to be replaced with >> if (tcp_send_flag(...) < 0) or if (tcp_send_flag(...) >= 0) in the >> callers. > > Ah, no, sorry, you already took care of this in the new > tcp_send_flag(). Then there must be something else... > >>> */ >>> -static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >>> +static int tcp_prepare_flags(struct ctx *c, struct tcp_tap_conn *conn, >>> + int flags, struct tcphdr *th, char *data, >>> + size_t *optlen) >>> { >>> - struct tcp_flags_t *payload; >>> struct tcp_info tinfo = { 0 }; >>> socklen_t sl = sizeof(tinfo); >>> int s = conn->sock; >>> - size_t optlen = 0; >>> - struct tcphdr *th; >>> - struct iovec *iov; >>> - size_t l4len; >>> - char *data; >>> >>> if (SEQ_GE(conn->seq_ack_to_tap, conn->seq_from_tap) && >>> !flags && conn->wnd_to_tap) >>> @@ -1606,20 +1607,11 @@ static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >>> if (!tcp_update_seqack_wnd(c, conn, flags, &tinfo) && !flags) >>> return 0; >>> >>> - if (CONN_V4(conn)) >>> - iov = tcp4_l2_flags_iov[tcp4_flags_used++]; >>> - else >>> - iov = tcp6_l2_flags_iov[tcp6_flags_used++]; >>> - >>> - payload = iov[TCP_IOV_PAYLOAD].iov_base; >>> - th = &payload->th; >>> - data = payload->opts; >>> - >>> if (flags & SYN) { >>> int mss; >>> >>> /* Options: MSS, NOP and window scale (8 bytes) */ >>> - optlen = OPT_MSS_LEN + 1 + OPT_WS_LEN; >>> + *optlen = OPT_MSS_LEN + 1 + OPT_WS_LEN; >>> >>> *data++ = OPT_MSS; >>> *data++ = OPT_MSS_LEN; >>> @@ -1651,19 +1643,16 @@ static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >>> *data++ = conn->ws_to_tap; >>> } else if (!(flags & RST)) { >>> flags |= ACK; >>> + *optlen = 0; >> >> *optlen also needs to be set to 0 if (flags & RST), say: >> >> } else { >> *optlen = 0; >> >> if (!(flags & RST)) >> flags |= ACK; >> } >> >>> } >>> >>> - th->doff = (sizeof(*th) + optlen) / 4; >>> + th->doff = (sizeof(*th) + *optlen) / 4; >>> >>> th->ack = !!(flags & ACK); >>> th->rst = !!(flags & RST); >>> th->syn = !!(flags & SYN); >>> th->fin = !!(flags & FIN); >>> >>> - l4len = tcp_l2_buf_fill_headers(c, conn, iov, optlen, NULL, >>> - conn->seq_to_tap); >>> - iov[TCP_IOV_PAYLOAD].iov_len = l4len; >>> - >>> if (th->ack) { >>> if (SEQ_GE(conn->seq_ack_to_tap, conn->seq_from_tap)) >>> conn_flag(c, conn, ~ACK_TO_TAP_DUE); >>> @@ -1678,6 +1667,41 @@ static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >>> if (th->fin || th->syn) >>> conn->seq_to_tap++; >>> >>> + return 1; >>> +} >>> + >>> +/** >>> + * tcp_send_flag() - Send segment with flags to tap (no payload) >>> + * @c: Execution context >>> + * @conn: Connection pointer >>> + * @flags: TCP flags: if not set, send segment only if ACK is due >>> + * >>> + * Return: negative error code on connection reset, 0 otherwise >>> + */ >>> +static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >>> +{ >>> + struct tcp_flags_t *payload; >>> + struct iovec *iov; >>> + size_t optlen; >>> + size_t l4len; >>> + int ret; >>> + >>> + if (CONN_V4(conn)) >>> + iov = tcp4_l2_flags_iov[tcp4_flags_used++]; >>> + else >>> + iov = tcp6_l2_flags_iov[tcp6_flags_used++]; >> >> We increase the counters here, but we don't decrease them back if we >> hit if (ret <= 0) return ret; later. >> >>> + >>> + payload = iov[TCP_IOV_PAYLOAD].iov_base; >>> + >>> + ret = tcp_prepare_flags(c, conn, flags, &payload->th, >>> + payload->opts, &optlen); >>> + if (ret <= 0) >>> + return ret; >>> + >>> + l4len = tcp_l2_buf_fill_headers(c, conn, iov, optlen, NULL, >>> + conn->seq_to_tap); >>> + iov[TCP_IOV_PAYLOAD].iov_len = l4len; >>> + >>> if (flags & DUP_ACK) { >>> struct iovec *dup_iov; >>> int i; >> >> Here's the diff I have so far, I'm not necessarily recommending any of >> that, it was just a quick try: >> >> diff --git a/tcp.c b/tcp.c >> index 6800209..6bff6bc 100644 >> --- a/tcp.c >> +++ b/tcp.c >> @@ -1644,6 +1644,8 @@ static int tcp_prepare_flags(struct ctx *c, struct tcp_tap_conn *conn, >> } else if (!(flags & RST)) { >> flags |= ACK; >> *optlen = 0; >> + } else { >> + *optlen = 0; >> } >> >> th->doff = (sizeof(*th) + *optlen) / 4; >> @@ -1695,8 +1697,14 @@ static int tcp_send_flag(struct ctx *c, struct tcp_tap_conn *conn, int flags) >> >> ret = tcp_prepare_flags(c, conn, flags, &payload->th, >> payload->opts, &optlen); >> - if (ret <= 0) >> + if (ret <= 0) { >> + if (CONN_V4(conn)) >> + tcp4_flags_used--; >> + else >> + tcp6_flags_used--; >> + >> return ret; >> + } >> >> l4len = tcp_l2_buf_fill_headers(c, conn, iov, optlen, NULL, >> conn->seq_to_tap); >> @@ -1738,7 +1746,7 @@ static void tcp_rst_do(struct ctx *c, struct tcp_tap_conn *conn) >> if (conn->events == CLOSED) >> return; >> >> - if (!tcp_send_flag(c, conn, RST)) >> + if (tcp_send_flag(c, conn, RST) >= 0) >> conn_event(c, conn, CLOSED); >> } >> >> @@ -2111,7 +2119,7 @@ static void tcp_conn_from_tap(struct ctx *c, sa_family_t af, >> } else { >> tcp_get_sndbuf(conn); >> >> - if (tcp_send_flag(c, conn, SYN | ACK)) >> + if (tcp_send_flag(c, conn, SYN | ACK) < 0) >> goto cancel; >> >> conn_event(c, conn, TAP_SYN_ACK_SENT); >> @@ -2282,9 +2290,11 @@ static int tcp_data_from_sock(struct ctx *c, struct tcp_tap_conn *conn) >> >> if (!len) { >> if ((conn->events & (SOCK_FIN_RCVD | TAP_FIN_SENT)) == SOCK_FIN_RCVD) { >> - if ((ret = tcp_send_flag(c, conn, FIN | ACK))) { >> + int rc = tcp_send_flag(c, conn, FIN | ACK); >> + >> + if (rc < 0) { >> tcp_rst(c, conn); >> - return ret; >> + return rc; >> } >> >> conn_event(c, conn, TAP_FIN_SENT); >> @@ -2716,7 +2726,7 @@ static void tcp_connect_finish(struct ctx *c, struct tcp_tap_conn *conn) >> return; >> } >> >> - if (tcp_send_flag(c, conn, SYN | ACK)) >> + if (tcp_send_flag(c, conn, SYN | ACK) < 0) >> return; >> >> conn_event(c, conn, TAP_SYN_ACK_SENT); >> >