From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id 176815A004F for ; Fri, 12 Jul 2024 15:34:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1720791285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=emVmmie2ViRJTAzESnkNbOt1++mFRDIFw68LBthEQ6I=; b=fBNlmVqEWEXiTTltSEwV8b7i2Hg/8vFr0xeyrYWhrm4Mw+tSAoDnPieAnmDsOiFBS2HjXc iOL9pbebvxnngBkSU2O9xCaIPSqTgFHAYGeIFVW2uklaxMhkB1TyllgXlJ02DiyarqbtZA 8VWLXWN3XZrh+HH34dIsowAAgZElM7k= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-438-XUbi0t_NPnS4KcgtPszRnw-1; Fri, 12 Jul 2024 09:34:44 -0400 X-MC-Unique: XUbi0t_NPnS4KcgtPszRnw-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-447fef8f3dbso21850251cf.0 for ; Fri, 12 Jul 2024 06:34:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720791283; x=1721396083; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=emVmmie2ViRJTAzESnkNbOt1++mFRDIFw68LBthEQ6I=; b=FLmSZUcpXZ2zvaYDYgvYm+DTBytGx5PBgMQn1bOEXZRWGzJN0LkqD84Q7l9BXAX2Oo J8p/vB5gYeo0ShCxaDgsgpu1CZdI2CsKZSMRTKgU2dSqB5Gyl4GM1ggeUuqp9UMJ/Jiv ROQqcT92scTqjI44bGzms4UsEqPtR2Rm5vrFDbIq+pJLYVSuPJnahir0nZBz6H1D4Amz xX1RnkIGnePA+x9POfeqgXc+YCd+IejEMp/cSsQjVjqGgk0ux71vxZF958FV7QalyR/R K3JaXtrS9w8bQoBLQlYDPRAAd/E/zk6tygO5KQwWxMp+0/P+lTI3iGRzn696KRAKo95b jAZw== X-Gm-Message-State: AOJu0Yxei/WvkIaH5kmIPo3HXs9Cz3QRaeOZ9i0T0bneTMxEP4LT/dPr T4i9jjjVmzF4dZQP1Kivf3/KUKrDswb1r/0mHUDBCKg0fousLr0/gY28mqswmU6iK2bSz2EWKMb 3htkAiXfRibrNbsgrrZcpowsXv7G9eAc4XM4uNc9RQCcvMxVqz6X0Nt/zsnds X-Received: by 2002:a05:622a:54e:b0:446:4c01:1f7a with SMTP id d75a77b69052e-447fa921710mr145882811cf.33.1720791283148; Fri, 12 Jul 2024 06:34:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGrjMJEQBNEkKDPUVHCxzoIZS2Iisl5t3Y3WAxPWYZ6LyN4trLr2tfN9XyoDYEckAXDOgorPw== X-Received: by 2002:a05:622a:54e:b0:446:4c01:1f7a with SMTP id d75a77b69052e-447fa921710mr145882561cf.33.1720791282753; Fri, 12 Jul 2024 06:34:42 -0700 (PDT) Received: from maya.cloud.tilaa.com (maya.cloud.tilaa.com. [164.138.29.33]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-447f9bd3212sm40881101cf.67.2024.07.12.06.34.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Jul 2024 06:34:42 -0700 (PDT) Date: Fri, 12 Jul 2024 15:34:07 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v7 21/27] udp: Handle "spliced" datagrams with per-flow sockets Message-ID: <20240712153407.4b894453@elisabeth> In-Reply-To: <20240705020724.3447719-22-david@gibson.dropbear.id.au> References: <20240705020724.3447719-1-david@gibson.dropbear.id.au> <20240705020724.3447719-22-david@gibson.dropbear.id.au> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: DQI6S4NBOWCANFACFJZYS2E3LLA6X3H7 X-Message-ID-Hash: DQI6S4NBOWCANFACFJZYS2E3LLA6X3H7 X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top, jmaloy@redhat.com X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri, 5 Jul 2024 12:07:18 +1000 David Gibson wrote: > When forwarding a datagram to a socket, we need to find a socket with a > suitable local address to send it. Currently we keep track of such sockets > in an array indexed by local port, but this can't properly handle cases > where we have multiple local addresses in active use. > > For "spliced" (socket to socket) cases, improve this by instead opening > a socket specifically for the target side of the flow. We connect() as > well as bind()ing that socket, so that it will only receive the flow's > reply packets, not anything else. We direct datagrams sent via that socket > using the addresses from the flow table, effectively replacing bespoke > addressing logic with the unified logic in fwd.c > > When we create the flow, we also take a duplicate of the originating > socket, and use that to deliver reply datagrams back to the origin, again > using addresses from the flow table entry. For some reason, after this patch (I bisected), I'm getting an EPOLLERR loop: pasta: epoll event on UDP socket 6 (events: 0x00000001) Flow 0 (NEW): FREE -> NEW Flow 0 (INI): NEW -> INI Flow 0 (INI): HOST [127.0.0.1]:47041 -> [0.0.0.0]:10001 => ? Flow 0 (TGT): INI -> TGT Flow 0 (TGT): HOST [127.0.0.1]:47041 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:47041 -> [127.0.0.1]:10001 Flow 0 (UDP flow): TGT -> TYPED Flow 0 (UDP flow): HOST [127.0.0.1]:47041 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:47041 -> [127.0.0.1]:10001 Flow 0 (UDP flow): Side 0 hash table insert: bucket: 97474 Flow 0 (UDP flow): TYPED -> ACTIVE Flow 0 (UDP flow): HOST [127.0.0.1]:47041 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:47041 -> [127.0.0.1]:10001 pasta: epoll event on UDP reply socket 116 (events: 0x00000008) pasta: epoll event on UDP reply socket 116 (events: 0x00000008) pasta: epoll event on UDP reply socket 116 (events: 0x00000008) [...repeated until I terminate the process] by sending one UDP datagram from the parent namespace with no "listening" process in the namespace, using the "spliced" path, something like this: echo a | nc -q1 -u localhost 10001 after running pasta with: ./pasta -u 10001 --trace -l /tmp/pasta.trace --log-size $((1 << 30)) I tried bigger/multiple datagrams, same result. Before this patch, I get something like this instead: 5.1018: pasta: epoll event on UDP socket 6 (events: 0x00000001) 5.1018: Flow 0 (NEW): FREE -> NEW 5.1018: Flow 0 (INI): NEW -> INI 5.1019: Flow 0 (INI): HOST [127.0.0.1]:41245 -> [0.0.0.0]:10001 => ? 5.1019: Flow 0 (TGT): INI -> TGT 5.1019: Flow 0 (TGT): HOST [127.0.0.1]:41245 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:41245 -> [127.0.0.1]:10001 5.1019: Flow 0 (UDP flow): TGT -> TYPED 5.1019: Flow 0 (UDP flow): HOST [127.0.0.1]:41245 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:41245 -> [127.0.0.1]:10001 5.1019: Flow 0 (UDP flow): Side 0 hash table insert: bucket: 111174 5.1019: Flow 0 (UDP flow): TYPED -> ACTIVE 5.1019: Flow 0 (UDP flow): HOST [127.0.0.1]:41245 -> [0.0.0.0]:10001 => SPLICE [127.0.0.1]:41245 -> [127.0.0.1]:10001 I didn't really investigate, though. -- Stefano