From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTP id 73EA15A0082 for ; Tue, 31 Jan 2023 19:44:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675190657; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gWCg9V+Hbma50nemzEd7Ur6lisMbL9saHwabGNK1PsA=; b=KUb4zrDXcceeB8bpfcjJJHwn/Kh0HSgC5dXnutRjIKt4d7IZw9f0resx1biLNihqWGh+0W Bq9CwsNGIBQG8gMbk2VCDCOSPwrwv69abbSRU6cA2MGIjEfTJFeeZ5OWUwWqpd2O+bBc13 2bKCvKo3F/tIWIU8axWMPHYTxzfmlvA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-640-ojpyQUBMM9uwfSYuqm_-Lw-1; Tue, 31 Jan 2023 13:44:15 -0500 X-MC-Unique: ojpyQUBMM9uwfSYuqm_-Lw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6E62A885628 for ; Tue, 31 Jan 2023 18:44:15 +0000 (UTC) Received: from maya.cloud.tilaa.com (ovpn-208-4.brq.redhat.com [10.40.208.4]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 42BF1C15BAD; Tue, 31 Jan 2023 18:44:15 +0000 (UTC) Date: Tue, 31 Jan 2023 19:43:52 +0100 From: Stefano Brivio To: Paul Holzinger Subject: Re: Race condition with pasta COMMAND... Message-ID: <20230131194352.69488545@elisabeth> In-Reply-To: References: <984f4c24-8fe4-71db-1fa6-174a1f5b055f@redhat.com> <20230131140246.33b39a31@elisabeth> Organization: Red Hat MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: GWQHZ6RLTU5SB6RH3VUMTMXF2HTQQJEQ X-Message-ID-Hash: GWQHZ6RLTU5SB6RH3VUMTMXF2HTQQJEQ X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.3 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue, 31 Jan 2023 19:00:02 +0100 Paul Holzinger wrote: > Hi Stefano, >=20 > On 31/01/2023 14:02, Stefano Brivio wrote: > > Hi Paul, > > > > On Mon, 30 Jan 2023 19:08:14 +0100 > > Paul Holzinger wrote: > > =20 > >> Hi all, > >> > >> while debugging some things I used `./pasta --config-net -- nslookup > >> google.com 1.1.1.1` to test dns. > >> The problem is that does not work because the nslookup process will be > >> executed before pasta is > >> ready with the netns setup, i.e. compare `./pasta --config-net -- ip a= `. =20 > > Thanks for the report. I also hit this a couple of months ago but I > > couldn't find yet the time to deal with it: > > > > https://bugs.passt.top/show_bug.cgi?id=3D37 > > =20 > >> So a workaround is to spawn a shell and sleep: `sh -c "sleep 1; nslook= up > >> google.com 1.1.1.1"` > >> However this is ugly and does not ensure that the netns is ready after > >> one second. As a user > >> I would expect pasta to wait until the setup is finished before it cal= ls > >> exec(). =20 > > Absolutely, yes. > > > > As I mentioned on that ticket, I *think* that the only way to make sure > > the setup is actually complete is to query back via netlink addresses > > and routes we configured -- simply waiting until we successfully sent > > netlink messages isn't enough, because it takes a (substantial) while > > until addresses and routes are actually available. > > =20 > Is there any reason why we would explicitly need to query netlink > after the setup is done? > With NLM_F_ACK=C2=A0 it should wait long enough, no? We use it like that = in > podman and never experienced an problem with the network not being > ready apart from ipv6 DAD and I don't think we need worry about this here= . Perhaps in practice yes, but we don't have a formal guarantee that any given routing table/trie/structure is actually synchronised just because we get an acknowledgement for the netlink message (that's all NLM_F_ACK says). In terms of RCU, by the time we read NLM_F_ACK in passt, we know that any pending (from the address/routing update perspective) read-side critical section must have exited. However, routing tables have generation IDs and there might be other mechanisms (current or future) that prevent us from making this assumption. Also, given multiple CPU threads on a machine, I don't think it's really that straightforward. > From=C2=A0 a quick test, at least for my use case it seems to be working = when > I hold the exec until the isolate_prefork() call. I tried that too in the past and I had some "failures", I don't remember if that was on "ip route show" or "ip address show". Still, it's by all means an improvement on the original, and if it works for you reliably that's a good sign, so I'd definitely welcome that patch. I'm just not convinced it's a complete fix, so I would also welcome a more involved patch doing the check via netlink. :) --=20 Stefano