From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RrWQqwgF; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id 0FABA5A0271 for ; Wed, 26 Feb 2025 11:15:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1740564916; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZYBBwX7luHX9LWUefIew09t4xxacTYhqpgGDugvl8Rs=; b=RrWQqwgFxHl5R7tuIHWpjAKCPCu40yH2yxrmXlKu2UU8S1MHfXIT1qQ987GqEK9ruO8fPq NX+hgW1uC9HW1fe+yTQ2ZErzIr5V+6kpPSIvI4SR98PMw+KBxPZCMnh8kL6GXYyb0kCmjy rI0jv8jzAtQVsq82KIYoJuEm0gCwT1k= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-arp5PnQ2MLeoMr94z9M9dQ-1; Wed, 26 Feb 2025 05:15:14 -0500 X-MC-Unique: arp5PnQ2MLeoMr94z9M9dQ-1 X-Mimecast-MFC-AGG-ID: arp5PnQ2MLeoMr94z9M9dQ_1740564913 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-438e4e9a53fso50880475e9.1 for ; Wed, 26 Feb 2025 02:15:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740564912; x=1741169712; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZYBBwX7luHX9LWUefIew09t4xxacTYhqpgGDugvl8Rs=; b=m5c9tGySQ5tEYIwd7gKr890hi1i7mZtd6PMu1umdU9DDbhQ+NAwT+MSEb+9EbR1rBR geRx7H4Uwl84mLvvLF3EY4es1xZ6d3LJ5mWz741BuijbuE7nKI5xJqF+ySNCXMR5JLXR Sg3EmmE6gvxnn+g795Z7usZii+3y3VhnDZvggckSrC5Pj+UhcjoHZ31lRcpBoljH9HQL hVIaEaM5Duqw6QD+kAWx9/v0V+uwU1uMi10mALrb5gsj7kDAfuDx+lEtrWkWwfKvQv/d XdhNeOVQl9W98TdfeLyblgoQp5WLDbM3PeaSNptXUjcXMgb0wOl0angmNc/gpeq02HSD Eg5A== X-Gm-Message-State: AOJu0Yw0dWFUGgReCBOo3D6umI9TAv4JSuvvJvt/UxbEG6/m/OotOZZx eHmbpe/7FoGoMJGPC/HClDW3DBq9/xHkoxmIoKaaj1ecs8n2mo8qJBC7utsaRatbBIIm+X21iIL qyD14bFuyll3rDWx/1i3ZqJYvXLqfhvyreINJhSr2fvGzd0ED5fr/RChw8w== X-Gm-Gg: ASbGncsGlq4CKbQAPFLLf6Ee45grqHOcNmHfEw6iQwSCcKbOztR/U+MZwUrACxuhJ6z DOurQviynrhzvvh9EAXaQ3/w03P7/Fzo3KGBS4HajyxWDkRcVxwjHOqrngLP5cBEHOHd2aLHqHQ c0tbL3LXHnTkWs5cjUQgZx5cUn3icQpcjQoelV/e6OIKxSums4H8I7qbcT4mFFXnYSKa/uhGyQi t+SZepsY0ue3g/oZJBS1B9Jbp7uPwYm6GENdrCytsZDss/vT9QgtueNM/Du4qimFAsyLo0sr8wV TPsnm3+S4X7O8sFcKNK3XC8Xv/UlPwMk65TwWR1IbwAj X-Received: by 2002:a05:600c:1c16:b0:439:8346:506b with SMTP id 5b1f17b1804b1-439aebb2e0fmr165764525e9.19.1740564912413; Wed, 26 Feb 2025 02:15:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IHAnM6Y4VzmAAsgSJasQ6vpXffq0v6mW/pc//yiU5aPrTk7hqhtXbVh24cXnb/cV6xC4KCArQ== X-Received: by 2002:a05:600c:1c16:b0:439:8346:506b with SMTP id 5b1f17b1804b1-439aebb2e0fmr165764235e9.19.1740564911967; Wed, 26 Feb 2025 02:15:11 -0800 (PST) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [176.103.220.4]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43aba571274sm15758995e9.31.2025.02.26.02.15.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2025 02:15:11 -0800 (PST) Date: Wed, 26 Feb 2025 11:15:07 +0100 From: Stefano Brivio To: David Gibson Subject: Re: tcp_rst() complications Message-ID: <20250226111507.166ed938@elisabeth> In-Reply-To: References: Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: qkfzFX-Hj09S1BuwWl_G2vSca4twwRx_nB1YBQzgfJM_1740564913 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: 3IGCKKSRPUP5WOGSYZIMKY5IHTWT2WKR X-Message-ID-Hash: 3IGCKKSRPUP5WOGSYZIMKY5IHTWT2WKR X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 26 Feb 2025 20:05:25 +1100 David Gibson wrote: > Amongst other things, I spotted some additional complications using > tcp_rst() in the migration path (some of which might also have > implications in other contexts). These might be things we can safely > ignore, at least for now, but I haven't thought through them enough to > be sure. > > 1) Sending RST to guest during migration > > The first issue is that tcp_rst() will send an actual RST to the guest > on the tap interface. During migration, that means we're sending to > the guest while it's suspended. At the very least that means we > probably have a much higher that usual chance of getting a queue full > failure writing to the tap interface, which could hit problem (2). > > But, beyond that, with vhost-user that means we're writing to guest > memory while the guest is suspended. Kind of the whole point of the > suspended time is that the guest memory doesn't change during it, so > I'm not sure what the consequences will be. If I recall correctly I checked this and something in the vhost-user code will tell us that the queue is not ready yet, done. Ideally we want to make sure we queue those, but queue sizes are finite and I don't think we can guarantee we can pile up 128k RST segments. Right now I would check that the functionality is not spectacularly broken (I looked into that briefly, QEMU didn't crash, guest kept running, but I didn't check further than that). If we miss a RST too bad, things will time out eventually. As a second step we could perhaps introduce a post-migration stage and move calling tcp_rst() to there if the connection is in a given state? > Now, at the moment I > think all our tcp_rst() calls are either on the source during rollback > (i.e. we're committed to resuming only on the source) or on the target > past the point of no return (i.e. we're committed to resuming only on > the target). I suspect that means we can get away with it, but I do > worry this could break something in qeme by violating that assumption. > > 2) tcp_rst() failures > > tcp_rst() can fail if tcp_send_flag() fails. In this case we *don't* > change the events to CLOSED. I _think_ that's a bug: even if we > weren't able to send the RST to the guest, we've already closed the > socket so the flow is dead. Moving to CLOSED state (and then removing > the flow entirely) should mean that we'll resend an RST if the guest > attempts to use the flow again later. > > But.. I was worried there might be some subtle reason for not changing > the event state in that case. Not very subtle: my original idea was that if we fail to send the RST, we should note that (by not moving to CLOSED) and try again from a timer. In practice I've never observed a tcp_send_flag() failure so I'm not sure if that mechanism even works. Moving the socket to CLOSED sounds totally okay to me, surely simpler, and probably more robust. -- Stefano