From patchwork Thu Nov 9 11:55:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai Huang X-Patchwork-Id: 163382 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b129:0:b0:403:3b70:6f57 with SMTP id q9csp381330vqs; Thu, 9 Nov 2023 04:01:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IGPsopjb/9Wdvtjui6/v98xQT8JRI3WMesZ9J3k2SIzJJW3E0PBDH33RlV7N2820wDcoOtw X-Received: by 2002:a05:6808:b0d:b0:3a7:8f94:743a with SMTP id s13-20020a0568080b0d00b003a78f94743amr1338667oij.10.1699531262979; Thu, 09 Nov 2023 04:01:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699531262; cv=none; d=google.com; s=arc-20160816; b=RLaJe4tvPOflq6zTVH5XzQADSiKqQhinQYznbY87Moy6klG7gWLWqrb7XUT9/t8aeZ m79R/SeVx88OzAIAqRbU0RHvwk4cqCAbIq1sH/2KFkCXR+BE4JGeHkXyl8mVyDeyZlP6 Vo+fOdSEEwTKsxoveLgdp2tsDEKkYN+rwqA34xefFnvQdXb23RSm+qPjMXifvCnHSyYl pWy9RaOaiFFqVFQbyfRLZt1DkPfW/2/ihFG3fmUAMGnOJZFAbtQzHAol8Heqht49ZfJt +RlBQ4gBL8ZOjja98cIGhvYgWpJPzHn6wciaxgVpqe8NbLxIDLVcJ1vFOXeWCz31IC6U CJVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3YimyGkA1tbp45hfDQeUvahsKPcV10CSSJ8Gp9TSnn8=; fh=WBgbLtMencYhgeHuu2sUs5b9THiYLgy17d2w1N+xuf4=; b=kr58pnMRaN2qFoAiV/A+ViEhhBBLL8hbliMobXqEBemznvSjwv9u10i/cm2biXGZQQ eFwx5Z/GG2+H9yBv1AWccgcUFhg94ZZ728nWgq1LMvDBrF6PMIg4eamFoDnlgoaFJraC 8XiEcQbzx1vdrtaQ29jyv7zYk99WnvTo4vUrBXdAH6szVvyPcbiRbuxi3dvbNte0LgaC 5OIU/p9a5MkqRsjpVbpViNe3ZUV1kmapZX16kF9DR7P1SLeJAXCeE4E5DmvVTihNnjYT T6oFrVzGDOOts2QhemZPBgSV2TOGC2mBbEjL5afCKxUdY1QbKKALwn3YDcHPEmOzbDTd fFHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KuZBZaMn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id bi21-20020a056808189500b003b2f5939549si6633989oib.265.2023.11.09.04.01.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Nov 2023 04:01:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KuZBZaMn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 432798236260; Thu, 9 Nov 2023 04:00:03 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234572AbjKIL73 (ORCPT + 32 others); Thu, 9 Nov 2023 06:59:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234554AbjKIL7G (ORCPT ); Thu, 9 Nov 2023 06:59:06 -0500 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 090C435A5; Thu, 9 Nov 2023 03:58:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1699531108; x=1731067108; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dMT+cVNkz5s3pqtS4rilxjvJDXZQPHh7uYPdNVf2Njg=; b=KuZBZaMn9IofNbX5ccsxoeiIr5FB1VGqDKVMqO/CbPiMqPLvHWQqmXAT hv7JGkRrJ21iZg39rjexzG+vktdJvQTktwxo2JtLq45MDHwqMx82Z4TJB mqqWbWuSRu5CNB3zW5nLT55OKlAHAnOspnArTtOrV3IPAOoe+ttKJz0Y/ czfL6nSoclfn++xXfY/1uBRLpZWXT9Quj+9WCfXfiMsutLZGSgJbd8i6H KJo8jjio5Na0hsa5lzBvigTX6T7udQ1hTljtGIU2RqcMWWxkWm303c/6g A7x49dh4qYXHsdaKiYu7FYCFptt95iYVNh1/X0aeBu6DgHvSjVBYjDUgo Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10888"; a="2936781" X-IronPort-AV: E=Sophos;i="6.03,289,1694761200"; d="scan'208";a="2936781" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2023 03:58:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10888"; a="766977056" X-IronPort-AV: E=Sophos;i="6.03,289,1694761200"; d="scan'208";a="766977056" Received: from shadphix-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.209.83.35]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2023 03:58:21 -0800 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: x86@kernel.org, dave.hansen@intel.com, kirill.shutemov@linux.intel.com, peterz@infradead.org, tony.luck@intel.com, tglx@linutronix.de, bp@alien8.de, mingo@redhat.com, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, rafael@kernel.org, david@redhat.com, dan.j.williams@intel.com, len.brown@intel.com, ak@linux.intel.com, isaku.yamahata@intel.com, ying.huang@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, nik.borisov@suse.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v15 21/23] x86/virt/tdx: Handle TDX interaction with ACPI S3 and deeper states Date: Fri, 10 Nov 2023 00:55:58 +1300 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Thu, 09 Nov 2023 04:00:03 -0800 (PST) X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782087693138339090 X-GMAIL-MSGID: 1782087693138339090 TDX cannot survive from S3 and deeper states. The hardware resets and disables TDX completely when platform goes to S3 and deeper. Both TDX guests and the TDX module get destroyed permanently. The kernel uses S3 to support suspend-to-ram, and S4 or deeper states to support hibernation. The kernel also maintains TDX states to track whether it has been initialized and its metadata resource, etc. After resuming from S3 or hibernation, these TDX states won't be correct anymore. Theoretically, the kernel can do more complicated things like resetting TDX internal states and TDX module metadata before going to S3 or deeper, and re-initialize TDX module after resuming, etc, but there is no way to save/restore TDX guests for now. Until TDX supports full save and restore of TDX guests, there is no big value to handle TDX module in suspend and hibernation alone. To make things simple, just choose to make TDX mutually exclusive with S3 and hibernation. Note the TDX module is initialized at runtime. To avoid having to deal with the fuss of determining TDX state at runtime, just choose TDX vs S3 and hibernation at kernel early boot. It's a bad user experience if the choice of TDX and S3/hibernation is done at runtime anyway, i.e., the user can experience being able to do S3/hibernation but later becoming unable to due to TDX being enabled. Disable TDX in kernel early boot when hibernation support is available. Currently there's no mechanism exposed by the hibernation code to allow other kernel code to disable hibernation once for all. Disable ACPI S3 when TDX is enabled by the BIOS. For now the user needs to disable TDX in the BIOS to use ACPI S3. A new kernel command line can be added in the future if there's a need to let user disable TDX host via kernel command line. Alternatively, the kernel could disable TDX when ACPI S3 is supported and request the user to disable S3 to use TDX. But there's no existing kernel command line to do that, and BIOS doesn't always have an option to disable S3. Signed-off-by: Kai Huang Reviewed-by: Kirill A. Shutemov --- v14 -> v15: - Simplify the error message when hibernation_available() returns true by removing "Use 'nohibernate' kernel command line part". Instead, explain how to resolve in the Documentation patch. (Rafael) - Simplify the comment around hibernation_available(). (Rafael) - Also guide acpi_suspend_lowlevel with CONFIG_SUSPEND. v13 -> v14: - New patch --- arch/x86/virt/vmx/tdx/tdx.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 53a87034ad59..cc21a0f25bee 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -26,6 +26,8 @@ #include #include #include +#include +#include #include #include #include @@ -1402,6 +1404,15 @@ static int __init tdx_init(void) return -ENODEV; } + /* + * At this point, hibernation_available() indicates whether or + * not hibernation support has been permanently disabled. + */ + if (hibernation_available()) { + pr_err("initialization failed: Hibernation support is enabled\n"); + return -ENODEV; + } + err = register_memory_notifier(&tdx_memory_nb); if (err) { pr_err("initialization failed: register_memory_notifier() failed (%d)\n", @@ -1417,6 +1428,11 @@ static int __init tdx_init(void) return -ENODEV; } +#if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND) + pr_info("Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.\n"); + acpi_suspend_lowlevel = NULL; +#endif + /* * Just use the first TDX KeyID as the 'global KeyID' and * leave the rest for TDX guests.