From patchwork Mon Nov 21 00:26:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai Huang X-Patchwork-Id: 23482 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1322594wrr; Sun, 20 Nov 2022 16:29:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf7uOK9P/uKSPWb0MDhBsfuXxzh29i1jbH5h09cUYxbhGK25cOrUMs4Ph4xeoBUdpf+Xo7O5 X-Received: by 2002:a05:6a00:1624:b0:573:993b:ea6d with SMTP id e4-20020a056a00162400b00573993bea6dmr3474831pfc.10.1668990541979; Sun, 20 Nov 2022 16:29:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668990541; cv=none; d=google.com; s=arc-20160816; b=Os5QM2X9eyI9OUta71dx9WGeXnsbr7q3khqNJL8JbMojZEGZ+gjk+80xbOR5xnR71H gFeAuId58yHw+goqRWN/ZEBYY+VZ3B+mVT0GtFCbWGmyVPP32oq8k//OVC6KTNBAPBXI 1Rxt5ShlL3zNkyofKY4XQSrgOv+xJEM9S5hmTInq22l1m+J1RtM5P+Bt6Yq3dzFWf95v aih0Q39W5i8QsXpEKsv6arvpdyx+bZeEr7NjBueBUmi6B9PTB0ebrVwSBG9G0chYeUzk HtucPA2IBlGzE0hFJZFx/yt3EKjxzs5mZ9n2550wt5FfRGTqI0AqeJ2n/iCq4M2C7Do5 EhOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8XoJQcUICegriGuR63P2iubKECjIBDjWK5SijeeW2NE=; b=DqmEPjof27sAOHY0BvprfDN5rT5O5JoI39RdZC8YoCP2OueUopXZXfczxZb5Ma8Hlv adzT6xiY2RpHEmxsmRbohGMfcPV/1LtU5e9p7gYCcIGLURo1lYrwO2O8TDEkW5DoQxNM C5xg82t8uTqqXds3EoHfMZK2Gb8Ick07k72mGZTzMOgJzlgJHF0hLzq338b5sXTLAgTv Ehy+lcqLvLofr79hxBU11KCD2ds4D6zvOzPJCiy1Naf+imdgMqC9ILgbZErdkiKvkPCZ yhFIkIGXKuZIAWmb+gjucCdfrwhlFtm2vjjPUuEm4eejRew8bNZhyJTKGOCCqyg4u7R6 fVOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zrudpnn8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x23-20020a634a17000000b00476e938c09asi9873714pga.280.2022.11.20.16.28.49; Sun, 20 Nov 2022 16:29:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zrudpnn8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229853AbiKUA2M (ORCPT + 99 others); Sun, 20 Nov 2022 19:28:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229762AbiKUA1V (ORCPT ); Sun, 20 Nov 2022 19:27:21 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E7352DAA1; Sun, 20 Nov 2022 16:27:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668990441; x=1700526441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hwCT3RUBvAvk7hoIlEmR+DbUGKtOmJbWdTENnqoyMek=; b=Zrudpnn80cZTcD+rAl188EsY3CbVD3b7z66ufsiUGtmrtXbnvKGc84r+ DzwmxPQ4PQd0Q0TJcger+qUuh3g1p46hG8eDjOg3/GLIljC4dxtqkqz3w HP+qAA4rOxg073fWCjcAjRbjqsmtXe/Gf0ERTf5Pz2Ak7A/woeDGXod3J kb3qBTTuNMPtaaw38lClHcmWpwpo9UMIsxkWuaJkVGc+hJFSi172Ww7Zn Colvi7qW+Zha+cjTUKwagPelAFBxbl9gAVKj6oTU8WNAXQqEokIp2q+3y uV6EEMb/uKvbQQV7/fUL5DpnbZzpV2XGNGSzIotjn/e2dCjzZw7yA9OrC g==; X-IronPort-AV: E=McAfee;i="6500,9779,10537"; a="399732300" X-IronPort-AV: E=Sophos;i="5.96,180,1665471600"; d="scan'208";a="399732300" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2022 16:27:21 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10537"; a="729825266" X-IronPort-AV: E=Sophos;i="5.96,180,1665471600"; d="scan'208";a="729825266" Received: from tomnavar-mobl.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.209.176.15]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2022 16:27:16 -0800 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, seanjc@google.com, pbonzini@redhat.com, dave.hansen@intel.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, kirill.shutemov@linux.intel.com, ying.huang@intel.com, reinette.chatre@intel.com, len.brown@intel.com, tony.luck@intel.com, peterz@infradead.org, ak@linux.intel.com, isaku.yamahata@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v7 06/20] x86/virt/tdx: Shut down TDX module in case of error Date: Mon, 21 Nov 2022 13:26:28 +1300 Message-Id: <48505089b645019a734d85c2c29f3c8ae2dbd6bd.1668988357.git.kai.huang@intel.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750063426863176062?= X-GMAIL-MSGID: =?utf-8?q?1750063426863176062?= TDX supports shutting down the TDX module at any time during its lifetime. After the module is shut down, no further TDX module SEAMCALL leaf functions can be made to the module on any logical cpu. Shut down the TDX module in case of any error during the initialization process. It's pointless to leave the TDX module in some middle state. Shutting down the TDX module requires calling TDH.SYS.LP.SHUTDOWN on all BIOS-enabled CPUs, and the SEMACALL can run concurrently on different CPUs. Implement a mechanism to run SEAMCALL concurrently on all online CPUs and use it to shut down the module. Later logical-cpu scope module initialization will use it too. Reviewed-by: Isaku Yamahata Signed-off-by: Kai Huang --- v6 -> v7: - No change. v5 -> v6: - Removed the seamcall() wrapper to previous patch (Dave). - v3 -> v5 (no feedback on v4): - Added a wrapper of __seamcall() to print error code if SEAMCALL fails. - Made the seamcall_on_each_cpu() void. - Removed 'seamcall_ret' and 'tdx_module_out' from 'struct seamcall_ctx', as they must be local variable. - Added the comments to tdx_init() and one paragraph to changelog to explain the caller should handle VMXON. - Called out after shut down, no "TDX module" SEAMCALL can be made. --- arch/x86/virt/vmx/tdx/tdx.c | 43 +++++++++++++++++++++++++++++++++---- arch/x86/virt/vmx/tdx/tdx.h | 5 +++++ 2 files changed, 44 insertions(+), 4 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index b06c1a2bc9cb..5db1a05cb4bd 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include #include #include @@ -124,15 +126,27 @@ bool platform_tdx_enabled(void) return !!tdx_keyid_num; } +/* + * Data structure to make SEAMCALL on multiple CPUs concurrently. + * @err is set to -EFAULT when SEAMCALL fails on any cpu. + */ +struct seamcall_ctx { + u64 fn; + u64 rcx; + u64 rdx; + u64 r8; + u64 r9; + atomic_t err; +}; + /* * Wrapper of __seamcall() to convert SEAMCALL leaf function error code * to kernel error code. @seamcall_ret and @out contain the SEAMCALL * leaf function return code and the additional output respectively if * not NULL. */ -static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, - u64 *seamcall_ret, - struct tdx_module_output *out) +static int seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, + u64 *seamcall_ret, struct tdx_module_output *out) { u64 sret; @@ -166,6 +180,25 @@ static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, } } +static void seamcall_smp_call_function(void *data) +{ + struct seamcall_ctx *sc = data; + int ret; + + ret = seamcall(sc->fn, sc->rcx, sc->rdx, sc->r8, sc->r9, NULL, NULL); + if (ret) + atomic_set(&sc->err, -EFAULT); +} + +/* + * Call the SEAMCALL on all online CPUs concurrently. Caller to check + * @sc->err to determine whether any SEAMCALL failed on any cpu. + */ +static void seamcall_on_each_cpu(struct seamcall_ctx *sc) +{ + on_each_cpu(seamcall_smp_call_function, sc, true); +} + /* * Detect and initialize the TDX module. * @@ -181,7 +214,9 @@ static int init_tdx_module(void) static void shutdown_tdx_module(void) { - /* TODO: Shut down the TDX module */ + struct seamcall_ctx sc = { .fn = TDH_SYS_LP_SHUTDOWN }; + + seamcall_on_each_cpu(&sc); } static int __tdx_enable(void) diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index 92a8de957dc7..215cc1065d78 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -12,6 +12,11 @@ /* MSR to report KeyID partitioning between MKTME and TDX */ #define MSR_IA32_MKTME_KEYID_PARTITIONING 0x00000087 +/* + * TDX module SEAMCALL leaf functions + */ +#define TDH_SYS_LP_SHUTDOWN 44 + /* * Do not put any hardware-defined TDX structure representations below * this comment!