From patchwork Wed Jan 24 12:55:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 191620 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1062530dyi; Wed, 24 Jan 2024 07:26:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IGmegpXkigJqVlDQsShCTxkP31BbblO05XcyQq6ngXmR7NavtUBMfbqGfavNwjEW8YZW8jD X-Received: by 2002:a17:906:4089:b0:a2e:d5c7:425c with SMTP id u9-20020a170906408900b00a2ed5c7425cmr920866ejj.30.1706109968469; Wed, 24 Jan 2024 07:26:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706109968; cv=pass; d=google.com; s=arc-20160816; b=ZTdV2/V8O4g6+WDpY3E5pxnGQfws0t4y2yLnyVHyiA25X+0jcJXucC3IFCb2ph5sla 9Q8ze0tj2b3IG+JbiHe4HuSUHZgx3xFFd4YpPX8vD6EJqk4G7jJcUMkXLYlBnXwxH062 3bqVOuf6AZ8I3qk5wBdASothdFtepMMBNJKz5Ew2VMGIlgmOakyZWMGgJQ9PEbkVQbeY bN9Kljbni8lnjgvGKiw1Jxpsul+POAZP0Cboq1EEqTGEldb9SrNqaeo+6F6X+J9g9y19 hQwcj4cGH3dBIZksGos6SRVOgMC4i2mcxfQI1uev9p47j/SXAyOXH42pPPjG9xeNdjhJ F31w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=lb5bqckJmnO/jWp23RWiUNEZb+E44+q4pNL7PsWVEwM=; fh=OvJRnOqsMTm9XoNmEwebcqh9Ud7yh1CTeKAP84ols98=; b=XxerJobga/qiIEJE1NicQg7ETA+4POtNHpO8gibR0E1EDOO3wyhafB9aex/oKGJ+Oh ZnJi0rrFhlMa2ei5pu7H9hbstbfVczsc6bf4gdrueqoYFIwkrd/1A2DZ1CfNXmDQSuF9 3NPDLOoYEvZDKAW5p/pCkjFLq/jZumDpK7LKveipwUBYN0ZM4K5RVDV/tkTUFa9fEcsB AZZn8LJh/+kzy/mcSWT+upk7Of6EvmjwwiIdvkfESy2Okc5bx30LV96SZrPT/rQl3F5n SJiB//BQ2ODBUoQR5LcLe2gDVfwLPc3QoHZF6pwK/L38GXVVD2byPAAlizlNpjTZLYLv ACiA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QfFO3UYo; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-37027-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37027-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id c9-20020a17090603c900b00a30cbd3e03esi1194eja.120.2024.01.24.07.26.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 07:26:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37027-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QfFO3UYo; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-37027-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37027-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D3DF71F28308 for ; Wed, 24 Jan 2024 12:58:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1D6117C081; Wed, 24 Jan 2024 12:56:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QfFO3UYo" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DD477A70A for ; Wed, 24 Jan 2024 12:56:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=134.134.136.31 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101014; cv=none; b=Ud6KIvEXQZMy8HGcEwu2rnLz1PcZWkk25DaDgtP0X2q4r/nD95H7Evgwog434MFpajCLWuTpesEOOYynsYWM4GPZv9ysZKKwcV5hC4LbVWqqL320/FJaoxOuvEJkNWejisTcVRaJKC1XfZbxHxHhrhb2ivPYAEM+b1aqZ2Jhy9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101014; c=relaxed/simple; bh=R0O6sMf8J5nkax+miymCAK7F6PZpI1weCFL0hJKn81Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Mb+TYQNO3Yf4sf3Kb08dP6Z2umJoa8e6mpfE5K71ugxGkdANiD7T6qVeSpBuYOMSLanDJISevIRfHG4kvjc7oxUNF8HF3bZmiWKdDtdS4gFNLJSsInRwMHgtMBZGpuWtOawlw3DPBUcTb46Nj3KwxktktgMKIph+kB1xMJu2b60= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QfFO3UYo; arc=none smtp.client-ip=134.134.136.31 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706101013; x=1737637013; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R0O6sMf8J5nkax+miymCAK7F6PZpI1weCFL0hJKn81Q=; b=QfFO3UYoayln5w+eTghgxtGNB/fWxZ8Zi0/O1YWF9bvdmF3ALG0KYbZq 9jT7snsnNYc4qw+DAiXEecrzWIfrNY+tj6Tx5EJPTYygv95CYYQKfcSGj sP27lNsMg11SqWfvKYCZ/Wzpkc6ud4ZRk7Ynen9MxidLVTmi2tjNxNXj8 zrrPJeJsCo2uOqxDF2gTkS1nDHWKV/10nwxNlaqW2qB1vrtGKTygPHNZF kvvrIDklZrqAFlL4vaI+axaVyaLFYAgkBFctLAyTz5cXAOYif05Pw0aw+ v9vXHzte50WKxu24TRvBgNqcrUXjiBxM4AvM8DfZ6rCcyN8Aa0xfYskZr w==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="466110104" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="466110104" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2024 04:56:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="735924071" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="735924071" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga003.jf.intel.com with ESMTP; 24 Jan 2024 04:56:47 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id 8B190832; Wed, 24 Jan 2024 14:56:02 +0200 (EET) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: "Rafael J. Wysocki" , Peter Zijlstra , Adrian Hunter , Kuppuswamy Sathyanarayanan , Elena Reshetova , Jun Nakajima , Rick Edgecombe , Tom Lendacky , "Kalra, Ashish" , Sean Christopherson , "Huang, Kai" , Baoquan He , kexec@lists.infradead.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv6 09/16] x86/mm: Adding callbacks to prepare encrypted memory for kexec Date: Wed, 24 Jan 2024 14:55:50 +0200 Message-ID: <20240124125557.493675-10-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124125557.493675-1-kirill.shutemov@linux.intel.com> References: <20240124125557.493675-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788985966628342875 X-GMAIL-MSGID: 1788985966628342875 AMD SEV and Intel TDX guests allocate shared buffers for performing I/O. This is done by allocating pages normally from the buddy allocator and then converting them to shared using set_memory_decrypted(). On kexec, the second kernel is unaware of which memory has been converted in this manner. It only sees E820_TYPE_RAM. Accessing shared memory as private is fatal. Therefore, the memory state must be reset to its original state before starting the new kernel with kexec. The process of converting shared memory back to private occurs in two steps: - enc_kexec_stop_conversion() stops new conversions. - enc_kexec_unshare_mem() unshares all existing shared memory, reverting it back to private. Signed-off-by: Kirill A. Shutemov Reviewed-by: Nikolay Borisov --- arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/crash.c | 5 +++++ arch/x86/kernel/reboot.c | 12 ++++++++++++ arch/x86/kernel/x86_init.c | 4 ++++ 4 files changed, 23 insertions(+) diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index c9503fe2d13a..3196ff20a29e 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -154,6 +154,8 @@ struct x86_guest { int (*enc_status_change_finish)(unsigned long vaddr, int npages, bool enc); bool (*enc_tlb_flush_required)(bool enc); bool (*enc_cache_flush_required)(void); + void (*enc_kexec_stop_conversion)(bool crash); + void (*enc_kexec_unshare_mem)(void); }; /** diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index b6b044356f1b..6585a5f2c2ba 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -107,6 +107,11 @@ void native_machine_crash_shutdown(struct pt_regs *regs) crash_smp_send_stop(); + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { + x86_platform.guest.enc_kexec_stop_conversion(true); + x86_platform.guest.enc_kexec_unshare_mem(); + } + cpu_emergency_disable_virtualization(); /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 830425e6d38e..0574d4ad6b41 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -716,6 +717,14 @@ static void native_machine_emergency_restart(void) void native_machine_shutdown(void) { + /* + * Call enc_kexec_stop_conversion() while all CPUs are still active and + * interrupts are enabled. This will allow all in-flight memory + * conversions to finish cleanly. + */ + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_stop_conversion(false); + /* Stop the cpus and apics */ #ifdef CONFIG_X86_IO_APIC /* @@ -752,6 +761,9 @@ void native_machine_shutdown(void) #ifdef CONFIG_X86_64 x86_platform.iommu_shutdown(); #endif + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_unshare_mem(); } static void __machine_emergency_restart(int emergency) diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index f0f54e109eb9..b95206ebc621 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -135,6 +135,8 @@ static int enc_status_change_prepare_noop(unsigned long vaddr, int npages, bool static int enc_status_change_finish_noop(unsigned long vaddr, int npages, bool enc) { return 0; } static bool enc_tlb_flush_required_noop(bool enc) { return false; } static bool enc_cache_flush_required_noop(void) { return false; } +static void enc_kexec_stop_conversion_noop(bool crash) {} +static void enc_kexec_unshare_mem_noop(void) {} static bool is_private_mmio_noop(u64 addr) {return false; } struct x86_platform_ops x86_platform __ro_after_init = { @@ -158,6 +160,8 @@ struct x86_platform_ops x86_platform __ro_after_init = { .enc_status_change_finish = enc_status_change_finish_noop, .enc_tlb_flush_required = enc_tlb_flush_required_noop, .enc_cache_flush_required = enc_cache_flush_required_noop, + .enc_kexec_stop_conversion = enc_kexec_stop_conversion_noop, + .enc_kexec_unshare_mem = enc_kexec_unshare_mem_noop, }, };