From patchwork Tue Feb 27 18:16:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 207443 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2973013dyb; Tue, 27 Feb 2024 13:23:28 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXYP+v6aCu9MFiBQNdyo6Ik9VX3dt5q0BvKdAOB59ge4DFaAJJYYhLx+3K7m1Iaxrfo5MTSbVL/WfteNPG3+Z5gt1xR8A== X-Google-Smtp-Source: AGHT+IF1XDPdEzJV7/pLLcX8PlZ7jzAuYPcdXQG1yDP7nGX+xzTFU21YNwl3mZDOyjliHoY+UAmf X-Received: by 2002:aa7:d510:0:b0:566:414d:d70d with SMTP id y16-20020aa7d510000000b00566414dd70dmr1833868edq.17.1709069008519; Tue, 27 Feb 2024 13:23:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709069008; cv=pass; d=google.com; s=arc-20160816; b=bwpkh4Ove7buFWJPiicDrcUv5cRoFW21vFZtXytpMV1ZusQkhBjlt1Lfo5zgJCJhTf htZa0UwgMoThLe3qjIIQPmRaOJGth6zrAAsd8eWMQAphvjPmY0g+swFKtahdWk5SgsBp 5pkdhCpJtYjXoQTMAjWmFFe+g+6cJWlkyBbVD4EHJjwMIdQA08SjOLREltxGqm/qbBuQ FAbXfflAH9GKO4HD22+2m2Nlgk7ihSDvMB0al0lqLtUgDOhALdh5+Nj/tGSIqXvyfHZL ubeHNFGGFS6q4dIpBjyFx+jy982wGUAJ09VeoB89vDXhMDBZZ6yi8UeZG6ji+vAdQ25V E3Zg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=WD2JGEweFTEcyVG1Zgdz/cxp/owXWYDMRqs2tjc+ULE=; fh=/Ak5C2ws7sfmtivYsSuDzWsPTaU4mDhR0lpxQ6qr3D0=; b=t+BT/rUBaOGoTtjpjQd0crzrtqkyxsTJdb9kMi2ju9j/y5au3kM6IKgZzoVXANcI/K 9+typCzR2D5aZlXfgA/8iF/xDdRGLdyXszMIA0MLBdxJWrAjcO7wLGwBN7wzy94q9N+8 iz54HEtpvHSMDuLozk8/N60wmJk1CFX+HFGQyiC1jyKbsOtAgGCrrc7MFZhZpUF2amWU ioPuXnDfPnLOJHMyFFc8U08G7sRnmbsQd+nTf4+uKX4nLK1RLq7FhVIx5Yb5D2wgQuqN zuttqgrelOGQk4n51zMBA68d6HBFN37Tl2MxZ887YcMdS5tmqbogaXLubljQwLTyad3l GunA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KYZTuWPE; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-83821-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83821-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id f24-20020a056402161800b005661c8bfc04si979932edv.94.2024.02.27.13.23.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 13:23:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-83821-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KYZTuWPE; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-83821-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83821-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3BD781F2BFAA for ; Tue, 27 Feb 2024 18:20:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D178514E2C8; Tue, 27 Feb 2024 18:17:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KYZTuWPE" Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C55B14EFE5 for ; Tue, 27 Feb 2024 18:17:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709057827; cv=none; b=iCtgi/tKufqhwt8EYc3gfXziQGEcrXCh+1Bh3bmpoY7dPGjwkZk/pEn/QkzfnlM+ElDRx6Dg/G+Wk337KIr46OA6m5iFn/Ci2dpT3rPJNaGrYZUAX5GSPPSKzfKwakqSOxccFxVsKGbbNPQESMkeDxmwLtaZXGNDnlhQ0jW7LrM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709057827; c=relaxed/simple; bh=284d6MdNJnzxJrEwLF3i2F8vgHGQ2onD2qQDE8fELP4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MimrH4HeXcquKoyI3Sc2Fp3V/sh0HbqUlcSWIXFL+VZnoUki3phuC8i7oR20U8Hc/sMoght6NlG/7arg9Km4vZj4xyt+N/3bsNRFF1Gxppayj8deTTU9Abnbj62q1oCXMmH4Bo0Gdza8zL5UauosOX1iMZy7DswnCyaxFAGZqU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KYZTuWPE; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709057825; x=1740593825; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=284d6MdNJnzxJrEwLF3i2F8vgHGQ2onD2qQDE8fELP4=; b=KYZTuWPEjos9SPn43G7nSmV/3MtzmTo1I3m9VE04FpKeWC+0MbWs5bOk OIxhIh54SoZscwgkUG3wv3SYxb9W88pj4Tt4Vp/cLNrmtQwZwskXU2kq+ 0E4zlfIymp8LK8JoB8LOoNBWuu0E9wIduOdwMhFEHBEJqMwVZNEwqoV73 r0baZJ+x9jlZs1NnfhaUU6o9mBI3R+lZoKkv/6X0662iEBmwpAhQ0YJgW FlLcxzY4SJ8imU1apfF7dlHkja2KxIf7IWoJ4B1UnxuPcTxfoCQl+hH/N NHcRQOqL9f91aGvTo5I5f1bm/F6k0lF6KdCxfFt0gsBald50GmnCxZSzK g==; X-IronPort-AV: E=McAfee;i="6600,9927,10996"; a="20966191" X-IronPort-AV: E=Sophos;i="6.06,188,1705392000"; d="scan'208";a="20966191" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2024 10:16:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10996"; a="937032743" X-IronPort-AV: E=Sophos;i="6.06,188,1705392000"; d="scan'208";a="937032743" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga001.fm.intel.com with ESMTP; 27 Feb 2024 10:16:26 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id B16ED9AB; Tue, 27 Feb 2024 20:16:19 +0200 (EET) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: "Rafael J. Wysocki" , Peter Zijlstra , Adrian Hunter , Kuppuswamy Sathyanarayanan , Elena Reshetova , Jun Nakajima , Rick Edgecombe , Tom Lendacky , "Kalra, Ashish" , Sean Christopherson , "Huang, Kai" , Baoquan He , kexec@lists.infradead.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , Nikolay Borisov Subject: [PATCHv8 09/16] x86/mm: Adding callbacks to prepare encrypted memory for kexec Date: Tue, 27 Feb 2024 20:16:09 +0200 Message-ID: <20240227181616.3032032-10-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240227181616.3032032-1-kirill.shutemov@linux.intel.com> References: <20240227181616.3032032-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792088744860377859 X-GMAIL-MSGID: 1792088744860377859 AMD SEV and Intel TDX guests allocate shared buffers for performing I/O. This is done by allocating pages normally from the buddy allocator and then converting them to shared using set_memory_decrypted(). On kexec, the second kernel is unaware of which memory has been converted in this manner. It only sees E820_TYPE_RAM. Accessing shared memory as private is fatal. Therefore, the memory state must be reset to its original state before starting the new kernel with kexec. The process of converting shared memory back to private occurs in two steps: - enc_kexec_stop_conversion() stops new conversions. - enc_kexec_unshare_mem() unshares all existing shared memory, reverting it back to private. Signed-off-by: Kirill A. Shutemov Reviewed-by: Nikolay Borisov x --- arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/crash.c | 6 ++++++ arch/x86/kernel/reboot.c | 12 ++++++++++++ arch/x86/kernel/x86_init.c | 4 ++++ 4 files changed, 24 insertions(+) diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index c9503fe2d13a..3196ff20a29e 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -154,6 +154,8 @@ struct x86_guest { int (*enc_status_change_finish)(unsigned long vaddr, int npages, bool enc); bool (*enc_tlb_flush_required)(bool enc); bool (*enc_cache_flush_required)(void); + void (*enc_kexec_stop_conversion)(bool crash); + void (*enc_kexec_unshare_mem)(void); }; /** diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index b6b044356f1b..3001f4857ed7 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -124,6 +124,12 @@ void native_machine_crash_shutdown(struct pt_regs *regs) #ifdef CONFIG_HPET_TIMER hpet_disable(); #endif + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { + x86_platform.guest.enc_kexec_stop_conversion(true); + x86_platform.guest.enc_kexec_unshare_mem(); + } + crash_save_cpu(regs, safe_smp_processor_id()); } diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 830425e6d38e..0574d4ad6b41 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -716,6 +717,14 @@ static void native_machine_emergency_restart(void) void native_machine_shutdown(void) { + /* + * Call enc_kexec_stop_conversion() while all CPUs are still active and + * interrupts are enabled. This will allow all in-flight memory + * conversions to finish cleanly. + */ + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_stop_conversion(false); + /* Stop the cpus and apics */ #ifdef CONFIG_X86_IO_APIC /* @@ -752,6 +761,9 @@ void native_machine_shutdown(void) #ifdef CONFIG_X86_64 x86_platform.iommu_shutdown(); #endif + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) && kexec_in_progress) + x86_platform.guest.enc_kexec_unshare_mem(); } static void __machine_emergency_restart(int emergency) diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index f0f54e109eb9..b95206ebc621 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -135,6 +135,8 @@ static int enc_status_change_prepare_noop(unsigned long vaddr, int npages, bool static int enc_status_change_finish_noop(unsigned long vaddr, int npages, bool enc) { return 0; } static bool enc_tlb_flush_required_noop(bool enc) { return false; } static bool enc_cache_flush_required_noop(void) { return false; } +static void enc_kexec_stop_conversion_noop(bool crash) {} +static void enc_kexec_unshare_mem_noop(void) {} static bool is_private_mmio_noop(u64 addr) {return false; } struct x86_platform_ops x86_platform __ro_after_init = { @@ -158,6 +160,8 @@ struct x86_platform_ops x86_platform __ro_after_init = { .enc_status_change_finish = enc_status_change_finish_noop, .enc_tlb_flush_required = enc_tlb_flush_required_noop, .enc_cache_flush_required = enc_cache_flush_required_noop, + .enc_kexec_stop_conversion = enc_kexec_stop_conversion_noop, + .enc_kexec_unshare_mem = enc_kexec_unshare_mem_noop, }, };