From patchwork Fri May 19 16:30:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 96495 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1367598vqo; Fri, 19 May 2023 09:39:05 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5bh+qoz/sI1iwK8Kwn2cySpvOpunfwBxAPWb1YREzwi30n36OPWgq9Gxd4osbO+XKNp9D/ X-Received: by 2002:a17:90b:3016:b0:253:37a9:178 with SMTP id hg22-20020a17090b301600b0025337a90178mr2624735pjb.45.1684514345386; Fri, 19 May 2023 09:39:05 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1684514345; cv=pass; d=google.com; s=arc-20160816; b=1D67lxDOnLU7pd350umAWPdWwu3//aEH1cSozpgybAq0PfXP/+kHBbCI0X/5F5FRIj DRnIv+ie67bgJpAb75pfFHjrd8zVI0yomDVtgdxYYhsmyukvPwgiYJxB05LEBLievpzA GaLs1TsmTJeIM64fgwrbjM+qFKlAyEgVWyVY3E8ES/1hlowtqrVjWT98txyEWXaSHhRg 6RPG4smTX/B+AGbaeKgYstFh3fAefCRonoDxh1U/WsQbS5M2QBhJC5qeRvZ5W7OT/DCv b+iF2OUy+52pc/6t+bP/QVfvemfl0IriQba82AOhDGlpA+QYFM4XS2sKzbiOY8iA1UBI b//w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MRl98XliT95INHRWHlCFvP+XcTziK7Bl8ILB8nXUGFo=; b=p3yZrjh6UjNtig5L/MTyY5wt8VeU/Dfue9cbx9aKqGg1FxY9zcal4fw/8617QywLgu LHTM748haqr/TdMFkVYnckQAQPNgIEjS7u7S89rt/WeDwEyKDE8LFhGwPbesJeNupESC qUbXquftJ7jKFgMz64pggL+qg0MvJhf58RfL8AclAmGgOjKFdgtAocL7z+3CEozRrGuJ Ly15RDdMrzae3dEc2kOPx3oEaY5d4rfdur5vr7mSJPHPNekzxZGKglSQ4zNYrOdD/abF 6M+d+JriZjKUTktkFrTgoz8NrIF02oDs9WO51+TLoYAcNpXMM8GIRwOGXuD2FkOX67zc hnYg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=vvMulTe3; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g64-20020a636b43000000b005321c4f2f54si3738710pgc.844.2023.05.19.09.38.50; Fri, 19 May 2023 09:39:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=vvMulTe3; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230526AbjESQbq (ORCPT + 99 others); Fri, 19 May 2023 12:31:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230518AbjESQbo (ORCPT ); Fri, 19 May 2023 12:31:44 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2079.outbound.protection.outlook.com [40.107.223.79]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 988991718 for ; Fri, 19 May 2023 09:31:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HSNRQk34ppTuZ9+Ezbkm+KvNYc5CcpcX2yANahyThzz2oRnxT1Lu+XvYNXa02QYnd+7v5/iC34flklluutzMDGjCgsiJHX5aaoPtyg69fJ14GBhiyqaO7E5ZlA0euZ/lhr+J6nZC55l5LjEQHQ39olcPOno/gUTGGDUDBIX5Y3VBWr5a86gGa5H2qXF1aY7IbmkfIa9+gW/Ea6Hgqlwz2pIxzNSGSPlVmH++5i+6YByyK6kp2gRD0ftW+w9ECabh/PektIDXzmfJ8I4BEYp2IR1sfJuB2Vr/uI+h2BN7DtN4UimP6thTs9WL53Yn3JBNoAyaWZVjPDJJj1mphugeWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MRl98XliT95INHRWHlCFvP+XcTziK7Bl8ILB8nXUGFo=; b=BN5JeWqXnn2lGhobUI6inLNETU1ObwztKzT0O3m3mJEATJbEOjoIEjn9QxH5V5mo+1cY4jQkNMfha5lBbXLMCINevkj1SKBpKg9ES4XUwKBHEP7pbX50NEKTAjbBcURda37ZZmHVmGk6ECMhOXqZ0tH5CwU4Pjj8CXcmxG5BaxnhtOLVm7/PeUoqyo9gKB41pn+H86C8CzYMQcKMt8IM3sXoTQAQ+1j7hRwPl9YJHuvuLf7CSiUtgL0ARbM9Qv+hcBG64PbMXIYsKdAo2X6zWSGHzjkrmYtJvK5//fWDkBoJWtVm8RptE0Igqe/J0a9XwBDeCvIJm7SyfYqHcD16xA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MRl98XliT95INHRWHlCFvP+XcTziK7Bl8ILB8nXUGFo=; b=vvMulTe39pDz0yXXP+2/NIgHO3LKzytwlXjNP4/DPw+5Gz5E7IlRnTRF//85gIc/G7XkFDgyKzxeKD6PhqvmQGNuRENXLoqBr+tiG/aeAkVZ+lKsJwydp4efJdxqQ6/oTy/SCGxXn6bdfFNXlU/KMbHg5955foxwineXZUV+xhw= Received: from SJ0PR05CA0210.namprd05.prod.outlook.com (2603:10b6:a03:330::35) by CY5PR12MB6060.namprd12.prod.outlook.com (2603:10b6:930:2b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.19; Fri, 19 May 2023 16:31:13 +0000 Received: from DM6NAM11FT087.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:330:cafe::7e) by SJ0PR05CA0210.outlook.office365.com (2603:10b6:a03:330::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.7 via Frontend Transport; Fri, 19 May 2023 16:31:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT087.mail.protection.outlook.com (10.13.172.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.21 via Frontend Transport; Fri, 19 May 2023 16:31:12 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 19 May 2023 11:31:09 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [RESEND PATCH v8 5/6] x86/sev: Add SNP-specific unaccepted memory support Date: Fri, 19 May 2023 11:30:12 -0500 Message-ID: <262831c7c519fedb1bdb3b51bf3019d49e68ff9e.1684513813.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: References: <20230518231434.26080-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT087:EE_|CY5PR12MB6060:EE_ X-MS-Office365-Filtering-Correlation-Id: 2dba92e1-7264-4b72-ee6b-08db588675d4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xXBEQR5N8UW98zj3T4QfTh2t8DOQd6WgtgMYmj0EXvM4Pr2Cu2EWp+ctJRLlBlg0guC2YEbBQTFMdZvRkxtGcxPXerpNpFqZTOzwl24gaweZSaTiYofrqQJByNINU0SaE2puzPAMYJnEq1v+6B+O8nX6CVBfD+291rCnFA4WMUAXpGthW0XRFEhqijM9B4Z1pgJYhXNodORdbszi3HYnESeaBjbZAUKr1RaBitUZTZXJIA9L96bWfND3cfav7q38CYiilAoF3Qli4PMiIvzsbrKmf3JxwaQxsqxUWZk6XCp9Ibp3zw1dVXugKNhmlr5TEvKFsrCbk1DWziJN8qeLgGn9+z3t85NEkwOMMbFyByGVNCs+HhkiTjCCbx4yqFwMpj2zLggJNirNvmy+B1HyBZtcAQLGk0yeOEZhS6z/t90ak+TUlxMW+yfPGeyK3STl+TLSDuNS87mmia5x0s1DMGQFPgOeg27r8ldY0fDzb73q0g2zlsASSOxE61eBaInaa2a67DX48pzR0QH9EgiLFg1ZjfiuRV9S5XzxSYVNx6XqMaVBFJ+60tZQq3InmpfyM3c9PzJmmYM8TokyaNMHVzFkgTWBewciGZQQFvPOse7zRl9Y7k8YyxyslysvcgxeH6PTQG8mM5a2dXn6b141DaSmcqtNq/C+WMGxXPhk7TANCfPQUw7fJwutXQr+mPVcMANWL+phjluxC4b2Obxz1Dg/vaw3R8f8IC2UHVxVJfeDFT4yCe3GZ05fuwMTFEFQfK2jCvV1LpCQZLk1PrBgcQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(396003)(346002)(376002)(136003)(451199021)(36840700001)(40470700004)(46966006)(26005)(40460700003)(36756003)(40480700001)(47076005)(36860700001)(336012)(426003)(86362001)(83380400001)(82310400005)(2616005)(82740400003)(356005)(16526019)(186003)(81166007)(110136005)(54906003)(7416002)(478600001)(30864003)(2906002)(316002)(8936002)(4326008)(8676002)(41300700001)(5660300002)(70586007)(70206006)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2023 16:31:12.7714 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2dba92e1-7264-4b72-ee6b-08db588675d4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT087.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6060 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_FILL_THIS_FORM_SHORT,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766341313797317093?= X-GMAIL-MSGID: =?utf-8?q?1766341313797317093?= Add SNP-specific hooks to the unaccepted memory support in the boot path (__accept_memory()) and the core kernel (accept_memory()) in order to support booting SNP guests when unaccepted memory is present. Without this support, SNP guests will fail to boot and/or panic() when unaccepted memory is present in the EFI memory map. The process of accepting memory under SNP involves invoking the hypervisor to perform a page state change for the page to private memory and then issuing a PVALIDATE instruction to accept the page. Since the boot path and the core kernel paths perform similar operations, move the pvalidate_pages() and vmgexit_psc() functions into sev-shared.c to avoid code duplication. Create the new header file arch/x86/boot/compressed/sev.h because adding the function declaration to any of the existing SEV related header files pulls in too many other header files, causing the build to fail. Signed-off-by: Tom Lendacky --- arch/x86/Kconfig | 2 + arch/x86/boot/compressed/mem.c | 3 + arch/x86/boot/compressed/sev.c | 54 ++++++++++- arch/x86/boot/compressed/sev.h | 23 +++++ arch/x86/include/asm/sev.h | 3 + arch/x86/include/asm/unaccepted_memory.h | 3 + arch/x86/kernel/sev-shared.c | 103 +++++++++++++++++++++ arch/x86/kernel/sev.c | 112 +++-------------------- 8 files changed, 204 insertions(+), 99 deletions(-) create mode 100644 arch/x86/boot/compressed/sev.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5c72067c06d4..b9c451f75d5e 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1543,11 +1543,13 @@ config X86_MEM_ENCRYPT config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD + depends on EFI_STUB select DMA_COHERENT_POOL select ARCH_USE_MEMREMAP_PROT select INSTRUCTION_DECODER select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT + select UNACCEPTED_MEMORY help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory diff --git a/arch/x86/boot/compressed/mem.c b/arch/x86/boot/compressed/mem.c index 8df3d988ae69..c8f2353f6894 100644 --- a/arch/x86/boot/compressed/mem.c +++ b/arch/x86/boot/compressed/mem.c @@ -3,6 +3,7 @@ #include "error.h" #include "misc.h" #include "tdx.h" +#include "sev.h" #include /* @@ -36,6 +37,8 @@ void arch_accept_memory(phys_addr_t start, phys_addr_t end) /* Platform-specific memory-acceptance call goes here */ if (early_is_tdx_guest()) tdx_accept_memory(start, end); + else if (sev_snp_enabled()) + snp_accept_memory(start, end); else error("Cannot accept memory: unknown platform\n"); } diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 014b89c89088..09dc8c187b3c 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -115,7 +115,7 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, /* Include code for early handlers */ #include "../../kernel/sev-shared.c" -static inline bool sev_snp_enabled(void) +bool sev_snp_enabled(void) { return sev_status & MSR_AMD64_SEV_SNP_ENABLED; } @@ -181,6 +181,58 @@ static bool early_setup_ghcb(void) return true; } +static phys_addr_t __snp_accept_memory(struct snp_psc_desc *desc, + phys_addr_t pa, phys_addr_t pa_end) +{ + struct psc_hdr *hdr; + struct psc_entry *e; + unsigned int i; + + hdr = &desc->hdr; + memset(hdr, 0, sizeof(*hdr)); + + e = desc->entries; + + i = 0; + while (pa < pa_end && i < VMGEXIT_PSC_MAX_ENTRY) { + hdr->end_entry = i; + + e->gfn = pa >> PAGE_SHIFT; + e->operation = SNP_PAGE_STATE_PRIVATE; + if (IS_ALIGNED(pa, PMD_SIZE) && (pa_end - pa) >= PMD_SIZE) { + e->pagesize = RMP_PG_SIZE_2M; + pa += PMD_SIZE; + } else { + e->pagesize = RMP_PG_SIZE_4K; + pa += PAGE_SIZE; + } + + e++; + i++; + } + + if (vmgexit_psc(boot_ghcb, desc)) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pvalidate_pages(desc); + + return pa; +} + +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + struct snp_psc_desc desc = {}; + unsigned int i; + phys_addr_t pa; + + if (!boot_ghcb && !early_setup_ghcb()) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pa = start; + while (pa < end) + pa = __snp_accept_memory(&desc, pa, end); +} + void sev_es_shutdown_ghcb(void) { if (!boot_ghcb) diff --git a/arch/x86/boot/compressed/sev.h b/arch/x86/boot/compressed/sev.h new file mode 100644 index 000000000000..fc725a981b09 --- /dev/null +++ b/arch/x86/boot/compressed/sev.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * AMD SEV header for early boot related functions. + * + * Author: Tom Lendacky + */ + +#ifndef BOOT_COMPRESSED_SEV_H +#define BOOT_COMPRESSED_SEV_H + +#ifdef CONFIG_AMD_MEM_ENCRYPT + +bool sev_snp_enabled(void); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); + +#else + +static inline bool sev_snp_enabled(void) { return false; } +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } + +#endif + +#endif diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index e21e1c5397c1..86e1296e87f5 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -206,6 +206,7 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -229,6 +230,8 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } + +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } #endif #endif diff --git a/arch/x86/include/asm/unaccepted_memory.h b/arch/x86/include/asm/unaccepted_memory.h index 72b354f992bb..ed3fcd3ac9dd 100644 --- a/arch/x86/include/asm/unaccepted_memory.h +++ b/arch/x86/include/asm/unaccepted_memory.h @@ -3,12 +3,15 @@ #include #include +#include static inline void arch_accept_memory(phys_addr_t start, phys_addr_t end) { /* Platform-specific memory-acceptance call goes here */ if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) { tdx_accept_memory(start, end); + } else if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + snp_accept_memory(start, end); } else { panic("Cannot accept memory: unknown platform\n"); } diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index 3a5b0c9c4fcc..be312db48a49 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -12,6 +12,9 @@ #ifndef __BOOT_COMPRESSED #define error(v) pr_err(v) #define has_cpuflag(f) boot_cpu_has(f) +#else +#undef WARN +#define WARN(condition, format...) (!!(condition)) #endif /* I/O parameters for CPUID-related helpers */ @@ -991,3 +994,103 @@ static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info) cpuid_ext_range_max = fn->eax; } } + +static void pvalidate_pages(struct snp_psc_desc *desc) +{ + struct psc_entry *e; + unsigned long vaddr; + unsigned int size; + unsigned int i; + bool validate; + int rc; + + for (i = 0; i <= desc->hdr.end_entry; i++) { + e = &desc->entries[i]; + + vaddr = (unsigned long)pfn_to_kaddr(e->gfn); + size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; + validate = (e->operation == SNP_PAGE_STATE_PRIVATE) ? true : false; + + rc = pvalidate(vaddr, size, validate); + if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { + unsigned long vaddr_end = vaddr + PMD_SIZE; + + for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { + rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + if (rc) + break; + } + } + + if (rc) { + WARN(1, "Failed to validate address 0x%lx ret %d", vaddr, rc); + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); + } + } +} + +static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) +{ + int cur_entry, end_entry, ret = 0; + struct snp_psc_desc *data; + struct es_em_ctxt ctxt; + + vc_ghcb_invalidate(ghcb); + + /* Copy the input desc into GHCB shared buffer */ + data = (struct snp_psc_desc *)ghcb->shared_buffer; + memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); + + /* + * As per the GHCB specification, the hypervisor can resume the guest + * before processing all the entries. Check whether all the entries + * are processed. If not, then keep retrying. Note, the hypervisor + * will update the data memory directly to indicate the status, so + * reference the data->hdr everywhere. + * + * The strategy here is to wait for the hypervisor to change the page + * state in the RMP table before guest accesses the memory pages. If the + * page state change was not successful, then later memory access will + * result in a crash. + */ + cur_entry = data->hdr.cur_entry; + end_entry = data->hdr.end_entry; + + while (data->hdr.cur_entry <= data->hdr.end_entry) { + ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); + + /* This will advance the shared buffer data points to. */ + ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_PSC, 0, 0); + + /* + * Page State Change VMGEXIT can pass error code through + * exit_info_2. + */ + if (WARN(ret || ghcb->save.sw_exit_info_2, + "SNP: PSC failed ret=%d exit_info_2=%llx\n", + ret, ghcb->save.sw_exit_info_2)) { + ret = 1; + goto out; + } + + /* Verify that reserved bit is not set */ + if (WARN(data->hdr.reserved, "Reserved bit is set in the PSC header\n")) { + ret = 1; + goto out; + } + + /* + * Sanity check that entry processing is not going backwards. + * This will happen only if hypervisor is tricking us. + */ + if (WARN(data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry, +"SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", + end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry)) { + ret = 1; + goto out; + } + } + +out: + return ret; +} diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 8802a75e1c20..ea2546e5130f 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -657,38 +657,6 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void pvalidate_pages(struct snp_psc_desc *desc) -{ - struct psc_entry *e; - unsigned long vaddr; - unsigned int size; - unsigned int i; - bool validate; - int rc; - - for (i = 0; i <= desc->hdr.end_entry; i++) { - e = &desc->entries[i]; - - vaddr = (unsigned long)pfn_to_kaddr(e->gfn); - size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; - validate = (e->operation == SNP_PAGE_STATE_PRIVATE) ? true : false; - - rc = pvalidate(vaddr, size, validate); - if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { - unsigned long vaddr_end = vaddr + PMD_SIZE; - - for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { - rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); - if (rc) - break; - } - } - - if (WARN(rc, "Failed to validate address 0x%lx ret %d", vaddr, rc)) - sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); - } -} - static void early_set_pages_state(unsigned long vaddr, unsigned long paddr, unsigned long npages, enum psc_op op) { @@ -796,72 +764,6 @@ void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op WARN(1, "invalid memory op %d\n", op); } -static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) -{ - int cur_entry, end_entry, ret = 0; - struct snp_psc_desc *data; - struct es_em_ctxt ctxt; - - vc_ghcb_invalidate(ghcb); - - /* Copy the input desc into GHCB shared buffer */ - data = (struct snp_psc_desc *)ghcb->shared_buffer; - memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); - - /* - * As per the GHCB specification, the hypervisor can resume the guest - * before processing all the entries. Check whether all the entries - * are processed. If not, then keep retrying. Note, the hypervisor - * will update the data memory directly to indicate the status, so - * reference the data->hdr everywhere. - * - * The strategy here is to wait for the hypervisor to change the page - * state in the RMP table before guest accesses the memory pages. If the - * page state change was not successful, then later memory access will - * result in a crash. - */ - cur_entry = data->hdr.cur_entry; - end_entry = data->hdr.end_entry; - - while (data->hdr.cur_entry <= data->hdr.end_entry) { - ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); - - /* This will advance the shared buffer data points to. */ - ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_PSC, 0, 0); - - /* - * Page State Change VMGEXIT can pass error code through - * exit_info_2. - */ - if (WARN(ret || ghcb->save.sw_exit_info_2, - "SNP: PSC failed ret=%d exit_info_2=%llx\n", - ret, ghcb->save.sw_exit_info_2)) { - ret = 1; - goto out; - } - - /* Verify that reserved bit is not set */ - if (WARN(data->hdr.reserved, "Reserved bit is set in the PSC header\n")) { - ret = 1; - goto out; - } - - /* - * Sanity check that entry processing is not going backwards. - * This will happen only if hypervisor is tricking us. - */ - if (WARN(data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry, -"SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", - end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry)) { - ret = 1; - goto out; - } - } - -out: - return ret; -} - static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, unsigned long vaddr_end, int op) { @@ -966,6 +868,20 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages) set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); } +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long vaddr; + unsigned int npages; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + vaddr = (unsigned long)__va(start); + npages = (end - start) >> PAGE_SHIFT; + + set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); +} + static int snp_set_vmsa(void *va, bool vmsa) { u64 attrs;