From patchwork Tue Jun 6 14:51:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103908 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3462851vqr; Tue, 6 Jun 2023 08:06:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4WxIjUjLlmbvyQcDIbWn7n/VwIaOqb8CTO2jiK2CketIDw5rRRQZ/K43vJHA7rDpENWihq X-Received: by 2002:a05:6214:628:b0:625:aa1a:b6d9 with SMTP id a8-20020a056214062800b00625aa1ab6d9mr2187117qvx.59.1686064014523; Tue, 06 Jun 2023 08:06:54 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686064014; cv=pass; d=google.com; s=arc-20160816; b=J5TY0iZBMs+saS507zk66svKq8X8WZn+BgnfL5QUVkzcXuWoQ+udDh0+RcPWA72nk5 gtz45KaNT9+4ovl/hOZlMpXBNZs/2QY0UTIFUVvjBbs9m16AhBrDDQdXrVqNwB98DaR8 xYQeHE+faokQzqUyZwCt7RDBaal7f154KPe4L17JP/UClD4k1MXKKVBXFw7tmMRbDHGU z7YaPlZ3XckuTQjYjVKUSSX3CdsiKHnZIVwC8ax1Bm0eDTqfq2VbU6MJc8v2dFL7OK3f yXn8CA+E47Jv3Qf41jja5waESdJxkYharcEuDpMgefu30ROH7+z0WzmTcR/QnPNY9w7L wKdw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=T3BWNRDBjL5IO3E4gbxcdh48HaLOHpW9frVn6bWGU9Q=; b=DTilb2DZxOcDEElmHmweQKUSPSM4pUe7cHAPzd7z4bclp3I/9E8+Hue6QzlixchC16 KjZRMQEsPi+k9pweHcpflXKPmRYXdHQU+yY6R2U9ibA58yN5CMx8gnVNDL0lbrpuxCKG UA4BgqtBPQqDiat+N8HfWef3IQ7x0VM7iTF4zh5SRS39fv8Os3eymITCY5OWivuAnlxy yIH1RJXtTEQa0yJUNTob/jcTCdW4qIiHcQV9vXmnvq8ZxRbfmLLlwTw19wtuyvh+C7p0 YxGB3Ucdm7ILNSq+Z3Cu4y7yiluVMEKciz7XTSgXBVR73L7R6Ln/QUt0n0+xLNIoUjWV dVnQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=bUz29Hgw; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jz8-20020a0562140e6800b006239d6bb9a2si6267728qvb.512.2023.06.06.08.06.39; Tue, 06 Jun 2023 08:06:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=bUz29Hgw; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238413AbjFFOwL (ORCPT + 99 others); Tue, 6 Jun 2023 10:52:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238419AbjFFOwB (ORCPT ); Tue, 6 Jun 2023 10:52:01 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7C131719 for ; Tue, 6 Jun 2023 07:51:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ihiab1lLw4hII/ZJdKmIeVa9XgS5/mKBx8znIgC0QY0xbbgoLQgz8+AzuC0LOF6CUekE0SjY7hA5onkEkoXqUCo6hNDCpa0V+0X7wo2i3cALvqFRtmRLNNjxdx2jL+5hS9pO3hDyVh/az1VvlRMFnAWzjUkLXzEbsWkVqwFOmAudYwJrft+LC2/jIVWR4egrTEOcOV9Jyqm172VGd7juYJvqi2HNCabKXVuWzUpVDtkWPO7ueOVwqE/QujW9ziStt+YA9TggT9eaYSXMDAfGznucShVqY4oTOGYmPStFFw0CE9szNDa6/+GMz7sOaNiTBSnrKYlLyFcRyvqX3ak3Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T3BWNRDBjL5IO3E4gbxcdh48HaLOHpW9frVn6bWGU9Q=; b=TNUvHldIGCRtvjbs6+sCL/EGVjujXMOjTT6PPCd5NpX36Jv3c1unnztvXEn1YQrUqEwox6JVjdeaQSOtJPyjahRItdqjX/Oh6c08wqQxc3IHCubzi3VM2nREYGZK9EhSYVspU3q66XHkx/GtSMFgUbJXBsn/Jp/Vn8ezo6su8ZWttEWRzTZsNSSjnRagscRb52cPJT96pbvXVIdeLANAW21VnvEmvMNv2nXAPG/FVs8MTpK9OtS1yFCp6fwCT1N7tb+TYnr7uwyLgfEuM5aUAdhHB4hGFUXGuQiavGYKjAud0oluXaJ2dOth8KHcAz9Ubqhdzkab70jKNtFBHvTlEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T3BWNRDBjL5IO3E4gbxcdh48HaLOHpW9frVn6bWGU9Q=; b=bUz29HgwXfQvWmcaet+hlmNZniHTIaJ0xLuRm4pBmFM9x7Y1KfX/KO6fYJp8UroFWtPsI9hyXQI0h2ZQh3Rd7BNO2c7MbwARFI3BAjQeIfnMBWre9ad253Z4X/AO6JWoXgV6skWaZVUV+GD+F8vXUN7x0JaLT9PApn6ucwZmZm8= Received: from SJ0PR03CA0175.namprd03.prod.outlook.com (2603:10b6:a03:338::30) by MN2PR12MB4456.namprd12.prod.outlook.com (2603:10b6:208:266::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Tue, 6 Jun 2023 14:51:49 +0000 Received: from CO1NAM11FT027.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:338:cafe::72) by SJ0PR03CA0175.outlook.office365.com (2603:10b6:a03:338::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:51:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT027.mail.protection.outlook.com (10.13.174.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.36 via Frontend Transport; Tue, 6 Jun 2023 14:51:49 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:51:47 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v9 1/6] x86/sev: Fix calculation of end address based on number of pages Date: Tue, 6 Jun 2023 09:51:22 -0500 Message-ID: <6a6e4eea0e1414402bac747744984fa4e9c01bb6.1686063086.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT027:EE_|MN2PR12MB4456:EE_ X-MS-Office365-Filtering-Correlation-Id: 06356530-4f1b-4b5b-966d-08db669d8ed1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sIk5ngNJGGn10KaQ+1nM/MChgT0khBXhJq+9YZ4KpByjIbsF4mblajx0mU1wYckIXh6HnsT2aBg7qanRVsZ6rc+Zrtk00xWaMN4p1M6YZRFzXxYAxyKQQFgRHAlhEMdiK5GO1aGGEJolTYb+uVlJMuwYVfvYM9lKIU7itvWqN32q/OARUpoR1l72xw4a+jXXQeep3gz+zaema1UTqHcDoy/jBYR3/QSKTpqxXOoMoa3jqBMp6tTb10LedUj8MdRVlm0Ze3ZnvMD9as4uQPT+2OOc455O5gOFCsE/zJpUNBdKMenfkegzrktUSxxSZ7WjOvhzJcQNWvPsmXvh/RqQ/RbC8qHeaQ5H1jQVuUJloSKtNXx6BmpYX3mf2TpLUTzHZ4WaQyGohQlsGqdz4r4GtjVwe05DJpeeb4BNqpmDqBdw7G3ftd+M/0UfSuyu+EYRN1nUX2iEyGCInXc7EONQAhL6ygNlF7wWfy7IGIuyYRrbzTHq9qIrhvBal4G+p9yAMiGeK6u6LiWNf/oqIffb+P1co8cDgnOdjbbuoG4lZRlRLn8Sh6XiizliLySi9pLhisTi5OmXV6snKeJNuZWhpea/layrXRW4pCDTWQw052iMf58gD6EcQtGeAuQA2GPPbH/zrt+NKjAI0tVZnznipitymBmFgIugXpabRV+PjaEs8uRo9R1i1LRZ4Z1hXWoQ3C1kR5BHG8bWf4YJbxQ18SlGqu0aluinwRJa44eqNnua1YiBmnBHwj9l2ROagIK7iW/E2Eh1ZdSrb2tQEaICLQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(396003)(376002)(136003)(451199021)(36840700001)(46966006)(40470700004)(40480700001)(186003)(54906003)(70586007)(110136005)(81166007)(478600001)(82740400003)(356005)(8676002)(8936002)(4326008)(41300700001)(316002)(70206006)(2616005)(426003)(16526019)(336012)(36860700001)(83380400001)(47076005)(40460700003)(6666004)(26005)(86362001)(5660300002)(7416002)(2906002)(82310400005)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:51:49.3649 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06356530-4f1b-4b5b-966d-08db669d8ed1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT027.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4456 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767966259917945101?= X-GMAIL-MSGID: =?utf-8?q?1767966259917945101?= When calculating an end address based on an unsigned int number of pages, any value greater than or equal to 0x100000 that is shift PAGE_SHIFT bits results in a 0 value, resulting in an invalid end address. Change the number of pages variable in various routines from an unsigned int to an unsigned long to calculate the end address correctly. Fixes: 5e5ccff60a29 ("x86/sev: Add helper for validating pages in early enc attribute changes") Fixes: dc3f3d2474b8 ("x86/mm: Validate memory when changing the C-bit") Signed-off-by: Tom Lendacky --- arch/x86/include/asm/sev.h | 16 ++++++++-------- arch/x86/kernel/sev.c | 14 +++++++------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 13dc2a9d23c1..7ca5c9ec8b52 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -192,12 +192,12 @@ struct snp_guest_request_ioctl; void setup_ghcb(void); void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, - unsigned int npages); + unsigned long npages); void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, - unsigned int npages); + unsigned long npages); void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op); -void snp_set_memory_shared(unsigned long vaddr, unsigned int npages); -void snp_set_memory_private(unsigned long vaddr, unsigned int npages); +void snp_set_memory_shared(unsigned long vaddr, unsigned long npages); +void snp_set_memory_private(unsigned long vaddr, unsigned long npages); void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); @@ -212,12 +212,12 @@ static inline int pvalidate(unsigned long vaddr, bool rmp_psize, bool validate) static inline int rmpadjust(unsigned long vaddr, bool rmp_psize, unsigned long attrs) { return 0; } static inline void setup_ghcb(void) { } static inline void __init -early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, unsigned int npages) { } +early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, unsigned long npages) { } static inline void __init -early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, unsigned int npages) { } +early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, unsigned long npages) { } static inline void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op) { } -static inline void snp_set_memory_shared(unsigned long vaddr, unsigned int npages) { } -static inline void snp_set_memory_private(unsigned long vaddr, unsigned int npages) { } +static inline void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) { } +static inline void snp_set_memory_private(unsigned long vaddr, unsigned long npages) { } static inline void snp_set_wakeup_secondary_cpu(void) { } static inline bool snp_init(struct boot_params *bp) { return false; } static inline void snp_abort(void) { } diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index b031244d6d2d..108bbae59c35 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -645,7 +645,7 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void pvalidate_pages(unsigned long vaddr, unsigned int npages, bool validate) +static void pvalidate_pages(unsigned long vaddr, unsigned long npages, bool validate) { unsigned long vaddr_end; int rc; @@ -662,7 +662,7 @@ static void pvalidate_pages(unsigned long vaddr, unsigned int npages, bool valid } } -static void __init early_set_pages_state(unsigned long paddr, unsigned int npages, enum psc_op op) +static void __init early_set_pages_state(unsigned long paddr, unsigned long npages, enum psc_op op) { unsigned long paddr_end; u64 val; @@ -701,7 +701,7 @@ static void __init early_set_pages_state(unsigned long paddr, unsigned int npage } void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, - unsigned int npages) + unsigned long npages) { /* * This can be invoked in early boot while running identity mapped, so @@ -723,7 +723,7 @@ void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long padd } void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, - unsigned int npages) + unsigned long npages) { /* * This can be invoked in early boot while running identity mapped, so @@ -879,7 +879,7 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); } -static void set_pages_state(unsigned long vaddr, unsigned int npages, int op) +static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) { unsigned long vaddr_end, next_vaddr; struct snp_psc_desc *desc; @@ -904,7 +904,7 @@ static void set_pages_state(unsigned long vaddr, unsigned int npages, int op) kfree(desc); } -void snp_set_memory_shared(unsigned long vaddr, unsigned int npages) +void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) { if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return; @@ -914,7 +914,7 @@ void snp_set_memory_shared(unsigned long vaddr, unsigned int npages) set_pages_state(vaddr, npages, SNP_PAGE_STATE_SHARED); } -void snp_set_memory_private(unsigned long vaddr, unsigned int npages) +void snp_set_memory_private(unsigned long vaddr, unsigned long npages) { if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return; From patchwork Tue Jun 6 14:51:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103931 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3480159vqr; Tue, 6 Jun 2023 08:30:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5Fg1hu8fZJgUbdnEQYugqZZlyIvsFW9ST8/23tDjUuVviriaCH3IojohMB/hm6+3HILibC X-Received: by 2002:a05:6a00:23c4:b0:65e:d5da:b3c6 with SMTP id g4-20020a056a0023c400b0065ed5dab3c6mr2470351pfc.7.1686065452545; Tue, 06 Jun 2023 08:30:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686065452; cv=pass; d=google.com; s=arc-20160816; b=RfPbCp/Uncg9N3CG2n+cnVLlI6UQHKkAEw1kCJLu3lvzyjfUnQTXKiWnhgHpBMj/NM rW10xJydG2rxSWut1AMniaivMymRGm1LFCFZ5bER+a5GGmbVnMQBTh2dlsJu6bFwUi5p fwdXkQaCV97uFwFBlpno9oJkaiegSp33DA5deH9YOooIlA9zPBpX4yZQ/HrvIF99uJNd +81db2p9r06p8XsQkUE2h6dWy6EaJhlgw5nDUZ6loKsdr1AA7+oKdgA/KNwtBYc88j/f Mw8DOecAdWYskYaunL7e74VNsHUzDAynYy6ftfKq4ffef5fiMJoSLqp2lYJzABDQ+E2y iV3A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=si0Y0hViSqZXgGsXUM4bYNOJ+CCiV+oRKTAmQV4GmTU=; b=beVshkJXPR2VFXMK/tmx2gMoy1nwND09mP7meJ9fKjhO65avxKSKmxI9szav/Xv2ZA iELGXXWMtHV/CuW+wO9Ddwtssq6zxkZpVaRiXaow33PKLwumXMLbdNsNrSdNnHJHFA0u OSqFKsQE8u69jMKWAsqqZqzsIRO/IU5TcZJjODcW37XNynpR61rKQTmgPrRtWtoVjwip kJrWdYb6NFuwWty+6jft7FwSSUL34Eaqi3KY22bORtcSVUyqtSSJQwr+2Tz7M1fCgQw6 wgRuOZZuCFDJdSlTzZ3c6W2lajo4cwR2VFqOHtCwRmCXhHN6b+TulT1awiaYGa8nbifP 3diw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ehUJZovd; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q14-20020aa7960e000000b006617e66da0csi46444pfg.15.2023.06.06.08.30.37; Tue, 06 Jun 2023 08:30:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ehUJZovd; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238396AbjFFOwY (ORCPT + 99 others); Tue, 6 Jun 2023 10:52:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237676AbjFFOwI (ORCPT ); Tue, 6 Jun 2023 10:52:08 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2051.outbound.protection.outlook.com [40.107.244.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E26A710F4 for ; Tue, 6 Jun 2023 07:52:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C2N9lHAS3pkRjIW38Hcbd1rDqC1abhxrdqAO7ufmucn1306/gGa79AlFA6MYxpjB+CJrslWknBMIdyvkMgGRcQai673YT0s3ggRRO84Ko2vuArjHz5h/d15AQVBHAJ8Ll2MyEEMsWIdxpzRmp3TzaPBgdCUYNhU5nYYxQ4EukxKNuqW1Hxv/u9K6zav9nwJl14aXyUqmZcN9tuIVAC6XTS49U1k2dlb2Oi54H2R5Hg83+J3Bp3d5me0p7JEqj/UgU7EQjANEMl+OmMoawg9mj+IVeKFli3UhaO96b7m1cnwJQ+grHSHYiwuZfU5lR9Y+Dh+o9pkbtRkMcjXEwnC7eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=si0Y0hViSqZXgGsXUM4bYNOJ+CCiV+oRKTAmQV4GmTU=; b=gu8gaqjNi4Qp4rnvF/pdKy2lvuWqTnrRatYyZH6Ro0rE34qB5mCBw89/TmE1tjBN+P2bjE7c063OU7bHPkrEV3st2toqXUDCSwjHhCtVe9Zmu1iihoODFHrjYA+txXVMTaBWHflYADTUtUghb2Td7iFp3RW5Bz5868ddrvIs6GFdIJPbe+YL+vohPWF6cX3gUx+yEGDDOnYydjhvPf7CM2nfnGnZ5TEYAorFn41829Jgv6lMx1GGiGybFpv7qAPx2QiFnegfQJ8D7DDS7pSXFt3sUPJunZMgdsc8qRkUD6Szk4apUaiDSTJKJ8cbW766D++LaatWvOIl+mgTQ/fGrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=si0Y0hViSqZXgGsXUM4bYNOJ+CCiV+oRKTAmQV4GmTU=; b=ehUJZovdPXqLyz9qYNZXjMUSwEY5/Kltsdd3yl0SicHh2DvRCL9x/FEEOGOOWhhZSNX4y/psewCZMsye25KCbaLnDiJCEA4Dh8AMhyWrInPOskgh3xtbMHPixAiJhUq+M+lvO0fEwmp0fypMJhw4FwiMbKNMIwUi/goU7fjglAc= Received: from MW4PR04CA0386.namprd04.prod.outlook.com (2603:10b6:303:81::31) by MW6PR12MB9020.namprd12.prod.outlook.com (2603:10b6:303:240::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Tue, 6 Jun 2023 14:51:59 +0000 Received: from CO1NAM11FT006.eop-nam11.prod.protection.outlook.com (2603:10b6:303:81:cafe::14) by MW4PR04CA0386.outlook.office365.com (2603:10b6:303:81::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Tue, 6 Jun 2023 14:51:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT006.mail.protection.outlook.com (10.13.174.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:51:58 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:51:56 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v9 2/6] x86/sev: Put PSC struct on the stack in prep for unaccepted memory support Date: Tue, 6 Jun 2023 09:51:23 -0500 Message-ID: <938865ea7f5ad88d4daa846b97c9414723b5639a.1686063086.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT006:EE_|MW6PR12MB9020:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e9fa361-3a9f-4413-5025-08db669d9458 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sq7LYwBA86WDWGPFP6F3vTNA9msSGUsWcnhQHltXWguctqvTR1jZD71gsJellpBwwRr3Kokjzl4YJ9iPhgnPKBu+ZDCcDkhqAIT4245c8brJugcDm1DaZ+/xd0jUGH9TaWvJVdsZ8q7EM6pGeyTkKsbqZm4JjWKjDp72GzdaYtajSGo2VirVLRhlpxAY7Uygsdp1s7kS/QttskIfl3+6VHZs/E9m0ZNTwY+zQL8hm4kwOqtUu4Dqkv6voenRMcn051hIiuOJYGQc+7vZ7YX9l4nD4TEaI+PSMODXw6leQVn8jTiLXcqKwkR8kCNeDqP3nD0qIQg/IgH07im8wUyJie6KVI+S/IeLlz18WW/3fO+DTcQC3RkClcquL46eV/JB7ezb6xI/0+MloTPLCPukgFWVzNlSEGFHDHDXDLKgVpvrfXGsOWxZ/MhlBHnAQ5PAvBwDE9rH0VPaGqQgFYhQRdgC8rByMG8ebRYbyie7l+zxVh37ej78A0xlMyPjSqkwQfHC21ydhojmzQS+K5m+KJLpIA+JjI5K9arUEHnSl1L/iAPnh4/3MKSAwdyFcrhIXIsvCVL+7iPlOFqk7/pvTnG1js42pyIsNQ8sZ2gnkG/+3bzdZ3x8+IJYHMHlvk5jcU7oP8XDd2cqYy0c+ZbipeYf0mri+40Vn7qhlY8UVhsYYdQ76FJ/G/k7IIhrdYjKRYkGZ3ZDYTtAwntSBgvjqbKUrAUGpx294uZMWF7KDxAN6OoQ5ZsG5j7qe1e1JnX4 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230028)(4636009)(396003)(39860400002)(376002)(136003)(346002)(451199021)(36840700001)(40470700004)(46966006)(40460700003)(83380400001)(186003)(47076005)(426003)(2906002)(336012)(2616005)(36756003)(86362001)(82310400005)(82740400003)(81166007)(40480700001)(356005)(36860700001)(966005)(316002)(41300700001)(6666004)(5660300002)(8936002)(8676002)(110136005)(54906003)(478600001)(4326008)(70206006)(70586007)(26005)(16526019)(7416002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:51:58.6209 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3e9fa361-3a9f-4413-5025-08db669d9458 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT006.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB9020 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767967767754158300?= X-GMAIL-MSGID: =?utf-8?q?1767967767754158300?= In advance of providing support for unaccepted memory, switch from using kmalloc() for allocating the Page State Change (PSC) structure to using a local variable that lives on the stack. This is needed to avoid a possible recursive call into set_pages_state() if the kmalloc() call requires (more) memory to be accepted, which would result in a hang. The current size of the PSC struct is 2,032 bytes. To make the struct more stack friendly, reduce the number of PSC entries from 253 down to 64, resulting in a size of 520 bytes. This is a nice compromise on struct size and total PSC requests while still allowing parallel PSC operations across vCPUs. If the reduction in PSC entries results in any kind of performance issue (that is not seen at the moment), use of a larger static PSC struct, with fallback to the smaller stack version, can be investigated. For more background info on this decision, see the subthread in the Link: tag below. Signed-off-by: Tom Lendacky Link: https://lore.kernel.org/lkml/658c455c40e8950cb046dd885dd19dc1c52d060a.1659103274.git.thomas.lendacky@amd.com --- arch/x86/include/asm/sev-common.h | 9 +++++++-- arch/x86/kernel/sev.c | 10 ++-------- 2 files changed, 9 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h index 0759af9b1acf..b463fcbd4b90 100644 --- a/arch/x86/include/asm/sev-common.h +++ b/arch/x86/include/asm/sev-common.h @@ -106,8 +106,13 @@ enum psc_op { #define GHCB_HV_FT_SNP BIT_ULL(0) #define GHCB_HV_FT_SNP_AP_CREATION BIT_ULL(1) -/* SNP Page State Change NAE event */ -#define VMGEXIT_PSC_MAX_ENTRY 253 +/* + * SNP Page State Change NAE event + * The VMGEXIT_PSC_MAX_ENTRY determines the size of the PSC structure, which + * is a local stack variable in set_pages_state(). Do not increase this value + * without evaluating the impact to stack usage. + */ +#define VMGEXIT_PSC_MAX_ENTRY 64 struct psc_hdr { u16 cur_entry; diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 108bbae59c35..7b0144acd7bf 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -882,11 +882,7 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) { unsigned long vaddr_end, next_vaddr; - struct snp_psc_desc *desc; - - desc = kmalloc(sizeof(*desc), GFP_KERNEL_ACCOUNT); - if (!desc) - panic("SNP: failed to allocate memory for PSC descriptor\n"); + struct snp_psc_desc desc; vaddr = vaddr & PAGE_MASK; vaddr_end = vaddr + (npages << PAGE_SHIFT); @@ -896,12 +892,10 @@ static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) next_vaddr = min_t(unsigned long, vaddr_end, (VMGEXIT_PSC_MAX_ENTRY * PAGE_SIZE) + vaddr); - __set_pages_state(desc, vaddr, next_vaddr, op); + __set_pages_state(&desc, vaddr, next_vaddr, op); vaddr = next_vaddr; } - - kfree(desc); } void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) From patchwork Tue Jun 6 14:51:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103914 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3466407vqr; Tue, 6 Jun 2023 08:11:14 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4ebiDxIa/X2ZGMO459d2TSGSU17z30WmckqWauPrQacMkpxeqJFr2ktSg9LonoCiWpO6rN X-Received: by 2002:a05:622a:1492:b0:3f9:ae23:5763 with SMTP id t18-20020a05622a149200b003f9ae235763mr2902995qtx.61.1686064273928; Tue, 06 Jun 2023 08:11:13 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686064273; cv=pass; d=google.com; s=arc-20160816; b=jr5+Gdena8VbEkCnT10vAZLX7paFUYgCs7Ir8H6aPiJo4kKwAM7+uGrsfqARAaH2mH YETmTrqcoLJuc+8PkiwSWLMG2VtkmT3RVYC+0kIW9fxZDXG3w6qmvwPtQjqoPcsBR6z9 kSuElXwoEcklEgffvmHt9nyIuy5N10VXX7/sIQJ8aqbAU8qaFD++ljq3wWJREqq0FPR6 keSsu/xbwGAf9LWvu/w5YkI58x+Qo0qpwVFwt2MbfBQPDiGakXnt53PQdIidvcXsGZMk +xb9NgrsguYID5ZBdeE/8GEh4z+idsMvWpfrAkBa300Ckq7p1DNJVs9iYSdF1rTNpdeo hVWw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ttXehVqZk6zYQofdpqGfIuBmlp/XNPqRNVhMzzATcOs=; b=P1WOrv1c7zy8YD4zjKPQpYF6Oe0A2+8y7asnCuHgqb+B2Qckn8zXvq9I7XCWa21mN9 WQdZiBQmX4Hshc2iShagHQ1Qe3aVLA47kTBgCGBxFQqs+KbuJjT9VujN1FaWqwRPs1/i JMFv/3zXOZArAbFiSbc5gVd7V+pILXnhKpNBEA33wXBJIU0p8Hgcoc8SaFICg0HtYOnd WOKSSnUS9pMacY/DItvIWv0hQVEGtfmwcrmWQ6uFrIWSESXirw/1E/HES7ECXFTdxe1N yK6jiuXsfNMY+35gzLcB7rYJObhbcIEIDg1zAHtW9cSYaFe2DPPGJ8MM2pOQobqW9eID jfXA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ns+0RXia; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jz8-20020a0562140e6800b006239d6bb9a2si6267728qvb.512.2023.06.06.08.10.58; Tue, 06 Jun 2023 08:11:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ns+0RXia; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237849AbjFFOwq (ORCPT + 99 others); Tue, 6 Jun 2023 10:52:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238476AbjFFOwW (ORCPT ); Tue, 6 Jun 2023 10:52:22 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2055.outbound.protection.outlook.com [40.107.223.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E49001712 for ; Tue, 6 Jun 2023 07:52:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=grJfLwnPcbOE84w+4X/9qFjPR9K1y7VGJIsVQxQKnMQjN05mvGb9DraDr+ekc2+uSRTCZrbWjG5en/gVuGocz/Vt4EFdrZtoXUOe+XuO1cXtvM3QQkzePhfl+3/OXENSwjv9zJbyau/G341HXVKnNIuOUO4HZoWafEYk+JS2sZk9orut7hS8+wJUIr53y6kmbl5cAqMnbmM3UYDsqU5LX/GeRu/Jcu/tYXukoXRA4Lccp2Jn7XhrU/uyHE6jfBsyzNetoRvCKgoafb/Leb5pRVwxUcTyAyL2i/ZDdLZzJraak3c6kiSQ0J0OcY9pCs9S3NYW13ic2bXRcRSFHz1mmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ttXehVqZk6zYQofdpqGfIuBmlp/XNPqRNVhMzzATcOs=; b=PmZEfOhxm+u7wjLt6vIQycZDLiS7OyfVu8zanYE4Q+FFuT8m4+ttbPsc5hqrQy5yE7J36+cponr9wU/SLutp9TI0Ud5chJ1zBwiL1e1/abw/kb8/BLZEIw44PG91xq/cfVRIp+jYA6SZoHRTEg8gDxHvmGnZARpOGLhLZ3DaRagOKYHpAj/qocFWdToxR6z1d7kOjI0cSUO7f1mFCciysOf7FAs7YrdUDisYimLRTOCtblclF47D3PejElAvqEefs8k9imSwIBUPBi3o0pNlDNmwKQnADrYY5o73h0qMFfmqo7dVB2Q4cG1D0LWgA2vmmf9OIDhfKb2Z7tOC1CaOkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ttXehVqZk6zYQofdpqGfIuBmlp/XNPqRNVhMzzATcOs=; b=Ns+0RXiazmJMYchGPIK71eWOQvFx49cVBc5voabHHrpswKLe9YTDmrUmH6zYF0re9PTpUv96Gv/Tj3UmV1Sc5c9OlhOSh/+6LFBBhIVyDl+MjY/Tfm/YoMd+tzZoOeAMhtHa0nHHRCHchUs+sypjP1sPG4KcreUjlZZht8/vpC4= Received: from MW4PR03CA0340.namprd03.prod.outlook.com (2603:10b6:303:dc::15) by DM4PR12MB5312.namprd12.prod.outlook.com (2603:10b6:5:39d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Tue, 6 Jun 2023 14:52:07 +0000 Received: from CO1NAM11FT100.eop-nam11.prod.protection.outlook.com (2603:10b6:303:dc:cafe::cf) by MW4PR03CA0340.outlook.office365.com (2603:10b6:303:dc::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:52:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT100.mail.protection.outlook.com (10.13.175.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.36 via Frontend Transport; Tue, 6 Jun 2023 14:52:07 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:52:05 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v9 3/6] x86/sev: Allow for use of the early boot GHCB for PSC requests Date: Tue, 6 Jun 2023 09:51:24 -0500 Message-ID: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT100:EE_|DM4PR12MB5312:EE_ X-MS-Office365-Filtering-Correlation-Id: bb00c4cf-53b4-4ba4-ea92-08db669d997f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hI8O12Qn5Vc81F+4IkNvdVAdR8Iv6Fcul7x7/nlGq1BXUlxjVrfwf2MNBwg+uxcONbg7nJjuZDgm0J4dBwB0lLewxB3Vf2Hy8/SskK1Gc4iq6W+vzeVlbCJkV5XJH4XHplQFdKWwSoOosVEmRKcP3wjehQ4YONltn4kHYrsNMJiRb7plgtphC0rlpAWrONd7lbfTSWX51mGeep5Lrm5426OkoYgV+yZWvOGYSy+j+IwywAtgGYPUIl6xWjR1l4sKgBpxXVvDjDHzddzYib+Rv2UmrqHXzz4WqwEFyOWCD9ausW6FRu1C1n5GU73l3lmlkA2fcky21OPwRTAlZ7GH0T1HslnFQKNpL7cY6IRrv0XMTLwISzzYE/3kmQhfZOIuZdn48/Lj3kHV7FwMkw1B2AjLfl7FuyIdK0nv3vUeG4aPuhifgPKzq5R+QMF2DlCBZtjEYywSFU4faZiZJr7mzJpsAO/Pg37nH/1JURFdynlosVBNGRXU1/BkcTNcZcZrXZ4+FG/CrpFNUINPAQT0WYLQruCXYJ0sINbEqWr+FUFWQJAi9ZRjesr/Z/m0erBONbFj3g9Z37UUT5B8R8MQlKj/w4QH+AVx1VwdQPVrF+JvJGSkOBYwU48XfoGi9942Xz0I86AVU6ceuJ4ndXdVmQ4fCN85wbOCejmEveEmVrAHZ+78h3q6SB//pZAYwMVhEerHxXljI/nk5qMnjSWd2g3URwB+m6SHGsjhumcL/+x7nR137YYbVpvPQ87epMlwfAsEPzTERR/JEiXWORYTUg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(136003)(376002)(396003)(39860400002)(451199021)(40470700004)(46966006)(36840700001)(54906003)(110136005)(40460700003)(478600001)(40480700001)(8936002)(5660300002)(7416002)(8676002)(36756003)(86362001)(2906002)(70206006)(4326008)(81166007)(82740400003)(70586007)(316002)(82310400005)(356005)(41300700001)(2616005)(26005)(36860700001)(16526019)(47076005)(83380400001)(186003)(6666004)(336012)(426003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:52:07.2846 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb00c4cf-53b4-4ba4-ea92-08db669d997f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT100.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5312 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767966531795425026?= X-GMAIL-MSGID: =?utf-8?q?1767966531795425026?= Using a GHCB for a page stage change (as opposed to the MSR protocol) allows for multiple pages to be processed in a single request. In prep for early PSC requests in support of unaccepted memory, update the invocation of vmgexit_psc() to be able to use the early boot GHCB and not just the per-CPU GHCB structure. In order to use the proper GHCB (early boot vs per-CPU), set a flag that indicates when the per-CPU GHCBs are available and registered. For APs, the per-CPU GHCBs are created before they are started and registered upon startup, so this flag can be used globally for the BSP and APs instead of creating a per-CPU flag. This will allow for a significant reduction in the number of MSR protocol page state change requests when accepting memory. Signed-off-by: Tom Lendacky --- arch/x86/kernel/sev.c | 61 +++++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 23 deletions(-) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 7b0144acd7bf..973756c89dac 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -119,7 +119,19 @@ static DEFINE_PER_CPU(struct sev_es_save_area *, sev_vmsa); struct sev_config { __u64 debug : 1, - __reserved : 63; + + /* + * A flag used by __set_pages_state() that indicates when the + * per-CPU GHCB has been created and registered and thus can be + * used by the BSP instead of the early boot GHCB. + * + * For APs, the per-CPU GHCB is created before they are started + * and registered upon startup, so this flag can be used globally + * for the BSP and APs. + */ + ghcbs_initialized : 1, + + __reserved : 62; }; static struct sev_config sev_cfg __read_mostly; @@ -662,7 +674,7 @@ static void pvalidate_pages(unsigned long vaddr, unsigned long npages, bool vali } } -static void __init early_set_pages_state(unsigned long paddr, unsigned long npages, enum psc_op op) +static void early_set_pages_state(unsigned long paddr, unsigned long npages, enum psc_op op) { unsigned long paddr_end; u64 val; @@ -756,26 +768,13 @@ void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op WARN(1, "invalid memory op %d\n", op); } -static int vmgexit_psc(struct snp_psc_desc *desc) +static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) { int cur_entry, end_entry, ret = 0; struct snp_psc_desc *data; - struct ghcb_state state; struct es_em_ctxt ctxt; - unsigned long flags; - struct ghcb *ghcb; - /* - * __sev_get_ghcb() needs to run with IRQs disabled because it is using - * a per-CPU GHCB. - */ - local_irq_save(flags); - - ghcb = __sev_get_ghcb(&state); - if (!ghcb) { - ret = 1; - goto out_unlock; - } + vc_ghcb_invalidate(ghcb); /* Copy the input desc into GHCB shared buffer */ data = (struct snp_psc_desc *)ghcb->shared_buffer; @@ -832,20 +831,18 @@ static int vmgexit_psc(struct snp_psc_desc *desc) } out: - __sev_put_ghcb(&state); - -out_unlock: - local_irq_restore(flags); - return ret; } static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, unsigned long vaddr_end, int op) { + struct ghcb_state state; struct psc_hdr *hdr; struct psc_entry *e; + unsigned long flags; unsigned long pfn; + struct ghcb *ghcb; int i; hdr = &data->hdr; @@ -875,8 +872,20 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, i++; } - if (vmgexit_psc(data)) + local_irq_save(flags); + + if (sev_cfg.ghcbs_initialized) + ghcb = __sev_get_ghcb(&state); + else + ghcb = boot_ghcb; + + if (!ghcb || vmgexit_psc(ghcb, data)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + if (sev_cfg.ghcbs_initialized) + __sev_put_ghcb(&state); + + local_irq_restore(flags); } static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) @@ -884,6 +893,10 @@ static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) unsigned long vaddr_end, next_vaddr; struct snp_psc_desc desc; + /* Use the MSR protocol when a GHCB is not available. */ + if (!boot_ghcb) + return early_set_pages_state(__pa(vaddr), npages, op); + vaddr = vaddr & PAGE_MASK; vaddr_end = vaddr + (npages << PAGE_SHIFT); @@ -1261,6 +1274,8 @@ void setup_ghcb(void) if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) snp_register_per_cpu_ghcb(); + sev_cfg.ghcbs_initialized = true; + return; } From patchwork Tue Jun 6 14:51:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103900 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3455258vqr; Tue, 6 Jun 2023 07:58:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ75HvrSOrLkFwL24kyYa934xWR16DACLn5qDqCKlCA8psodeFQY/+HHJ92Wo5zBjivl4N6h X-Received: by 2002:ac8:5885:0:b0:3f3:a0c0:cd6f with SMTP id t5-20020ac85885000000b003f3a0c0cd6fmr2820298qta.9.1686063514745; Tue, 06 Jun 2023 07:58:34 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686063514; cv=pass; d=google.com; s=arc-20160816; b=qf1959RXtktXESP6m+dXWe7tmLD2Wzn22dLjkcQzt3xMLx+11y39b53g0t5BwARyVo QmD+uLKYA+TodAJlunrDBzqC8KDNEPwVX7H8bzdD9/xGN7gD38DxEyD4ZaDvvu9TD0jX SC6eYmIa7+nMHrh9X2bhk5fktOnmPFjcCU3UhuoHIwC2hhzNJDXbi95sdGovHR9/M1pn xCVC+84GNumgAX9elbF5Au0YNb0zr51KMOoN7hBg0isHyjgy5FSacl16nqITDNaQcKu9 S0YtncbG2raRvpZp4Eu+e7RUcAPOuxyzkFsnbJi4MPr2GD2vayrLq6TGSscax3eZVV7X UVMg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MWgBmrc/3MsRAqnwKmUfMfGVs279AeRT62omDfjvr4o=; b=paLuVLQdaO0qe/bzy4ATDBC5ar2Bpe2Ot5H/lT0LSfYx9KVGuT+SFHWZKYQaOcR+Gg hfEqnXnmR1SpnbdZoUKIC5+tBxPfrRW8iGM/RVNfa7odj+dtq3reEdhFCGiDL79pSafo o8JsRsqPjdBGLy+1UHgV5COBoRGdk2X5642fIdPCe0suzUWEuAdl7d2We9ehrbNXYSLr 9N9uVPerdNR3obJ0FPic2MQGV7prCIWjl42AA/BMPVBUuV1MOcW7qYl3tHwsPq87zRFl ubdPVywkgQN75Ox8c4d/SR2Zvh9OTpximf6h9YRqR94bqCOATIfPnWE4dJxnyZ8nlAAT F24g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=yw8LrMOm; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g10-20020ac87f4a000000b003f69abc8e60si427944qtk.339.2023.06.06.07.58.20; Tue, 06 Jun 2023 07:58:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=yw8LrMOm; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238449AbjFFOw4 (ORCPT + 99 others); Tue, 6 Jun 2023 10:52:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238492AbjFFOwe (ORCPT ); Tue, 6 Jun 2023 10:52:34 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2046.outbound.protection.outlook.com [40.107.243.46]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CB2A172C for ; Tue, 6 Jun 2023 07:52:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J3qzCKNhIyu8FLgDX3jV5+0ctVSP6Lj8F9ziHC6UXA2/jDDHQud5H8fayXBaqnoNbNYu782l0scS3WvCSsXAcCiSpHtJIUJcQzHIhEwJKhSa6Jd7O/GWwf0Kx6rProxNmpoKS0dbu8hrhmiqJ5BLABhpmVz0v8bHf9cTwzrwdUKNOvH4zh7GuuoM49tQiRTILdkXVuDKiX5h/i9ONdDFHS0ZGdjy3BYaNcWqe+xIwhvna5k7beZwXFPgefpJJcVPBeUtg3HvFjCP2fKy/S4oryP2NUvJ58be7m7PSTEmOeX0v+LxlYDyk92W+xc2LWGGIzj4eQ018sv2EYakpTEnfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MWgBmrc/3MsRAqnwKmUfMfGVs279AeRT62omDfjvr4o=; b=WqwyGwuK/K6hQ/sgPngR8x5dmmKxywEidQWp+vSVQ10piqwKr1O4Xl4TJdnrpoX5cLLf2yDKTTKJqQ3bOiXV/kLGRSPz4xyR+z13KaOjdSyJzsOsKo91ckJyzlKOzckE3RXTGF3sx0W72fzjFWuyhDgzUo4UcBKm4erE6KdSvyIGThI+ZHTU2cTExoXNYtxDORHIfYvE9BAXmlWLlPesnOGf7B05OcGy1qbIdDRu46EwGHwUnW92g22076L0IubA4a17c5knsEs5b9Nm+gkXhG3/ynUE0xs4ktVXH2OoAD51msNo/GRx9Fo6No4/IhIQy60U1OiixSINbZIIjCQFBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MWgBmrc/3MsRAqnwKmUfMfGVs279AeRT62omDfjvr4o=; b=yw8LrMOmPapvk77IUOiTuEKqx3RD1yD85pzrHP4SOSpMbYoKY2J4QDX97FDF6iSGiJOmGxuZYz61epiSDygEjXwv3N3uYt2qrm64wwhLGBdCTy3SypVtu1/ZP/O4eMFghYzvi9w3+rUI7fVc7p0TF6iikbNMGwfWkd+d5mBQDFE= Received: from MW4PR03CA0079.namprd03.prod.outlook.com (2603:10b6:303:b6::24) by SN7PR12MB8772.namprd12.prod.outlook.com (2603:10b6:806:341::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.27; Tue, 6 Jun 2023 14:52:19 +0000 Received: from CO1NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b6:cafe::c7) by MW4PR03CA0079.outlook.office365.com (2603:10b6:303:b6::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:52:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT009.mail.protection.outlook.com (10.13.175.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:52:19 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:52:15 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v9 4/6] x86/sev: Use large PSC requests if applicable Date: Tue, 6 Jun 2023 09:51:25 -0500 Message-ID: <050d17b460dfc237b51d72082e5df4498d3513cb.1686063086.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT009:EE_|SN7PR12MB8772:EE_ X-MS-Office365-Filtering-Correlation-Id: b796b3d9-907a-4afd-f148-08db669da0cb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iJ2YAJNZ4gE+PwNLKI6HDQGtkJdN9uEZN7HvzNgBAwltvaqMB90EYliASTzEBEZve7921r5S6Tmupm8XUM8Wre9nvIvo5V1FvX6pRpUyV50qiOtmLbNcC+/uOEYHFrr5TK90RCqgR9wwNuRcUrkCN+dAZkCiIoZj90yFCdf89jy1Gi42KcBm8OpfK5wO9Mzy0yGgDUc7ll9wDgZLGBsBTwrPfV63lPEczjP8np+NZUxRvdfB/+Z9VFVqmOoO1khAyN9T/UYL2oFFBKZep/JqYhSeHkEWWpqWyzzJqG6qnlJuYnv8Xr7UdBOTS4xlII1aJfxSk83n+1j4fv8nhHo9aQO1t0l79vyBPIAHvXh8XNnATEUsS8a5rlViecmrhSv0cPikOB9rqwoqstqtzFoTgZk3GamPdhKtg/K7HRv3rOMzq0Om2Di0IrpENkeRSy9aoxish75Xt10HogA5lw7faZBP/mmdpwDyST5jRUYhynC3meWZk7qY4ustiyl84RJT3/qWmBcsN2pLIKy4f9QG+//4uoYP8tx3HgQLEN3LCIaCcQziFzZT0Q9dHpKfoYw1uo+rfzIEhpLV3DCVqJcbLzYa13B7XTvzyV1yKyfXDsbefBZwRoHZoQ+LmiGQvBVIxc0nzajyw4Q0PTxDUagiBor2b35KB9vRFQ26yxPg/E0UZmCcPyYJdMR41vRRdv9s7LQcInj1ZG25NPNw22zgWoLB7dOsDIsvbeTfyY336vmqOwWY09MR7tRIxlvZbzrb7RxNwcR4dsMQ6NBWuk7ESw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(376002)(136003)(39860400002)(346002)(451199021)(40470700004)(36840700001)(46966006)(40460700003)(6666004)(478600001)(36860700001)(47076005)(426003)(36756003)(336012)(83380400001)(16526019)(82310400005)(186003)(81166007)(26005)(86362001)(82740400003)(2616005)(356005)(40480700001)(4326008)(41300700001)(316002)(70206006)(70586007)(7416002)(2906002)(8676002)(8936002)(5660300002)(54906003)(110136005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:52:19.5269 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b796b3d9-907a-4afd-f148-08db669da0cb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8772 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_FILL_THIS_FORM_SHORT,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767965736313130019?= X-GMAIL-MSGID: =?utf-8?q?1767965736313130019?= In advance of providing support for unaccepted memory, request 2M Page State Change (PSC) requests when the address range allows for it. By using a 2M page size, more PSC operations can be handled in a single request to the hypervisor. The hypervisor will determine if it can accommodate the larger request by checking the mapping in the nested page table. If mapped as a large page, then the 2M page request can be performed, otherwise the 2M page request will be broken down into 512 4K page requests. This is still more efficient than having the guest perform multiple PSC requests in order to process the 512 4K pages. In conjunction with the 2M PSC requests, attempt to perform the associated PVALIDATE instruction of the page using the 2M page size. If PVALIDATE fails with a size mismatch, then fallback to validating 512 4K pages. To do this, page validation is modified to work with the PSC structure and not just a virtual address range. Signed-off-by: Tom Lendacky --- arch/x86/include/asm/sev.h | 4 ++ arch/x86/kernel/sev.c | 125 ++++++++++++++++++++++++------------- 2 files changed, 84 insertions(+), 45 deletions(-) diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 7ca5c9ec8b52..e21e1c5397c1 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -80,11 +80,15 @@ extern void vc_no_ghcb(void); extern void vc_boot_ghcb(void); extern bool handle_vc_boot_ghcb(struct pt_regs *regs); +/* PVALIDATE return codes */ +#define PVALIDATE_FAIL_SIZEMISMATCH 6 + /* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 /* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1 #define RMPADJUST_VMSA_PAGE_BIT BIT(16) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 973756c89dac..17b3d003b2ea 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -657,32 +657,58 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void pvalidate_pages(unsigned long vaddr, unsigned long npages, bool validate) +static void pvalidate_pages(struct snp_psc_desc *desc) { - unsigned long vaddr_end; + struct psc_entry *e; + unsigned long vaddr; + unsigned int size; + unsigned int i; + bool validate; int rc; - vaddr = vaddr & PAGE_MASK; - vaddr_end = vaddr + (npages << PAGE_SHIFT); + for (i = 0; i <= desc->hdr.end_entry; i++) { + e = &desc->entries[i]; + + vaddr = (unsigned long)pfn_to_kaddr(e->gfn); + size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; + validate = e->operation == SNP_PAGE_STATE_PRIVATE; + + rc = pvalidate(vaddr, size, validate); + if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { + unsigned long vaddr_end = vaddr + PMD_SIZE; + + for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { + rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + if (rc) + break; + } + } - while (vaddr < vaddr_end) { - rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); if (WARN(rc, "Failed to validate address 0x%lx ret %d", vaddr, rc)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); - - vaddr = vaddr + PAGE_SIZE; } } -static void early_set_pages_state(unsigned long paddr, unsigned long npages, enum psc_op op) +static void early_set_pages_state(unsigned long vaddr, unsigned long paddr, + unsigned long npages, enum psc_op op) { unsigned long paddr_end; u64 val; + int ret; + + vaddr = vaddr & PAGE_MASK; paddr = paddr & PAGE_MASK; paddr_end = paddr + (npages << PAGE_SHIFT); while (paddr < paddr_end) { + if (op == SNP_PAGE_STATE_SHARED) { + /* Page validation must be rescinded before changing to shared */ + ret = pvalidate(vaddr, RMP_PG_SIZE_4K, false); + if (WARN(ret, "Failed to validate address 0x%lx ret %d", paddr, ret)) + goto e_term; + } + /* * Use the MSR protocol because this function can be called before * the GHCB is established. @@ -703,7 +729,15 @@ static void early_set_pages_state(unsigned long paddr, unsigned long npages, enu paddr, GHCB_MSR_PSC_RESP_VAL(val))) goto e_term; - paddr = paddr + PAGE_SIZE; + if (op == SNP_PAGE_STATE_PRIVATE) { + /* Page validation must be performed after changing to private */ + ret = pvalidate(vaddr, RMP_PG_SIZE_4K, true); + if (WARN(ret, "Failed to validate address 0x%lx ret %d", paddr, ret)) + goto e_term; + } + + vaddr += PAGE_SIZE; + paddr += PAGE_SIZE; } return; @@ -728,10 +762,7 @@ void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long padd * Ask the hypervisor to mark the memory pages as private in the RMP * table. */ - early_set_pages_state(paddr, npages, SNP_PAGE_STATE_PRIVATE); - - /* Validate the memory pages after they've been added in the RMP table. */ - pvalidate_pages(vaddr, npages, true); + early_set_pages_state(vaddr, paddr, npages, SNP_PAGE_STATE_PRIVATE); } void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, @@ -746,11 +777,8 @@ void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) return; - /* Invalidate the memory pages before they are marked shared in the RMP table. */ - pvalidate_pages(vaddr, npages, false); - /* Ask hypervisor to mark the memory pages shared in the RMP table. */ - early_set_pages_state(paddr, npages, SNP_PAGE_STATE_SHARED); + early_set_pages_state(vaddr, paddr, npages, SNP_PAGE_STATE_SHARED); } void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op) @@ -834,10 +862,11 @@ static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) return ret; } -static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, - unsigned long vaddr_end, int op) +static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, + unsigned long vaddr_end, int op) { struct ghcb_state state; + bool use_large_entry; struct psc_hdr *hdr; struct psc_entry *e; unsigned long flags; @@ -851,27 +880,37 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, memset(data, 0, sizeof(*data)); i = 0; - while (vaddr < vaddr_end) { - if (is_vmalloc_addr((void *)vaddr)) + while (vaddr < vaddr_end && i < ARRAY_SIZE(data->entries)) { + hdr->end_entry = i; + + if (is_vmalloc_addr((void *)vaddr)) { pfn = vmalloc_to_pfn((void *)vaddr); - else + use_large_entry = false; + } else { pfn = __pa(vaddr) >> PAGE_SHIFT; + use_large_entry = true; + } e->gfn = pfn; e->operation = op; - hdr->end_entry = i; - /* - * Current SNP implementation doesn't keep track of the RMP page - * size so use 4K for simplicity. - */ - e->pagesize = RMP_PG_SIZE_4K; + if (use_large_entry && IS_ALIGNED(vaddr, PMD_SIZE) && + (vaddr_end - vaddr) >= PMD_SIZE) { + e->pagesize = RMP_PG_SIZE_2M; + vaddr += PMD_SIZE; + } else { + e->pagesize = RMP_PG_SIZE_4K; + vaddr += PAGE_SIZE; + } - vaddr = vaddr + PAGE_SIZE; e++; i++; } + /* Page validation must be rescinded before changing to shared */ + if (op == SNP_PAGE_STATE_SHARED) + pvalidate_pages(data); + local_irq_save(flags); if (sev_cfg.ghcbs_initialized) @@ -879,6 +918,7 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, else ghcb = boot_ghcb; + /* Invoke the hypervisor to perform the page state changes */ if (!ghcb || vmgexit_psc(ghcb, data)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); @@ -886,29 +926,28 @@ static void __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, __sev_put_ghcb(&state); local_irq_restore(flags); + + /* Page validation must be performed after changing to private */ + if (op == SNP_PAGE_STATE_PRIVATE) + pvalidate_pages(data); + + return vaddr; } static void set_pages_state(unsigned long vaddr, unsigned long npages, int op) { - unsigned long vaddr_end, next_vaddr; struct snp_psc_desc desc; + unsigned long vaddr_end; /* Use the MSR protocol when a GHCB is not available. */ if (!boot_ghcb) - return early_set_pages_state(__pa(vaddr), npages, op); + return early_set_pages_state(vaddr, __pa(vaddr), npages, op); vaddr = vaddr & PAGE_MASK; vaddr_end = vaddr + (npages << PAGE_SHIFT); - while (vaddr < vaddr_end) { - /* Calculate the last vaddr that fits in one struct snp_psc_desc. */ - next_vaddr = min_t(unsigned long, vaddr_end, - (VMGEXIT_PSC_MAX_ENTRY * PAGE_SIZE) + vaddr); - - __set_pages_state(&desc, vaddr, next_vaddr, op); - - vaddr = next_vaddr; - } + while (vaddr < vaddr_end) + vaddr = __set_pages_state(&desc, vaddr, vaddr_end, op); } void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) @@ -916,8 +955,6 @@ void snp_set_memory_shared(unsigned long vaddr, unsigned long npages) if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return; - pvalidate_pages(vaddr, npages, false); - set_pages_state(vaddr, npages, SNP_PAGE_STATE_SHARED); } @@ -927,8 +964,6 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages) return; set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); - - pvalidate_pages(vaddr, npages, true); } static int snp_set_vmsa(void *va, bool vmsa) From patchwork Tue Jun 6 14:51:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103951 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3488494vqr; Tue, 6 Jun 2023 08:42:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5hsxZECKp3Vh8JE2vW78NywXO9E3fZWaN8cccXaEYwMW2yI4u+uaQoJl02igSIpfUPkvA6 X-Received: by 2002:a05:622a:104c:b0:3f3:d9b8:994f with SMTP id f12-20020a05622a104c00b003f3d9b8994fmr2492833qte.66.1686066147947; Tue, 06 Jun 2023 08:42:27 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686066147; cv=pass; d=google.com; s=arc-20160816; b=TCklAem8z937af/g8DM5Zg6nv+E2z40MvBmtSjTc6qYHxistwq7HAPvsSd/R1AsZTW 4HsXzwKgtWo5RgiD2/AsWBwZ+ZlHVLbWxvrOQ9IYLEQnxs5matp+AmGlFeCJiTbiqz49 KrDAHwPpgj4fb19XQnt41Qf+fOstPE+dy7NLtIJ1228egW5hqTdSn2T/amIw3BApUflf 7N3yeoDEW9bZPPq3Mca0x5deGP75Q+AoAJcGpbfEQZ4X243m1zb7gwHgnpcsyB1qRwMV Y5/Zv4Kip01roBL/JeCuuNvhCvNo83pWni9smZLW8zvYnV+v98trKIYDJrHNxP5kB6kE nGvg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XcHcxgB8hmsPJvHDrFtnkzlRV/I3Gc+1vg7l0zhuuF4=; b=Khf4W+hImTQX/XbbxpJxs2R5OvcdNGkA0980Scd96HhvJLY+NEz2U3hTN6SEcRGVoR c9ULv9a+2+TvJY9zaHJxz2Xf5jO8egay2fXKNShGh9hYtZ8qWCsj94fqHQiXlDtJaGl0 hurMwFBCzq5ekWwgrYW8Y/NOESMqmQCggdrrwpg7ECv5xucScJiKFFJeQ42nbN1GhTFX Lz109Bb/Mg5AZWFt3g1E2wl5CAk3Lb/BTqBuWHUfrAWEK+Opw1U8JHQdZ+TP/c4kdTKn E8DZaLeB0I/XmFQ3yymdYJbJCbDkDqgjAX8lg6oU4n0J1L2hfX0e+ANks9K4PG6V5PDS o9Xw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ULUktT5m; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m15-20020a05622a054f00b003f4e6c38525si6340429qtx.590.2023.06.06.08.42.13; Tue, 06 Jun 2023 08:42:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=ULUktT5m; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238481AbjFFOxM (ORCPT + 99 others); Tue, 6 Jun 2023 10:53:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238480AbjFFOwv (ORCPT ); Tue, 6 Jun 2023 10:52:51 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 463C31BCF for ; Tue, 6 Jun 2023 07:52:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hl4/SzWq2ZjIfN0Stg9WujsvKEskSoORHXNPDUpg5wQJc8pB2gT5gJnrZmGqqjxCMPqOTtyR7ta2Gef78rhaL8JvGpSyK3kPUfzHPQGKUZ/zbI2BykTNBmsjSc6OdIf+jE1FVzr/CI6Uj+N6va5mLZIb2b5kFHqHo6MlOCRWmCEmpD8OutGebmUk251ucBrMJqaMOoi/E5NH/LsDDvZpcw/FzBoTqvHQ5rIvYU2tVjfaaxW/3vVKPJtDcdn4rIadxpVLkcI4kizDBugOIzORFy7pIQuuMvFLXSfndSHct9qRNxjhBnFCvuRRzKYLvZ4BPX2O6mvQFyw9gc9HR/1Dvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XcHcxgB8hmsPJvHDrFtnkzlRV/I3Gc+1vg7l0zhuuF4=; b=TLVKI1d781gaABM5v9nW43aWU7gpdqfmCOSoqbIfGqiFxktTZWvNDQVLmyl2Ve59q3IIS+ZLzKdZfiFT/SfIaJnPwKFh/eu2RHCC4a8LuRYg6XGqx8F+/uzdeYKAqe2dpisQJcZzNFbJ5/VjrpZg4oGmkLy07SQern42efNaG1vW96byERj0MuTa6pLub/iFJHG4+El47ny3zhEKi//RrVAeNtZJWt+/1fZiQ1NzFpMZ7aqP3sD9TA8axyMGFJBn63mWZ2nz8PAkoAN4fvj/dle21T5pMsxSDyUxujqgL6WKWsWKVxR7Cn1L7bFpRMIUNMY6uQUpMrSt2kAvsAocyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XcHcxgB8hmsPJvHDrFtnkzlRV/I3Gc+1vg7l0zhuuF4=; b=ULUktT5m8do1hFCvTv/STxZMtRiwGI1rhzTMy1H8bmkurRtgmX1UWb4ozTydg3rcJbT5c76a7uyZULsN5T1KuIW0IXVow/z0zv60kz+5Z7ZDJPOWo+o6Pyhl2QV4xBHbzpe1lOCb45e3BNJvKhUlzm2tDO7wx2DxiH95wiTwczQ= Received: from MW2PR16CA0068.namprd16.prod.outlook.com (2603:10b6:907:1::45) by DM4PR12MB6472.namprd12.prod.outlook.com (2603:10b6:8:bc::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Tue, 6 Jun 2023 14:52:27 +0000 Received: from CO1NAM11FT116.eop-nam11.prod.protection.outlook.com (2603:10b6:907:1:cafe::a4) by MW2PR16CA0068.outlook.office365.com (2603:10b6:907:1::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Tue, 6 Jun 2023 14:52:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT116.mail.protection.outlook.com (10.13.174.243) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.36 via Frontend Transport; Tue, 6 Jun 2023 14:52:27 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:52:24 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra Subject: [PATCH v9 5/6] x86/sev: Add SNP-specific unaccepted memory support Date: Tue, 6 Jun 2023 09:51:26 -0500 Message-ID: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT116:EE_|DM4PR12MB6472:EE_ X-MS-Office365-Filtering-Correlation-Id: a597f66c-c6f3-478a-830b-08db669da546 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F/tr2sC7YBRng3s1k8b6lXSMMXE3/aO/gB/Ukm6rzG4fuxZSuT/xDFbmV4brYqd6uoS63tbLTKZTW7M6D/LDnubR8jplhx3AcwS4myxilUGsOK3jCY9wfWYONyNTQkagKZnXRWHv+ppi47sEYT06MO69r73cPBS0gFqPgT7VGDqLQIJkPjUWRGQhYGZ5M5bRyrcxwtzLFrATcCdhiNuN0MUvpvXWpOGmG5WfuL++4gc+lq5AbJ+ZnBajRqQAtX1Y9QI/DbFIQQiRUpY+ARPbLqf4rNEOjL2eLBRlQ7nijDx/Kl2C33hKzezh63qM5/TLAcLHj4zc1gUpOyU9Xs34fu0Hfri9dcwdoTISI3vRM/mWOQ/gtklXLAxjI3XqZ3Go8LVch7diBZ/Hvp5lyzDjwvt816rhHUL8Oz8MfVAGqNE6JhoP3O32xM2KhZQf2hF69UNVB0PnAqxTCmNrkfHLcEgxiViSn3Qam7IWvuFMXWPPv5aD0pO5jowo2Uy3PX7du/dcjqFe5+MBK5E0ueLrBTc+8mXwD/qhkxHVP+ubU1h+WIb457WtSOb3PFzsPtZkXt2oxXAZjT4yTBC41hcNm5pismuHksztVLUN0kCGogkyJLZ85utc7rc/crr+B/VEAEiFPx0PVaiOL2Y5AkPJth7nPItM30FGiI3A+0ROZWUCBhjQSQkOhzHc3VZdxjYqea1aGrIEF/YBkaA4Kdyff4AD1vz9TtJyPRi0FEk1rjuL0hrF4jpldySxxrn9MeGNcFJhKPN9I771lX5d2+bcvQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(136003)(376002)(39860400002)(396003)(451199021)(36840700001)(46966006)(40470700004)(8676002)(8936002)(478600001)(110136005)(54906003)(41300700001)(5660300002)(316002)(16526019)(26005)(7416002)(70586007)(70206006)(4326008)(2616005)(186003)(83380400001)(40460700003)(2906002)(47076005)(30864003)(426003)(336012)(81166007)(82740400003)(356005)(40480700001)(36860700001)(86362001)(82310400005)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:52:27.0411 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a597f66c-c6f3-478a-830b-08db669da546 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT116.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6472 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_FILL_THIS_FORM_SHORT,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767968497394148952?= X-GMAIL-MSGID: =?utf-8?q?1767968497394148952?= Add SNP-specific hooks to the unaccepted memory support in the boot path (__accept_memory()) and the core kernel (accept_memory()) in order to support booting SNP guests when unaccepted memory is present. Without this support, SNP guests will fail to boot and/or panic() when unaccepted memory is present in the EFI memory map. The process of accepting memory under SNP involves invoking the hypervisor to perform a page state change for the page to private memory and then issuing a PVALIDATE instruction to accept the page. Since the boot path and the core kernel paths perform similar operations, move the pvalidate_pages() and vmgexit_psc() functions into sev-shared.c to avoid code duplication. Create the new header file arch/x86/boot/compressed/sev.h because adding the function declaration to any of the existing SEV related header files pulls in too many other header files, causing the build to fail. Signed-off-by: Tom Lendacky --- arch/x86/Kconfig | 2 + arch/x86/boot/compressed/mem.c | 3 + arch/x86/boot/compressed/sev.c | 54 ++++++++++- arch/x86/boot/compressed/sev.h | 23 +++++ arch/x86/include/asm/sev.h | 3 + arch/x86/include/asm/unaccepted_memory.h | 3 + arch/x86/kernel/sev-shared.c | 103 +++++++++++++++++++++ arch/x86/kernel/sev.c | 112 +++-------------------- 8 files changed, 204 insertions(+), 99 deletions(-) create mode 100644 arch/x86/boot/compressed/sev.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5c72067c06d4..b9c451f75d5e 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1543,11 +1543,13 @@ config X86_MEM_ENCRYPT config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD + depends on EFI_STUB select DMA_COHERENT_POOL select ARCH_USE_MEMREMAP_PROT select INSTRUCTION_DECODER select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT + select UNACCEPTED_MEMORY help Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory diff --git a/arch/x86/boot/compressed/mem.c b/arch/x86/boot/compressed/mem.c index f04b29f3572f..3c1609245f2a 100644 --- a/arch/x86/boot/compressed/mem.c +++ b/arch/x86/boot/compressed/mem.c @@ -3,6 +3,7 @@ #include "error.h" #include "misc.h" #include "tdx.h" +#include "sev.h" #include /* @@ -37,6 +38,8 @@ void arch_accept_memory(phys_addr_t start, phys_addr_t end) if (early_is_tdx_guest()) { if (!tdx_accept_memory(start, end)) panic("TDX: Failed to accept memory\n"); + } else if (sev_snp_enabled()) { + snp_accept_memory(start, end); } else { error("Cannot accept memory: unknown platform\n"); } diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 014b89c89088..09dc8c187b3c 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -115,7 +115,7 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, /* Include code for early handlers */ #include "../../kernel/sev-shared.c" -static inline bool sev_snp_enabled(void) +bool sev_snp_enabled(void) { return sev_status & MSR_AMD64_SEV_SNP_ENABLED; } @@ -181,6 +181,58 @@ static bool early_setup_ghcb(void) return true; } +static phys_addr_t __snp_accept_memory(struct snp_psc_desc *desc, + phys_addr_t pa, phys_addr_t pa_end) +{ + struct psc_hdr *hdr; + struct psc_entry *e; + unsigned int i; + + hdr = &desc->hdr; + memset(hdr, 0, sizeof(*hdr)); + + e = desc->entries; + + i = 0; + while (pa < pa_end && i < VMGEXIT_PSC_MAX_ENTRY) { + hdr->end_entry = i; + + e->gfn = pa >> PAGE_SHIFT; + e->operation = SNP_PAGE_STATE_PRIVATE; + if (IS_ALIGNED(pa, PMD_SIZE) && (pa_end - pa) >= PMD_SIZE) { + e->pagesize = RMP_PG_SIZE_2M; + pa += PMD_SIZE; + } else { + e->pagesize = RMP_PG_SIZE_4K; + pa += PAGE_SIZE; + } + + e++; + i++; + } + + if (vmgexit_psc(boot_ghcb, desc)) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pvalidate_pages(desc); + + return pa; +} + +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + struct snp_psc_desc desc = {}; + unsigned int i; + phys_addr_t pa; + + if (!boot_ghcb && !early_setup_ghcb()) + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC); + + pa = start; + while (pa < end) + pa = __snp_accept_memory(&desc, pa, end); +} + void sev_es_shutdown_ghcb(void) { if (!boot_ghcb) diff --git a/arch/x86/boot/compressed/sev.h b/arch/x86/boot/compressed/sev.h new file mode 100644 index 000000000000..fc725a981b09 --- /dev/null +++ b/arch/x86/boot/compressed/sev.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * AMD SEV header for early boot related functions. + * + * Author: Tom Lendacky + */ + +#ifndef BOOT_COMPRESSED_SEV_H +#define BOOT_COMPRESSED_SEV_H + +#ifdef CONFIG_AMD_MEM_ENCRYPT + +bool sev_snp_enabled(void); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); + +#else + +static inline bool sev_snp_enabled(void) { return false; } +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } + +#endif + +#endif diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index e21e1c5397c1..86e1296e87f5 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -206,6 +206,7 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio); +void snp_accept_memory(phys_addr_t start, phys_addr_t end); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -229,6 +230,8 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } + +static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { } #endif #endif diff --git a/arch/x86/include/asm/unaccepted_memory.h b/arch/x86/include/asm/unaccepted_memory.h index 572514e36fde..f5937e9866ac 100644 --- a/arch/x86/include/asm/unaccepted_memory.h +++ b/arch/x86/include/asm/unaccepted_memory.h @@ -3,6 +3,7 @@ #include #include +#include static inline void arch_accept_memory(phys_addr_t start, phys_addr_t end) { @@ -10,6 +11,8 @@ static inline void arch_accept_memory(phys_addr_t start, phys_addr_t end) if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) { if (!tdx_accept_memory(start, end)) panic("TDX: Failed to accept memory\n"); + } else if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + snp_accept_memory(start, end); } else { panic("Cannot accept memory: unknown platform\n"); } diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index 3a5b0c9c4fcc..2eabccde94fb 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -12,6 +12,9 @@ #ifndef __BOOT_COMPRESSED #define error(v) pr_err(v) #define has_cpuflag(f) boot_cpu_has(f) +#else +#undef WARN +#define WARN(condition, format...) (!!(condition)) #endif /* I/O parameters for CPUID-related helpers */ @@ -991,3 +994,103 @@ static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info) cpuid_ext_range_max = fn->eax; } } + +static void pvalidate_pages(struct snp_psc_desc *desc) +{ + struct psc_entry *e; + unsigned long vaddr; + unsigned int size; + unsigned int i; + bool validate; + int rc; + + for (i = 0; i <= desc->hdr.end_entry; i++) { + e = &desc->entries[i]; + + vaddr = (unsigned long)pfn_to_kaddr(e->gfn); + size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; + validate = e->operation == SNP_PAGE_STATE_PRIVATE; + + rc = pvalidate(vaddr, size, validate); + if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { + unsigned long vaddr_end = vaddr + PMD_SIZE; + + for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { + rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); + if (rc) + break; + } + } + + if (rc) { + WARN(1, "Failed to validate address 0x%lx ret %d", vaddr, rc); + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); + } + } +} + +static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) +{ + int cur_entry, end_entry, ret = 0; + struct snp_psc_desc *data; + struct es_em_ctxt ctxt; + + vc_ghcb_invalidate(ghcb); + + /* Copy the input desc into GHCB shared buffer */ + data = (struct snp_psc_desc *)ghcb->shared_buffer; + memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); + + /* + * As per the GHCB specification, the hypervisor can resume the guest + * before processing all the entries. Check whether all the entries + * are processed. If not, then keep retrying. Note, the hypervisor + * will update the data memory directly to indicate the status, so + * reference the data->hdr everywhere. + * + * The strategy here is to wait for the hypervisor to change the page + * state in the RMP table before guest accesses the memory pages. If the + * page state change was not successful, then later memory access will + * result in a crash. + */ + cur_entry = data->hdr.cur_entry; + end_entry = data->hdr.end_entry; + + while (data->hdr.cur_entry <= data->hdr.end_entry) { + ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); + + /* This will advance the shared buffer data points to. */ + ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_PSC, 0, 0); + + /* + * Page State Change VMGEXIT can pass error code through + * exit_info_2. + */ + if (WARN(ret || ghcb->save.sw_exit_info_2, + "SNP: PSC failed ret=%d exit_info_2=%llx\n", + ret, ghcb->save.sw_exit_info_2)) { + ret = 1; + goto out; + } + + /* Verify that reserved bit is not set */ + if (WARN(data->hdr.reserved, "Reserved bit is set in the PSC header\n")) { + ret = 1; + goto out; + } + + /* + * Sanity check that entry processing is not going backwards. + * This will happen only if hypervisor is tricking us. + */ + if (WARN(data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry, +"SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", + end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry)) { + ret = 1; + goto out; + } + } + +out: + return ret; +} diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 17b3d003b2ea..ea2546e5130f 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -657,38 +657,6 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void pvalidate_pages(struct snp_psc_desc *desc) -{ - struct psc_entry *e; - unsigned long vaddr; - unsigned int size; - unsigned int i; - bool validate; - int rc; - - for (i = 0; i <= desc->hdr.end_entry; i++) { - e = &desc->entries[i]; - - vaddr = (unsigned long)pfn_to_kaddr(e->gfn); - size = e->pagesize ? RMP_PG_SIZE_2M : RMP_PG_SIZE_4K; - validate = e->operation == SNP_PAGE_STATE_PRIVATE; - - rc = pvalidate(vaddr, size, validate); - if (rc == PVALIDATE_FAIL_SIZEMISMATCH && size == RMP_PG_SIZE_2M) { - unsigned long vaddr_end = vaddr + PMD_SIZE; - - for (; vaddr < vaddr_end; vaddr += PAGE_SIZE) { - rc = pvalidate(vaddr, RMP_PG_SIZE_4K, validate); - if (rc) - break; - } - } - - if (WARN(rc, "Failed to validate address 0x%lx ret %d", vaddr, rc)) - sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); - } -} - static void early_set_pages_state(unsigned long vaddr, unsigned long paddr, unsigned long npages, enum psc_op op) { @@ -796,72 +764,6 @@ void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op WARN(1, "invalid memory op %d\n", op); } -static int vmgexit_psc(struct ghcb *ghcb, struct snp_psc_desc *desc) -{ - int cur_entry, end_entry, ret = 0; - struct snp_psc_desc *data; - struct es_em_ctxt ctxt; - - vc_ghcb_invalidate(ghcb); - - /* Copy the input desc into GHCB shared buffer */ - data = (struct snp_psc_desc *)ghcb->shared_buffer; - memcpy(ghcb->shared_buffer, desc, min_t(int, GHCB_SHARED_BUF_SIZE, sizeof(*desc))); - - /* - * As per the GHCB specification, the hypervisor can resume the guest - * before processing all the entries. Check whether all the entries - * are processed. If not, then keep retrying. Note, the hypervisor - * will update the data memory directly to indicate the status, so - * reference the data->hdr everywhere. - * - * The strategy here is to wait for the hypervisor to change the page - * state in the RMP table before guest accesses the memory pages. If the - * page state change was not successful, then later memory access will - * result in a crash. - */ - cur_entry = data->hdr.cur_entry; - end_entry = data->hdr.end_entry; - - while (data->hdr.cur_entry <= data->hdr.end_entry) { - ghcb_set_sw_scratch(ghcb, (u64)__pa(data)); - - /* This will advance the shared buffer data points to. */ - ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_PSC, 0, 0); - - /* - * Page State Change VMGEXIT can pass error code through - * exit_info_2. - */ - if (WARN(ret || ghcb->save.sw_exit_info_2, - "SNP: PSC failed ret=%d exit_info_2=%llx\n", - ret, ghcb->save.sw_exit_info_2)) { - ret = 1; - goto out; - } - - /* Verify that reserved bit is not set */ - if (WARN(data->hdr.reserved, "Reserved bit is set in the PSC header\n")) { - ret = 1; - goto out; - } - - /* - * Sanity check that entry processing is not going backwards. - * This will happen only if hypervisor is tricking us. - */ - if (WARN(data->hdr.end_entry > end_entry || cur_entry > data->hdr.cur_entry, -"SNP: PSC processing going backward, end_entry %d (got %d) cur_entry %d (got %d)\n", - end_entry, data->hdr.end_entry, cur_entry, data->hdr.cur_entry)) { - ret = 1; - goto out; - } - } - -out: - return ret; -} - static unsigned long __set_pages_state(struct snp_psc_desc *data, unsigned long vaddr, unsigned long vaddr_end, int op) { @@ -966,6 +868,20 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages) set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); } +void snp_accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long vaddr; + unsigned int npages; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + vaddr = (unsigned long)__va(start); + npages = (end - start) >> PAGE_SHIFT; + + set_pages_state(vaddr, npages, SNP_PAGE_STATE_PRIVATE); +} + static int snp_set_vmsa(void *va, bool vmsa) { u64 attrs; From patchwork Tue Jun 6 14:51:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 103943 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3484868vqr; Tue, 6 Jun 2023 08:36:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7eKUEGzkzvFXu7malCyVdrDh4DGYGwTlIP2v9dkXUi+7LehW92zFRPJGlVeM1sN8krFcOj X-Received: by 2002:a05:620a:4455:b0:75b:23a1:433 with SMTP id w21-20020a05620a445500b0075b23a10433mr3462872qkp.73.1686065812727; Tue, 06 Jun 2023 08:36:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1686065812; cv=pass; d=google.com; s=arc-20160816; b=MWSnhkhDERsiveGxLp2zprv+I1Ua3sZpg3fX6jeCzNcCNFzPnE0jVv+C/cKkSivmDh U3DmV+5GgGgSiHYXZs6iJgPGVSXt3yQI0DkrJ3EyPMhyAQxbDofhoHPWO66hQer7iAaO bGc1XU7EsJk8dLbLYgTN7AkKJ5ZnydkEE0P+T80DtH8NZtseamYIR7wFNjrcnWfsqJex If+WgZmrdVZPa+93M/KiuvNtXOT/KLRitP8G7VlNgw5Q1fQZs8MKT/+um2aFDPYsYP2C 5R7rt/1xjIHuDqxawbav9rgyOjtexilqvPzWCEiqjW46Vmr3gBafWRFBj8Rlrvs2Bm5x xNRw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3Nxajyk8eKxvej9jI08aDPnLBj0Vz+sgI0IjX4exDQY=; b=TFYrjUwWOhc0x57zTu0Cu3MNRkdMKgZ/M9ElMrvt7JrLWl1HD1mcIOxGA8BFd9mt4+ 3JzMwLBuTvxYHiz8cBGkRFJdRm1YLM+03SFmPGkHtzQ/8eijTxRrk9oP+F3CFlKxvN9D qgV3Wkz9NNzGfh/vBSCpfamepqtkOndyqapdbAf/Trew67jv6AIa9RssxR9/gHvhecSM TJqCoXefo6wOy2kp9jyGCQZc40Gl1mKkuc1AX5iWYyaDGA5Pecss+H1CwIJpvbVxfdtb EQ9O1HRSqGflRYZZuWVqmvQkedG7p2gs5CyC2zO33LvukgKCS08fTFImd6dL6EcuamMW bKaA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=5KQ2+3Qd; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w10-20020ae9e50a000000b0075b233a919fsi820442qkf.657.2023.06.06.08.36.38; Tue, 06 Jun 2023 08:36:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=5KQ2+3Qd; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238506AbjFFOxU (ORCPT + 99 others); Tue, 6 Jun 2023 10:53:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237822AbjFFOw5 (ORCPT ); Tue, 6 Jun 2023 10:52:57 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2062.outbound.protection.outlook.com [40.107.237.62]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5ED321BE3 for ; Tue, 6 Jun 2023 07:52:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LjRAw6u70MiEZtDz/ZOYWvVSJudsyRBnxb7+t0/x2/JLI2sLAyg9HNu2CZ4LPJA11OXsfDsvbVBeCrZQ1jEbcPLF3Lq6xJiDcOEum8S3GNmaKT/VwHD5VbnT5KOxFHMimhZtrpW4rFwhNo3qrwKP4CvurPvBMLzVPNWw9whOIhHU7Nh1lfy/bfYEtTZ85GDNA0r18SfqP6/EUt9wLjaEsHdIu84ELTpzg011gjsghdYW9yumMytev+xyd3tGBA5mm1JfQIWQz2yvK9l08ZxL+quXFxKJAbxVU5veHf3cNH2A2rZ+SDvRDhEyY4HFBVUsttl7C8sggGNMu5w8Ye/cBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3Nxajyk8eKxvej9jI08aDPnLBj0Vz+sgI0IjX4exDQY=; b=aCAnCr9wNGb1WKtNc0FFjGhCjB2u15YbdfR4pkMZyqj7Hcf0lDFed0xgUvbIRPcpTtk35N/3oGM3SvQXRricNe50HP7opZCBc/+JtHGn9h2+P/PrWodd9VbbEPAHSceNgtNxGqItGVpef5wHKixEpepdUrJhn6gMaZDbRs/xj/T4f27TtjzWmJ859WloDSTlEgsHz5I2ep/zqNhpf45QeaBJ7Gs66Gy1OgbKBV+6H1uki16X1uf40enU0nbP79ocRvtDZiPDXzAk8D7ysSr95kPkz/1S5Yaw9B3bj9r5lLjC567a67rcW8ZnRhyapv+so6KiQNa1H7vIcqRl+MRFdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3Nxajyk8eKxvej9jI08aDPnLBj0Vz+sgI0IjX4exDQY=; b=5KQ2+3Qd9sjCahiH1dvU2tJauRIRgQNvY1wlkJC+vJHixKODDq81IrA+nah6ay69JfHIR5r6/K1r9tvJ7bdP0xHZH+EXxEbg5fSoIzrdLrzAxskGLpUNwtg/UjSBIfo0mVoOW2c1oKsPR89kZmy1Ff3k/K6Wa22XMzNuWqB50sw= Received: from MW4PR04CA0208.namprd04.prod.outlook.com (2603:10b6:303:86::33) by BY5PR12MB5512.namprd12.prod.outlook.com (2603:10b6:a03:1df::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Tue, 6 Jun 2023 14:52:36 +0000 Received: from CO1NAM11FT045.eop-nam11.prod.protection.outlook.com (2603:10b6:303:86:cafe::c) by MW4PR04CA0208.outlook.office365.com (2603:10b6:303:86::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:52:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT045.mail.protection.outlook.com (10.13.175.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6455.33 via Frontend Transport; Tue, 6 Jun 2023 14:52:35 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 6 Jun 2023 09:52:33 -0500 From: Tom Lendacky To: , CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "Kirill A. Shutemov" , "H. Peter Anvin" , Michael Roth , Joerg Roedel , Dionna Glaze , Andy Lutomirski , Peter Zijlstra , Ard Biescheuvel , "Min M. Xu" , Gerd Hoffmann , James Bottomley , Tom Lendacky , Jiewen Yao , Erdem Aktas , "Kirill A. Shutemov" Subject: [PATCH v9 6/6] x86/efi: Safely enable unaccepted memory in UEFI Date: Tue, 6 Jun 2023 09:51:27 -0500 Message-ID: <0d5f3d9a20b5cf361945b7ab1263c36586a78a42.1686063086.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: <20230606142637.5171-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT045:EE_|BY5PR12MB5512:EE_ X-MS-Office365-Filtering-Correlation-Id: 222dbb1a-d3dd-441f-716a-08db669daa5e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mg1XlaZ+wAp2LyUBnv+F7UUptxSFQRAC+2LGBQNlyhDI90sy4wavLeATqJERniRfM0LziIj3BSZXVL6Pc6JzdT3DGhQapnsXrycWjadmPYnp5bHEKHAfO6Y2nlCYAywuyMmXDb0lv+oeCDws0d2CgJ6M3KkuIWGQvH6cGRKp1YEbORUTjZMlBOTwELdb3JK4ibJDK+wBAiObkisTe23sj7v5RvSXDV+iDlQZKFkObK7PAR+sD8xhciuPQCwxceW5GKnVKq5MS87uCeM9vp0w/rC1UUNUEYXspf7CnOQX52Infol3A8XRYo8gp/58JFxr19bz4AJk0mB7CEiRnS7cc1uR8/+vZv8Ax/Z5RCoFGaAMpo20zgIsVKB/GphBXlwmDOXWFzDQWB0HsPm+2lNwOLBRWhWduf1O7XUPpG/6iqjIuwhf9hpv5V+6kU8o6MDTsMEPMRF9DBmbZEqINL7etEtKzGLeQXt+M/boKUuh3qffRMxamjASEbQqKsjyJQqdz+lT28BhQeFjvmLXNq5LU8eLA7Z08vZwajxdjSGPWaBGdIOk07fcUgOQ8qfnbWalRoclrMcfSNvQDFQWTLlnyq69ohA54KzdAnnl1FE4bbDqIIXwfwLj2lZBs8aKuweDjKdXwGbEn6jLu9mqTN8VLQ+UgMAKjClN7ZND4PP1v7MytdjysLLvl5FZ2Y6QAjnlVA7AqgYpkEOylORJmDG22uM23aBdSKciRlgvXANzE7D2KECGrFsrce2xrgc3fwcuW6Y4ZpjVsgJan1T3WSjq/w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(346002)(376002)(39860400002)(396003)(451199021)(46966006)(36840700001)(40470700004)(41300700001)(316002)(478600001)(40460700003)(40480700001)(83380400001)(2906002)(36756003)(7416002)(70586007)(70206006)(356005)(8676002)(86362001)(5660300002)(8936002)(4326008)(81166007)(82740400003)(54906003)(110136005)(6666004)(82310400005)(36860700001)(47076005)(336012)(426003)(16526019)(2616005)(186003)(26005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 14:52:35.5865 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 222dbb1a-d3dd-441f-716a-08db669daa5e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT045.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB5512 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767968145827966139?= X-GMAIL-MSGID: =?utf-8?q?1767968145827966139?= From: Dionna Glaze The UEFI v2.9 specification includes a new memory type to be used in environments where the OS must accept memory that is provided from its host. Before the introduction of this memory type, all memory was accepted eagerly in the firmware. In order for the firmware to safely stop accepting memory on the OS's behalf, the OS must affirmatively indicate support to the firmware. This is only a problem for AMD SEV-SNP, since Linux has had support for it since 5.19. The other technology that can make use of unaccepted memory, Intel TDX, does not yet have Linux support, so it can strictly require unaccepted memory support as a dependency of CONFIG_TDX and not require communication with the firmware. Enabling unaccepted memory requires calling a 0-argument enablement protocol before ExitBootServices. This call is only made if the kernel is compiled with UNACCEPTED_MEMORY=y This protocol will be removed after the end of life of the first LTS that includes it, in order to give firmware implementations an expiration date for it. When the protocol is removed, firmware will strictly infer that a SEV-SNP VM is running an OS that supports the unaccepted memory type. At the earliest convenience, when unaccepted memory support is added to Linux, SEV-SNP may take strict dependence in it. After the firmware removes support for the protocol, this patch should be reverted. [tl: address some checkscript warnings] Cc: Ard Biescheuvel Cc: "Min M. Xu" Cc: Gerd Hoffmann Cc: James Bottomley Cc: Tom Lendacky Cc: Jiewen Yao Cc: Erdem Aktas Cc: "Kirill A. Shutemov" Cc: Dave Hansen Cc: Borislav Petkov Signed-off-by: Dionna Glaze Signed-off-by: Tom Lendacky Reviewed-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/x86-stub.c | 36 +++++++++++++++++++++++++ include/linux/efi.h | 3 +++ 2 files changed, 39 insertions(+) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 8d17cee8b98e..e2193dbe1f66 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -26,6 +26,17 @@ const efi_dxe_services_table_t *efi_dxe_table; u32 image_offset __section(".data"); static efi_loaded_image_t *image = NULL; +typedef union sev_memory_acceptance_protocol sev_memory_acceptance_protocol_t; +union sev_memory_acceptance_protocol { + struct { + efi_status_t (__efiapi * allow_unaccepted_memory)( + sev_memory_acceptance_protocol_t *); + }; + struct { + u32 allow_unaccepted_memory; + } mixed_mode; +}; + static efi_status_t preserve_pci_rom_image(efi_pci_io_protocol_t *pci, struct pci_setup_rom **__rom) { @@ -310,6 +321,29 @@ setup_memory_protection(unsigned long image_base, unsigned long image_size) #endif } +static void setup_unaccepted_memory(void) +{ + efi_guid_t mem_acceptance_proto = OVMF_SEV_MEMORY_ACCEPTANCE_PROTOCOL_GUID; + sev_memory_acceptance_protocol_t *proto; + efi_status_t status; + + if (!IS_ENABLED(CONFIG_UNACCEPTED_MEMORY)) + return; + + /* + * Enable unaccepted memory before calling exit boot services in order + * for the UEFI to not accept all memory on EBS. + */ + status = efi_bs_call(locate_protocol, &mem_acceptance_proto, NULL, + (void **)&proto); + if (status != EFI_SUCCESS) + return; + + status = efi_call_proto(proto, allow_unaccepted_memory); + if (status != EFI_SUCCESS) + efi_err("Memory acceptance protocol failed\n"); +} + static const efi_char16_t apple[] = L"Apple"; static void setup_quirks(struct boot_params *boot_params, @@ -908,6 +942,8 @@ asmlinkage unsigned long efi_main(efi_handle_t handle, setup_quirks(boot_params, bzimage_addr, buffer_end - buffer_start); + setup_unaccepted_memory(); + status = exit_boot(boot_params, handle); if (status != EFI_SUCCESS) { efi_err("exit_boot() failed!\n"); diff --git a/include/linux/efi.h b/include/linux/efi.h index 9864f9c00da2..8c5abcf70a05 100644 --- a/include/linux/efi.h +++ b/include/linux/efi.h @@ -437,6 +437,9 @@ void efi_native_runtime_setup(void); #define DELLEMC_EFI_RCI2_TABLE_GUID EFI_GUID(0x2d9f28a2, 0xa886, 0x456a, 0x97, 0xa8, 0xf1, 0x1e, 0xf2, 0x4f, 0xf4, 0x55) #define AMD_SEV_MEM_ENCRYPT_GUID EFI_GUID(0x0cf29b71, 0x9e51, 0x433a, 0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75) +/* OVMF protocol GUIDs */ +#define OVMF_SEV_MEMORY_ACCEPTANCE_PROTOCOL_GUID EFI_GUID(0xc5a010fe, 0x38a7, 0x4531, 0x8a, 0x4a, 0x05, 0x00, 0xd2, 0xfd, 0x16, 0x49) + typedef struct { efi_guid_t guid; u64 table;