From patchwork Mon Jan 16 14:25:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44199 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232559wrn; Mon, 16 Jan 2023 06:54:19 -0800 (PST) X-Google-Smtp-Source: AMrXdXu/nmTPOcD3tX16dCn7zd0YP3mkrvvJjJoHN6OT4Iw5gJ0njwkwH49XnLwBIaE5F/i7Br4Y X-Received: by 2002:a17:90a:6449:b0:229:9369:9d94 with SMTP id y9-20020a17090a644900b0022993699d94mr495762pjm.30.1673880859219; Mon, 16 Jan 2023 06:54:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880859; cv=none; d=google.com; s=arc-20160816; b=qGKOg2SqC89EsUMtRd4qcclo1NaeE+2Ob7/pFRV4OXlXQYVKKVAHWliyuDoyc0/cBK vhMPainht/6GHJ1mGhmWBxnHGwgYR2xRBcNGHMOBhDtYdVLgh9Q78BgEAsDtpDsK5ZbK 1kXLzTXDXkC9svJFtma2ETWe0ycjDqVWyOUnDb8y3m9wcgiVf8ANs/qsVRd+HBUYI/ZE rconjvrlLnMawcZjqlWSOvhwyNtk3r27CXKFqCu2TU3CyJdWAz+rmEZ2ZUbsuNDt91AX UFJBYPUf0QF9Ckh0QbV6Qk3SM/QB8FQLe9Rbs7j6Ad0UvFitONQW+AwCoMg5krHwZfeP 4gow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=rHjspetb6Sj1qMw4IETTuTuaCFumb5EYx3riiNCTYv8=; b=msVNQkGTdiBZjW59J6VS6c+Fxjh9FW/tCZtTLPCC/QU+hG9QCyN/zv4V+XpPHdr/BA y7+2pc1Umf04NxbnpqLsKnlYE/qg0w+g7rjV4rLeLaPCgGsM8W1BHKb2rh1suaz1OvKW toT806Ht6OxZi1b0mor8aLufSrm1gJ4alaB3Qo4YyEopdM28GURjcg39FovvlwzHjmgA 9lLTcMtQZd75QA6DKKKaC81F5Ten9ANblo39qsFvyQe2CVrITYHRC2tn6eAYBgTq2Ftz UwSLcz+Ec685fRXYut+VE6+uLpPWzvaZ0Zpfc4L89SV9YjHghgzbXRN2S0jz73KOp6WG y6hw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=cxz2ZJ+y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q4-20020a17090a938400b002266735a4b8si9811045pjo.81.2023.01.16.06.54.07; Mon, 16 Jan 2023 06:54:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=cxz2ZJ+y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231516AbjAPOxB (ORCPT + 99 others); Mon, 16 Jan 2023 09:53:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231209AbjAPOwN (ORCPT ); Mon, 16 Jan 2023 09:52:13 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52AF123DAD for ; Mon, 16 Jan 2023 06:38:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=rHjspetb6Sj1qMw4IETTuTuaCFumb5EYx3riiNCTYv8=; b=cxz2ZJ+y57FVH0axH7GJ3/SDvk SfIli5aMNXUCBqSZ5s0hpQ541ETtZeXdeKeGN56Ebrzzk0C4N9eLXGVlRGHVrQ5JzV49oWAUTb+Qe Kn4rI0WMLi8kkdccgmv96vW1qIQzU99aMQ1+XNeo6itRotmYQ0j8+xlhlEUBQ0ztiZvuG2cxAChr6 22QxIhtpXZ6ZvAkrMLo5HT7ZYfsHVWzIJ2yKMB5H3w81SPg3IvAOF/XqgpTenKxT3AR/Anre1jbBa 5T4FbzmfMt67wCGdvro8HFHivVFFtpRbdX2x3wnviudr35z2uOGNZgbxfcNisa/ZFz7Fzu5vDAznP UN6EINCw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHQcT-008oZ5-S7; Mon, 16 Jan 2023 14:37:54 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E59A8300C0C; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id ADBD820B75F3D; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.589522290@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:34 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" , jroedel@suse.de Subject: [PATCH v2 1/7] x86/boot: Remove verify_cpu() from secondary_startup_64() References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191296241935759?= X-GMAIL-MSGID: =?utf-8?q?1755191296241935759?= The boot trampolines from trampoline_64.S have code flow like: 16bit BIOS SEV-ES 64bit EFI trampoline_start() sev_es_trampoline_start() trampoline_start_64() verify_cpu() | | switch_to_protected: <---------------' v | pa_trampoline_compat() v | startup_32() <-----------------------------------------------' | v startup_64() | v tr_start() := head_64.S:secondary_startup_64() Since AP bringup always goes through the 16bit BIOS path (EFI doesn't touch the APs), there is already a verify_cpu() invocation. Removing the verify_cpu() invocation from secondary_startup_64() renders the whole secondary_startup_64_no_verify() thing moot, so remove that too. Cc: jroedel@suse.de Cc: hpa@zytor.com Fixes: e81dc127ef69 ("x86/callthunks: Add call patching for call depth tracking") Reported-by: Joan Bruguera Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Ingo Molnar --- arch/x86/include/asm/realmode.h | 1 - arch/x86/kernel/head_64.S | 16 ---------------- arch/x86/realmode/init.c | 6 ------ 3 files changed, 23 deletions(-) --- a/arch/x86/include/asm/realmode.h +++ b/arch/x86/include/asm/realmode.h @@ -73,7 +73,6 @@ extern unsigned char startup_32_smp[]; extern unsigned char boot_gdt[]; #else extern unsigned char secondary_startup_64[]; -extern unsigned char secondary_startup_64_no_verify[]; #endif static inline size_t real_mode_size_needed(void) --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -143,22 +143,6 @@ SYM_CODE_START(secondary_startup_64) * after the boot processor executes this code. */ - /* Sanitize CPU configuration */ - call verify_cpu - - /* - * The secondary_startup_64_no_verify entry point is only used by - * SEV-ES guests. In those guests the call to verify_cpu() would cause - * #VC exceptions which can not be handled at this stage of secondary - * CPU bringup. - * - * All non SEV-ES systems, especially Intel systems, need to execute - * verify_cpu() above to make sure NX is enabled. - */ -SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) - UNWIND_HINT_EMPTY - ANNOTATE_NOENDBR - /* * Retrieve the modifier (SME encryption mask if SME is active) to be * added to the initial pgdir entry that will be programmed into CR3. --- a/arch/x86/realmode/init.c +++ b/arch/x86/realmode/init.c @@ -74,12 +74,6 @@ static void __init sme_sev_setup_real_mo th->flags |= TH_FLAGS_SME_ACTIVE; if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { - /* - * Skip the call to verify_cpu() in secondary_startup_64 as it - * will cause #VC exceptions when the AP can't handle them yet. - */ - th->start = (u64) secondary_startup_64_no_verify; - if (sev_es_setup_ap_jump_table(real_mode_header)) panic("Failed to get/update SEV-ES AP Jump Table"); } From patchwork Mon Jan 16 14:25:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44194 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232282wrn; Mon, 16 Jan 2023 06:53:39 -0800 (PST) X-Google-Smtp-Source: AMrXdXuOKcd0snrZg+9NQARzT2VgISFvjvuXmykt8wcMNdSnOvzNgTUJ8Sj/OLACm+XcLdKn0EIC X-Received: by 2002:a05:6a20:a6a0:b0:b6:99c7:9283 with SMTP id ba32-20020a056a20a6a000b000b699c79283mr16755471pzb.12.1673880819570; Mon, 16 Jan 2023 06:53:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880819; cv=none; d=google.com; s=arc-20160816; b=k67iEm6nPA6bNms459RS7sfAMJ6MojfgUzqhq6hUO6FT2T3evT1JF7CZ2Zw8jFQoJ1 Mi9Qh4tasv0WMQWtwk6bcjgrEoOwStCrjpnefXRS29oZStlc6Gf2zTv/Dy9Ezc59/tLb rAZy93R+YsSfCBJb2ZDAVLEI+LcrCT8hxA4NjCEZcmsJeXhjfmrBRpyJT8obovucEU2p jl0djtQ3+pDwLpLTKsY1DmXzCZwotj66GvSOTXrK49ZOaLIx8msgnwzSQHxD163BLDPS ZFmJIiq8G2H30xOs+ay2/Ep5IWJnsRWDX2szGuZfp+CV7k2l1PrRLQ/zB6sCwoMc86BL nSqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=/Xw5T+5vUqgsMfvHO/2XJeMLoksLNzw5fViFdsAKWFI=; b=Ejs0RcJaIFY1uhRhZTTqZqGvX1ywsGdjLhO+S1i86qmN7GC+A8hZqa1T2tio9q2zgX MRoNYRp2ZwmyjgfSQ3shMFjTYh+eJ7Mk+9bssuoNF5rsf/Y/D8RcDP0uWMoPTRmBzh2d AMtQVQU8igz/yiDGOC/LwSEZxP2CKJvL/tmd01EtqWMVXvAEN6HJgC28/bgboCQsK8Ho U+Q7PxQwRSrkyx/lfrwS7mTJDbo1xFPJLwlaFCaLJMGAtDaff2jtnJfPAx+CIXjlwGU2 /TpdgIHCg9bzoQIaB6sABncW3pYHYQmPrsX2IczcPdO06Uh9lva6dCIWoIYq8dUsna24 5VPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=Eq3y0scg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h124-20020a636c82000000b0049ca25d4963si29420469pgc.468.2023.01.16.06.53.27; Mon, 16 Jan 2023 06:53:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=Eq3y0scg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232282AbjAPOwo (ORCPT + 99 others); Mon, 16 Jan 2023 09:52:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230183AbjAPOwM (ORCPT ); Mon, 16 Jan 2023 09:52:12 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CDC92BF16 for ; Mon, 16 Jan 2023 06:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=/Xw5T+5vUqgsMfvHO/2XJeMLoksLNzw5fViFdsAKWFI=; b=Eq3y0scguUgdkMj79Zzk6GCrvF hQAhEQP/E7cAerOW9szVqcnkFmKfzWUaiZxup/I+SMwXzVRopqfn4zPHPbmkQufcFo19M0uELMYv/ kYPxOMfLbqjIwv9QIGVc5oiVosXskQwBXNQfwBy/jDHeBBuxrkRaeFIhaA+Tm2XLDqfwvxvnRyH0O rvOYW2PlqyMs+3qT9hF1XykM1+ON8wTIDA4x8JJJGyRjveBRsW6uZuMNrQ9P/htuomhOyIDx2ZWUH ITNdFmVF/slE93TC/wZdRxX36dsgHyfS5wX7wOPPamJ7fIEFgb5fD7j0P6Cn79iKWZ/XYGoYdt6+A rE5ri56A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHQcT-008oZ7-Sz; Mon, 16 Jan 2023 14:37:54 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E3A5C3007DA; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id AF96620306BCC; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.649204101@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:35 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 2/7] x86/boot: Delay sev_verify_cbit() a bit References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191254114727505?= X-GMAIL-MSGID: =?utf-8?q?1755191254114727505?= Per the comment it is important to call sev_verify_cbit() before the first RET instruction, this means we can delay calling this until more of the CPU state is set up, specifically delay this until GS is 'sane' such that per-cpu variables work. Fixes: e81dc127ef69 ("x86/callthunks: Add call patching for call depth tracking") Reported-by: Joan Bruguera Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/kernel/head_64.S | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -185,19 +185,6 @@ SYM_CODE_START(secondary_startup_64) addq phys_base(%rip), %rax /* - * For SEV guests: Verify that the C-bit is correct. A malicious - * hypervisor could lie about the C-bit position to perform a ROP - * attack on the guest by writing to the unencrypted stack and wait for - * the next RET instruction. - * %rsi carries pointer to realmode data and is callee-clobbered. Save - * and restore it. - */ - pushq %rsi - movq %rax, %rdi - call sev_verify_cbit - popq %rsi - - /* * Switch to new page-table * * For the boot CPU this switches to early_top_pgt which still has the @@ -265,6 +252,19 @@ SYM_CODE_START(secondary_startup_64) */ movq initial_stack(%rip), %rsp + /* + * For SEV guests: Verify that the C-bit is correct. A malicious + * hypervisor could lie about the C-bit position to perform a ROP + * attack on the guest by writing to the unencrypted stack and wait for + * the next RET instruction. + * %rsi carries pointer to realmode data and is callee-clobbered. Save + * and restore it. + */ + pushq %rsi + movq %rax, %rdi + call sev_verify_cbit + popq %rsi + /* Setup and Load IDT */ pushq %rsi call early_setup_idt From patchwork Mon Jan 16 14:25:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44201 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232616wrn; Mon, 16 Jan 2023 06:54:26 -0800 (PST) X-Google-Smtp-Source: AMrXdXsqxCgssywioKFnQ6qUVe2Xf9Q7lUysv00/FR3u7hrH7VWMmoES7tY3GZKhYN38rMoe0M+C X-Received: by 2002:a17:902:a503:b0:194:996e:3285 with SMTP id s3-20020a170902a50300b00194996e3285mr2694064plq.68.1673880866234; Mon, 16 Jan 2023 06:54:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880866; cv=none; d=google.com; s=arc-20160816; b=w0uJiPqt1Bb11D1xAQhzNmXhahZJmZqVhXFrg7lS+sVWay5OVjgenPlmxb2Z8hP3hd aTQkBgAqoJT6/JcRdz7JeJaWqlzqTstZOuJz6N+B52ZBV3bcV6EoD7M59dyzgbqdugQQ +Ds9Xx8eMV5CPCLXevdsfLPOswXzuCl/nQBlc3kXnG/P43z0P9wnjkIAyQny8D2hzn6+ AA+I3PHY2MlPclVa8L8mn5/+osv7q3aT/TwyrGDIKci/nXDZoNtbH6Z63ATlUxrzC3d+ edWdeU9u3nZe/dztxRh5uwFGCq9EROc/kh7JNa/I3rcnQNRmQcP5RLyjFb7ErLtbT2SV 4Zyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=ts2f9zMhByBtAAtY3UBCAh5kpNSswewgiYAUfY8HRdU=; b=zb6ZpOZOLNhpv3K0MXRxaSAN9kgGG/3lFcHz4y2rNjOjEhMWaD5/Ln7GbIZgyoELao vz1V1m4r9r+Tw6mRbiA+eNr8+agnu7zFqA2CnFCxV/t8VZRzhwNrcV7416KG1Ha16lM8 igUF44MGlZpwjaCrWukHZaZfi82kQLIGqj7Emh2rDqNvV1/vtXli8JCKgkRi6ASWgMSt KIGUOOqPeEMezG4cdmKaZ7JrxoHlIx5Ur3+swqql5n7IACxy9zOK06c9wa557qrZMEvv C1k94cY4SYmZRhNqmGbCeEgkwCbEuY+Lvh5ytb4/lK3oXFbdYbd9FZ1T9JF3dSf0c6yO RYXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=aGBhaLZs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x1-20020a170902ec8100b00189e2b9e630si31266577plg.270.2023.01.16.06.54.12; Mon, 16 Jan 2023 06:54:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=aGBhaLZs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232433AbjAPOxJ (ORCPT + 99 others); Mon, 16 Jan 2023 09:53:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230257AbjAPOwO (ORCPT ); Mon, 16 Jan 2023 09:52:14 -0500 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CE1A2BF30 for ; Mon, 16 Jan 2023 06:38:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=ts2f9zMhByBtAAtY3UBCAh5kpNSswewgiYAUfY8HRdU=; b=aGBhaLZsy7tRmU8DfE6K+TDDO0 vuZ+yzEJlLmyIkJpKj5j6TVZcKHJw7LV5zhoweNmIAXEhyqNbDctZ6rxIemADC20CHJlks7N8JIx+ K5F4VyyMc1wR1Ib2pi1CQk44Pzj8XTpGUHCP4oZt5ZuUo1cEJm9pBXD/3o3j4D+/yj051n7fxMsXX 9AcOJ4ti/xsLZ0KgZKwSKrmlvgRdCt+MP/E3zPoSa5b3jgTI5IW4LJAlgGIKa96qCBr3IpBr3kI5N Tg9J6WHAyn2j5+4GcLAyMQZQrgoS+GXww4z8hdfHou6kXyg3O02NeUUJTaNy52X7cIFx8XWm84+1+ NZWTDZ8g==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1pHQc7-005csw-18; Mon, 16 Jan 2023 14:37:32 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id DE246300652; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id B311720EF0A20; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.708895882@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:36 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 3/7] x86/power: De-paravirt restore_processor_state() References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191303141876349?= X-GMAIL-MSGID: =?utf-8?q?1755191303141876349?= Since Xen PV doesn't use restore_processor_state(), and we're going to have to avoid CALL/RET until at least GS is restored, de-paravirt the easy bits. Fixes: e81dc127ef69 ("x86/callthunks: Add call patching for call depth tracking") Reported-by: Joan Bruguera Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Juergen Gross --- arch/x86/power/cpu.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) --- a/arch/x86/power/cpu.c +++ b/arch/x86/power/cpu.c @@ -197,25 +197,25 @@ static void notrace __restore_processor_ struct cpuinfo_x86 *c; if (ctxt->misc_enable_saved) - wrmsrl(MSR_IA32_MISC_ENABLE, ctxt->misc_enable); + native_wrmsrl(MSR_IA32_MISC_ENABLE, ctxt->misc_enable); /* * control registers */ /* cr4 was introduced in the Pentium CPU */ #ifdef CONFIG_X86_32 if (ctxt->cr4) - __write_cr4(ctxt->cr4); + native_write_cr4(ctxt->cr4); #else /* CONFIG X86_64 */ - wrmsrl(MSR_EFER, ctxt->efer); - __write_cr4(ctxt->cr4); + native_wrmsrl(MSR_EFER, ctxt->efer); + native_write_cr4(ctxt->cr4); #endif - write_cr3(ctxt->cr3); - write_cr2(ctxt->cr2); - write_cr0(ctxt->cr0); + native_write_cr3(ctxt->cr3); + native_write_cr2(ctxt->cr2); + native_write_cr0(ctxt->cr0); /* Restore the IDT. */ - load_idt(&ctxt->idt); + native_load_idt(&ctxt->idt); /* * Just in case the asm code got us here with the SS, DS, or ES @@ -230,7 +230,7 @@ static void notrace __restore_processor_ * handlers or in complicated helpers like load_gs_index(). */ #ifdef CONFIG_X86_64 - wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base); + native_wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base); #else loadsegment(fs, __KERNEL_PERCPU); #endif @@ -246,15 +246,15 @@ static void notrace __restore_processor_ loadsegment(ds, ctxt->es); loadsegment(es, ctxt->es); loadsegment(fs, ctxt->fs); - load_gs_index(ctxt->gs); + native_load_gs_index(ctxt->gs); /* * Restore FSBASE and GSBASE after restoring the selectors, since * restoring the selectors clobbers the bases. Keep in mind * that MSR_KERNEL_GS_BASE is horribly misnamed. */ - wrmsrl(MSR_FS_BASE, ctxt->fs_base); - wrmsrl(MSR_KERNEL_GS_BASE, ctxt->usermode_gs_base); + native_wrmsrl(MSR_FS_BASE, ctxt->fs_base); + native_wrmsrl(MSR_KERNEL_GS_BASE, ctxt->usermode_gs_base); #else loadsegment(gs, ctxt->gs); #endif From patchwork Mon Jan 16 14:25:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44200 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232576wrn; Mon, 16 Jan 2023 06:54:22 -0800 (PST) X-Google-Smtp-Source: AMrXdXuTGWEStrfLoARIaoG5l9NbnpwRzr45F742OjLEYZLZDNqiBi8CTndicFKamNtss1mqAR9p X-Received: by 2002:a05:6a20:690d:b0:b5:97de:d7e7 with SMTP id q13-20020a056a20690d00b000b597ded7e7mr48698410pzj.18.1673880861754; Mon, 16 Jan 2023 06:54:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880861; cv=none; d=google.com; s=arc-20160816; b=zn0aqm6O/JAN7P+a8mjzT8wkMsIguxqE9NfcFMl51hy1VPITppYt2yjDYQ4azVoExM 1PHQVIJ6Ea7Yb8McMeop21yELmjIBt3E4Qje4i5NzcjxIht30DOaBa904hrTTa3ymvQx xvG5QhMl/1CFlzSOR8LxTdfK/1nyiKir5BrmqXyabrYebSIzByD+qHw126z+/QKsW/bU NWTxfmK/a5ED7q5eVLjIVuNbV531LeRTHIOwORiLoRfscTMAI4k8Ws76wn+jVPNxnzqs YtZhvVvG6KEF1jg9xOqxH4y5x0d23yy9AelS3kocL/esVWl3bPUouNxRNDb9Rn1/wTJs PrfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=dtG3CCLzMA0i0qOsNCGAACCWIWjzH+RX76nYJr/7nUg=; b=KV7RTR3hVChvYyBul0MZ8OyQD3VUGn0+dcx7WIftVGBXQQtdaXsoZwiZpYnxm2zqL4 DqlXWCXgjXtFVTYA3TFKzlgIaH/7gvLNMcCVq2Zv+TrWB4wXY357XZfcF+ytob+Xhd0t KU7+reijNLRTbXqnCZZ6qUKFNEIGp0LG/pe+50QaT7d+YS4nXKUvQ2X3XyKfzDylk8Jf x8cVY8cpwbizJByv2SCA/yk8wmEioZf9QxEdjVBj1eKqIz4PWys6Dh2E084Q/Jmy6rkY 3eEKb5mXCSKY540jghui85gCuARbv/QnLOqdlBXnWAB9+fmWm2TbObpKqu6N0CfY03Tp hPEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=mNSQfQeU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q8-20020a63cc48000000b0046b1dabcb34si30135426pgi.765.2023.01.16.06.54.09; Mon, 16 Jan 2023 06:54:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=mNSQfQeU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232398AbjAPOxG (ORCPT + 99 others); Mon, 16 Jan 2023 09:53:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230218AbjAPOwN (ORCPT ); Mon, 16 Jan 2023 09:52:13 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 895D72BF1B for ; Mon, 16 Jan 2023 06:38:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=dtG3CCLzMA0i0qOsNCGAACCWIWjzH+RX76nYJr/7nUg=; b=mNSQfQeUAAjFw9ESss496f+Z7Q u0TgyafjRhE1gQwftI2iBVWQ3NaSLG34VdbAWazXouiYsRdKyzqy1wML0X2573JOCsV/N0PK5k2F2 TPov7/LEGoLwGuZcybAwU9tHaa3wffyOD9zxSarAfZ29Q4WErfXs+qrs4jgpQMpEdyO9/ORvp95nX EjaMEymmS+cwIurns0c3wEq3uE17tL2K8Hwq/Ex3y3gpAT4bxe9pp5k86FtmXoW5UtC3/6pfTmVtZ 8qPTNYgKrdqtoTo1pP63NdYMrqwT3UhPTSsaXrAPfezN6rdi70FJ8sxMqiPPuMxRvDSFpPFtdzk0Q gXratnhg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHQcT-008oZ6-SE; Mon, 16 Jan 2023 14:37:54 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E016530073F; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id B816220B75F29; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.768035056@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:37 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 4/7] x86/power: Inline write_cr[04]() References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191298758443345?= X-GMAIL-MSGID: =?utf-8?q?1755191298758443345?= Since we can't do CALL/RET until GS is restored and CR[04] pinning is of dubious value in this code path, simply write the stored values. Fixes: e81dc127ef69 ("x86/callthunks: Add call patching for call depth tracking") Reported-by: Joan Bruguera Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Kees Cook --- arch/x86/power/cpu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/arch/x86/power/cpu.c +++ b/arch/x86/power/cpu.c @@ -208,11 +208,11 @@ static void notrace __restore_processor_ #else /* CONFIG X86_64 */ native_wrmsrl(MSR_EFER, ctxt->efer); - native_write_cr4(ctxt->cr4); + asm volatile("mov %0,%%cr4": "+r" (ctxt->cr4) : : "memory"); #endif native_write_cr3(ctxt->cr3); native_write_cr2(ctxt->cr2); - native_write_cr0(ctxt->cr0); + asm volatile("mov %0,%%cr0": "+r" (ctxt->cr0) : : "memory"); /* Restore the IDT. */ native_load_idt(&ctxt->idt); From patchwork Mon Jan 16 14:25:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44198 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232556wrn; Mon, 16 Jan 2023 06:54:19 -0800 (PST) X-Google-Smtp-Source: AMrXdXthaZb5HmhIU84ZXohASZJeiOZaf02SgY3J9DtWj5BeFwOuznMIU9b30v24eS0zJsgb9LU8 X-Received: by 2002:a17:902:da91:b0:188:760f:d831 with SMTP id j17-20020a170902da9100b00188760fd831mr108290plx.7.1673880858926; Mon, 16 Jan 2023 06:54:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880858; cv=none; d=google.com; s=arc-20160816; b=eZKAg+ewIavfyQt7F37m92gzAiSL07KxyXBNav5fGLxvXV9cuO4kbOZUTmVpud7jnN QAqTLFcX+0wxrKCjP3p9AnNTRP4DhNx8ZxIIG8sA0JIA2tjD810BnuE2dKT3XiDcSkDU s3J+oKIDzo4YmxRA7WoF1kTGGxNGMj0i6CDYwPbwNolok5wI73c31mnPxyDBAQBZ9SS4 1YSF2wmWLdW6i0tdQPJrGfuXHnbysprXnDljkw9Bml1xOQZXEUzKgijhHKcz7Jt1eS4O oUxffBB1cZoH99cTssIoaQasByjwWGY3WHCjN8lB02tGVzqtChWCaNfPP2dLLbjne5rH 6uEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=Mc6JE/lSqhoI08ZCOd1MO1vmLEzmsp/ETRSpv4hRS+8=; b=PuM6Edf+r7xht7rFFDZF9abi5SxjQzhK2JkE4XAPVr/u4wkdu31WXYnAWgeWK13iSJ Y4qvzGy9HgS4zKR2XYTeedrSsiagyhZfNjcMLA58myOPfUV3gqBPz/pDViKbGWs2zquu NPPUXxbM0AJki9mnC64sq7zx+TWHY+nB0IUsaSXy2URuGGDNYC98E19p3ibi2CQj3e1M 15iam6+axgyPvWjQZty8tZmPF82uqAu2nhCKWqvtZcFjhRz1NMQrW40vhG5I2dWfS4YD czHusFPz+bybyiL/n2EjvScZ4DAtk1P9q1Z02MW3D4RfaJbVwSYEUi9uxwy4rRR2Z/U9 St8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=T1nrQTJt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t20-20020a170902b21400b0019491eabca0si1066455plr.98.2023.01.16.06.54.07; Mon, 16 Jan 2023 06:54:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=T1nrQTJt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231524AbjAPOw6 (ORCPT + 99 others); Mon, 16 Jan 2023 09:52:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229638AbjAPOwN (ORCPT ); Mon, 16 Jan 2023 09:52:13 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C43522BF13 for ; Mon, 16 Jan 2023 06:37:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=Mc6JE/lSqhoI08ZCOd1MO1vmLEzmsp/ETRSpv4hRS+8=; b=T1nrQTJttu/3Jnm9EJrYVDiMxC s34Y8QKh6Os9ywtC1Z1w2DiWPN6Xf9K5GkRtu6DtxWWsA8ej0VDBcu9iVcitzgXSTWCKMPsZsT/yv UyAjSANcObXIQvCe1J1cb2ZFm0PgpHHKh22ywjkdRZY49ET7hYiwVX9R/AkWzL156Ux4u1mcKbpW3 wubPrTdQVa+IM+KQkoM2pcN0SQ8YrHNTvKlSQpXlxOMZ0Z7Y2ts9qxviLI1ZgAHkTz7oQ5Z95SZu9 jTowvNgMdlt4jqLsw2QD7LpDyIe1S1Qwn8XSQdU41tuNFe9PXsjS6ESqii8W89Nd11lkxG5d+ERsF kQkbM/9A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHQcU-008oZF-RG; Mon, 16 Jan 2023 14:37:55 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 56162300C6F; Mon, 16 Jan 2023 15:37:39 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id B9FA620EF0A23; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.829076358@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:38 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 5/7] x86/callthunk: No callthunk for restore_processor_state() References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191295659186469?= X-GMAIL-MSGID: =?utf-8?q?1755191295659186469?= From: Joan Bruguera When resuming from suspend we don't have coherent CPU state, trying to do callthunks here isn't going to work. Specifically GS isn't set yet. Fixes: e81dc127ef69 ("x86/callthunks: Add call patching for call depth tracking") Signed-off-by: Joan Bruguera Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20230109040531.7888-1-joanbrugueram@gmail.com --- arch/x86/kernel/callthunks.c | 5 +++++ arch/x86/power/cpu.c | 3 +++ 2 files changed, 8 insertions(+) --- a/arch/x86/kernel/callthunks.c +++ b/arch/x86/kernel/callthunks.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -151,6 +152,10 @@ static bool skip_addr(void *dest) dest < (void*)hypercall_page + PAGE_SIZE) return true; #endif +#ifdef CONFIG_PM_SLEEP + if (dest == restore_processor_state) + return true; +#endif return false; } From patchwork Mon Jan 16 14:25:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44195 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232318wrn; Mon, 16 Jan 2023 06:53:43 -0800 (PST) X-Google-Smtp-Source: AMrXdXsTbAJNDZqv4IH3YYKRtO/CtIoua2umZRFeGqpMHkP3fI5p4yKIIo6svTD9ukFEm1xtdp/h X-Received: by 2002:a62:198a:0:b0:577:d10d:6eab with SMTP id 132-20020a62198a000000b00577d10d6eabmr87342344pfz.21.1673880822835; Mon, 16 Jan 2023 06:53:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880822; cv=none; d=google.com; s=arc-20160816; b=i5N4OBlSmjSuR8N6BxDL1d46vLsxFIHfPdVPoSU+taE8mJKaaoizqtctT077m/AhLG GZ/uK/jcv3qUuFhqcztkP0LSObpRKhGTu4uJ2+uIkWehSC/MZojAB4wxSyCE+IN2Kmkg qqspOLL+uA/SqZ8UO/8Z2hyemFI6PoQgRtrzAwJ8v7uwvE0maJmOahcFbTdwsT0ddWY6 LfD3ELi+T4UJj5s7hcu1IOo6yWpBaK83KQQvUiNm4hdmbl5Maow1aayWV+kDlyaByE2c AKnNGDCayJaOQrjP0CA5TpjijMVbtIOxD59fFZ+9XeHeIdniutS90PrA084S2DoZUWnn 2Jyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=eW1eJS20PBPNIc/x7hkOEbhV5Z70C96GEP7VlPQr3GE=; b=QFnsVF9DFNJsoBUDCF/5q8O+MnSjEM9KTDYW/CKON3kPW+KVzMVW7Qys11uJsodQtQ 1Xr0JgFGVUiPXqLyjuG4Jy1c5apnCM10bXtQoZrPD8ntsCUVMZmT5gSE5oPQTtnJ9j6b 4fmaQpjAsKE0xMRqatiBstIX+qdDBV3ZdSXMmkiTlZ3Ov/OK5SmKH1gaEEcHcwUqeHO5 Izo15ZgVsEbKHMH0VOT34x1QnZp6NSJo32Q6BHG8p1fex/FxOMKQIQYNqkJ191yO9xcf GRsiKKxkK5o//uEC3yxqsra7s7e4zIH3Ezd96j3Gkk+VJKh88JMpRXqFTCKs7VK2X7n0 3yJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=bkEUaHzN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z18-20020aa78892000000b00581c741f92asi31218490pfe.36.2023.01.16.06.53.30; Mon, 16 Jan 2023 06:53:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=bkEUaHzN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232298AbjAPOwr (ORCPT + 99 others); Mon, 16 Jan 2023 09:52:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231175AbjAPOwM (ORCPT ); Mon, 16 Jan 2023 09:52:12 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC0FB2BF17 for ; Mon, 16 Jan 2023 06:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=eW1eJS20PBPNIc/x7hkOEbhV5Z70C96GEP7VlPQr3GE=; b=bkEUaHzN3DU2Ozeq31w4gh/n2o R6F441bq+sbQaHOybPyF32cPvwYN0zd7oUr9R4K9RCdt8hDgWzWxaj2YBgmp4DkR9+D8F5Gs4dYOt 9lUNiL9h6dmbWadpBSYOt2SznC7hv64myxTtyscypeuWaItFMH/MD2H9N+RHOQ0ha2uXGYbfnTpp+ NghLUZyPGSsWGWKu+nqt7jQ7P4s9CMg2/V9GnkXxMdKljoX9C9dsDu4q8LkBDu52HDGHBkiTsolMw vvwHa2GRba1mlCVFQpftSw0QHpUi6oRrEyKnr2j/V09JyB9VCvK9zjxBViUdKEbhTQhWC7ktrQ+hh hJ6cgCPA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHQcU-008oZG-So; Mon, 16 Jan 2023 14:37:55 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 56119300C50; Mon, 16 Jan 2023 15:37:39 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id BEB6220D304B0; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.888786209@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:39 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 6/7] x86/power: Sprinkle some noinstr References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191257709913081?= X-GMAIL-MSGID: =?utf-8?q?1755191257709913081?= Ensure no compiler instrumentation sneaks in while restoring the CPU state. Specifically we can't handle CALL/RET until GS is restored. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/power/cpu.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) --- a/arch/x86/power/cpu.c +++ b/arch/x86/power/cpu.c @@ -192,7 +192,7 @@ static void fix_processor_context(void) * The asm code that gets us here will have restored a usable GDT, although * it will be pointing to the wrong alias. */ -static void notrace __restore_processor_state(struct saved_context *ctxt) +static __always_inline void __restore_processor_state(struct saved_context *ctxt) { struct cpuinfo_x86 *c; @@ -235,6 +235,13 @@ static void notrace __restore_processor_ loadsegment(fs, __KERNEL_PERCPU); #endif + /* + * Definitely wrong, but at this point we should have at least enough + * to do CALL/RET (consider SKL callthunks) and this avoids having + * to deal with the noinstr explosion for now :/ + */ + instrumentation_begin(); + /* Restore the TSS, RO GDT, LDT, and usermode-relevant MSRs. */ fix_processor_context(); @@ -276,10 +283,12 @@ static void notrace __restore_processor_ * because some of the MSRs are "emulated" in microcode. */ msr_restore_context(ctxt); + + instrumentation_end(); } /* Needed by apm.c */ -void notrace restore_processor_state(void) +void noinstr restore_processor_state(void) { __restore_processor_state(&saved_context); } From patchwork Mon Jan 16 14:25:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 44193 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1232272wrn; Mon, 16 Jan 2023 06:53:39 -0800 (PST) X-Google-Smtp-Source: AMrXdXvPfKDTXZsdz76HCd59lD0rVLRAgysKC/tpSovKYJi8xlOtXyo1/Lz+gwRzk7qS2wzd1BXf X-Received: by 2002:a17:90a:db0a:b0:229:4731:994d with SMTP id g10-20020a17090adb0a00b002294731994dmr10392155pjv.4.1673880818696; Mon, 16 Jan 2023 06:53:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673880818; cv=none; d=google.com; s=arc-20160816; b=yEJ8ize0ID3KPm44443Hikg6OvrEa+/rLWMBciWFm6XoihGjnSgQ6gbs+IT8Pj4i9S XtIHcmCvkDL3f8ukXq+eXJgpVlZWoGo5+VoczI7KCCzu1LujcL+nl1WI+NHe7H2h0A7u 7hqZZ4ArABh/lEgk8mvWceBva6t6rjUg5IhdzrJVYrCIAVel0JDpxkbvWEiPKqkHfuLQ 1xriwDGPPl0aSLNQ8jZ/218h+R0Z6pUpVNmPcUvwZUvTOPwD8aFIziShOa5qt2PW4QO+ HR+UjASyZ3fg6Vw7EKWOjbKzJKorTEH3U3JpohzaxK0IA1/MSSsDcFrEaMEDoEvOzxmE PLrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=w9+PAdJk73UH2Fl3uPAAs4wgk5kyO76VlH+ufUk/PX4=; b=L0QbieSvjQbFU5eZt8iv1Nhl8HzyRURXAcOlp2DPQBddeU1jjCNzxLEQWUDpYCnwfa CfG9RqsyJqB6T4OoztPavbgZlI5KorlLeq28mahpAjyrc4UnbPSTwDVS3Upvl8sRcEun ynrNMvIK8BbQJn0udM8kc7vnTmh8qC7d/kLt4FBLBhMLADZ0Rw9zCP9h4ymxCZklez8o tIdzHeQDr76/zU7A3d2HrjqGtAnsJff8ZwZdKBYPcRqCL0qzTPV9+Z8MEAHl3o36l4ca 7U+Jumf2JILf/I+ybTXeL7Xp63yv/SPPNEr6Pc7zuCdnW++aYRnyaaBKOKM4m8Sc6uxJ 9f7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=OB3RdkQh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h24-20020a63df58000000b004b45fca9020si22674163pgj.397.2023.01.16.06.53.26; Mon, 16 Jan 2023 06:53:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=OB3RdkQh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232114AbjAPOwj (ORCPT + 99 others); Mon, 16 Jan 2023 09:52:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230466AbjAPOwM (ORCPT ); Mon, 16 Jan 2023 09:52:12 -0500 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 241DD2BF0E for ; Mon, 16 Jan 2023 06:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=w9+PAdJk73UH2Fl3uPAAs4wgk5kyO76VlH+ufUk/PX4=; b=OB3RdkQh/z8SFHj5199h+5XxCG thtkjKdk+ckbxf7Iellx2ZelkT1vO3AvWAgAjOiq8fzpCJwR6AC2SkmBx8R0Lr5PfqM/r3BVOSesf iNTTF8J0Dh6dYSuq81Aos7ogKJWM8VOUSyQK3UNQsqCLKlLOI0+qQzX7Hru0M/PxVUOf8M3fiPwcL XUNEPBzCO1X90Eazd5t/tUXX6nnvwXa2NznHU7elGvs2Vf9jjgratRjuPAlLu2YOjbhnfusCUjccb 9glOoHYAYwuO41FGkPxDyw2EqP509qne4J5wYOpjcjdfHCeOJ6BFrHWxncvKtAlQKHBLhyYXy8uSD vFlY2FHg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1pHQc8-005csz-1j; Mon, 16 Jan 2023 14:37:33 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 56FD1300C9D; Mon, 16 Jan 2023 15:37:39 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id C252D20EF0A28; Mon, 16 Jan 2023 15:37:38 +0100 (CET) Message-ID: <20230116143645.948125465@infradead.org> User-Agent: quilt/0.66 Date: Mon, 16 Jan 2023 15:25:40 +0100 From: Peter Zijlstra To: x86@kernel.org, Joan Bruguera Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, Juergen Gross , "Rafael J. Wysocki" , xen-devel , Jan Beulich , Roger Pau Monne , Kees Cook , mark.rutland@arm.com, Andrew Cooper , =?utf-8?b?SsO2cmcgUsO2ZGVs?= , "H. Peter Anvin" Subject: [PATCH v2 7/7] PM / hibernate: Add minimal noinstr annotations References: <20230116142533.905102512@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755191253164681966?= X-GMAIL-MSGID: =?utf-8?q?1755191253164681966?= When resuming there must not be any code between swsusp_arch_suspend() and restore_processor_state() since the CPU state is ill defined at this point in time. Signed-off-by: Peter Zijlstra (Intel) --- kernel/power/hibernate.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) --- a/kernel/power/hibernate.c +++ b/kernel/power/hibernate.c @@ -280,6 +280,32 @@ __weak int arch_resume_nosmt(void) return 0; } +static noinstr int suspend_and_restore(void) +{ + int error; + + /* + * Strictly speaking swsusp_arch_suspend() should be noinstr too but it + * is typically written in asm, as such, assume it is good and shut up + * the validator. + */ + instrumentation_begin(); + error = swsusp_arch_suspend(); + instrumentation_end(); + + /* + * Architecture resume code 'returns' from the swsusp_arch_suspend() + * call and resumes execution here with some very dodgy machine state. + * + * Compiler instrumentation between these two calls (or in + * restore_processor_state() for that matter) will make life *very* + * interesting indeed. + */ + restore_processor_state(); + + return error; +} + /** * create_image - Create a hibernation image. * @platform_mode: Whether or not to use the platform driver. @@ -323,9 +349,7 @@ static int create_image(int platform_mod in_suspend = 1; save_processor_state(); trace_suspend_resume(TPS("machine_suspend"), PM_EVENT_HIBERNATE, true); - error = swsusp_arch_suspend(); - /* Restore control flow magically appears here */ - restore_processor_state(); + error = suspend_and_restore(); trace_suspend_resume(TPS("machine_suspend"), PM_EVENT_HIBERNATE, false); if (error) pr_err("Error %d creating image\n", error);