From patchwork Thu Jun 22 14:42:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111704 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5127003vqr; Thu, 22 Jun 2023 07:54:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4IyrqAewrP2+rfaQJMDIVomyE9gh8/9Pey7+zmrp2shVzz9xGDAlePQMWf7PyxlRc1Bcbb X-Received: by 2002:a17:902:ed44:b0:1b1:9218:6bf9 with SMTP id y4-20020a170902ed4400b001b192186bf9mr15240524plb.43.1687445689246; Thu, 22 Jun 2023 07:54:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687445689; cv=none; d=google.com; s=arc-20160816; b=uCk+HcAIYumfF0MXrsU7ToMja3KwPXOa/InYYJYN4rfqS+byXECA+bR2plNc2z3IWE 4ShDoFqDLmC4xsW3X1iLBck03eA/gSODY/87jfSV1wza7tkOLhG96O3lYL4Tdq8l0YYY TJvmJl7WGxY+jMwTAVfujj5uJ9hhtUPdMbpyN6D0bsWnmFXcoil3CQ1uydUiHU1eZIQQ bzWGie8U+5jktxthiLrWmbHL/pTAVOfhFHcxtXWtpLoy1WefyyZvAfTYFSyfAlu30/02 C3ViMPAVdo+wA8OL/3j+xDOOlTVpBLTiycnZuDHFqwBch/iNG8VTMCCUJkIFxsJST22n ulfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=FI1Ilf3TB6X2/hXhpNF04O2UYw1z64psa92hXMDi+uU=; b=aBj5hU6EU3ReYIynBZThMDReZ/xeYKYLOiN1AxwHMTEUdXHaolbopruZFhaBF2vcPp dYlSjgtN82nzYTqCsOK7xfZlLBJeNondviKrOOXfWD/Pj2f4WZOL8pZ8X9AW0qX+pkem godLAQd0613RiwAGmNKPJfYqhRCoDztjOzL2c8G/sSPEdf3DAOwIU6bwVHfmXMzQUync BiEQ4gitwC4x9VuZGPKvbI1dLNnUN5KhyCoYdDKEQ9nTBpt7fRZEKhhLrt+J8DMkfi9e agXzlApbtDYzMlYhGx6pIv58lgbnpSxbO752gKreaEDCC3ATQpVQQGBQhFIMxiRyinvt 5Ukg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b="P9y0b72/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c9-20020a170902b68900b001b22f31cce9si6339999pls.179.2023.06.22.07.54.37; Thu, 22 Jun 2023 07:54:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b="P9y0b72/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231962AbjFVOuy (ORCPT + 99 others); Thu, 22 Jun 2023 10:50:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231866AbjFVOug (ORCPT ); Thu, 22 Jun 2023 10:50:36 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0A891710 for ; Thu, 22 Jun 2023 07:50:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=FI1Ilf3TB6X2/hXhpNF04O2UYw1z64psa92hXMDi+uU=; b=P9y0b72/KnuWV4rOI284V4niL6 bwMBr6txDcKTZfI2iEfHs2xGzc/eWJhA03uX85c+n4Sb3ygdtxh8QExp0RxlQWMU3nN2/c7E6uHxN 0cAxuLq/lVGkHYAfET2O28dvzeaBT0DCD81LWUCzsKOrZ74nu8pXX5duHV5Z/kPxiuWpWgMeGcpkR qyPHy47E7o92Ur/nuv4xbRwe8CKPNBfiKe58MOg3+FjTG7c6on+cMT0MVSSrK4KtsOqqlzTj6oCk2 CpZP4Gy9bIwXpX9Wn9vM0leOnIODoXcP15tNloD2H9pvJEoEMrxo5N1ky7wGY9WfED8XDWAtxW8AM onUcA9jw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qCLdh-00FgfA-Rv; Thu, 22 Jun 2023 14:50:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 2C40D300338; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 15905209D8B39; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.360957723@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:19 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com Subject: [PATCH v2 1/6] x86/cfi: Extend {JMP,CAKK}_NOSPEC comment References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769415050918230469?= X-GMAIL-MSGID: =?utf-8?q?1769415050918230469?= With the introduction of kCFI these helpers are no longer equivalent to C indirect calls and should be used with care. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/include/asm/nospec-branch.h | 4 ++++ 1 file changed, 4 insertions(+) --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -234,6 +234,10 @@ * JMP_NOSPEC and CALL_NOSPEC macros can be used instead of a simple * indirect jmp/call which may be susceptible to the Spectre variant 2 * attack. + * + * NOTE: these do not take kCFI into account and are thus not comparable to C + * indirect calls, take care when using. The target of these should be an ENDBR + * instruction irrespective of kCFI. */ .macro JMP_NOSPEC reg:req #ifdef CONFIG_RETPOLINE From patchwork Thu Jun 22 14:42:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111727 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5142461vqr; Thu, 22 Jun 2023 08:13:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4w/9KZjJfMjIC22XJjMY6wuzbxpPStGMF1arjMQPF+6te17XlFD9+HCfdq9oogqToCL9F9 X-Received: by 2002:a05:6808:1987:b0:3a1:9540:feb1 with SMTP id bj7-20020a056808198700b003a19540feb1mr399174oib.14.1687446813389; Thu, 22 Jun 2023 08:13:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687446813; cv=none; d=google.com; s=arc-20160816; b=SbsPCwQLDPaTuaixrDOPwBW74zVsff3ZXiwNP67Ci2/7I+rz0F8hN18CzKOSVGwa3d Xa44ORCsYGAjmtSjxVh8Z6i3G9rLjBeyryjF6tSceZ5NI/7m0XKyVch68FtZu22IA8uA 2NlizuPG+5BLqCzjrFfOCSRQGWeCu22GVnuyssWWkkz2y1+NlDYqeYG9GPGdDM49D1r4 AzXAN4agpfmkPiWPvaIdg5CoXPMKeskskN9y30G9i2IkLjxxT5iSfDR5q8220LM8j2Py PauWqQeZndGuYWDBYPVhlb1n6wnY2tx1cv10/cidDuIccoItbFuDCoAFNq+uvF60+Xln RQqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=BTT+KoTXqCaNq9y13aV/Jw9DZObN70x0nFDB9z9hNio=; b=MOA5KdEkaeS3aD8HQC3A/8O9sW/6WyAPLg0fMLy07qNhPBBTk76XhO5RcNEUkeqepq 2cjAWkdpWOP8ALQEla+R28hByKTsL1avi8Z5qv1R+lKgi12TA/Y0kac7KLUXpIa/JtPB whANq/nuPtZ8VUGzl/+lRE1Ij2IzfKPbpxxSotJuxfoTbpHWcPBfJixbgImqdBZ3KWqY ZnFNwX/95rfFRUtfeo9klLwKIhYUeTeEjdeNaWBo44krMOBN4rLkhlZ8TmViEQTovpNK aMmDNdgVAMSe9Qcl9nZWeS9Rg88gSYQ9C0amSW88JqVlxqz779FcMc6qWm4aVvsfOt7+ uysA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b="fbvfH2/H"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u7-20020a17090ae00700b0025bf848104fsi2217503pjy.155.2023.06.22.08.13.20; Thu, 22 Jun 2023 08:13:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b="fbvfH2/H"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231946AbjFVOut (ORCPT + 99 others); Thu, 22 Jun 2023 10:50:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231403AbjFVOug (ORCPT ); Thu, 22 Jun 2023 10:50:36 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F16FA1995 for ; Thu, 22 Jun 2023 07:50:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=BTT+KoTXqCaNq9y13aV/Jw9DZObN70x0nFDB9z9hNio=; b=fbvfH2/HrnNfAVJ69LLHLM/iwi JHopjHpVTm/xSNYjjJbBfkdnDXhp1u+5qYEZbc5IkZjky+2ZsJGY+iFaRB3cXGtzB+VRbinoPb7Ae NQU4erYGvI7cq2fINx7mr3lkMpGzJ/3wAR8sVN/Nq1GFMaaQPd2uR1B6G5aU3fAuCpyFWsr2bVeso fjY6sp+vZe4i1aUc0irIpwEe/7HnqQMnIF83vXK6a624TlQCjR0K3GZlmpoRDbovKdlWqoPrT8sFw cGZaF1/rXWTilayNT6o95K6uZgbEyAyC1YhZ0I9+TsNHp7x9jrSlBYiUNTiPsEIrQGVdXu77Da++h jYYnKxvA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qCLdh-00FgfB-Rc; Thu, 22 Jun 2023 14:50:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 309B0300427; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 17D91209D8B3B; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.427441595@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:20 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com Subject: [PATCH v2 2/6] x86/alternative: Rename apply_ibt_endbr() References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769416229757459667?= X-GMAIL-MSGID: =?utf-8?q?1769416229757459667?= The current name doesn't reflect what it does very well. Signed-off-by: Peter Zijlstra (Intel) --- arch/um/kernel/um_arch.c | 2 +- arch/x86/include/asm/alternative.h | 2 +- arch/x86/include/asm/ibt.h | 2 +- arch/x86/kernel/alternative.c | 9 ++++++--- arch/x86/kernel/module.c | 2 +- 5 files changed, 10 insertions(+), 7 deletions(-) --- a/arch/um/kernel/um_arch.c +++ b/arch/um/kernel/um_arch.c @@ -437,7 +437,7 @@ void __init arch_cpu_finalize_init(void) os_check_bugs(); } -void apply_ibt_endbr(s32 *start, s32 *end) +void apply_seal_endbr(s32 *start, s32 *end) { } --- a/arch/x86/include/asm/alternative.h +++ b/arch/x86/include/asm/alternative.h @@ -96,7 +96,7 @@ extern void alternative_instructions(voi extern void apply_alternatives(struct alt_instr *start, struct alt_instr *end); extern void apply_retpolines(s32 *start, s32 *end); extern void apply_returns(s32 *start, s32 *end); -extern void apply_ibt_endbr(s32 *start, s32 *end); +extern void apply_seal_endbr(s32 *start, s32 *end); extern void apply_fineibt(s32 *start_retpoline, s32 *end_retpoine, s32 *start_cfi, s32 *end_cfi); --- a/arch/x86/include/asm/ibt.h +++ b/arch/x86/include/asm/ibt.h @@ -34,7 +34,7 @@ /* * Create a dummy function pointer reference to prevent objtool from marking * the function as needing to be "sealed" (i.e. ENDBR converted to NOP by - * apply_ibt_endbr()). + * apply_seal_endbr()). */ #define IBT_NOSEAL(fname) \ ".pushsection .discard.ibt_endbr_noseal\n\t" \ --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -803,7 +803,7 @@ static void __init_or_module poison_endb /* * Generated by: objtool --ibt */ -void __init_or_module noinline apply_ibt_endbr(s32 *start, s32 *end) +void __init_or_module noinline apply_seal_endbr(s32 *start, s32 *end) { s32 *s; @@ -818,7 +818,7 @@ void __init_or_module noinline apply_ibt #else -void __init_or_module apply_ibt_endbr(s32 *start, s32 *end) { } +void __init_or_module apply_seal_endbr(s32 *start, s32 *end) { } #endif /* CONFIG_X86_KERNEL_IBT */ @@ -1565,7 +1565,10 @@ void __init alternative_instructions(voi */ callthunks_patch_builtin_calls(); - apply_ibt_endbr(__ibt_endbr_seal, __ibt_endbr_seal_end); + /* + * Seal all functions that do not have their address taken. + */ + apply_seal_endbr(__ibt_endbr_seal, __ibt_endbr_seal_end); #ifdef CONFIG_SMP /* Patch to UP if other cpus not imminent. */ --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -358,7 +358,7 @@ int module_finalize(const Elf_Ehdr *hdr, } if (ibt_endbr) { void *iseg = (void *)ibt_endbr->sh_addr; - apply_ibt_endbr(iseg, iseg + ibt_endbr->sh_size); + apply_seal_endbr(iseg, iseg + ibt_endbr->sh_size); } if (locks) { void *lseg = (void *)locks->sh_addr; From patchwork Thu Jun 22 14:42:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111711 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5129712vqr; Thu, 22 Jun 2023 07:59:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68ET/S64jLFW0jmeD1Bs+Vij0Sr2hnTRfS+FtQKGZFwTR8E5CMSOx9gTzxgkLt7dg0jcGp X-Received: by 2002:a17:90a:69a2:b0:25b:f9ce:d8df with SMTP id s31-20020a17090a69a200b0025bf9ced8dfmr15309589pjj.8.1687445975090; Thu, 22 Jun 2023 07:59:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687445975; cv=none; d=google.com; s=arc-20160816; b=q+YGiw12dzU3pBGUL8dvTuM+bgpgAz/0TeSvhnoLyHklEkA427mU+9+MeECCDw5HFc rrAMtK8Gj/O6CI7Jr3iDZsNZYzCNdnGR9qMg62H8xJ8tTJqN2QDAJF0SXtWf8RkKX83E BIzPIzyUn2eVllSkl1NPvjW/fCauhhbjLIebpv5CmxCYO+NSY/kt1/r031UnisjMQE08 aJHkc3SwdR1DEZKJWNPMdK/b+jF0//TR9YrA/dFRM9MucfZldAQYMKKWgLGD0esKVbEA UnXoQaHeNy2LXZha/w+0ZDG+mKqmJTFiKC8PoiwdpThBzDT08LTGYN9W606yMttfTQ5Y RzHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=QJieZYDWzFCRvSf0wOeMBiP6eKWnKstpX1vra/Nb2RE=; b=ptJG6UIft/qS4WyyzPM4tdAygU4kx+AyEGiJe41LWzvk2liQCiCH1mfgVJyDjwo5Mj A2wBljq8GUnJxncsK54iuIClFtDOy8GwDB+KtZYLaKBZ2QyFWhluNSfBjr0coOwUS7Oz gCFGqBw/RGEZui6jzq+sCuECOvNTZPVOtfDKfSyRTBYWz8P1rhvCNih0snmpvSawmny0 KKN7Z8Z9ZhMcRLx9KgVerD0oc6gVSFzzfKadA/a816UBZZcG0yw/HKsei9/24enkbfe8 wAvbX/WYfRY/1aChWDtqL/e29/QiRt1Rq7Gmh0wL3n4pUpKR/SEJdXKs0C9iJhIeXTYN eBFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=t5FAyWdS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 4-20020a630204000000b0054fe7a4c49esi6618667pgc.824.2023.06.22.07.59.22; Thu, 22 Jun 2023 07:59:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=t5FAyWdS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231968AbjFVOvB (ORCPT + 99 others); Thu, 22 Jun 2023 10:51:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231881AbjFVOug (ORCPT ); Thu, 22 Jun 2023 10:50:36 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BACD19B7 for ; Thu, 22 Jun 2023 07:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=QJieZYDWzFCRvSf0wOeMBiP6eKWnKstpX1vra/Nb2RE=; b=t5FAyWdSCA8MhmTF70mRlJ8+bF PtTAVofzOV0VUJ4IiR6Mj3h2bermXWeNMQu6CTVlEUAHj7O9WATPVBktgOnDycDJWu9ZAh6nY7JXL Kci/BqJJMsb5tUV3YqtNL2Q/x7x0IZFV0DMecRpsvSf127dAPsa0FsRHtSaRyTX6RVBa49KwKbFkU f5rNyS8G8XvzWrM82W1/Q2pUh6uZ7SSeSYKMOKNY9ugxyEXvyyV/d2AZHHsChCjBAMmsdYuasMPTp Vl5cd+fqKKEiEuTe9gWblRPAtTOjqmUdvLcptOBXcBHu2oGQxRO2B6ohEXYnT7IbnHN/4TQrCzJnr kwc3bnog==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qCLdh-00FgfC-Ru; Thu, 22 Jun 2023 14:50:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 3390C3006D5; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 1C78A209D8B3D; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.494426891@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:21 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com Subject: [PATCH v2 3/6] x86/cfi: Extend ENDBR sealing to kCFI References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769415351008113374?= X-GMAIL-MSGID: =?utf-8?q?1769415351008113374?= Kees noted that IBT sealing could be extended to kCFI. Fundamentally it is the list of functions that do not have their address taken and are thus never called indirectly. It doesn't matter that objtool uses IBT infrastructure to determine this list, once we have it it can also be used to clobber kCFI hashes and seal kCFI indirect calls. Suggested-by: Kees Cook Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/kernel/alternative.c | 44 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -778,6 +778,8 @@ void __init_or_module noinline apply_ret #ifdef CONFIG_X86_KERNEL_IBT +static void poison_cfi(void *addr); + static void __init_or_module poison_endbr(void *addr, bool warn) { u32 endbr, poison = gen_endbr_poison(); @@ -802,6 +804,9 @@ static void __init_or_module poison_endb /* * Generated by: objtool --ibt + * + * Seal the functions for indirect calls by clobbering the ENDBR instructions + * and the kCFI hash value. */ void __init_or_module noinline apply_seal_endbr(s32 *start, s32 *end) { @@ -812,7 +817,7 @@ void __init_or_module noinline apply_sea poison_endbr(addr, true); if (IS_ENABLED(CONFIG_FINEIBT)) - poison_endbr(addr - 16, false); + poison_cfi(addr - 16); } } @@ -1177,6 +1182,41 @@ static void __apply_fineibt(s32 *start_r pr_err("Something went horribly wrong trying to rewrite the CFI implementation.\n"); } +static inline void poison_hash(void *addr) +{ + *(u32 *)addr = 0; +} + +static void poison_cfi(void *addr) +{ + switch (cfi_mode) { + case CFI_FINEIBT: + /* + * __cfi_\func: + * osp nopl (%rax) + * subl $0, %r10d + * jz 1f + * ud2 + * 1: nop + */ + poison_endbr(addr, false); + poison_hash(addr + fineibt_preamble_hash); + break; + + case CFI_KCFI: + /* + * __cfi_\func: + * movl $0, %eax + * .skip 11, 0x90 + */ + poison_hash(addr + 1); + break; + + default: + break; + } +} + #else static void __apply_fineibt(s32 *start_retpoline, s32 *end_retpoline, @@ -1184,6 +1224,8 @@ static void __apply_fineibt(s32 *start_r { } +static void poison_cfi(void *addr) { } + #endif void apply_fineibt(s32 *start_retpoline, s32 *end_retpoline, From patchwork Thu Jun 22 14:42:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111706 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5129067vqr; Thu, 22 Jun 2023 07:58:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6KQLx8Sq094No0E24smqFCT5k33DvNUYolk4B7kXS+2AqQCDyYWp++/TgwhOPrNelByUu3 X-Received: by 2002:a05:6a20:12d3:b0:122:7e90:61c2 with SMTP id v19-20020a056a2012d300b001227e9061c2mr10170823pzg.9.1687445909877; Thu, 22 Jun 2023 07:58:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687445909; cv=none; d=google.com; s=arc-20160816; b=qjUXRZFMPqcXKPag1JSJpxRD6rz4uUeEi406pRVWwnu5L+j8YKViOawmfJ9l0c6DrW S034gRhth33Gw1O3voUYfpSP8jO8Ker1qynx8xGDaPLtlgyiPDxeTYrz916x7a1huHgZ M6Q5GXEZquP2FXWWhL2mRBrQFbfju3Ht3ZL+uCPG8Y3HhtlCpxlx9WgrhLaxZoZEhodY gvTWU+s70iAUgWSB1JTLbY+X0t2s0D+zRy4UkzDMzlBgqc5H6TU0lRv6igB53B/KZdOz fZqF3YMz86KlnUchivk4iF1c9EhvhQiuSzLxdJFrjmlo4DtmCRDoHBtfRNR0Lf8qi3fU q6IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=LGsdeHWEpXVTOOsTrfyXxkvr7QjEX8WnxEDVS86hVu0=; b=Vqv1zsCAai+Z9SGOS6Zs2702c4BqkOM1xA1vrTIaxs+cKgQzGQz8xZI1eJhH4WcfhC aGSME/e7f5HHSyk9vP0L35/vQyudoCXJNu4tHOR72kdl8hmhWhifonOmAZRyQmtD++tC 34g6ONtCBCIydn1PkN4LozVQA7LEn+dGRpirvHtK1Ur/yQZrGaWKdo2WCj1GUzMKw+xF iXWIiL31dMqfm2AYoMoFvDB6c8rA+lCk7F7OlOsdduopDEUzkM2l9r1jTjn3Ora7xUoK pFLC/ErxUnRdHdOyZZziel5a7qfabHFc5djmR2BIwNzNeShiRCkzgUEzaPQSYs02F+KM pDDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=Vc1PCzlg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x28-20020aa7941c000000b00666b95e38efsi5223064pfo.34.2023.06.22.07.58.16; Thu, 22 Jun 2023 07:58:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=Vc1PCzlg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229813AbjFVOui (ORCPT + 99 others); Thu, 22 Jun 2023 10:50:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231661AbjFVOuf (ORCPT ); Thu, 22 Jun 2023 10:50:35 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A88CAE57 for ; Thu, 22 Jun 2023 07:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=LGsdeHWEpXVTOOsTrfyXxkvr7QjEX8WnxEDVS86hVu0=; b=Vc1PCzlgv53QVpsR+WDW+RQqb6 hK7KNjrcjl+OM0sSOaXWHo1TLrdWF6Ft+sYwXsr6/tvfbojEmVncYsiUxhSWg/ccffx5Vp5h+E65l iTOfakYBU57ARcTK5pLhxLX1TwdYs+p+rSAi35g9L7X/LffRI+NRdtOrMWDspVfMgzmS1h0vWfayf 92OwwNzhL4uiLlFDCTrtaT0x3lah4hMBUKxB/5tLAOxbTKp31oapEesD7UR1+ROiQbXFVLC2HgORi SQp/YPBNcgr0rmbQmi37UXBKkT/rDdl8fS3inb9MhS5FPUIHGwQtdpek14S6NsTFIbr4tFvPPh9nA mhbSdVtQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qCLdh-001CzP-1B; Thu, 22 Jun 2023 14:50:25 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 3999330075E; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 20EF6209D8B3E; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.561264520@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:22 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com Subject: [PATCH v2 4/6] x86/32: Remove schedule_tail_wrapper() References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769415282413465281?= X-GMAIL-MSGID: =?utf-8?q?1769415282413465281?= From: Brian Gerst The unwinder expects a return address at the very top of the kernel stack just below pt_regs and before any stack frame is created. Instead of calling a wrapper, set up a return address as if ret_from_fork() was called from the syscall entry code. Signed-off-by: Brian Gerst Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20230622120750.5549-2-brgerst@gmail.com --- arch/x86/entry/entry_32.S | 32 ++++++++++---------------------- 1 file changed, 10 insertions(+), 22 deletions(-) --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -720,26 +720,6 @@ SYM_CODE_END(__switch_to_asm) .popsection /* - * The unwinder expects the last frame on the stack to always be at the same - * offset from the end of the page, which allows it to validate the stack. - * Calling schedule_tail() directly would break that convention because its an - * asmlinkage function so its argument has to be pushed on the stack. This - * wrapper creates a proper "end of stack" frame header before the call. - */ -.pushsection .text, "ax" -SYM_FUNC_START(schedule_tail_wrapper) - FRAME_BEGIN - - pushl %eax - call schedule_tail - popl %eax - - FRAME_END - RET -SYM_FUNC_END(schedule_tail_wrapper) -.popsection - -/* * A newly forked process directly context switches into this address. * * eax: prev task we switched from @@ -748,7 +728,13 @@ SYM_FUNC_END(schedule_tail_wrapper) */ .pushsection .text, "ax" SYM_CODE_START(ret_from_fork) - call schedule_tail_wrapper + /* return address for the stack unwinder */ + pushl $.Lsyscall_32_done + FRAME_BEGIN + + pushl %eax + call schedule_tail + addl $4, %esp testl %ebx, %ebx jnz 1f /* kernel threads are uncommon */ @@ -757,7 +743,9 @@ SYM_CODE_START(ret_from_fork) /* When we fork, we trace the syscall return in the child, too. */ movl %esp, %eax call syscall_exit_to_user_mode - jmp .Lsyscall_32_done + + FRAME_END + RET /* kernel thread */ 1: movl %edi, %eax From patchwork Thu Jun 22 14:42:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111729 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5144925vqr; Thu, 22 Jun 2023 08:16:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ47mjDvupmfpKNTPQZpFwEkUD2DHScEZPbFtD1Z8kVw4ZEIcJxx+Z0iWYrJpI8FPJ73Nlla X-Received: by 2002:a17:903:188:b0:1b2:4b28:793c with SMTP id z8-20020a170903018800b001b24b28793cmr23167099plg.29.1687447009429; Thu, 22 Jun 2023 08:16:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687447009; cv=none; d=google.com; s=arc-20160816; b=z7NJhmvFtXLSU0+dyx5dvwwG87mLHDJWODpdx0Tbe6YHv3f+lU4ITRngJfphs8xpu6 aMbRrqswh6B6gm+W5VF1O/dptrtEEN8DioP07Qt7Nb59qjSbxjpd2XhCelDUnBlu8uFj u8C+6NS0MBdPofGLLzSZvsPE/JCdwZGuR7tHoY0GLCGdj3gc7CIzI3RhSYzNOoLj1gNa GU9ZAaor0PX/WAlZNVARlgWS4ynwTcxhc36GVvZtYBldH6p1j8EwUifj07WXhqLv/9yd F+bK+SAtRA/EzqfeiNjN3E9frEJf0LHhpZ3bZZq30Mk5ftRigXbvnEL7VvcEZqOXl2nn HPcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=MiQXciH6QFUFwWGYzaBS9NluMPpHBbDeC6AdeQsFuMU=; b=JstLuhKvTc668Y1xZhHY+lVvi7jNIzXnE+Qe42t5XuFaIu+7tl0F1CaUNj7GLkECHj SCQz/1MsGEMmjQcSuzHieEMWqgY5qgTSR8aEinYbImR2DoW6kl2kbRxIJAEHFhhidW/R AxIt8kyu1lvJ5u6nUJWQbDsulDPSgIk/08nzYdZs1Hiw0ZxglPhkORcKGdS09oR7Bp+0 rsfKYLj1mgjGzFjumTu1scIMmxhaY8HBBftXppu/a+CM/0Qx/WAgzorri17F5M9jWFMh 17Js/fbZslDgLLM/pnG0hIL/P/Txo8xftp6J9UC9014wWtfsL8MYFaYmelslmh9FaEIs hIuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=PtHif4Ew; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o8-20020a170902d4c800b001ab147e4543si7757665plg.418.2023.06.22.08.16.36; Thu, 22 Jun 2023 08:16:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=PtHif4Ew; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231922AbjFVOuk (ORCPT + 99 others); Thu, 22 Jun 2023 10:50:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231689AbjFVOuf (ORCPT ); Thu, 22 Jun 2023 10:50:35 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1C68E7E for ; Thu, 22 Jun 2023 07:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=MiQXciH6QFUFwWGYzaBS9NluMPpHBbDeC6AdeQsFuMU=; b=PtHif4EwoZQlLcVEpXtJWFpyNU GdRXNWxLm1HEbFx6U8QdziOOM12seJlecoI+n0+mMvvz+/ztcAigWigR+dwcmRxzZck1qORCqq8sT q9LWokguxzNnDdk1UiG+CModRKKD0c4DKPlJaFTWzm0C7l8IHM6hisuT5OhYEdnDwAXkW9f17rvQy mAu1O26d9XVgA8jeuPMbkBDAH+tKiaHWJLB1ejxjnjq2fbgxHbC/WyILXuGZu5R/oWOTCNeJ50V4x JeDs/HkDewn8XeSyWsT/9xXKCO8zkVNfxVgYDAq4vLVGHbgUSgZ5QalZci7uTz6TTRu6jQRYTgYgs UHklGcLg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qCLdh-001CzU-36; Thu, 22 Jun 2023 14:50:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 771CB300794; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 24D9B209D8B3C; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.629918727@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:23 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com Subject: [PATCH v2 5/6] x86: Rewrite ret_from_fork() in C References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769416435383935750?= X-GMAIL-MSGID: =?utf-8?q?1769416435383935750?= From: Brian Gerst When kCFI is enabled, special handling is needed for the indirect call to the kernel thread function. Rewrite the ret_from_fork() function in C so that the compiler can properly handle the indirect call. Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Brian Gerst Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20230622120750.5549-3-brgerst@gmail.com --- arch/x86/entry/entry_32.S | 30 +++++++----------------------- arch/x86/entry/entry_64.S | 35 ++++++++++------------------------- arch/x86/include/asm/switch_to.h | 4 +++- arch/x86/kernel/process.c | 22 +++++++++++++++++++++- 4 files changed, 41 insertions(+), 50 deletions(-) --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -727,37 +727,21 @@ SYM_CODE_END(__switch_to_asm) * edi: kernel thread arg */ .pushsection .text, "ax" -SYM_CODE_START(ret_from_fork) +SYM_CODE_START(ret_from_fork_asm) /* return address for the stack unwinder */ pushl $.Lsyscall_32_done FRAME_BEGIN - pushl %eax - call schedule_tail + /* prev already in EAX */ + movl %esp, %edx /* regs */ + movl %ebx, %ecx /* fn */ + pushl %edi /* fn_arg */ + call ret_from_fork addl $4, %esp - testl %ebx, %ebx - jnz 1f /* kernel threads are uncommon */ - -2: - /* When we fork, we trace the syscall return in the child, too. */ - movl %esp, %eax - call syscall_exit_to_user_mode - FRAME_END RET - - /* kernel thread */ -1: movl %edi, %eax - CALL_NOSPEC ebx - /* - * A kernel thread is allowed to return here after successfully - * calling kernel_execve(). Exit to userspace to complete the execve() - * syscall. - */ - movl $0, PT_EAX(%esp) - jmp 2b -SYM_CODE_END(ret_from_fork) +SYM_CODE_END(ret_from_fork_asm) .popsection SYM_ENTRY(__begin_SYSENTER_singlestep_region, SYM_L_GLOBAL, SYM_A_NONE) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -284,36 +284,21 @@ SYM_FUNC_END(__switch_to_asm) * r12: kernel thread arg */ .pushsection .text, "ax" - __FUNC_ALIGN -SYM_CODE_START_NOALIGN(ret_from_fork) +SYM_CODE_START(ret_from_fork_asm) UNWIND_HINT_END_OF_STACK ANNOTATE_NOENDBR // copy_thread CALL_DEPTH_ACCOUNT - movq %rax, %rdi - call schedule_tail /* rdi: 'prev' task parameter */ - testq %rbx, %rbx /* from kernel_thread? */ - jnz 1f /* kernel threads are uncommon */ + /* return address for the stack unwinder */ + pushq $swapgs_restore_regs_and_return_to_usermode + UNWIND_HINT_FUNC -2: - UNWIND_HINT_REGS - movq %rsp, %rdi - call syscall_exit_to_user_mode /* returns with IRQs disabled */ - jmp swapgs_restore_regs_and_return_to_usermode - -1: - /* kernel thread */ - UNWIND_HINT_END_OF_STACK - movq %r12, %rdi - CALL_NOSPEC rbx - /* - * A kernel thread is allowed to return here after successfully - * calling kernel_execve(). Exit to userspace to complete the execve() - * syscall. - */ - movq $0, RAX(%rsp) - jmp 2b -SYM_CODE_END(ret_from_fork) + movq %rax, %rdi /* prev */ + movq %rsp, %rsi /* regs */ + movq %rbx, %rdx /* fn */ + movq %r12, %rcx /* fn_arg */ + jmp ret_from_fork +SYM_CODE_END(ret_from_fork_asm) .popsection .macro DEBUG_ENTRY_ASSERT_IRQS_OFF --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -12,7 +12,9 @@ struct task_struct *__switch_to_asm(stru __visible struct task_struct *__switch_to(struct task_struct *prev, struct task_struct *next); -asmlinkage void ret_from_fork(void); +asmlinkage void ret_from_fork_asm(void); +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg); /* * This is the structure pointed to by thread.sp for an inactive task. The --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -134,6 +135,25 @@ static int set_new_tls(struct task_struc return do_set_thread_area_64(p, ARCH_SET_FS, tls); } +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg) +{ + schedule_tail(prev); + + /* Is this a kernel thread? */ + if (unlikely(fn)) { + fn(fn_arg); + /* + * A kernel thread is allowed to return here after successfully + * calling kernel_execve(). Exit to userspace to complete the + * execve() syscall. + */ + regs->ax = 0; + } + + syscall_exit_to_user_mode(regs); +} + int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) { unsigned long clone_flags = args->flags; @@ -149,7 +169,7 @@ int copy_thread(struct task_struct *p, c frame = &fork_frame->frame; frame->bp = encode_frame_pointer(childregs); - frame->ret_addr = (unsigned long) ret_from_fork; + frame->ret_addr = (unsigned long) ret_from_fork_asm; p->thread.sp = (unsigned long) fork_frame; p->thread.io_bitmap = NULL; p->thread.iopl_warn = 0; From patchwork Thu Jun 22 14:42:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 111709 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5129670vqr; Thu, 22 Jun 2023 07:59:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5PXlp/WOtmBHJIIBdm+5+AeYNVKDKnMqRIgJmrRUe9CaLGE4+8hxN71PEv7/Zl7gtTCzMG X-Received: by 2002:a17:902:db04:b0:1b6:4bc2:74bc with SMTP id m4-20020a170902db0400b001b64bc274bcmr17008900plx.2.1687445970319; Thu, 22 Jun 2023 07:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687445970; cv=none; d=google.com; s=arc-20160816; b=Iz8MGb38iJo+y1wRMH2XnvI6gOVoE7WI9Ld2R4c2Dd7NK+yTaLFyeSAihqxHIBYKa6 0HARvIq/7w5NaHOoCBxweC2hk+4ykQW4vk8PERNMrjzLYHeXtJmPro3WCu6iQ4Zogp6y kIHbz7vMgVsrXDNsvVqi+CZf16mcoV2UJpBroj9sVwgJL9hRwe38YmKMk6ocRA3skPB/ +mXm4Y8HyBUkNnFeKNP0MaC9XWkxeGvj1k6XG8ju1uIFsAnT4qZ0OMiGhksQGkBRO6iK o+AsyAJ/DaHR287wAiyCygB/0oufRhq8KZQtaUNdKXXR+53XaF//qzy58tMnHDn0L9DK v0mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=r2emQb1KTou2w3E9LTe7OUggXTTKEQJMuYJi6cUiILo=; b=f9oTvQ5RVoiFxH4kwy5yqUcUXr2CDvDYITmz68tDVgN3uScNtb6ZACkOdCtTwMAlob 0FwBzg1ClPvE3omEl4hnfkW5dGmPMFga88CyPvQay6m6dUBTGWhHwKpJXTXALiUbFuug bcBdxuBhrAcX8Y8WvSm0feQCntD1ayNvt/tKFf+joTepnv2+SQrSMXqTSxDjVocRzG21 +i3wJrLEhYaVjxfGiMXOt7hqQzkp6XYOjzEgNAUmu9BCw5csrl0N5VSSfPkX1L1qpd9L LDimXLwetp+4NNOCfNgAj6EktpmsY5i41LYTix/ZioLa7kbtMRt6lACrgSGWcy1mibMp XMBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=ojMRDX4Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b16-20020a170903229000b001a6e1b073cdsi7528516plh.639.2023.06.22.07.59.17; Thu, 22 Jun 2023 07:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=ojMRDX4Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231950AbjFVOvG (ORCPT + 99 others); Thu, 22 Jun 2023 10:51:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231913AbjFVOuj (ORCPT ); Thu, 22 Jun 2023 10:50:39 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2462FE7E for ; Thu, 22 Jun 2023 07:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=r2emQb1KTou2w3E9LTe7OUggXTTKEQJMuYJi6cUiILo=; b=ojMRDX4ZBEo3NUsxlNBhbZLvLt fkA47p2jhqe8lzrjurlZD2De4pLm2TykWc791yVF/zL0xFJOdfWxq++Veod7TQ+XhUdasBxrjX8nN OPAOuz0MCUzYEi5uA94oEvnGBs/JxIEIewilpDBPNdDupmEPS2IQkauXWZ1vv9BQ0B4FqjdrHr9/k uz//mQ/fof1UKUzFvRn3jAOYUN6rNV20xLBwELPIXwHorcoG27Mi7SDZOq2BCt3HWvJBLHzwNlkWM jBEnghKq3fIWQ5vkBpYn7CMxzJOb13s5sOeop9IS55tSD/6RdDNLVPyBCkK7oQ7E5V7V/TqIP49tf wSh7KL6A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qCLdh-00FgfE-V5; Thu, 22 Jun 2023 14:50:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 7716F300790; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 29E62209FB79B; Thu, 22 Jun 2023 16:50:24 +0200 (CEST) Message-ID: <20230622144321.696656240@infradead.org> User-Agent: quilt/0.66 Date: Thu, 22 Jun 2023 16:42:24 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, brgerst@gmail.com, "Milburn, Alyssa" Subject: [PATCH v2 6/6] x86/fineibt: Poison ENDBR at +0 References: <20230622144218.860926475@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769415345378388956?= X-GMAIL-MSGID: =?utf-8?q?1769415345378388956?= Alyssa noticed that when building the kernel with CFI_CLANG+IBT and booting on IBT enabled hardware to obtain FineIBT, the indirect functions look like: __cfi_foo: endbr64 subl $hash, %r10d jz 1f ud2 nop 1: foo: endbr64 This is because the compiler generates code for kCFI+IBT. In that case the caller does the hash check and will jump to +0, so there must be an ENDBR there. The compiler doesn't know about FineIBT at all; also it is possible to actually use kCFI+IBT when booting with 'cfi=kcfi' on IBT enabled hardware. Having this second ENDBR however makes it possible to elide the CFI check. Therefore, we should poison this second ENDBR when switching to FineIBT mode. Fixes: 931ab63664f0 ("x86/ibt: Implement FineIBT") Reported-by: "Milburn, Alyssa" Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Acked-by: Kees Cook Link: https://lore.kernel.org/r/20230615193722.194131053@infradead.org --- arch/x86/kernel/alternative.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -1063,6 +1063,17 @@ static int cfi_rewrite_preamble(s32 *sta return 0; } +static void cfi_rewrite_endbr(s32 *start, s32 *end) +{ + s32 *s; + + for (s = start; s < end; s++) { + void *addr = (void *)s + *s; + + poison_endbr(addr+16, false); + } +} + /* .retpoline_sites */ static int cfi_rand_callers(s32 *start, s32 *end) { @@ -1157,14 +1168,19 @@ static void __apply_fineibt(s32 *start_r return; case CFI_FINEIBT: + /* place the FineIBT preamble at func()-16 */ ret = cfi_rewrite_preamble(start_cfi, end_cfi); if (ret) goto err; + /* rewrite the callers to target func()-16 */ ret = cfi_rewrite_callers(start_retpoline, end_retpoline); if (ret) goto err; + /* now that nobody targets func()+0, remove ENDBR there */ + cfi_rewrite_endbr(start_cfi, end_cfi); + if (builtin) pr_info("Using FineIBT CFI\n"); return;