From patchwork Thu Jun 15 19:35:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 108693 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp881839vqr; Thu, 15 Jun 2023 12:58:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7rPdcxmUxyvtcoco9732082HGdeb8Ls3ZaMDasJxBtvEdi4N1sNjcm4j48HvTe7f3A8zXQ X-Received: by 2002:a17:907:7248:b0:974:61dc:107c with SMTP id ds8-20020a170907724800b0097461dc107cmr87010ejc.44.1686859117407; Thu, 15 Jun 2023 12:58:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686859117; cv=none; d=google.com; s=arc-20160816; b=t12HVR/JUIxtWFss+pMGFM2WgFAYCpm6LDDn+STkODHiqIPD9YtZDekxbRHbvU1sG6 ZvTWGHU51x078FW0WrwOUbyTC/0WkrNoMLGH3Dgervk2fijNFquoCiMtRkr5WjWoH3PX 16QqrILJwMGTWhQqZeez802iq/OxSiCXIkmnk8XOBw5ddBcZhBS0n+aS76TQIjYbhrMv xzXDoSnwflNZHzuED4/f6A5h8DcuVyZnME+FYTzvLjc59u7Kjrp2YJwa72sKHZE27SDe HlHc0gMCpC6i83Qu5m6jQOyLqUaHyfZjpMVJwyZVX6CG7xuJQ/T9WFzApvfmYiIhu1Ln axoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=BhX7pkXejcd+7jOdak4TeKh4gofIP/p6FvONT7yMD5Q=; b=qhFo0pmXLjKnbNmbdRZBk55KFNEXQMHeFlolFenUgrJyjBzHsQ2rww0eknmDEdQxQi DrZyji5a6bGgwOp/GUyja72xq+V1RRM1Agp5rqmANVohPSllTMiKj5WpkprEd6183xRW +PyC11c+5mKTzc0868zdUkxnXCRLIkyJ+ZtOw7i8rtP8xTpUrtwbEIon9RqZRlWNIpvY MiTmroo0tcMS2VEV9kmIRZxo7hpWc39baOfjoDjgszjW1ckR+ykFRp4KfaeVWsz0K94X LzfEWEL+A5It2KBWtYVjPmiUJ22wbrLi4PWY8xsgeyl8OxBpaUjG96XORJXUYPcfs6nd 8fkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b="Luncf/CU"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i21-20020a170906251500b0094f93169acdsi9519509ejb.181.2023.06.15.12.58.12; Thu, 15 Jun 2023 12:58:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b="Luncf/CU"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbjFOTkr (ORCPT + 99 others); Thu, 15 Jun 2023 15:40:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229694AbjFOTkn (ORCPT ); Thu, 15 Jun 2023 15:40:43 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C66B2954 for ; Thu, 15 Jun 2023 12:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=BhX7pkXejcd+7jOdak4TeKh4gofIP/p6FvONT7yMD5Q=; b=Luncf/CUiCXEJyaixzGv2qhSHY FpBEa0sP9GHb+kSvFbCdnhUAl6ex0IbnWIihLc+vrU5Pw52VWmVi1dYIF5rgUcq/yBbQ5fnBT7C0I 3wroRla56ysNYuNVj7s6Ut8oxsy9oQhuc+xhs6KBlF8eUg6x7JKUC0cX1ZsK5DDlHv12G98EZjay1 cpv9C9FULKKGfXA4bevZK24nVZmoUtTU5Yo4WNZjwGTMnKbQQWdG5EqCJVOkajdE1H0p+FBVsmvSu YWvrH7dT6MjxhkaDEfbxUFnp+PZROLzB6Mfn5k5SkMpHhbmSh+wE8JWtqmXcPcMz3ZWUAJUMwTP2D mPoNrRow==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1q9spY-00BtUT-1s; Thu, 15 Jun 2023 19:40:29 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1018830031B; Thu, 15 Jun 2023 21:40:27 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id E847C245F1E4B; Thu, 15 Jun 2023 21:40:26 +0200 (CEST) Message-ID: <20230615193722.127844423@infradead.org> User-Agent: quilt/0.66 Date: Thu, 15 Jun 2023 21:35:47 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, tim.c.chen@linux.intel.com Subject: [PATCH 1/2] x86/cfi: Fix ret_from_fork indirect calls References: <20230615193546.949657149@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768799985959414203?= X-GMAIL-MSGID: =?utf-8?q?1768799985959414203?= The ret_from_fork stub does an indirect call to the kthread function, but only knows about Retpolines. Instead of making the asm more complicated, punt to C and let the compiler figure it out. Specifically, this makes it a proper kCFI indirect call when needed (in fact, it is nearly impossible to code a kCFI indirect call in asm). This was the only callsite that was still calling func()+0 on regular indirect functions. Signed-off-by: Peter Zijlstra (Intel) Acked-by: Kees Cook Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen --- arch/x86/entry/entry_64.S | 6 ++++-- arch/x86/include/asm/switch_to.h | 2 ++ arch/x86/kernel/process_64.c | 5 +++++ 3 files changed, 11 insertions(+), 2 deletions(-) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -304,8 +304,10 @@ SYM_CODE_START_NOALIGN(ret_from_fork) 1: /* kernel thread */ UNWIND_HINT_END_OF_STACK - movq %r12, %rdi - CALL_NOSPEC rbx + movq %rbx, %rdi + movq %r12, %rsi + call kthread_from_fork + /* * A kernel thread is allowed to return here after successfully * calling kernel_execve(). Exit to userspace to complete the execve() --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -74,6 +74,8 @@ static inline void update_task_stack(str #endif } +extern void kthread_from_fork(int (*fn)(void *), void *); + static inline void kthread_frame_init(struct inactive_task_frame *frame, int (*fun)(void *), void *arg) { --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -544,6 +544,11 @@ void compat_start_thread(struct pt_regs } #endif +__visible noinstr void kthread_from_fork(int (*fn)(void *), void *arg) +{ + fn(arg); +} + /* * switch_to(x,y) should switch tasks from x to y. * From patchwork Thu Jun 15 19:35:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 108694 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp887650vqr; Thu, 15 Jun 2023 13:08:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4K/Q9qrBxQSmte3R42s/5qWlaDLX0aynPQxzj1gK2qO7vDWokwDS882fIKkHDLrqiWfbAK X-Received: by 2002:a17:907:6e92:b0:982:6bba:79ce with SMTP id sh18-20020a1709076e9200b009826bba79cemr136730ejc.12.1686859687890; Thu, 15 Jun 2023 13:08:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686859687; cv=none; d=google.com; s=arc-20160816; b=m9mch22d+su7s/gOY0TAk2HyZFAFtk/tLfAldLFjKxj6r+150TgTnSgUSndSHyM2oh CQKNsYsKH7L9oE20Wh8saPs/02TctBfFvpIG2DErcyUB+iZAXOy+dM3pcZVe2qaiuWth xxP+4W722RJ+KyOgn8V2p6qeqFxz4h11LlW36YO+1ljqurZbaPaGNRxdg89Ro6QMsBmh B9ly8jXBmucovmZ1MpMzrOQP65p5cqEfjc0IY0dCRfgxr9Iq/tpZN2JahSFoZBJ6Ijvs nkTTAgbwHPG94nYNMKwSQwDv72FR+jsmwlgB+Y+D8Nu8o52r0cS6cAxWnus5EKlJv6rr KjhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=5QR/G+Po7KVXkZ8pw8MI2J3h+jcGvJpJMtnZs0VY6Uw=; b=OdlXaxw2M2SHx6nuyW/Q4pltjWl1CfFD302PXI+HgwjLz6fTRFj0uSm158W37wlPIv f22IwQQZMTvMtP93nlBOYxCVgh+ABSy0NnDDehjsktxI7AjLlhVurqR9jNEO755aQXa3 UptdSpIf+gkF3aOKK8W/SmGd40oZjFMpWif8my0I9vmIvpitky3x97/bPPiP33Bs+XKw tPioxy/U40v/dV2edFYuwYX06wHJc60yQgacZJz55WVkwsccQ9BhQTxUkbXyP46ALyAT BilK36LjfIOo6VKf8/JLgFundw1DfWdvTMRslOY+rQLLk8Kk+aYa6Osabx4Tv2P9U0Se nJbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=VH+MQ2ph; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q19-20020a1709066b1300b0097455b9d069si10284359ejr.1033.2023.06.15.13.07.41; Thu, 15 Jun 2023 13:08:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=VH+MQ2ph; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231738AbjFOTku (ORCPT + 99 others); Thu, 15 Jun 2023 15:40:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229572AbjFOTkn (ORCPT ); Thu, 15 Jun 2023 15:40:43 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0724B2953 for ; Thu, 15 Jun 2023 12:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=5QR/G+Po7KVXkZ8pw8MI2J3h+jcGvJpJMtnZs0VY6Uw=; b=VH+MQ2phJAjRHH/4i2FQnrxJXJ Of7AucP+aR7hJKYULJXBW0cOEdrBvHqOiYedgFVW3NEmFTRF6gYU/6Z4VXDhaSEHsV9Oxop2SBzZa Txm9EE9XsckO/OjI7qy0ID7RJnAVBWX4dQEWjbxuaoKvngR39LRNmBIGV8vK+ISJtY0/dfrqoxFWI ZOjeicpPcAjtcdc9s5AS46W7bjzGwDmnD/8y93CdPyQi4EzgDPRy37QR9CIbR7XshqofHeqZzq/4U iosEOH2iSmO1XhZ1YTF8iamxZyFGPJo9rvgzJlwMD0VqkLvIZwZkrNpzOUZb2y8n5+IQWEG6z25VH aqWvcIwg==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1q9spY-00BtUU-1s; Thu, 15 Jun 2023 19:40:29 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 14AA73003E1; Thu, 15 Jun 2023 21:40:27 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id EC6C52461D7D2; Thu, 15 Jun 2023 21:40:26 +0200 (CEST) Message-ID: <20230615193722.194131053@infradead.org> User-Agent: quilt/0.66 Date: Thu, 15 Jun 2023 21:35:48 +0200 From: Peter Zijlstra To: x86@kernel.org, alyssa.milburn@linux.intel.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, samitolvanen@google.com, keescook@chromium.org, jpoimboe@kernel.org, joao@overdrivepizza.com, tim.c.chen@linux.intel.com Subject: [PATCH 2/2] x86/fineibt: Poison ENDBR at +0 References: <20230615193546.949657149@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768800583727399784?= X-GMAIL-MSGID: =?utf-8?q?1768800583727399784?= Alyssa noticed that when building the kernel with CFI_CLANG+IBT and booting on IBT enabled hardware obtain FineIBT, the indirect functions look like: __cfi_foo: endbr64 subl $hash, %r10d jz 1f ud2 nop 1: foo: endbr64 This is because clang currently does not supress ENDBR emission for functions it provides a __cfi prologue symbol for. Having this second ENDBR however makes it possible to elide the CFI check. Therefore, we should poison this second ENDBR (if present) when switching to FineIBT mode. Fixes: 931ab63664f0 ("x86/ibt: Implement FineIBT") Reported-by: "Milburn, Alyssa" Signed-off-by: Peter Zijlstra (Intel) Acked-by: Kees Cook Reported-by: "Milburn, Alyssa" Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Sami Tolvanen Acked-by: Kees Cook --- arch/x86/kernel/alternative.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -940,6 +940,17 @@ static int cfi_rewrite_preamble(s32 *sta return 0; } +static void cfi_rewrite_endbr(s32 *start, s32 *end) +{ + s32 *s; + + for (s = start; s < end; s++) { + void *addr = (void *)s + *s; + + poison_endbr(addr+16, false); + } +} + /* .retpoline_sites */ static int cfi_rand_callers(s32 *start, s32 *end) { @@ -1034,14 +1045,19 @@ static void __apply_fineibt(s32 *start_r return; case CFI_FINEIBT: + /* place the FineIBT preamble at func()-16 */ ret = cfi_rewrite_preamble(start_cfi, end_cfi); if (ret) goto err; + /* rewrite the callers to target func()-16 */ ret = cfi_rewrite_callers(start_retpoline, end_retpoline); if (ret) goto err; + /* now that nobody targets func()+0, remove ENDBR there */ + cfi_rewrite_endbr(start_cfi, end_cfi); + if (builtin) pr_info("Using FineIBT CFI\n"); return;