From patchwork Mon Oct 17 14:53:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 3526 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1493885wrs; Mon, 17 Oct 2022 07:57:29 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5vhkzLXDF9InuHQtheDqAGfORqRGPT4MUIL+SUuPdFuKIytdsQzjWu2RZSPT4ZVlwTkTJc X-Received: by 2002:a17:902:a718:b0:182:c500:d8da with SMTP id w24-20020a170902a71800b00182c500d8damr12388153plq.98.1666018648804; Mon, 17 Oct 2022 07:57:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666018648; cv=none; d=google.com; s=arc-20160816; b=UwCbKAvrL2bNSH4HY4UrClMcagmjwNboO3a4aii9+ltSt+HinsbUYhN+i3c/qE6pg2 tYPEAwJ+i+D9ESzBUlwYy/TdMa6BIq2koS/i1680CPVEWAEF0M8hw7p/Y7kfeywZQsBp h8wj04fM3T2I9o7bwlm5eU7yV62Hm3Zl0B4Rc8wTgZ8uAX+e4EoDxLW5vWYFyYjFbpck XzKjmDf/EbWzbz5kTM70i6hHsVJQbc0VoKJ469xcbZ/CzFcdv/HtI4em9AfZTCKcU2QU qkpx5ZzQ0mGUrMdhAIxj7lNiIKaX8ShS4l/12Yb1bVZxhUyTxoqD5sOivjFJn8h7WNQj CpKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=brruPqTqUU9fTZWrUqRiiP3PaFtkap8CCYjIT4ZdseI=; b=rLeL/02s+2UQ6sgB+Ic7tywBtjiVWtSSDBMNGFUfazIQ3osXYu84sE/nAfx25ASra2 xAVTtdX5ixZTmR2oIxzLQ+wkybbrohnnblf8AOHuir8jNp6VaXtTKfW4a1qxXBanOZ73 XGBSS1LTrH/FNJURCwo1CWBmqCRFIDlPKT0eMB8qUftVNggt+rk+mBcpuIiBUbW7lAif ycJzfYWPBGuW59CX8hFQs4zFDbReJBLhr8DZP/8eQ9rJf8grwDj/nFOmafQGJTFSHKfS UBq+t3dG04/R2auHjBe0YBf8/BbHUjNqmnU4jev8xpiiS6BLepXmMfgp9kviG8Fu1Fp0 QjxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=XhcETwL9; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=0LP8jHaX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y2-20020a17090a474200b0020a98093e0bsi11359925pjg.44.2022.10.17.07.57.15; Mon, 17 Oct 2022 07:57:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=XhcETwL9; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=0LP8jHaX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229742AbiJQOyg (ORCPT + 99 others); Mon, 17 Oct 2022 10:54:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230367AbiJQOxz (ORCPT ); Mon, 17 Oct 2022 10:53:55 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 176D768CEF; Mon, 17 Oct 2022 07:53:51 -0700 (PDT) Date: Mon, 17 Oct 2022 14:53:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1666018424; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=brruPqTqUU9fTZWrUqRiiP3PaFtkap8CCYjIT4ZdseI=; b=XhcETwL9/4Ecs6DeNtG1KKKYGxVa3tGl3S3+PW7rtbcLRSSR/ySOj1lyKdNS9SGERist9H De1ikr32ieh9bfgIh07oYibHlCAgzxdwzOg75/fFfqwtJaBQnKeGLrYrogfNNwxUj33Yb6 jXi099CpfcHmAjbXhIysxB9KXdNZnHfZilu1WctoYZUDncw0t28v+pnca0O0YsVvXurAob Ext7B7f8M8wifEOsqHRQu3+ITiMSvqDFccJghOSiynGlAjK5x1KD0qyvHzCMMp71M/WYN5 Nro2xuBFJBEODARdDjZj3tipF6I490E9hEhqm2Ortmrsg9Arij0BMZYTNetzvg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1666018424; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=brruPqTqUU9fTZWrUqRiiP3PaFtkap8CCYjIT4ZdseI=; b=0LP8jHaXnKb3gY7nHHc/oK4FNDd97hynmoYhZrUomBcmbft34QJCFDz1FYWLUbdXokDSD/ QIkB46YE6HdP/rCQ== From: "tip-bot2 for Peter Zijlstra" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/core] x86/returnthunk: Allow different return thunks Cc: "Peter Zijlstra (Intel)" , Thomas Gleixner , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20220915111147.680469665@infradead.org> References: <20220915111147.680469665@infradead.org> MIME-Version: 1.0 Message-ID: <166601842272.401.18030395061124314174.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746947170627624379?= X-GMAIL-MSGID: =?utf-8?q?1746947170627624379?= The following commit has been merged into the x86/core branch of tip: Commit-ID: 770ae1b709528a6a173b5c7b183818ee9b45e376 Gitweb: https://git.kernel.org/tip/770ae1b709528a6a173b5c7b183818ee9b45e376 Author: Peter Zijlstra AuthorDate: Thu, 15 Sep 2022 13:11:25 +02:00 Committer: Peter Zijlstra CommitterDate: Mon, 17 Oct 2022 16:41:14 +02:00 x86/returnthunk: Allow different return thunks In preparation for call depth tracking on Intel SKL CPUs, make it possible to patch in a SKL specific return thunk. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20220915111147.680469665@infradead.org --- arch/x86/include/asm/nospec-branch.h | 6 ++++++ arch/x86/kernel/alternative.c | 17 +++++++++++++---- arch/x86/kernel/ftrace.c | 2 +- arch/x86/kernel/static_call.c | 2 +- arch/x86/net/bpf_jit_comp.c | 2 +- 5 files changed, 22 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index c936ce9..f10ca33 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -208,6 +208,12 @@ extern void __x86_return_thunk(void); extern void zen_untrain_ret(void); extern void entry_ibpb(void); +#ifdef CONFIG_CALL_THUNKS +extern void (*x86_return_thunk)(void); +#else +#define x86_return_thunk (&__x86_return_thunk) +#endif + #ifdef CONFIG_RETPOLINE #define GEN(reg) \ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 963872d..04d1e3d 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -518,6 +518,11 @@ void __init_or_module noinline apply_retpolines(s32 *start, s32 *end) } #ifdef CONFIG_RETHUNK + +#ifdef CONFIG_CALL_THUNKS +void (*x86_return_thunk)(void) __ro_after_init = &__x86_return_thunk; +#endif + /* * Rewrite the compiler generated return thunk tail-calls. * @@ -533,14 +538,18 @@ static int patch_return(void *addr, struct insn *insn, u8 *bytes) { int i = 0; - if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) - return -1; + if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) { + if (x86_return_thunk == __x86_return_thunk) + return -1; - bytes[i++] = RET_INSN_OPCODE; + i = JMP32_INSN_SIZE; + __text_gen_insn(bytes, JMP32_INSN_OPCODE, addr, x86_return_thunk, i); + } else { + bytes[i++] = RET_INSN_OPCODE; + } for (; i < insn->length;) bytes[i++] = INT3_INSN_OPCODE; - return i; } diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 00eac45..4ac6692 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -359,7 +359,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) ip = trampoline + size; if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) - __text_gen_insn(ip, JMP32_INSN_OPCODE, ip, &__x86_return_thunk, JMP32_INSN_SIZE); + __text_gen_insn(ip, JMP32_INSN_OPCODE, ip, x86_return_thunk, JMP32_INSN_SIZE); else memcpy(ip, retq, sizeof(retq)); diff --git a/arch/x86/kernel/static_call.c b/arch/x86/kernel/static_call.c index aaaba85..5d3844a 100644 --- a/arch/x86/kernel/static_call.c +++ b/arch/x86/kernel/static_call.c @@ -52,7 +52,7 @@ static void __ref __static_call_transform(void *insn, enum insn_type type, case RET: if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) - code = text_gen_insn(JMP32_INSN_OPCODE, insn, &__x86_return_thunk); + code = text_gen_insn(JMP32_INSN_OPCODE, insn, x86_return_thunk); else code = &retinsn; break; diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 9962042..0df391e 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -432,7 +432,7 @@ static void emit_return(u8 **pprog, u8 *ip) u8 *prog = *pprog; if (cpu_feature_enabled(X86_FEATURE_RETHUNK)) { - emit_jump(&prog, &__x86_return_thunk, ip); + emit_jump(&prog, x86_return_thunk, ip); } else { EMIT1(0xC3); /* ret */ if (IS_ENABLED(CONFIG_SLS))