From patchwork Mon Oct 17 14:53:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 3514 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1493582wrs; Mon, 17 Oct 2022 07:56:49 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ydHk7mVmTmcqVDcm06ws4UgH93UI+of0R2SWxalIgmrnU0+CA7K8HluAzKR3fx6a+oZ65 X-Received: by 2002:a17:907:788:b0:741:4d1a:595d with SMTP id xd8-20020a170907078800b007414d1a595dmr8823131ejb.737.1666018608960; Mon, 17 Oct 2022 07:56:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666018608; cv=none; d=google.com; s=arc-20160816; b=MKWaC0UcNcVCOCdkyk6T28bDB9bslUxFx0FzUuF6Y0h3rdjnsqeMesa+qsc3leZ1fz WiONMs0hpX/qOvuZUuOjzBFYeoIuDcIzvlI98LsYWHdLU7IKi5Nntl01Y+eBMjSwFwsh GN7Jq8Y+i6VmLIY+vTHIBZpxV8z7nIVO5EiguzVxoGhnaHcw9v9l3slCMOSyimlMafOZ 3WkkMUtySDpHBpsxnNJPKZ9cghOFU9P3UHUFpy9k8ec5ZFelTVUk3XatX26pv3nvV/Nx 8rOyQl+32DtX0juAzLye+SYc+8os7eWIprs/wwpZy01nHWpOo4X1FhkC25E4tr+yAdTv vyIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=nr1cugH5Yu0i8Cjc289xP+gGWqESMqaJMC8BByvp6Lk=; b=goNMf9I1Lzh9E5wVEnXiscArk8p9Auh5HVP/Mne+h/jdIaUgB3XwXtCnpa4QfyRe+I Qe5f0x7IoijrHQ7Njd3iH0OQ0YAvc6xzxHuY9DbkFAtKkVA8kbbdImxg8Ddm5KRpAzgs Pd+5/JdozDSCWxBfkLZY3XoknN7gBeM9Vz10VKgWNlomcltWQmbPwKDX8MQzuOxDXc8h UC9rAlpGfCg5/spESr6I2yv1NzdYLOixdkIDiRFBc7ObMSCwddR3P3bupqEbjarRjE/B 7HdD/BcXlbjKzCvp9WWTL6wf2rpGIDlR7PLwsmMhbDP1pu2mj83itT4dfAB6TnuL0681 WzkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=XBrPOlAg; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g12-20020a1709065d0c00b007700c3ad033si10159224ejt.565.2022.10.17.07.56.23; Mon, 17 Oct 2022 07:56:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=XBrPOlAg; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229799AbiJQOzm (ORCPT + 99 others); Mon, 17 Oct 2022 10:55:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230345AbiJQOyF (ORCPT ); Mon, 17 Oct 2022 10:54:05 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F308367CA3; Mon, 17 Oct 2022 07:53:56 -0700 (PDT) Date: Mon, 17 Oct 2022 14:53:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1666018425; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nr1cugH5Yu0i8Cjc289xP+gGWqESMqaJMC8BByvp6Lk=; b=XBrPOlAg3nP7whdSBBxSlML+Do67zZsbFVkwniwgMHLhlErZ87Lgcg2Tvs7ORZ5/4XukBf +CgsvVdoqY4CE23sUsQ7Gu2+3omxGMbNLrXz/DCJ9jOTlz9XUoHL1aNdBFr+Hm277CswIm fNDDIgXC2y14Z1/UamM7/twqLaLWCmD25OF5W3/3hODz0mkciRNrWVZ4ZBchGAUG/XRABO NnDt6m8gG6BPTZ6E8LyPbsl1sWBEgDdjd8OxjMBZdm9HQVa8gXpYOvP5ErSjluiL+xSF6W 58q6tqTGdR1yR5giBJLKmuP4R6baAeYrrGkIRdPjeHbLRUhEcQBFw97q8vvt/g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1666018425; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nr1cugH5Yu0i8Cjc289xP+gGWqESMqaJMC8BByvp6Lk=; b=8aMEMoaUB6zVV31xtD0dRbAgAmMo5yyKTB7epiuGcdnMr6PdZazvqQkTaVRlTcfY47CxbG 2dsSh9jo2KG1ioAQ== From: "tip-bot2 for Thomas Gleixner" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/core] x86/modules: Add call patching Cc: Thomas Gleixner , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20220915111147.575673066@infradead.org> References: <20220915111147.575673066@infradead.org> MIME-Version: 1.0 Message-ID: <166601842378.401.4292798870470772226.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746947129005668912?= X-GMAIL-MSGID: =?utf-8?q?1746947129005668912?= The following commit has been merged into the x86/core branch of tip: Commit-ID: eaf44c816ed8d1ef94c354e3ed47d53cd5a5cb13 Gitweb: https://git.kernel.org/tip/eaf44c816ed8d1ef94c354e3ed47d53cd5a5cb13 Author: Thomas Gleixner AuthorDate: Thu, 15 Sep 2022 13:11:24 +02:00 Committer: Peter Zijlstra CommitterDate: Mon, 17 Oct 2022 16:41:13 +02:00 x86/modules: Add call patching As for the builtins create call thunks and patch the call sites to call the thunk on Intel SKL CPUs for retbleed mitigation. Note, that module init functions are ignored for sake of simplicity because loading modules is not something which is done in high frequent loops and the attacker has not really a handle on when this happens in order to launch a matching attack. The depth tracking will still work for calls into the builtins and because the call is not accounted it will underflow faster and overstuff, but that's mitigated by the saturating counter and the side effect is only temporary. Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20220915111147.575673066@infradead.org --- arch/x86/include/asm/alternative.h | 5 +++++ arch/x86/kernel/callthunks.c | 19 +++++++++++++++++++ arch/x86/kernel/module.c | 20 +++++++++++++++++++- 3 files changed, 43 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/alternative.h b/arch/x86/include/asm/alternative.h index 6b7bbd0..ef007fa 100644 --- a/arch/x86/include/asm/alternative.h +++ b/arch/x86/include/asm/alternative.h @@ -89,8 +89,13 @@ struct callthunk_sites { #ifdef CONFIG_CALL_THUNKS extern void callthunks_patch_builtin_calls(void); +extern void callthunks_patch_module_calls(struct callthunk_sites *sites, + struct module *mod); #else static __always_inline void callthunks_patch_builtin_calls(void) {} +static __always_inline void +callthunks_patch_module_calls(struct callthunk_sites *sites, + struct module *mod) {} #endif #ifdef CONFIG_SMP diff --git a/arch/x86/kernel/callthunks.c b/arch/x86/kernel/callthunks.c index e5275d6..7b9d998 100644 --- a/arch/x86/kernel/callthunks.c +++ b/arch/x86/kernel/callthunks.c @@ -249,3 +249,22 @@ void __init callthunks_patch_builtin_calls(void) thunks_initialized = true; mutex_unlock(&text_mutex); } + +#ifdef CONFIG_MODULES +void noinline callthunks_patch_module_calls(struct callthunk_sites *cs, + struct module *mod) +{ + struct core_text ct = { + .base = (unsigned long)mod->core_layout.base, + .end = (unsigned long)mod->core_layout.base + mod->core_layout.size, + .name = mod->name, + }; + + if (!thunks_initialized) + return; + + mutex_lock(&text_mutex); + callthunks_setup(cs, &ct); + mutex_unlock(&text_mutex); +} +#endif /* CONFIG_MODULES */ diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 43f0112..2fb9de2 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -254,7 +254,8 @@ int module_finalize(const Elf_Ehdr *hdr, { const Elf_Shdr *s, *text = NULL, *alt = NULL, *locks = NULL, *para = NULL, *orc = NULL, *orc_ip = NULL, - *retpolines = NULL, *returns = NULL, *ibt_endbr = NULL; + *retpolines = NULL, *returns = NULL, *ibt_endbr = NULL, + *calls = NULL; char *secstrings = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; for (s = sechdrs; s < sechdrs + hdr->e_shnum; s++) { @@ -274,6 +275,8 @@ int module_finalize(const Elf_Ehdr *hdr, retpolines = s; if (!strcmp(".return_sites", secstrings + s->sh_name)) returns = s; + if (!strcmp(".call_sites", secstrings + s->sh_name)) + calls = s; if (!strcmp(".ibt_endbr_seal", secstrings + s->sh_name)) ibt_endbr = s; } @@ -299,6 +302,21 @@ int module_finalize(const Elf_Ehdr *hdr, void *aseg = (void *)alt->sh_addr; apply_alternatives(aseg, aseg + alt->sh_size); } + if (calls || para) { + struct callthunk_sites cs = {}; + + if (calls) { + cs.call_start = (void *)calls->sh_addr; + cs.call_end = (void *)calls->sh_addr + calls->sh_size; + } + + if (para) { + cs.pv_start = (void *)para->sh_addr; + cs.pv_end = (void *)para->sh_addr + para->sh_size; + } + + callthunks_patch_module_calls(&cs, me); + } if (ibt_endbr) { void *iseg = (void *)ibt_endbr->sh_addr; apply_ibt_endbr(iseg, iseg + ibt_endbr->sh_size);