From patchwork Mon Oct 17 14:53:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 3505 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1493375wrs; Mon, 17 Oct 2022 07:56:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM67vOhSQfapkDrMLKz0Z7jg5tNhO9sNzOQUyt+6KX4FBDbQbai10bP3ZKIEjPxPb6ho/vnT X-Received: by 2002:a17:90b:4a49:b0:20d:5099:f5cc with SMTP id lb9-20020a17090b4a4900b0020d5099f5ccmr13927034pjb.137.1666018572930; Mon, 17 Oct 2022 07:56:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666018572; cv=none; d=google.com; s=arc-20160816; b=Py0/7+nkpVi9HIYvBLa2tBDAMdgFl88jhtKS24s1KsyqDB+1Wt9lb9QUEymYCMWhpl 804DoXOA+PRfI7VYYHhFmQJU905RaE1klsgoFcuFknXnjNtE5vIWYWlote59d7nTedJ/ VtNYBqBKzMKBPnLi6Vq+eu/wqWdubCzE5h/MtuGZBviIit6U3wPQYJQEMW0LL54t5UyV rBMIhu6lCVBZdCYaCaVOpu8pOlrY2kkkh97k+iszEYIpTJRgC1NnVdzOzJAOyKDp0sIc m3NEdbwBp7r5XoD5kw3rBp9ZokkNL1VrfjDt1YIfWDuPu8jjkXYbJ4MqLrUrKzNdGeRQ EIHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=iESZwYUsl9VoA6IO1HubGWuLJwmOnpSg5W5sq0VFvpY=; b=KvEBi1trxrNU7gfmWVye5KolPvMNHizuo+yzJrJD61DHHhC55TW0XHA2CCyrcpVpXn Or7Hw4gjYx3Et7Lour4Q/jv/PZVNiO9dbsqQaxzvF84eFpOYribBfPxYh8n3psvZLgsF 4SscIJknvSXwlXrgr2qTA8WewTcSGGYzxYd5QelEetBPFpteVxqgWcq3G7n9mIF+HhTb xQc+P14xygBcTiAJ7xM8fdIW/YOrD50mYmCMbIqUNRmro0Y+gd6FqU2yfPqZ9JXPXZc4 5MTEDrC6WdIuJ7AW+W4u8YGa8cfw3JIJ7vWWXB/YCqhnkXibVL3NdSEvcYyPlQCKj3e3 j27Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PG0VFA76; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h19-20020a056a00231300b00563b702a8e5si14101908pfh.10.2022.10.17.07.55.59; Mon, 17 Oct 2022 07:56:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PG0VFA76; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231278AbiJQOzI (ORCPT + 99 others); Mon, 17 Oct 2022 10:55:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230351AbiJQOyB (ORCPT ); Mon, 17 Oct 2022 10:54:01 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F301C7646; Mon, 17 Oct 2022 07:53:57 -0700 (PDT) Date: Mon, 17 Oct 2022 14:53:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1666018430; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iESZwYUsl9VoA6IO1HubGWuLJwmOnpSg5W5sq0VFvpY=; b=PG0VFA76O1XYhaJnSTqSv9Ybwy6Z2TLurVWS35R8xqpaTMKWLeW9lQ934SJggY5jfcaAwl eeq1pHnProYSusosXb6h4fYFIfHIqEezjFt/bG8+qD+CFLRYJ3U1mmMv6GSvjrNUrSEr37 kaKxT1gTNOMUDcZg62qOso51o6Sd4fz26p+VEgTr4O/tY3Ra1Q3+hbtNJZgoM1zyuN+Tzn mtHPSQnhhZ5IhlqgvPkTWdySdTSRGJNFR3D66zK1I5+YBih/itxtBVd3JOIVm3h/kluXjn VpvgNSzREv6q3mkGyBQGnCDYj8emxY9nuchefs6MiA88Mq2MY343jSN2N8Cs/Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1666018430; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iESZwYUsl9VoA6IO1HubGWuLJwmOnpSg5W5sq0VFvpY=; b=3iWFNB49as2W4iJ5CuFkEnoytp/x0lSUlN0ahOFoxSKOtpDgywdXUTqXMOsz2qoQNJaMGj mUghQYI4pywObKAw== From: "tip-bot2 for Thomas Gleixner" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/core] x86/retbleed: Add X86_FEATURE_CALL_DEPTH Cc: Thomas Gleixner , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20220915111147.056176424@infradead.org> References: <20220915111147.056176424@infradead.org> MIME-Version: 1.0 Message-ID: <166601842920.401.2168961504911360398.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746947091215451216?= X-GMAIL-MSGID: =?utf-8?q?1746947091215451216?= The following commit has been merged into the x86/core branch of tip: Commit-ID: 80e4c1cd42fff110bfdae8fce7ac4f22465f9664 Gitweb: https://git.kernel.org/tip/80e4c1cd42fff110bfdae8fce7ac4f22465f9664 Author: Thomas Gleixner AuthorDate: Thu, 15 Sep 2022 13:11:19 +02:00 Committer: Peter Zijlstra CommitterDate: Mon, 17 Oct 2022 16:41:11 +02:00 x86/retbleed: Add X86_FEATURE_CALL_DEPTH Intel SKL CPUs fall back to other predictors when the RSB underflows. The only microcode mitigation is IBRS which is insanely expensive. It comes with performance drops of up to 30% depending on the workload. A way less expensive, but nevertheless horrible mitigation is to track the call depth in software and overeagerly fill the RSB when returns underflow the software counter. Provide a configuration symbol and a CPU misfeature bit. Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20220915111147.056176424@infradead.org --- arch/x86/Kconfig | 19 +++++++++++++++++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 9 ++++++++- 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index e368fc0..6ae7fa4 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2523,6 +2523,25 @@ config CPU_UNRET_ENTRY help Compile the kernel with support for the retbleed=unret mitigation. +config CALL_DEPTH_TRACKING + bool "Mitigate RSB underflow with call depth tracking" + depends on CPU_SUP_INTEL && HAVE_CALL_THUNKS + select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE + select CALL_THUNKS + default y + help + Compile the kernel with call depth tracking to mitigate the Intel + SKL Return-Speculation-Buffer (RSB) underflow issue. The + mitigation is off by default and needs to be enabled on the + kernel command line via the retbleed=stuff option. For + non-affected systems the overhead of this option is marginal as + the call depth tracking is using run-time generated call thunks + in a compiler generated padding area and call patching. This + increases text size by ~5%. For non affected systems this space + is unused. On affected SKL systems this results in a significant + performance gain over the IBRS mitigation. + + config CPU_IBPB_ENTRY bool "Enable IBPB on kernel entry" depends on CPU_SUP_AMD && X86_64 diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index b71f4f2..aefd081 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -304,6 +304,7 @@ #define X86_FEATURE_UNRET (11*32+15) /* "" AMD BTB untrain return */ #define X86_FEATURE_USE_IBPB_FW (11*32+16) /* "" Use IBPB during runtime firmware calls */ #define X86_FEATURE_RSB_VMEXIT_LITE (11*32+17) /* "" Fill RSB on VM exit when EIBRS is enabled */ +#define X86_FEATURE_CALL_DEPTH (11*32+18) /* "" Call depth tracking for RSB stuffing */ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 33d2cd0..bbb03b2 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -69,6 +69,12 @@ # define DISABLE_UNRET (1 << (X86_FEATURE_UNRET & 31)) #endif +#ifdef CONFIG_CALL_DEPTH_TRACKING +# define DISABLE_CALL_DEPTH_TRACKING 0 +#else +# define DISABLE_CALL_DEPTH_TRACKING (1 << (X86_FEATURE_CALL_DEPTH & 31)) +#endif + #ifdef CONFIG_INTEL_IOMMU_SVM # define DISABLE_ENQCMD 0 #else @@ -101,7 +107,8 @@ #define DISABLED_MASK8 (DISABLE_TDX_GUEST) #define DISABLED_MASK9 (DISABLE_SGX) #define DISABLED_MASK10 0 -#define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET) +#define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ + DISABLE_CALL_DEPTH_TRACKING) #define DISABLED_MASK12 0 #define DISABLED_MASK13 0 #define DISABLED_MASK14 0