From patchwork Mon Dec 5 23:32:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kim Phillips X-Patchwork-Id: 30025 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2529963wrr; Mon, 5 Dec 2022 15:36:21 -0800 (PST) X-Google-Smtp-Source: AA0mqf5DWHHHzZIxP3MaB3/dtATopadYDo01kO351Lxzi9Jo5xlKtLZVh8mjFStIVbu0qCELXVje X-Received: by 2002:a17:902:7584:b0:189:6292:990a with SMTP id j4-20020a170902758400b001896292990amr51950811pll.171.1670283380834; Mon, 05 Dec 2022 15:36:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1670283380; cv=pass; d=google.com; s=arc-20160816; b=keg41WdyaYXk9xuDUl6o0pnFf0DFYnjADnMBU5zfowxVijY7MFwV7P0ugXTBwh8A0T AZkF8l4+IYmzEyJG7q87DjCFx+xuIYhCqehtnI3gacXydXE7n04jFgBR2VSfZmvsBlyj 6DoRf4UnQKosAyfy6IGXx7Xru9pvh6qwNlHT8uDSYgDCnZuvVuE+KV+v5Go4bHfZtyZR FFB4KISscD1ZymCI6+ht/GrwWCNsS1vUe811A3SPh6zqKNY7eX11vCnf3+ytrVgaOoVX lt4IPv8eTbQC+vUP0c/LJPDZaeZW6ybncLNwssB+GIuGKKjfmrvOAGfmZ7UoE22sJv6h XP1A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QtbEVcIXKPcNUDK5Ed3i+L8em6ijJ026n5PAVD1LpfY=; b=xP50kyWogVL5V1gIyOOkbOJWujo3Trtaw6BbTqyl0i8eBscvdyymgBOFMY+f9dAtvH dAmS4+rK80YGuaJPChF34idjAPUKobDX2CkW9c2CAa1knaK7vSIjWWFOUFGiXQbJMb9x vg23L2oU96XvI0KL6MGGUgMW5BpRL+q2gG0goJ4/Y0ov6ftImhuuvY/gzuzsGT8cpMS2 9wpwFExEpOXGAKVPcsHotrpnUfyzsHks52qU4fGSJCpqOsMh/Vh87V+ga32aVGk1OLo4 Mc+Qv7S5y4vJvuaJuae0Po+bCJxGyWYTJkG5XJX4RDMyYW0QGwHzT2eps0Qbg22wDF5Q czdw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=d27I1oWs; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x189-20020a6386c6000000b00478a51cc009si6563995pgd.873.2022.12.05.15.36.07; Mon, 05 Dec 2022 15:36:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=d27I1oWs; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232830AbiLEXej (ORCPT + 99 others); Mon, 5 Dec 2022 18:34:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232851AbiLEXe2 (ORCPT ); Mon, 5 Dec 2022 18:34:28 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2078.outbound.protection.outlook.com [40.107.220.78]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A0611F9D2; Mon, 5 Dec 2022 15:34:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HtI+LEhRCaZ39hZwvQPF2zPN3jS3N3yNGEXBzCG3KSyyVpt3kNgQhDiSgIMLYBDbAZwYEG3KjTDiVHVBxvFpJRsTUmMKw10j4ESA+upri2ZFuHuIVWqwSIfqiwgUq/fZqE/dcHT11ziXWOAMj4TK+A4wncQ+mlg74PPSZY8FyPZit890pmXGfeDVyaCm5qs20Ms5DpnMvUxsCD9Oh3ci6L5WkBbg1ys7E0wwwu23421h59nsxtweSV0Od0+zujkLx5CV3j9UlBL7FL5PLfguPCEL/SpOMom06NaYpUD9HuR0HGkpG+oQD8NgiJo4uBFs09EIuJjgroscDpGD559xUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QtbEVcIXKPcNUDK5Ed3i+L8em6ijJ026n5PAVD1LpfY=; b=Zo8DPNW9pcGj4In+tJfrg91aj/OaSLUyoOwJq8PWfAFeL8DJfm1AvRp4HMe1GAHbGxu9A80BIRcsF4YxZCYUUKWFUyjGkQYeHWsBE7HQdwRJzbPWPx5xK4yRxiFbWHlyAPP63GIrX0iDoXPv4cZwhjzY8nKLFuvU3002mV6n3u3iPGcoQLbI6mO5mt54cYgL0xhkWBa0iQYV+6XLwAy1OiOsWlv6BTR3TMZE1/ejba1AIDDMZqh9F5tqQ8x/MFROzAUa+XyjCegt3TPj/NYW3uTUElSyTtsd4dsdlXuK/bWDhPVswAdQAh/ghkIXptI0+Z+KhQjy/+RSmbYh8pAQig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QtbEVcIXKPcNUDK5Ed3i+L8em6ijJ026n5PAVD1LpfY=; b=d27I1oWs4zGNT9F24X0qS0EhzhO2viKKNnEbJNYVOir4+Qm2UeXmX4Z5uplVmyandR8IoAkYOP2R9hfIvQGx9Ltlny6uopJV5uGA6flfLKUDsu2mJfj7tX12HHwq4Rqwl0svI6EYY4Zfi1Lmm+4p9Zz96liHHXXqnSm26uA+ySk= Received: from DS7PR05CA0027.namprd05.prod.outlook.com (2603:10b6:5:3b9::32) by SJ0PR12MB6989.namprd12.prod.outlook.com (2603:10b6:a03:448::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5880.14; Mon, 5 Dec 2022 23:34:07 +0000 Received: from DS1PEPF0000E645.namprd02.prod.outlook.com (2603:10b6:5:3b9:cafe::6a) by DS7PR05CA0027.outlook.office365.com (2603:10b6:5:3b9::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5901.8 via Frontend Transport; Mon, 5 Dec 2022 23:34:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS1PEPF0000E645.mail.protection.outlook.com (10.167.17.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5880.8 via Frontend Transport; Mon, 5 Dec 2022 23:34:07 +0000 Received: from fritz.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 5 Dec 2022 17:34:04 -0600 From: Kim Phillips To: CC: Kim Phillips , Babu Moger , Borislav Petkov , Borislav Petkov , Boris Ostrovsky , Dave Hansen , "H. Peter Anvin" , Ingo Molnar , Joao Martins , Jonathan Corbet , "Konrad Rzeszutek Wilk" , Paolo Bonzini , Sean Christopherson , Thomas Gleixner , David Woodhouse , Greg Kroah-Hartman , Juergen Gross , Peter Zijlstra , Tony Luck , Tom Lendacky , Alexey Kardashevskiy , , , Subject: [PATCH v5 6/7] x86/cpu: Support AMD Automatic IBRS Date: Mon, 5 Dec 2022 17:32:34 -0600 Message-ID: <20221205233235.622491-7-kim.phillips@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221205233235.622491-1-kim.phillips@amd.com> References: <20221205233235.622491-1-kim.phillips@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0000E645:EE_|SJ0PR12MB6989:EE_ X-MS-Office365-Filtering-Correlation-Id: 86d44ee2-d085-467a-55a9-08dad7193444 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s5h0yqurSRhuPkdKVzW7eXE0mfVZV7AEdHZjx8OojtMLMrPodd/jUwO2H0fll1NJfgfRyMxaihixZjibGegNvMW7lCftxxmqG7Ydw1aMuYBAIpxaWb/aA5dJqp5b7LSYlJDmDF9es52OM76Rm0L+mE/8p6iuMp9BbreIeF/hDNRxmQKJsJxgXocRdKpzOX1g6qYB1Xx66WVi/VUXFX9im9aJ9qWMsuAZMsh8mCRD7/wc8iu74AUrjAy80/Aot7rhiIiMGBz7gy9RKYsRaSw4YdSN0sITW+CCFL8MEeyzfyVBGFlsjeM1q+jBkJoO6GLxNwb7spcDiyArkyJKEja45jURmDlpwivA3F8fuYzR5QZ8X2E72qU0NM3QRi2sDCQ9IYVbyEnOpl9CItXtA49JK8DeTxlT6RVYV/VyiKhRdD15juaJGfSyKKicvkluu1+pIGZJGmw7MMd9ERwfu06B58LGhZnLOVCrAONtzAXdP4tm+2xenVwY8E3oYLspSegNoWsTRXGxcTMLoKZkumhZA/Om85hI5bZSKuefOUuASx6vQiicWHBjkSuFpj30szxojd3Hv6+zRyIfRAXjB1eK2TnO5YD7mSs27ixO8khLZjFnLZ77lICClAB53Qj7x4bNR53/AqKqkyhM4zqwiOBKNQQOwg0ZOUKqJAOEAoP2UUsx7gVRZCcIlCgDLfw0AiZv4jPqcgyQt8L7q3ZDYekCz1H0ccKmDRW0WfSdyfBV4H4= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(346002)(376002)(396003)(39860400002)(136003)(451199015)(36840700001)(46966006)(40470700004)(186003)(16526019)(336012)(41300700001)(356005)(2616005)(81166007)(82740400003)(1076003)(426003)(47076005)(2906002)(83380400001)(82310400005)(36860700001)(36756003)(7416002)(5660300002)(8936002)(40460700003)(44832011)(6666004)(40480700001)(7696005)(478600001)(86362001)(70586007)(70206006)(8676002)(4326008)(26005)(316002)(6916009)(54906003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2022 23:34:07.6613 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 86d44ee2-d085-467a-55a9-08dad7193444 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0000E645.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6989 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751419066436237601?= X-GMAIL-MSGID: =?utf-8?q?1751419066436237601?= The AMD Zen4 core supports a new feature called Automatic IBRS. It is a "set-and-forget" feature that means that, like Intel's Enhanced IBRS, h/w manages its IBRS mitigation resources automatically across CPL transitions. The feature is advertised by CPUID_Fn80000021_EAX bit 8 and is enabled by setting MSR C000_0080 (EFER) bit 21. Enable Automatic IBRS by default if the CPU feature is present. It typically provides greater performance over the incumbent generic retpolines mitigation. Reuse the SPECTRE_V2_EIBRS spectre_v2_mitigation enum. AMD Automatic IBRS and Intel Enhanced IBRS have similar bugs.c enablement. Add NO_EIBRS_PBRSB to cpu_vuln_whitelist, since AMD Automatic IBRS isn't affected by PBRSB-eIBRS. The kernel command line option spectre_v2=eibrs is used to select AMD Automatic IBRS, if available. Signed-off-by: Kim Phillips --- Documentation/admin-guide/hw-vuln/spectre.rst | 6 +++--- .../admin-guide/kernel-parameters.txt | 6 +++--- arch/x86/include/asm/msr-index.h | 2 ++ arch/x86/kernel/cpu/bugs.c | 20 +++++++++++-------- arch/x86/kernel/cpu/common.c | 19 ++++++++++-------- 5 files changed, 31 insertions(+), 22 deletions(-) diff --git a/Documentation/admin-guide/hw-vuln/spectre.rst b/Documentation/admin-guide/hw-vuln/spectre.rst index c4dcdb3d0d45..3fe6511c5405 100644 --- a/Documentation/admin-guide/hw-vuln/spectre.rst +++ b/Documentation/admin-guide/hw-vuln/spectre.rst @@ -610,9 +610,9 @@ kernel command line. retpoline,generic Retpolines retpoline,lfence LFENCE; indirect branch retpoline,amd alias for retpoline,lfence - eibrs enhanced IBRS - eibrs,retpoline enhanced IBRS + Retpolines - eibrs,lfence enhanced IBRS + LFENCE + eibrs Enhanced/Auto IBRS + eibrs,retpoline Enhanced/Auto IBRS + Retpolines + eibrs,lfence Enhanced/Auto IBRS + LFENCE ibrs use IBRS to protect kernel Not specifying this option is equivalent to diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 42af9ca0127e..3e8322acf665 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5698,9 +5698,9 @@ retpoline,generic - Retpolines retpoline,lfence - LFENCE; indirect branch retpoline,amd - alias for retpoline,lfence - eibrs - enhanced IBRS - eibrs,retpoline - enhanced IBRS + Retpolines - eibrs,lfence - enhanced IBRS + LFENCE + eibrs - Enhanced/Auto IBRS + eibrs,retpoline - Enhanced/Auto IBRS + Retpolines + eibrs,lfence - Enhanced/Auto IBRS + LFENCE ibrs - use IBRS to protect kernel Not specifying this option is equivalent to diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 37ff47552bcb..6843d6241203 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -25,6 +25,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) #define EFER_LME (1<<_EFER_LME) @@ -33,6 +34,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /* Intel MSRs. Some also available on other CPUs */ diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 84c14e11db90..fc8fab82cc44 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -1222,9 +1222,9 @@ static const char * const spectre_v2_strings[] = { [SPECTRE_V2_NONE] = "Vulnerable", [SPECTRE_V2_RETPOLINE] = "Mitigation: Retpolines", [SPECTRE_V2_LFENCE] = "Mitigation: LFENCE", - [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced IBRS", - [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced IBRS + LFENCE", - [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced IBRS + Retpolines", + [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced / Automatic IBRS", + [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced / Automatic IBRS + LFENCE", + [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced / Automatic IBRS + Retpolines", [SPECTRE_V2_IBRS] = "Mitigation: IBRS", }; @@ -1293,7 +1293,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) cmd == SPECTRE_V2_CMD_EIBRS_LFENCE || cmd == SPECTRE_V2_CMD_EIBRS_RETPOLINE) && !boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) { - pr_err("%s selected but CPU doesn't have eIBRS. Switching to AUTO select\n", + pr_err("%s selected but CPU doesn't have Enhanced or Automatic IBRS. Switching to AUTO select\n", mitigation_options[i].option); return SPECTRE_V2_CMD_AUTO; } @@ -1479,8 +1479,12 @@ static void __init spectre_v2_select_mitigation(void) pr_err(SPECTRE_V2_EIBRS_EBPF_MSG); if (spectre_v2_in_ibrs_mode(mode)) { - x86_spec_ctrl_base |= SPEC_CTRL_IBRS; - write_spec_ctrl_current(x86_spec_ctrl_base, true); + if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) { + msr_set_bit(MSR_EFER, _EFER_AUTOIBRS); + } else { + x86_spec_ctrl_base |= SPEC_CTRL_IBRS; + write_spec_ctrl_current(x86_spec_ctrl_base, true); + } } switch (mode) { @@ -1564,8 +1568,8 @@ static void __init spectre_v2_select_mitigation(void) /* * Retpoline protects the kernel, but doesn't protect firmware. IBRS * and Enhanced IBRS protect firmware too, so enable IBRS around - * firmware calls only when IBRS / Enhanced IBRS aren't otherwise - * enabled. + * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't + * otherwise enabled. * * Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because * the user might select retpoline on the kernel command line and if diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 8d28cd7c9072..57efc4b6eac3 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1226,8 +1226,8 @@ static const __initconst struct x86_cpu_id cpu_vuln_whitelist[] = { VULNWL_AMD(0x12, NO_MELTDOWN | NO_SSB | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO), /* FAMILY_ANY must be last, otherwise 0x0f - 0x12 matches won't work */ - VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO), - VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO), + VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB), + VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB), /* Zhaoxin Family 7 */ VULNWL(CENTAUR, 7, X86_MODEL_ANY, NO_SPECTRE_V2 | NO_SWAPGS | NO_MMIO), @@ -1338,8 +1338,16 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) !cpu_has(c, X86_FEATURE_AMD_SSB_NO)) setup_force_cpu_bug(X86_BUG_SPEC_STORE_BYPASS); - if (ia32_cap & ARCH_CAP_IBRS_ALL) + /* + * AMD's AutoIBRS is equivalent to Intel's eIBRS - use the Intel feature + * flag and protect from vendor-specific bugs via the whitelist. + */ + if ((ia32_cap & ARCH_CAP_IBRS_ALL) || cpu_has(c, X86_FEATURE_AUTOIBRS)) { setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED); + if (!cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) && + !(ia32_cap & ARCH_CAP_PBRSB_NO)) + setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB); + } if (!cpu_matches(cpu_vuln_whitelist, NO_MDS) && !(ia32_cap & ARCH_CAP_MDS_NO)) { @@ -1401,11 +1409,6 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) setup_force_cpu_bug(X86_BUG_RETBLEED); } - if (cpu_has(c, X86_FEATURE_IBRS_ENHANCED) && - !cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) && - !(ia32_cap & ARCH_CAP_PBRSB_NO)) - setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB); - if (cpu_matches(cpu_vuln_whitelist, NO_MELTDOWN)) return;