Message ID | 20230227210526.83182-1-itazur@amazon.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp2651327wrd; Mon, 27 Feb 2023 13:17:28 -0800 (PST) X-Google-Smtp-Source: AK7set9m6YZIIfs+IOVXZM7zXBQ6pGWFbbF3W6ULzytYAXqn4e3jTCQQdeWM+2KlppltSPkZXWZe X-Received: by 2002:a17:903:120d:b0:19c:bcb4:cacb with SMTP id l13-20020a170903120d00b0019cbcb4cacbmr287878plh.56.1677532648330; Mon, 27 Feb 2023 13:17:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677532648; cv=none; d=google.com; s=arc-20160816; b=P426J8wD8l4zYC87YSUOH8RpuuB3MRQgW9X+jKQbiAQykU0T6WgOUFHWG7YA/7f2bZ g5XRVnojQdVFvcp32ZQvwvM0mDrEdNzHcqWPW5ZCgmfEGtwXIkgxzCxjhRlI3DQpVhES TnX46sVTjWg0TAFuPZ/qz6eC/1grlJsy8sBaKJyhLAP3pg/Ut8fQaSM0K6LAH8gi+reh QjSFDNhU8gvfoQVHBaMyVWdZLncqTME9nHv/57m1zHpljmZbiCFvXcVGCCOybtI29v1P YsmvGeD11638PWc5ipRlxTCGNT9QD4ovQSIt1Yk8oKwmorrKDN3+IBFNzB//PnqviudZ Bg1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=2KiKvFK2pjl8jx0HGi3QbZP7fKXFC2HYLTg5JQbTjrI=; b=ADf/bLjZjfe710pWQgW0NpalbuCKoY3Hfjp+6pRWBYCKAwDHisy2Pdwd6VsSt3cykh 5U5UaBWfeQpiyE0V+W2fwIQNhP9VapVUtVxGfVDspNhDEn2gHqO9a2CuA31wVQPcZQli +9Xd938PA4rH/WRbksdXxNgY0oSkg5p5iybFN5Egqg82tcSipTB6c1+zPX+f8HkUD+Sk 6/Xy7kgEIuaW2trthQM7PYM2yR0Q6pc5mm5Hl7EtdR/wlx4vsjtr7DReHYLA6ejKOxT5 7YSipRgeiRnSHKISPlSQ5ak0F5LIxARK8qwGeKrcdenZpqlaubpNK09d1I+7rCgWADlV s7XA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=SND8uTU4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p6-20020a1709026b8600b0019cc6eac51bsi7277187plk.1.2023.02.27.13.17.15; Mon, 27 Feb 2023 13:17:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=SND8uTU4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229904AbjB0VOk (ORCPT <rfc822;wenzhi022@gmail.com> + 99 others); Mon, 27 Feb 2023 16:14:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229800AbjB0VOi (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 27 Feb 2023 16:14:38 -0500 Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB70E23D85; Mon, 27 Feb 2023 13:14:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1677532476; x=1709068476; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=2KiKvFK2pjl8jx0HGi3QbZP7fKXFC2HYLTg5JQbTjrI=; b=SND8uTU4z93mnunsWeyCUw+sFXFoEPzgWKhmLRCcIAb3XsEeR0uVLUuw SCJ9d4IZv2qEsc9inQSOJGyZ9/vW7T/e1OwIkDRuuiXcGdhjG5nREqis0 zWmj+llWVQUNKcnMksg0T+5G2rE4ug/FRMYijSXBXd8rRX0QrylAwgRD7 k=; X-IronPort-AV: E=Sophos;i="5.98,220,1673913600"; d="scan'208";a="187220727" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-d40ec5a9.us-west-2.amazon.com) ([10.25.36.210]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 21:05:49 +0000 Received: from EX13MTAUWB002.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-pdx-2a-m6i4x-d40ec5a9.us-west-2.amazon.com (Postfix) with ESMTPS id A4C7140D3F; Mon, 27 Feb 2023 21:05:47 +0000 (UTC) Received: from EX19D002ANA003.ant.amazon.com (10.37.240.141) by EX13MTAUWB002.ant.amazon.com (10.43.161.202) with Microsoft SMTP Server (TLS) id 15.0.1497.45; Mon, 27 Feb 2023 21:05:47 +0000 Received: from b0f1d8753182.ant.amazon.com (10.95.130.142) by EX19D002ANA003.ant.amazon.com (10.37.240.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.24; Mon, 27 Feb 2023 21:05:42 +0000 From: Takahiro Itazuri <itazur@amazon.com> To: <kvm@vger.kernel.org>, <x86@kernel.org> CC: Sean Christopherson <seanjc@google.com>, Paolo Bonzini <pbonzini@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, <linux-kernel@vger.kernel.org>, "Takahiro Itazuri" <zulinx86@gmail.com>, Takahiro Itazuri <itazur@amazon.com> Subject: [PATCH 0/2] KVM: x86: Propagate AMD-specific IBRS bits to guests Date: Mon, 27 Feb 2023 21:05:24 +0000 Message-ID: <20230227210526.83182-1-itazur@amazon.com> X-Mailer: git-send-email 2.38.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.95.130.142] X-ClientProxiedBy: EX19D031UWC001.ant.amazon.com (10.13.139.241) To EX19D002ANA003.ant.amazon.com (10.37.240.141) X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759020474018266058?= X-GMAIL-MSGID: =?utf-8?q?1759020474018266058?= |
Series |
KVM: x86: Propagate AMD-specific IBRS bits to guests
|
|
Message
Takahiro Itazuri
Feb. 27, 2023, 9:05 p.m. UTC
VMMs retrieve supported CPUID features via KVM_GET_SUPPORTED_CPUID to construct CPUID information to be passed to KVM_SET_CPUID2. Most CPUID feature bits related to speculative attacks are propagated from host CPUID. But AMD processors have AMD-specific IBRS related bits in CPUID Fn8000_0008_EBX (ref: AMD64 Architecture Programmer's Manual Volume 3: General-Purpose and System Instructions) and some bits are not propagated to guests. Enable propagation of these bits to guests, so that VMMs don't have to enable them explicitly based on host CPUID. Takahiro Itazuri (2): x86/cpufeatures: Add AMD-specific IBRS bits KVM: x86: Propagate AMD-specific IBRS related bits arch/x86/include/asm/cpufeatures.h | 3 +++ arch/x86/kvm/cpuid.c | 5 +++-- 2 files changed, 6 insertions(+), 2 deletions(-)
Comments
On Mon, Feb 27, 2023 at 09:05:24PM +0000, Takahiro Itazuri wrote: > VMMs retrieve supported CPUID features via KVM_GET_SUPPORTED_CPUID to > construct CPUID information to be passed to KVM_SET_CPUID2. Most CPUID > feature bits related to speculative attacks are propagated from host > CPUID. But AMD processors have AMD-specific IBRS related bits in CPUID > Fn8000_0008_EBX (ref: AMD64 Architecture Programmer's Manual Volume 3: > General-Purpose and System Instructions) and some bits are not > propagated to guests. > > Enable propagation of these bits to guests, so that VMMs don't have to > enable them explicitly based on host CPUID. How hard is it for the VMMs to enable them?
Date: Mon, 27 Feb 2023 22:40:21 +0100 From: Borislav Petkov <bp@alien8.de> > On Mon, Feb 27, 2023 at 09:05:24PM +0000, Takahiro Itazuri wrote: > > VMMs retrieve supported CPUID features via KVM_GET_SUPPORTED_CPUID to > > construct CPUID information to be passed to KVM_SET_CPUID2. Most CPUID > > feature bits related to speculative attacks are propagated from host > > CPUID. But AMD processors have AMD-specific IBRS related bits in CPUID > > Fn8000_0008_EBX (ref: AMD64 Architecture Programmer's Manual Volume 3: > > General-Purpose and System Instructions) and some bits are not > > propagated to guests. > > > > Enable propagation of these bits to guests, so that VMMs don't have to > > enable them explicitly based on host CPUID. > > How hard is it for the VMMs to enable them? Actually it is not so hard. What VMMs need to do is: 1. Get host CPUID value. 2. Check if these bits are set. 3. Modify the return value of KVM_GET_SUPPORTED_CPUID based on step 2. 4. Pass it to KVM_SET_CPUID2. If these bits are propagated to guests same as other bits, VMMs can skip the above process. https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt > This ioctl returns x86 cpuid features which are supported by both the > hardware and kvm in its default configuration. Userspace can use the > information returned by this ioctl to construct cpuid information (for > KVM_SET_CPUID2) that is consistent with hardware, kernel, and > userspace capabilities, and with user requirements (for example, the > user may wish to constrain cpuid to emulate older hardware, or for > feature consistency across a cluster). VMMs trust to some extent that KVM_GET_SUPPORTED_CPUID returns cpuid information consistent with hardware, although they should not for some leaves (like CPU topoligy). IMHO, propagating these bits without VMM actions would be helpful since guests come to know IBRS related information of processors by default and applies mitigations properly based on that information. Best regards, Takahiro Itazuri
On Tue, Feb 28, 2023 at 06:13:45PM +0000, Takahiro Itazuri wrote: > VMMs trust to some extent that KVM_GET_SUPPORTED_CPUID returns cpuid > information consistent with hardware, although they should not for some > leaves (like CPU topoligy). IMHO, propagating these bits without VMM > actions would be helpful since guests come to know IBRS related > information of processors by default and applies mitigations properly > based on that information. I'd prefer if VMMs did supply whatever they prefer to the guests instead. None of those bits are used in the kernel for mitigations, as you've realized. Thx.
Date: Tue, 28 Feb 2023 20:24:12 +0100 From: Borislav Petkov <bp@alien8.de> > I'd prefer if VMMs did supply whatever they prefer to the guests > instead. None of those bits are used in the kernel for mitigations, as > you've realized. It is true that the kernel does not use those bits at all, but any codes could be run inside guests. One of examples is the following spectre/meltdown checker scipt used as de facto standard. https://github.com/speed47/spectre-meltdown-checker/blob/master/spectre-meltdown-checker.sh#L2768 Best regards, Takahiro Itazuri
On Tue, Feb 28, 2023 at 07:41:53PM +0000, Takahiro Itazuri wrote: > It is true that the kernel does not use those bits at all, but any > codes could be run inside guests. So you mean we should stick *all* CPUID leafs in there just because anything can run in guests? What is the hypervisor then for? > One of examples is the following spectre/meltdown checker scipt used as > de facto standard. Really? Says who? $ grep -r . /sys/devices/system/cpu/vulnerabilities/ gives you all you need to know. And if something's missing, then I'm listening. Thx.
Date: Tue, 28 Feb 2023 21:45:56 +0100 From: Borislav Petkov <bp@alien8.de> > So you mean we should stick *all* CPUID leafs in there just because > anything can run in guests? > > What is the hypervisor then for? I'm still a kernel newbie and I don't have a strong opinion for that. I just thought it would be helpful if the KVM_GET_SUPPORTED_CPUID API returns the same security information as the host, as long as it is harmless. I'm inclined to withdraw this patch if it is not worth enough. > Really? Says who? > > $ grep -r . /sys/devices/system/cpu/vulnerabilities/ > > gives you all you need to know. > > And if something's missing, then I'm listening. "De facto standard" was too much. I apologize for my incorrect expression and poor English. What I wanted to say is that the script was introduced as a useful tool by Intel and SLES and it provides some additional information (IBRS always-on in this case). https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/secure-coding/spectre-and-meltdown-checker-script.html https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-spectre.html Best regards, Takahiro Itazuri
On Tue, Feb 28, 2023 at 10:24:16PM +0000, Takahiro Itazuri wrote: > I'm still a kernel newbie and I don't have a strong opinion for that. > I just thought it would be helpful if the KVM_GET_SUPPORTED_CPUID API > returns the same security information as the host, as long as it is > harmless. Not harmless - cpufeatures.h should contain flags which the kernel uses and not *every* CPUID bit out there. If you want to advertize flags to guests, see arch/x86/kvm/reverse_cpuid.h and the KVM-only feature flags. You can add them there. > https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-spectre.html Well, I was against adding that to the documentation when I was at SUSE but ...
On Tue, Feb 28, 2023, Borislav Petkov wrote: > On Tue, Feb 28, 2023 at 10:24:16PM +0000, Takahiro Itazuri wrote: > > I'm still a kernel newbie and I don't have a strong opinion for that. > > I just thought it would be helpful if the KVM_GET_SUPPORTED_CPUID API > > returns the same security information as the host, as long as it is > > harmless. > > Not harmless - cpufeatures.h should contain flags which the kernel uses > and not *every* CPUID bit out there. I thought that the consensus was that adding unused-by-the-kernel flags to cpufeatures.h is ok so long as the feature is hidden from /proc/cpuinfo and the kernel already dedicates a word to the CPUID leaf?
On 3/6/23 22:16, Sean Christopherson wrote: >> Not harmless - cpufeatures.h should contain flags which the kernel uses >> and not*every* CPUID bit out there. > > I thought that the consensus was that adding unused-by-the-kernel flags to > cpufeatures.h is ok so long as the feature is hidden from /proc/cpuinfo and the > kernel already dedicates a word to the CPUID leaf? Yeah, I understand adding no new CPUID leaf just for KVM, but you don't gain anything really from not having X86_FEATURE_* defines. Paolo
On 2/27/23 22:40, Borislav Petkov wrote: > On Mon, Feb 27, 2023 at 09:05:24PM +0000, Takahiro Itazuri wrote: >> VMMs retrieve supported CPUID features via KVM_GET_SUPPORTED_CPUID to >> construct CPUID information to be passed to KVM_SET_CPUID2. Most CPUID >> feature bits related to speculative attacks are propagated from host >> CPUID. But AMD processors have AMD-specific IBRS related bits in CPUID >> Fn8000_0008_EBX (ref: AMD64 Architecture Programmer's Manual Volume 3: >> General-Purpose and System Instructions) and some bits are not >> propagated to guests. >> >> Enable propagation of these bits to guests, so that VMMs don't have to >> enable them explicitly based on host CPUID. > > How hard is it for the VMMs to enable them? Let me rephrase the second paragraph of Takahiro's commit message: "Tell the VMMs that they can pass the bits to the guests, instead of having to second-guess that the hypervisor does not have to do anything to support these bits". In general, userspace should not second guess the hypervisor. There are some rare cases in which QEMU (and probably the proprietary hypervisors at Google and Amazon) does that, but in general you want it to trust information coming from the kernel. New CPUID bits are quite frequent, and sometimes also stupidly difficult to get right, so if filtering CPUID can be done in the kernel you won't have to do the same change N times in _all_ userspaces that use KVM. Paolo
On Mon, Mar 06, 2023 at 01:16:25PM -0800, Sean Christopherson wrote: > I thought that the consensus was that adding unused-by-the-kernel flags to > cpufeatures.h is ok so long as the feature is hidden from /proc/cpuinfo and the > kernel already dedicates a word to the CPUID leaf? I guess we should finally write it down in Documentation/x86/cpuinfo.rst And in case there's no dedicated word, it should be resorted to KVM-only feature flags. In any case, I'd like for baremetal CPUID stuff to be decoupled from KVM's machinery as far as possible as both have different goals wrt feature flags. Thx.
On 3/6/23 22:44, Borislav Petkov wrote: >> I thought that the consensus was that adding unused-by-the-kernel flags to >> cpufeatures.h is ok so long as the feature is hidden from /proc/cpuinfo and the >> kernel already dedicates a word to the CPUID leaf? > I guess we should finally write it down in Documentation/x86/cpuinfo.rst > > And in case there's no dedicated word, it should be resorted to KVM-only > feature flags. > > In any case, I'd like for baremetal CPUID stuff to be decoupled from > KVM's machinery as far as possible as both have different goals wrt > feature flags. It's very rare that KVM can provide a CPUID feature if the kernel has masked it, so if the kernel needs to know about a feature word than KVM most likely needs to know what kind of massaging the kernel has done. Paolo
On Mon, Mar 06, 2023 at 10:47:18PM +0100, Paolo Bonzini wrote: > It's very rare that KVM can provide a CPUID feature if the kernel has > masked it, I'm talking about pure hw feature bits which don't need any enablement. Like AVX512 insns subset support or something else which the hw does without the need for the kernel. Those should be KVM-only if baremetal doesn't use them.
On Mon, Mar 06, 2023, Borislav Petkov wrote: > On Mon, Mar 06, 2023 at 10:47:18PM +0100, Paolo Bonzini wrote: > > It's very rare that KVM can provide a CPUID feature if the kernel has > > masked it, > > I'm talking about pure hw feature bits which don't need any enablement. > Like AVX512 insns subset support or something else which the hw does > without the need for the kernel. > > Those should be KVM-only if baremetal doesn't use them. I don't see what such a rule buys us beyond complexity and, IMO, unnecessary maintenance burden. As Paolo pointed out, when there's an existing word, the only "cost" is the existence of the #define. The bit is still present in the capabilities, and KVM relies on this! And as mentioned in the tangent about reworking cpufeatures[*], I get a _lot_ of value out of cpufeatures.h being fully populated for known bits (in defined words). Forcing KVM to #define bits in reverse_cpuid.h just because the kernel doesn't need the macro will inevitably lead to confusion for KVM developers, both when writing new code and when reading existing code. [*] https://lore.kernel.org/all/Y8nhtjFcsB63UsmQ@google.com
On Tue, Mar 07, 2023 at 10:49:22AM -0800, Sean Christopherson wrote: > I don't see what such a rule buys us beyond complexity and, IMO, unnecessary > maintenance burden. As Paolo pointed out, when there's an existing word, the Maybe I wasn't clear enough - I don't mind existing words. What I mind is adding new ones only for KVM's sake.
On Tue, Mar 07, 2023, Borislav Petkov wrote: > On Tue, Mar 07, 2023 at 10:49:22AM -0800, Sean Christopherson wrote: > > I don't see what such a rule buys us beyond complexity and, IMO, unnecessary > > maintenance burden. As Paolo pointed out, when there's an existing word, the > > Maybe I wasn't clear enough - I don't mind existing words. What I mind > is adding new ones only for KVM's sake. Ah, gotcha. We're on the same page then. Thanks!
On Tue, Mar 07, 2023 at 11:28:26AM -0800, Sean Christopherson wrote:
> Ah, gotcha. We're on the same page then. Thanks!
Yeah, now someone needs to document it.. :-)
I'll put it on my TODO.