[1/3] KVM: VMX: Don't rely _only_ on CPUID to enforce XCR0 restrictions for ECREATE
Message ID | 20230405005911.423699-2-seanjc@google.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp256224vqo; Tue, 4 Apr 2023 18:06:23 -0700 (PDT) X-Google-Smtp-Source: AKy350bviX9rnSY6MafeCytFoa+TZVJrOpIxoT3stfmnw9esJxlFr64Cm62xL/tI/VdTTGYO/JWn X-Received: by 2002:a17:903:88b:b0:1a1:e01e:7279 with SMTP id kt11-20020a170903088b00b001a1e01e7279mr4435172plb.4.1680656783539; Tue, 04 Apr 2023 18:06:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680656783; cv=none; d=google.com; s=arc-20160816; b=E/Bqc50iwM8vkqkrIrFeQ6fit6z+Xk6gboS1nezipXvgcH7TzTmzn4kUbWwB3KTzUn V4h2rfWuYpksv20tJMHcBV9rkGjRE3AoArwEqGLeVEjCzijuUtbz9dsk2hEe40GqKiAa YHdwsVmZNW61FwRDbzJMFpGviLNaJtQ3YxNIRn5FVIziTTLfCVpNau2Faf2rxxMQXQ62 I7DpIanc8OgmakstavbErzJXZh4a8TV1WSF/L/liZvnmp9D6H01dF8U+SoDftndHjvvC aW9RqMZeERYTHId9l1Pxfvg+Th0zA2baJBLL8+L7TVWf0lQROKyX4Ij6KSnD3+hKFrM4 0NzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=PLGWC8bjiCbqzAe+2LaZO4qsM4GvEpJuy8h3g8GekVY=; b=j4WLQsalh039tIVwHph0Zn2kqAT2IfOo4EZX4c3sRv8XolYx1LvDQLoL+6Xlx92vMZ vSokI2o6AFwUsiD2KxUGcxnEWog5sOSXnfH9HrURrR4QWAuRdZf9KFZTcTPm8KPgl1AY 7JqIWPssZpnzQ6m1exY5oCz0yT/W4tJFCCJVlRkbwKS/7DiMpvY5wwshQaBdU1fvGKBA iisldnMA6N5fliLSbwDsRxXrG+VDPAVKaRV6CITwqGc4+I4r/J8Dg8HdKllOPDrQEN/j zcn0MgzpXeyNs5wkbgVH7V0FnY2mqJ61DCEQX3hr9SCRi/vgzikwzJ+zdOxpBbn6LppO x9jA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b="gVF/Uw+g"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k9-20020a63f009000000b004a4eae7c943si11319536pgh.535.2023.04.04.18.06.10; Tue, 04 Apr 2023 18:06:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b="gVF/Uw+g"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236647AbjDEA7T (ORCPT <rfc822;lkml4gm@gmail.com> + 99 others); Tue, 4 Apr 2023 20:59:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236441AbjDEA7R (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 4 Apr 2023 20:59:17 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 209CF1BEA for <linux-kernel@vger.kernel.org>; Tue, 4 Apr 2023 17:59:16 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id m12-20020a62f20c000000b0062612a76a08so15330951pfh.2 for <linux-kernel@vger.kernel.org>; Tue, 04 Apr 2023 17:59:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680656355; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=PLGWC8bjiCbqzAe+2LaZO4qsM4GvEpJuy8h3g8GekVY=; b=gVF/Uw+gSqX3uWJdb16W3jVtNtrHQr6D5L84MS4xLmhm6IUvTa+93i6w1I25DZf4jn Dd1bzqEFttWNNPRb2DarphJkl8zKFFr8zKOA40L4od5qv9oEiDk6r831QpejBKnHq82l QYDAmQMYSzK6Lsq3wcLlP9WOuQ7+7wabq1HlZ0KW799j4S/VLiKMFNovN864xvln0wsK zbj9mWBagl/UDX0m4r9S1vwH1AESsE7ei8c2dOcn16xkglwtPbFQu+18tgu1fsvxVtO7 2pTCFma2FYLBy6j4N/P0bLvYFixRklR/5aUe3TcS4tttEcpVZUXYMwkjQd4DH/EW1NSA 6A+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680656355; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PLGWC8bjiCbqzAe+2LaZO4qsM4GvEpJuy8h3g8GekVY=; b=3wVAgyJuqG+jIuTxKssxjjf14FNoScjr4bj6ZywZY/kVk3aYVmHMts0utgUsXbIsvm zutmfbxhJHZufFdrDkWaYNsi7h/iGK9Qp3W4796sG8WyFA4J3N4rJbUba/N3qh/d8RRn ZdsqLjyBv1W7a6YvoQK8dquNNel9oL5KjdO7gLAtClUdUI9dBqH10WZ109UnR34PWW/U PK/+OYhnIGHAMHTTkljBblQXp04ovRPWWAi2H6o9iudFvW6XMkG4idiPs9SLoWvfEOig SWkt0yF98ZKy9cktBnGKdlRQYrlWG81MWUALwWAQDg50DFqEkAuYYX0qGmhBTO0ourAq NI4g== X-Gm-Message-State: AAQBX9cHALs2Y5Tx40rn2X/bMD0xrbghcXXlqHjtV8QNTdO28anwAfHb v9m+QxhQpHA/xcgSFZ5abPmohBNhPwQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a63:4c1f:0:b0:4fb:1f2a:e6c6 with SMTP id z31-20020a634c1f000000b004fb1f2ae6c6mr1346195pga.2.1680656355579; Tue, 04 Apr 2023 17:59:15 -0700 (PDT) Reply-To: Sean Christopherson <seanjc@google.com> Date: Tue, 4 Apr 2023 17:59:09 -0700 In-Reply-To: <20230405005911.423699-1-seanjc@google.com> Mime-Version: 1.0 References: <20230405005911.423699-1-seanjc@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230405005911.423699-2-seanjc@google.com> Subject: [PATCH 1/3] KVM: VMX: Don't rely _only_ on CPUID to enforce XCR0 restrictions for ECREATE From: Sean Christopherson <seanjc@google.com> To: Sean Christopherson <seanjc@google.com>, Paolo Bonzini <pbonzini@redhat.com> Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Kai Huang <kai.huang@intel.com> Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-7.7 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762296367738413450?= X-GMAIL-MSGID: =?utf-8?q?1762296367738413450?= |
Series |
[1/3] KVM: VMX: Don't rely _only_ on CPUID to enforce XCR0 restrictions for ECREATE
|
|
Commit Message
Sean Christopherson
April 5, 2023, 12:59 a.m. UTC
Explicitly check the vCPU's supported XCR0 when determining whether or not
the XFRM for ECREATE is valid. Checking CPUID works because KVM updates
guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed
XCR0, but that is rather subtle and KVM should not modify guest CPUID
except for modeling true runtime behavior (allowed XFRM is most definitely
not "runtime" behavior).
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/vmx/sgx.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Comments
On Tue, 2023-04-04 at 17:59 -0700, Sean Christopherson wrote: > Explicitly check the vCPU's supported XCR0 when determining whether or not > the XFRM for ECREATE is valid. Checking CPUID works because KVM updates > guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed > XCR0, but that is rather subtle and KVM should not modify guest CPUID > except for modeling true runtime behavior (allowed XFRM is most definitely > not "runtime" behavior). > > Signed-off-by: Sean Christopherson <seanjc@google.com> > --- > arch/x86/kvm/vmx/sgx.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c > index aa53c98034bf..362a31b19b0e 100644 > --- a/arch/x86/kvm/vmx/sgx.c > +++ b/arch/x86/kvm/vmx/sgx.c > @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, > (u32)attributes & ~sgx_12_1->eax || > (u32)(attributes >> 32) & ~sgx_12_1->ebx || > (u32)xfrm & ~sgx_12_1->ecx || > - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { > + (u32)(xfrm >> 32) & ~sgx_12_1->edx || > + xfrm & ~vcpu->arch.guest_supported_xcr0) { Perhaps this change is needed even without patch 2? This is because when CPUID 0xD doesn't exist, guest_supported_xcr0 is 0. But when CPUID 0xD doesn't exist, IIUC currently KVM doesn't clear SGX in CPUID, and sgx_12_1->ecx is always set to 0x3. __handle_encls_ereate() doesn't check CPUID 0xD either, so w/o above explicit check xfrm against guest_supported_xcr0, it seems guest can successfully run ECREATE when it doesn't have CPUID 0xD? > kvm_inject_gp(vcpu, 0); > return 1; > } > -- > 2.40.0.348.gf938b09366-goog >
On Wed, Apr 05, 2023, Huang, Kai wrote: > On Tue, 2023-04-04 at 17:59 -0700, Sean Christopherson wrote: > > Explicitly check the vCPU's supported XCR0 when determining whether or not > > the XFRM for ECREATE is valid. Checking CPUID works because KVM updates > > guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed > > XCR0, but that is rather subtle and KVM should not modify guest CPUID > > except for modeling true runtime behavior (allowed XFRM is most definitely > > not "runtime" behavior). > > > > Signed-off-by: Sean Christopherson <seanjc@google.com> > > --- > > arch/x86/kvm/vmx/sgx.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c > > index aa53c98034bf..362a31b19b0e 100644 > > --- a/arch/x86/kvm/vmx/sgx.c > > +++ b/arch/x86/kvm/vmx/sgx.c > > @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, > > (u32)attributes & ~sgx_12_1->eax || > > (u32)(attributes >> 32) & ~sgx_12_1->ebx || > > (u32)xfrm & ~sgx_12_1->ecx || > > - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { > > + (u32)(xfrm >> 32) & ~sgx_12_1->edx || > > + xfrm & ~vcpu->arch.guest_supported_xcr0) { > > Perhaps this change is needed even without patch 2? > > This is because when CPUID 0xD doesn't exist, guest_supported_xcr0 is 0. But > when CPUID 0xD doesn't exist, IIUC currently KVM doesn't clear SGX in CPUID, and > sgx_12_1->ecx is always set to 0x3. Hrm, that's a bug in this patch. Drat. More below. > __handle_encls_ereate() doesn't check CPUID 0xD either, so w/o above explicit > check xfrm against guest_supported_xcr0, it seems guest can successfully run > ECREATE when it doesn't have CPUID 0xD? ECREATE doesn't have a strict dependency on CPUID 0xD or XSAVE. This exact scenario is called out in the SDM: Legal values for SECS.ATTRIBUTES.XFRM conform to these requirements: * XFRM[1:0] must be set to 0x3. * If the processor does support XSAVE, XFRM must contain a value that would be legal if loaded into XCR0. * If the processor does not support XSAVE, or if the system software has not enabled XSAVE, then XFRM[63:2] must be zero. So the above needs to be either xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) or (xfrm & ~XFEATURE_MASK_FPSSE & ~vcpu->arch.guest_supported_xcr0) I think I prefer the first one as I find it slightly more obvious that FP+SSE are allowed in addition to the XCR0 bits.
On Wed, 2023-04-05 at 18:44 -0700, Sean Christopherson wrote: > On Wed, Apr 05, 2023, Huang, Kai wrote: > > On Tue, 2023-04-04 at 17:59 -0700, Sean Christopherson wrote: > > > Explicitly check the vCPU's supported XCR0 when determining whether or not > > > the XFRM for ECREATE is valid. Checking CPUID works because KVM updates > > > guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed > > > XCR0, but that is rather subtle and KVM should not modify guest CPUID > > > except for modeling true runtime behavior (allowed XFRM is most definitely > > > not "runtime" behavior). > > > > > > Signed-off-by: Sean Christopherson <seanjc@google.com> > > > --- > > > arch/x86/kvm/vmx/sgx.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c > > > index aa53c98034bf..362a31b19b0e 100644 > > > --- a/arch/x86/kvm/vmx/sgx.c > > > +++ b/arch/x86/kvm/vmx/sgx.c > > > @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, > > > (u32)attributes & ~sgx_12_1->eax || > > > (u32)(attributes >> 32) & ~sgx_12_1->ebx || > > > (u32)xfrm & ~sgx_12_1->ecx || > > > - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { > > > + (u32)(xfrm >> 32) & ~sgx_12_1->edx || > > > + xfrm & ~vcpu->arch.guest_supported_xcr0) { > > > > Perhaps this change is needed even without patch 2? > > > > This is because when CPUID 0xD doesn't exist, guest_supported_xcr0 is 0. But > > when CPUID 0xD doesn't exist, IIUC currently KVM doesn't clear SGX in CPUID, and > > sgx_12_1->ecx is always set to 0x3. > > Hrm, that's a bug in this patch. Drat. More below. > > > __handle_encls_ereate() doesn't check CPUID 0xD either, so w/o above explicit > > check xfrm against guest_supported_xcr0, it seems guest can successfully run > > ECREATE when it doesn't have CPUID 0xD? > > ECREATE doesn't have a strict dependency on CPUID 0xD or XSAVE. This exact scenario > is called out in the SDM: > > Legal values for SECS.ATTRIBUTES.XFRM conform to these requirements: > * XFRM[1:0] must be set to 0x3. > * If the processor does support XSAVE, XFRM must contain a value that would > be legal if loaded into XCR0. > * If the processor does not support XSAVE, or if the system software has not > enabled XSAVE, then XFRM[63:2] must be zero. > > So the above needs to be either > > xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) > > or > > (xfrm & ~XFEATURE_MASK_FPSSE & ~vcpu->arch.guest_supported_xcr0) > > > I think I prefer the first one as I find it slightly more obvious that FP+SSE are > allowed in addition to the XCR0 bits. The above check doesn't verify xfrm is a super set of 0x3. I think we verify that per SDM: 39.7.3 Processor Extended States and ENCLS[ECREATE] The ECREATE leaf function of the ENCLS instruction enforces a number of consistency checks described earlier. The execution of ENCLS[ECREATE] leaf function results in a #GP(0) in any of the following cases: • SECS.ATTRIBUTES.XFRM[1:0] is not 3. • The processor does not support XSAVE and any of the following is true: — SECS.ATTRIBUTES.XFRM[63:2] is not 0. — SECS.SSAFRAMESIZE is 0. • The processor supports XSAVE and any of the following is true: — XSETBV would fault on an attempt to load XFRM into XCR0. — XFRM[63]=1. — The SSAFRAME is too small to hold required, enabled states ... And in the ECREATE pseudo code, the relevant parts seem to be: (* Check lower 2 bits of XFRM are set *) IF ( ( DS:TMP_SECS.ATTRIBUTES.XFRM BitwiseAND 03H) ≠ 03H) THEN #GP(0); FI; IF (XFRM is illegal) THEN #GP(0); FI; The first part is clear, but the second part is vague. I am not sure in hardware behaviour, whether XCR0 is actually checked in ECREATE. It's more likely XCRO is actually checked in EENTER. But I think it's just fine to also check against XCR0 here.
On Thu, Apr 06, 2023, Huang, Kai wrote: > On Wed, 2023-04-05 at 18:44 -0700, Sean Christopherson wrote: > > On Wed, Apr 05, 2023, Huang, Kai wrote: > > > On Tue, 2023-04-04 at 17:59 -0700, Sean Christopherson wrote: > > > > Explicitly check the vCPU's supported XCR0 when determining whether or not > > > > the XFRM for ECREATE is valid. Checking CPUID works because KVM updates > > > > guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed > > > > XCR0, but that is rather subtle and KVM should not modify guest CPUID > > > > except for modeling true runtime behavior (allowed XFRM is most definitely > > > > not "runtime" behavior). > > > > > > > > Signed-off-by: Sean Christopherson <seanjc@google.com> > > > > --- > > > > arch/x86/kvm/vmx/sgx.c | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c > > > > index aa53c98034bf..362a31b19b0e 100644 > > > > --- a/arch/x86/kvm/vmx/sgx.c > > > > +++ b/arch/x86/kvm/vmx/sgx.c > > > > @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, > > > > (u32)attributes & ~sgx_12_1->eax || > > > > (u32)(attributes >> 32) & ~sgx_12_1->ebx || > > > > (u32)xfrm & ~sgx_12_1->ecx || > > > > - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { > > > > + (u32)(xfrm >> 32) & ~sgx_12_1->edx || > > > > + xfrm & ~vcpu->arch.guest_supported_xcr0) { > > > > > > Perhaps this change is needed even without patch 2? > > > > > > This is because when CPUID 0xD doesn't exist, guest_supported_xcr0 is 0. But > > > when CPUID 0xD doesn't exist, IIUC currently KVM doesn't clear SGX in CPUID, and > > > sgx_12_1->ecx is always set to 0x3. > > > > Hrm, that's a bug in this patch. Drat. More below. > > > > > __handle_encls_ereate() doesn't check CPUID 0xD either, so w/o above explicit > > > check xfrm against guest_supported_xcr0, it seems guest can successfully run > > > ECREATE when it doesn't have CPUID 0xD? > > > > ECREATE doesn't have a strict dependency on CPUID 0xD or XSAVE. This exact scenario > > is called out in the SDM: > > > > Legal values for SECS.ATTRIBUTES.XFRM conform to these requirements: > > * XFRM[1:0] must be set to 0x3. > > * If the processor does support XSAVE, XFRM must contain a value that would > > be legal if loaded into XCR0. > > * If the processor does not support XSAVE, or if the system software has not > > enabled XSAVE, then XFRM[63:2] must be zero. > > > > So the above needs to be either > > > > xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) > > > > or > > > > (xfrm & ~XFEATURE_MASK_FPSSE & ~vcpu->arch.guest_supported_xcr0) > > > > > > I think I prefer the first one as I find it slightly more obvious that FP+SSE are > > allowed in addition to the XCR0 bits. > > The above check doesn't verify xfrm is a super set of 0x3. I think we verify > that per SDM: Oooh, right. It's not that FP+SSE are always allowed, it's that FP+SSE must always be _set_. So this? xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) || (xfrm & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE > 39.7.3 Processor Extended States and ENCLS[ECREATE] > > The ECREATE leaf function of the ENCLS instruction enforces a number of > consistency checks described earlier. The execution of ENCLS[ECREATE] leaf > function results in a #GP(0) in any of the following cases: > • SECS.ATTRIBUTES.XFRM[1:0] is not 3. > • The processor does not support XSAVE and any of the following is true: > — SECS.ATTRIBUTES.XFRM[63:2] is not 0. > — SECS.SSAFRAMESIZE is 0. > • The processor supports XSAVE and any of the following is true: > — XSETBV would fault on an attempt to load XFRM into XCR0. > — XFRM[63]=1. > — The SSAFRAME is too small to hold required, enabled states ... > > > And in the ECREATE pseudo code, the relevant parts seem to be: > > (* Check lower 2 bits of XFRM are set *) > IF ( ( DS:TMP_SECS.ATTRIBUTES.XFRM BitwiseAND 03H) ≠ 03H) > THEN #GP(0); FI; > > IF (XFRM is illegal) > THEN #GP(0); FI; > > The first part is clear, but the second part is vague. > > I am not sure in hardware behaviour, whether XCR0 is actually checked in > ECREATE. It's more likely XCRO is actually checked in EENTER. > > But I think it's just fine to also check against XCR0 here. ECREATE doesn't check XCR0, it checks that XFRM represents a legal XCR0 values for the platform, which in KVM is tracked as guest_supported_xcr0.
On Thu, 2023-04-06 at 12:12 -0700, Sean Christopherson wrote: > On Thu, Apr 06, 2023, Huang, Kai wrote: > > On Wed, 2023-04-05 at 18:44 -0700, Sean Christopherson wrote: > > > On Wed, Apr 05, 2023, Huang, Kai wrote: > > > > On Tue, 2023-04-04 at 17:59 -0700, Sean Christopherson wrote: > > > > > Explicitly check the vCPU's supported XCR0 when determining whether or not > > > > > the XFRM for ECREATE is valid. Checking CPUID works because KVM updates > > > > > guest CPUID.0x12.1 to restrict the leaf to a subset of the guest's allowed > > > > > XCR0, but that is rather subtle and KVM should not modify guest CPUID > > > > > except for modeling true runtime behavior (allowed XFRM is most definitely > > > > > not "runtime" behavior). > > > > > > > > > > Signed-off-by: Sean Christopherson <seanjc@google.com> > > > > > --- > > > > > arch/x86/kvm/vmx/sgx.c | 3 ++- > > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > > > diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c > > > > > index aa53c98034bf..362a31b19b0e 100644 > > > > > --- a/arch/x86/kvm/vmx/sgx.c > > > > > +++ b/arch/x86/kvm/vmx/sgx.c > > > > > @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, > > > > > (u32)attributes & ~sgx_12_1->eax || > > > > > (u32)(attributes >> 32) & ~sgx_12_1->ebx || > > > > > (u32)xfrm & ~sgx_12_1->ecx || > > > > > - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { > > > > > + (u32)(xfrm >> 32) & ~sgx_12_1->edx || > > > > > + xfrm & ~vcpu->arch.guest_supported_xcr0) { > > > > > > > > Perhaps this change is needed even without patch 2? > > > > > > > > This is because when CPUID 0xD doesn't exist, guest_supported_xcr0 is 0. But > > > > when CPUID 0xD doesn't exist, IIUC currently KVM doesn't clear SGX in CPUID, and > > > > sgx_12_1->ecx is always set to 0x3. > > > > > > Hrm, that's a bug in this patch. Drat. More below. > > > > > > > __handle_encls_ereate() doesn't check CPUID 0xD either, so w/o above explicit > > > > check xfrm against guest_supported_xcr0, it seems guest can successfully run > > > > ECREATE when it doesn't have CPUID 0xD? > > > > > > ECREATE doesn't have a strict dependency on CPUID 0xD or XSAVE. This exact scenario > > > is called out in the SDM: > > > > > > Legal values for SECS.ATTRIBUTES.XFRM conform to these requirements: > > > * XFRM[1:0] must be set to 0x3. > > > * If the processor does support XSAVE, XFRM must contain a value that would > > > be legal if loaded into XCR0. > > > * If the processor does not support XSAVE, or if the system software has not > > > enabled XSAVE, then XFRM[63:2] must be zero. > > > > > > So the above needs to be either > > > > > > xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) > > > > > > or > > > > > > (xfrm & ~XFEATURE_MASK_FPSSE & ~vcpu->arch.guest_supported_xcr0) > > > > > > > > > I think I prefer the first one as I find it slightly more obvious that FP+SSE are > > > allowed in addition to the XCR0 bits. > > > > The above check doesn't verify xfrm is a super set of 0x3. I think we verify > > that per SDM: > > Oooh, right. It's not that FP+SSE are always allowed, it's that FP+SSE must always > be _set_. So this? > > xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) || > (xfrm & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE Looks good. I'll try to get some test done with this code change. > > > 39.7.3 Processor Extended States and ENCLS[ECREATE] > > > > The ECREATE leaf function of the ENCLS instruction enforces a number of > > consistency checks described earlier. The execution of ENCLS[ECREATE] leaf > > function results in a #GP(0) in any of the following cases: > > • SECS.ATTRIBUTES.XFRM[1:0] is not 3. > > • The processor does not support XSAVE and any of the following is true: > > — SECS.ATTRIBUTES.XFRM[63:2] is not 0. > > — SECS.SSAFRAMESIZE is 0. > > • The processor supports XSAVE and any of the following is true: > > — XSETBV would fault on an attempt to load XFRM into XCR0. > > — XFRM[63]=1. > > — The SSAFRAME is too small to hold required, enabled states ... > > > > > > And in the ECREATE pseudo code, the relevant parts seem to be: > > > > (* Check lower 2 bits of XFRM are set *) > > IF ( ( DS:TMP_SECS.ATTRIBUTES.XFRM BitwiseAND 03H) ≠ 03H) > > THEN #GP(0); FI; > > > > IF (XFRM is illegal) > > THEN #GP(0); FI; > > > > The first part is clear, but the second part is vague. > > > > I am not sure in hardware behaviour, whether XCR0 is actually checked in > > ECREATE. It's more likely XCRO is actually checked in EENTER. > > > > But I think it's just fine to also check against XCR0 here. > > ECREATE doesn't check XCR0, it checks that XFRM represents a legal XCR0 values > for the platform, which in KVM is tracked as guest_supported_xcr0. Yes agreed.
> > > > Oooh, right. It's not that FP+SSE are always allowed, it's that FP+SSE must always > > be _set_. So this? > > > > xfrm & ~(vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FPSSE) || > > (xfrm & XFEATURE_MASK_FPSSE) != XFEATURE_MASK_FPSSE > > Looks good. > > I'll try to get some test done with this code change. > Tested this series with your above code change by running simple SGX app in the guest. For this particular case, tested with ECREATE with xfrm = 0x1 in the guest, and guest can receive #GP. So for the entire series: Reviewed-by: Kai Huang <kai.huang@intel.com> Tested-by: Kai Huang <kai.huang@intel.com> >
diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index aa53c98034bf..362a31b19b0e 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -175,7 +175,8 @@ static int __handle_encls_ecreate(struct kvm_vcpu *vcpu, (u32)attributes & ~sgx_12_1->eax || (u32)(attributes >> 32) & ~sgx_12_1->ebx || (u32)xfrm & ~sgx_12_1->ecx || - (u32)(xfrm >> 32) & ~sgx_12_1->edx) { + (u32)(xfrm >> 32) & ~sgx_12_1->edx || + xfrm & ~vcpu->arch.guest_supported_xcr0) { kvm_inject_gp(vcpu, 0); return 1; }