From patchwork Fri Jan 13 22:01:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43628 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1507wrn; Fri, 13 Jan 2023 14:06:08 -0800 (PST) X-Google-Smtp-Source: AMrXdXvAtFpm96Rw5KAwYAneWy3eXiYXldCiE5slo+Lc+i4OSrRxz3lHs5Kbf5QSiXP1iRDyQzbp X-Received: by 2002:a17:906:308b:b0:7ae:cda1:76d0 with SMTP id 11-20020a170906308b00b007aecda176d0mr64205459ejv.15.1673647568319; Fri, 13 Jan 2023 14:06:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647568; cv=pass; d=google.com; s=arc-20160816; b=Nc2fLYCGb0zx/sjHf2FCxop7E1rorHp6bkRSYvh+4hN+02A4Juj5p6NdBJMQphWEIK N+ytnLmPUfhEjggglD8F1SPiXLOd+opajpzF+qI75T/W3wH0pcODfbGskhDsrOtxUwtI 1xzRiglhtY9VwUxqBBv9Xj9X8TP9LcU3WEiegWPHncOfqcGEuaOvXcFClPDWLtVWVp7a WbjoFm6z0E+fpzhNhYJrqHyjyeQiNLzfICrGCbAaEYUdIAw/SEFbkXkzJHBrHa5X8Pdu 2G6J+CuoP1on4fUj81saaHfd1YFeAFzsRs11CYzENFgaouKKW8t3/9GBFSJoKl4YC43/ 609w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+sk5rrMO/llRPsazlOXAoiWlkhec7l5GhOumZ697aSE=; b=SZKjyxEYrt9E0CI545XAI/TpvXCqrlpZ2PT94qClijHWigwfvoZoeR7zE7vrLbuEDt Epzx8h7PQxMY54DfYV2JuFcYUJFVgSn/turJT4pk7THzwNMevECLWzyPDV/rlfJ7anvB sv2cTbKtecUArLMP1wp9UOrl588QaCirU6iObI33Zd1wOuUSghml+H1bFdIWw1bo7BhR M3rc2H536BKx46gisEChvUW17AoNSfUBEvXSDeczvdNE9QuA4FahJ1mVRe/KxCZGfrxA KYOTRSXaDYf2fjz75jJm02lP+fyEXwnIpHwZDVgm2kNU8btcUwCYXJqKpKO+IkIiNssZ kKDA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=pOB7Od80; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xb13-20020a170907070d00b0081a9c3ca4c3si22744098ejb.374.2023.01.13.14.05.43; Fri, 13 Jan 2023 14:06:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=pOB7Od80; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231237AbjAMWCN (ORCPT + 99 others); Fri, 13 Jan 2023 17:02:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229505AbjAMWCD (ORCPT ); Fri, 13 Jan 2023 17:02:03 -0500 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2058.outbound.protection.outlook.com [40.107.244.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4ECE56C06B; Fri, 13 Jan 2023 14:02:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lAqROyt20HvJUNWqK0HUpWJ+rLS9VdhUdtqUHTIStyIbdb5544Xzb2Bayg0YmXXSv16jWcU1p3C+w3YGO0003t/hju3ER2Ti0P1FAvmE/wQ2Oxt7q61/AXqSS6OJ0UjcaMg7txfybVujnpU83Pa2LDITOS3KJeIGP9/F81cXoLQ37LeojXdQynwNoSfv7jSe0J4aTUl229OKRrnyj0ke+at+eheVOaFVkv+TG7VNL48KUuZERPODDfvj+TkPMFcQUZCeHB2TG9SbW/UmAr2BIqapJBfs9TPh3fJ5J3v5LREyvEOvycB6HX35TTR4cxG8Wt4UQBhgWS9hx9qQhZg3+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+sk5rrMO/llRPsazlOXAoiWlkhec7l5GhOumZ697aSE=; b=IuwYdDQakQYfRKpxcwUn1SXivjplooX805VsDAWMbAAvRbI5cLLlqe47VxAmuiBol8g2WIxevrE55wPRLvrBIwJ8+JZ5fMepQuQh+s61uA2i3ZTj0XGl0FBqWEyL9Y5TD3E0Y/bbLjJvDIWRd2udGrZeZV88kd/hvvj5mqT4Z+gSQ7uTkgMA5sXGbC99VOASnxjjbvi7ZkYq4BDOJ3B2rTAUUn8Dl3RA2Vt1mNBORwBQvUTMkHHc27c/FyA7jfHDzS+QEzb8OlVEN6dU2pnkaDv+bmqcfm6FicYsBVDtf9mxUBvfQQ88VJf6xJjjiMcjEGCDzkb9xKfRLwaZKaGXtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+sk5rrMO/llRPsazlOXAoiWlkhec7l5GhOumZ697aSE=; b=pOB7Od80VCnWGJCHYw5qwxxltvKxfZTvTUF/zBBEvfbGNKcjSPo43/XF45UZByfN5HiF4w79LV0hsuIr3WWm6/Hvx0vtLFnl1p8knehAFVrmTAiEcdqB7u5wfHDqJy/kKSKglA4vD2sbMqRw+S8NsXWIxQhcV3D2MIBUN5G9+OHjO2+2N1XXFeaP0RVUj30WylaJg8Xxpo1docBXQycy+SsXxPvapiOM1drF35xApkQ8bO3fqQKRu37OsgDeUIz27EI8b5F5nh0Xrj8oYoe/3wY3iTYRoVlW2l/Xl97rfJ/TC/h7lDYKB4iPhEKXA45bCd2C0hixUo8riFIfnhOm5g== Received: from MW4PR04CA0054.namprd04.prod.outlook.com (2603:10b6:303:6a::29) by MW3PR12MB4345.namprd12.prod.outlook.com (2603:10b6:303:59::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.19; Fri, 13 Jan 2023 22:01:58 +0000 Received: from CO1NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6a:cafe::1e) by MW4PR04CA0054.outlook.office365.com (2603:10b6:303:6a::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.16 via Frontend Transport; Fri, 13 Jan 2023 22:01:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1NAM11FT009.mail.protection.outlook.com (10.13.175.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:01:58 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:49 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:49 -0800 From: Kechen Lu To: , , CC: , , , , , Subject: [RFC PATCH v5 1/6] KVM: x86: only allow exits disable before vCPUs created Date: Fri, 13 Jan 2023 22:01:09 +0000 Message-ID: <20230113220114.2437-2-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT009:EE_|MW3PR12MB4345:EE_ X-MS-Office365-Filtering-Correlation-Id: febc1451-96fb-4f9f-553e-08daf5b1cae2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V1Qjdr9x9djLI0NXWbs/p2ECWEuHn96+l84VO5oiQyu2Ka8xaU5BQ8iBoXiaKPRKy/oOQcNWi9DBaAP3Au7AoJtSMFfw46YoTilu9HhHu+KEw2ZY++QIPtob7ileLyyGrp6Y98GScNBI/DZUvTgZ13Iynw52vzEhYhH4lytp6B2yFMNW4+Sx+vzADfKiiwQ6q+BH+/zUjERnmDT+HXeXiHf0c1iMq8fQOVcAlmNJAA8bKeB4F8YRfRtBkqjw5C+U7I/l38TuhbbZvAF4GlVxixAhQ/AjHSepUQs5wYaR9OUyuW0Qw9dJLLOL9mfLXch+b/a657qZ+vK/w4DN1OHVtLRC0tb2eqU5n9nCzn3eCrMougQEQb6iWZDkKa8CXR5JU7GlvWlK66qvn4LYVrYATBDkkldEcuvw3QEPuiBTrbwMJcSupzyc/E9GPPzVQkFn4NQJy7gPbVlGud+iaNoEB/0IGQW6QPQEdeRLhf0qw+umH1wKdwePi1iX3bcuZT3UNVdyy+fWPMuQvvy09ExBS92cR/mxY+JkptENtnBgwPTIII2vc/72a7KtSWn5zlGsEjmgt6mANTSuwVp08iZi/3NKI2QbZRbHsywOCNux9PaHMrc9v2pOegOZJ5Oohk4gJRJ4I0m1zJBgqICUFwCfYNPpmovBfoMkj8Ht3EfVumHv+UaS4e9IhPExh/GsvHASVAE2siwPgNL3cdzlhOequg== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(346002)(376002)(136003)(39860400002)(396003)(451199015)(40470700004)(46966006)(36840700001)(82310400005)(26005)(36756003)(6666004)(16526019)(186003)(7696005)(82740400003)(356005)(7636003)(478600001)(86362001)(40480700001)(40460700003)(83380400001)(2616005)(47076005)(1076003)(36860700001)(336012)(426003)(5660300002)(8936002)(316002)(4326008)(70206006)(70586007)(2906002)(8676002)(110136005)(54906003)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:01:58.7256 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: febc1451-96fb-4f9f-553e-08daf5b1cae2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4345 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946672548729607?= X-GMAIL-MSGID: =?utf-8?q?1754946672548729607?= From: Sean Christopherson Since VMX and SVM both would never update the control bits if exits are disable after vCPUs are created, only allow setting exits disable flag before vCPU creation. Fixes: 4d5422cea3b6 ("KVM: X86: Provide a capability to disable MWAIT intercepts") Signed-off-by: Sean Christopherson Signed-off-by: Kechen Lu Cc: stable@vger.kernel.org --- Documentation/virt/kvm/api.rst | 1 + arch/x86/kvm/x86.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 9807b05a1b57..fb0fcc566d5a 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7087,6 +7087,7 @@ branch to guests' 0x200 interrupt vector. :Architectures: x86 :Parameters: args[0] defines which exits are disabled :Returns: 0 on success, -EINVAL when args[0] contains invalid exits + or if any vCPU has already been created Valid bits in args[0] are:: diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index da4bbd043a7b..c8ae9c4f9f08 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6227,6 +6227,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (cap->args[0] & ~KVM_X86_DISABLE_VALID_EXITS) break; + mutex_lock(&kvm->lock); + if (kvm->created_vcpus) + goto disable_exits_unlock; + if ((cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) && kvm_can_mwait_in_guest()) kvm->arch.mwait_in_guest = true; @@ -6237,6 +6241,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (cap->args[0] & KVM_X86_DISABLE_EXITS_CSTATE) kvm->arch.cstate_in_guest = true; r = 0; +disable_exits_unlock: + mutex_unlock(&kvm->lock); break; case KVM_CAP_MSR_PLATFORM_INFO: kvm->arch.guest_can_read_msr_platform_info = cap->args[0]; From patchwork Fri Jan 13 22:01:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43632 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2313wrn; Fri, 13 Jan 2023 14:08:13 -0800 (PST) X-Google-Smtp-Source: AMrXdXvsxKpeTMX86VtGsTV98qvMpjKWMThUpxe0Gt5pRYs4MASvHOVJfMwrQUPcz2SiXwNviMvo X-Received: by 2002:a17:906:9f0a:b0:7ad:88f8:7663 with SMTP id fy10-20020a1709069f0a00b007ad88f87663mr93512635ejc.43.1673647693249; Fri, 13 Jan 2023 14:08:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647693; cv=pass; d=google.com; s=arc-20160816; b=kTHQdwbj+dcopXdTgnzLLiTGoY+dpUN5Kfm8TVOam1YaxwRXLeNBDFaLowBB3dKKD7 R85KlJwPMlsIRB7HOcrnzoPB4WVuhJsx7axa5VRPY33PtKsYodhsKM2wn96swQ/hpugP LmCGdcP+zOSGpZThTfvHcCy5ogSkuOvG9znzvUIt7VivLCCRbwuP4n2w7MZcPABlsfuZ NRPZlUsYF0lNVWLEZy4eQq4hRWKi1xCPd7N3bRJ/EfCKEmfNtny6E+fyNGd5dfhFQkG5 AoFpiTH2Byln1gBXI7FcvPFq0GRg8UYQCiIMwxwmHRGScOgihCD9zEscTufS6y+rT4AP WJwQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ifqlyX1+U3lHcUzvf1Q9I0DTmIcUk92W/dHamwnNkIU=; b=Y958120qI1aagbHlCFMyZ7KfHeVD/1x3Jy9+13OWrpljces3hg+sSnYIWLD1x6Ddsk af00wkl0C2CbCe1OdQdBr3hRbhAV0yr7VydE+Yoho6K9qvufFgU9VOVgrTCuGQDdXxaC og49QRwl+HQr32gw9jdSb+XLDxCJYkkpqQhmy7PDEhywcL9lnfnx4BKsCvqYR1j+gGPG 96nLxL/PpowTrwIi6GJExivmtU9KJyG3EeY6cgMZU0Jfujf9ssvKf1otObWGpQDU+D+S +i032FCxfDpf1RcyIYhya9rtR8iGCPtZGW1jD7g1ewkfuuCAQk1cf9sCCGAbvR53Xbb7 Ut3A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=TcwM9VkB; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ds21-20020a170907725500b0084cbb065b06si26353787ejc.112.2023.01.13.14.07.49; Fri, 13 Jan 2023 14:08:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=TcwM9VkB; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231337AbjAMWCR (ORCPT + 99 others); Fri, 13 Jan 2023 17:02:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231334AbjAMWCJ (ORCPT ); Fri, 13 Jan 2023 17:02:09 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2044.outbound.protection.outlook.com [40.107.220.44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F27C6CFF8; Fri, 13 Jan 2023 14:02:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bY3c1f/Q7GK+Qyt07ynL486uRnHUT2BrYxzcRPf3mvZslAfhcxpoozPnYhZ+sUhPBSgORITtPA3IMkqgEAQgeUsK6bjCuLAyKmlPRuo4cYEF0p637tLmkgG+fQ/fKsLGz8/BPgXo75SiIbSkk7YiOgELEFio+8mK49J5IlW102OsCYGL89czasFEehkYeHeKcXyYGkRTY9VC3DEcLEmydGArX8JJ26MHAzu+VeRdl626i25ekfBp1yZ9FvcaBLLkXozhaC8cg7H284LdFo7P0hc2Z15s82UFrcQV0o00XfQzuQrnX1OCESdl08BigUrxOVth/ZTVxhNiYDSkVWq4eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ifqlyX1+U3lHcUzvf1Q9I0DTmIcUk92W/dHamwnNkIU=; b=cgb7R5j8p2aHaincMScJNx/oLoMha1+HUiu7UwZAaB3nxHy2VSCdXRAcuPYaDdU7zHH8nI0uVX7Quj4DH2imIvZOjMfWQPJi73teM9pZciAVRKlBB1kpX13Ex51xSR3zSVLsaJ+UD9yiQbWzXyHblW9T8sodwf1KOtc/ux4AO7chWlWrCj6n35vxsB0QA/PGXhpmyLbrZGDwMZVKKrg5BIsCdIu+t8pxaWkWro1Nhze1UZBpQcN5yu0uZPpr4r/MNKNKrRBWR0Nu7pFj9fbrxnBPkzq89Z2HpcxVwIg8Jiveb7fm4Nk68s4bJGVyswewYCphqsJwJ0lE91MwvvJpkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ifqlyX1+U3lHcUzvf1Q9I0DTmIcUk92W/dHamwnNkIU=; b=TcwM9VkBKGAEzzhOBNCYBg/IFAzHgu/zLMJexyL2eAxnIT+kPs7wok5mux2bvQM71VTpqb03WHLS+JnyyH/83vWtLkklEdm/5rp02kCFu9IQKZstvUf7xM4Np30FGPYDaz4D0bnw+8HMbXtvbYNqe9WAGfle82HVwqVAu8gGASzxizF/F5ZpgaK+BwI03hVJtgEGkw4AKpLJHAaGiBqa2/kO7PZdNKqiqGeSzwuscnWYSM2ClY5nlyutFaw6sDHjry8O2IHdaLhbXxD2ynEki2DOeQShKvDvGKQ//mz/V82sTqOvNzHBwbnfDAdjvXwo/o8VMI27kwxYPNSXuQjRyQ== Received: from DM6PR06CA0051.namprd06.prod.outlook.com (2603:10b6:5:54::28) by MW4PR12MB7032.namprd12.prod.outlook.com (2603:10b6:303:20a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.19; Fri, 13 Jan 2023 22:02:04 +0000 Received: from DM6NAM11FT027.eop-nam11.prod.protection.outlook.com (2603:10b6:5:54:cafe::3f) by DM6PR06CA0051.outlook.office365.com (2603:10b6:5:54::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.14 via Frontend Transport; Fri, 13 Jan 2023 22:02:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT027.mail.protection.outlook.com (10.13.172.205) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:02:04 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:52 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:52 -0800 From: Kechen Lu To: , , CC: , , , , Subject: [RFC PATCH v5 2/6] KVM: x86: Move *_in_guest power management flags to vCPU scope Date: Fri, 13 Jan 2023 22:01:10 +0000 Message-ID: <20230113220114.2437-3-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT027:EE_|MW4PR12MB7032:EE_ X-MS-Office365-Filtering-Correlation-Id: 41d83069-4c1c-4191-d926-08daf5b1ce61 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z6VH4hzmpEoSCYWzbNsItXyU4lgDkKd641wjDKAo9pnrydPOnwJ8dJs0vCx797C5vFFxzAAd0qfDwRQPUymIv4juYL+mFaU8WlIigrzu6dJyYwvOUaIJJ6WJWT5Z+7NoM8BvM5oYHfoKz8JNQprWxW9eU4nT+VbYy8mZuLqqNWkImzJYcKdThVBMLwLsrHear+NzBlfrvZmiw2KrMXjkulckHCd6zLPSR8OyJ85AUiAaZE+cBn7WeBiLFbsz7hSDE+j+KY+Ljwh0MvGo1XUaTMCwNvZE+AioPb86bTrfMyBgr3gXcmLJq6330fZXwn7njA6g49c/taOi9hZ3qt90lycV7Hugyhq/yn4CU1OCmaCawA33zYzS0nkb6HG4oppnGS2V4zlD6gHRUKiIp2FF6KJWcXr2qMsskPi/p+ViOisaAy4FxsQpk10uJgJlODVv9Nl0VEZKCf1K2McoTGgg9W4OPt4lK9+Y7RD3jtTunlZqVAJ0oePlOI0rjLToSxWKMjcuHyGXeb0ofQSut9G6rxcgtze1sFCz0Ooql1NsuwV++Q6Sxi6/nPoBdZUJtBnKzCEqgsxVJPGpQiZege4NROWlBqgEzdolWFfAj0p+w42kyBLAm3HL0RWbwzgbBucn6YZelQr7f/mVBmRfbJBAdsISaht0n/+8vr/NO31IsrkRDXSb1nx9S5Q6pm6prMs5AH08IUForuvBsOL1D51Z/Q== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(346002)(376002)(136003)(396003)(451199015)(40470700004)(36840700001)(46966006)(356005)(82740400003)(8936002)(30864003)(5660300002)(40460700003)(82310400005)(41300700001)(426003)(47076005)(478600001)(40480700001)(4326008)(316002)(8676002)(70586007)(70206006)(336012)(36756003)(86362001)(36860700001)(83380400001)(54906003)(2906002)(2616005)(7636003)(1076003)(6666004)(26005)(186003)(16526019)(7696005)(110136005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:02:04.5319 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 41d83069-4c1c-4191-d926-08daf5b1ce61 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT027.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7032 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946803439153935?= X-GMAIL-MSGID: =?utf-8?q?1754946803439153935?= Make the runtime disabled mwait/hlt/pause/cstate exits flags vCPU scope to allow finer-grained, per-vCPU control. The VM-scoped control is only allowed before vCPUs are created, thus preserving the existing behavior is a simple matter of snapshotting the flags at vCPU creation. Signed-off-by: Kechen Lu Suggested-by: Sean Christopherson Reviewed-by: Sean Christopherson --- arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/kvm/cpuid.c | 4 ++-- arch/x86/kvm/lapic.c | 7 +++---- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/svm.c | 12 ++++++------ arch/x86/kvm/vmx/vmx.c | 16 ++++++++-------- arch/x86/kvm/x86.c | 6 +++++- arch/x86/kvm/x86.h | 16 ++++++++-------- 8 files changed, 39 insertions(+), 31 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6aaae18f1854..41b998234a04 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1009,6 +1009,11 @@ struct kvm_vcpu_arch { #if IS_ENABLED(CONFIG_HYPERV) hpa_t hv_root_tdp; #endif + + bool mwait_in_guest; + bool hlt_in_guest; + bool pause_in_guest; + bool cstate_in_guest; }; struct kvm_lpage_info { diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 596061c1610e..20e427dc608c 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -283,8 +283,8 @@ static void __kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu, struct kvm_cpuid_e best->ebx = xstate_required_size(vcpu->arch.xcr0, true); best = __kvm_find_kvm_cpuid_features(vcpu, entries, nent); - if (kvm_hlt_in_guest(vcpu->kvm) && best && - (best->eax & (1 << KVM_FEATURE_PV_UNHALT))) + if (kvm_hlt_in_guest(vcpu) && + best && (best->eax & (1 << KVM_FEATURE_PV_UNHALT))) best->eax &= ~(1 << KVM_FEATURE_PV_UNHALT); if (!kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT)) { diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 4efdb4a4d72c..8f74f9a80aa5 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -151,14 +151,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) { return pi_inject_timer && kvm_vcpu_apicv_active(vcpu) && - (kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm)); + (kvm_mwait_in_guest(vcpu) || kvm_hlt_in_guest(vcpu)); } bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu) { - return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest(vcpu->kvm) || - kvm_can_post_timer_interrupt(vcpu)); + return kvm_x86_ops.set_hv_timer && + !(kvm_mwait_in_guest(vcpu) || kvm_can_post_timer_interrupt(vcpu)); } static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index add65dd59756..ed26b6de3007 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -721,7 +721,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, pause_count12 = svm->pause_filter_enabled ? svm->nested.ctl.pause_filter_count : 0; pause_thresh12 = svm->pause_threshold_enabled ? svm->nested.ctl.pause_filter_thresh : 0; - if (kvm_pause_in_guest(svm->vcpu.kvm)) { + if (kvm_pause_in_guest(&svm->vcpu)) { /* use guest values since host doesn't intercept PAUSE */ vmcb02->control.pause_filter_count = pause_count12; vmcb02->control.pause_filter_thresh = pause_thresh12; @@ -1012,7 +1012,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.event_inj = svm->nested.ctl.event_inj; vmcb12->control.event_inj_err = svm->nested.ctl.event_inj_err; - if (!kvm_pause_in_guest(vcpu->kvm)) { + if (!kvm_pause_in_guest(vcpu)) { vmcb01->control.pause_filter_count = vmcb02->control.pause_filter_count; vmcb_mark_dirty(vmcb01, VMCB_INTERCEPTS); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9a194aa1a75a..dc7176605e01 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1014,7 +1014,7 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) struct vmcb_control_area *control = &svm->vmcb->control; int old = control->pause_filter_count; - if (kvm_pause_in_guest(vcpu->kvm)) + if (kvm_pause_in_guest(vcpu)) return; control->pause_filter_count = __grow_ple_window(old, @@ -1035,7 +1035,7 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) struct vmcb_control_area *control = &svm->vmcb->control; int old = control->pause_filter_count; - if (kvm_pause_in_guest(vcpu->kvm)) + if (kvm_pause_in_guest(vcpu)) return; control->pause_filter_count = @@ -1229,12 +1229,12 @@ static void init_vmcb(struct kvm_vcpu *vcpu) svm_set_intercept(svm, INTERCEPT_RDPRU); svm_set_intercept(svm, INTERCEPT_RSM); - if (!kvm_mwait_in_guest(vcpu->kvm)) { + if (!kvm_mwait_in_guest(vcpu)) { svm_set_intercept(svm, INTERCEPT_MONITOR); svm_set_intercept(svm, INTERCEPT_MWAIT); } - if (!kvm_hlt_in_guest(vcpu->kvm)) + if (!kvm_hlt_in_guest(vcpu)) svm_set_intercept(svm, INTERCEPT_HLT); control->iopm_base_pa = __sme_set(iopm_base); @@ -1278,7 +1278,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu) svm->nested.vmcb12_gpa = INVALID_GPA; svm->nested.last_vmcb12_gpa = INVALID_GPA; - if (!kvm_pause_in_guest(vcpu->kvm)) { + if (!kvm_pause_in_guest(vcpu)) { control->pause_filter_count = pause_filter_count; if (pause_filter_thresh) control->pause_filter_thresh = pause_filter_thresh; @@ -4362,7 +4362,7 @@ static void svm_handle_exit_irqoff(struct kvm_vcpu *vcpu) static void svm_sched_in(struct kvm_vcpu *vcpu, int cpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) shrink_ple_window(vcpu); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index fc9008dbed33..019a20029878 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1689,7 +1689,7 @@ static void vmx_clear_hlt(struct kvm_vcpu *vcpu) * then the instruction is already executing and RIP has already been * advanced. */ - if (kvm_hlt_in_guest(vcpu->kvm) && + if (kvm_hlt_in_guest(vcpu) && vmcs_read32(GUEST_ACTIVITY_STATE) == GUEST_ACTIVITY_HLT) vmcs_write32(GUEST_ACTIVITY_STATE, GUEST_ACTIVITY_ACTIVE); } @@ -4412,10 +4412,10 @@ static u32 vmx_exec_control(struct vcpu_vmx *vmx) exec_control &= ~(CPU_BASED_CR3_LOAD_EXITING | CPU_BASED_CR3_STORE_EXITING | CPU_BASED_INVLPG_EXITING); - if (kvm_mwait_in_guest(vmx->vcpu.kvm)) + if (kvm_mwait_in_guest(&vmx->vcpu)) exec_control &= ~(CPU_BASED_MWAIT_EXITING | CPU_BASED_MONITOR_EXITING); - if (kvm_hlt_in_guest(vmx->vcpu.kvm)) + if (kvm_hlt_in_guest(&vmx->vcpu)) exec_control &= ~CPU_BASED_HLT_EXITING; return exec_control; } @@ -4515,7 +4515,7 @@ static u32 vmx_secondary_exec_control(struct vcpu_vmx *vmx) } if (!enable_unrestricted_guest) exec_control &= ~SECONDARY_EXEC_UNRESTRICTED_GUEST; - if (kvm_pause_in_guest(vmx->vcpu.kvm)) + if (kvm_pause_in_guest(&vmx->vcpu)) exec_control &= ~SECONDARY_EXEC_PAUSE_LOOP_EXITING; if (!kvm_vcpu_apicv_active(vcpu)) exec_control &= ~(SECONDARY_EXEC_APIC_REGISTER_VIRT | @@ -4661,7 +4661,7 @@ static void init_vmcs(struct vcpu_vmx *vmx) vmcs_write16(LAST_PID_POINTER_INDEX, kvm->arch.max_vcpu_ids - 1); } - if (!kvm_pause_in_guest(kvm)) { + if (!kvm_pause_in_guest(&vmx->vcpu)) { vmcs_write32(PLE_GAP, ple_gap); vmx->ple_window = ple_window; vmx->ple_window_dirty = true; @@ -5833,7 +5833,7 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) */ static int handle_pause(struct kvm_vcpu *vcpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) grow_ple_window(vcpu); /* @@ -7379,7 +7379,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_CS, MSR_TYPE_RW); vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_ESP, MSR_TYPE_RW); vmx_disable_intercept_for_msr(vcpu, MSR_IA32_SYSENTER_EIP, MSR_TYPE_RW); - if (kvm_cstate_in_guest(vcpu->kvm)) { + if (kvm_cstate_in_guest(vcpu)) { vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); @@ -7935,7 +7935,7 @@ static void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu) static void vmx_sched_in(struct kvm_vcpu *vcpu, int cpu) { - if (!kvm_pause_in_guest(vcpu->kvm)) + if (!kvm_pause_in_guest(vcpu)) shrink_ple_window(vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c8ae9c4f9f08..9a77b55142c6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11634,6 +11634,10 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) #if IS_ENABLED(CONFIG_HYPERV) vcpu->arch.hv_root_tdp = INVALID_PAGE; #endif + vcpu->arch.mwait_in_guest = vcpu->kvm->arch.mwait_in_guest; + vcpu->arch.hlt_in_guest = vcpu->kvm->arch.hlt_in_guest; + vcpu->arch.pause_in_guest = vcpu->kvm->arch.pause_in_guest; + vcpu->arch.cstate_in_guest = vcpu->kvm->arch.cstate_in_guest; r = static_call(kvm_x86_vcpu_create)(vcpu); if (r) @@ -12885,7 +12889,7 @@ bool kvm_can_do_async_pf(struct kvm_vcpu *vcpu) kvm_is_exception_pending(vcpu))) return false; - if (kvm_hlt_in_guest(vcpu->kvm) && !kvm_can_deliver_async_pf(vcpu)) + if (kvm_hlt_in_guest(vcpu) && !kvm_can_deliver_async_pf(vcpu)) return false; /* diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 9de72586f406..b8e49a9d353d 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -351,24 +351,24 @@ static inline u64 nsec_to_cycles(struct kvm_vcpu *vcpu, u64 nsec) __rem; \ }) -static inline bool kvm_mwait_in_guest(struct kvm *kvm) +static inline bool kvm_mwait_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.mwait_in_guest; + return vcpu->arch.mwait_in_guest; } -static inline bool kvm_hlt_in_guest(struct kvm *kvm) +static inline bool kvm_hlt_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.hlt_in_guest; + return vcpu->arch.hlt_in_guest; } -static inline bool kvm_pause_in_guest(struct kvm *kvm) +static inline bool kvm_pause_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.pause_in_guest; + return vcpu->arch.pause_in_guest; } -static inline bool kvm_cstate_in_guest(struct kvm *kvm) +static inline bool kvm_cstate_in_guest(struct kvm_vcpu *vcpu) { - return kvm->arch.cstate_in_guest; + return vcpu->arch.cstate_in_guest; } static inline bool kvm_notify_vmexit_enabled(struct kvm *kvm) From patchwork Fri Jan 13 22:01:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43629 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1703wrn; Fri, 13 Jan 2023 14:06:36 -0800 (PST) X-Google-Smtp-Source: AMrXdXsqyTXGvoucf54naA+c2YGMG4RuXHpONVSkmRFtMAqN7tjB78aCtKj8ZFcAvulTcOv5gcWS X-Received: by 2002:a17:907:6292:b0:846:56fe:6c68 with SMTP id nd18-20020a170907629200b0084656fe6c68mr80440490ejc.21.1673647596119; Fri, 13 Jan 2023 14:06:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647596; cv=pass; d=google.com; s=arc-20160816; b=bGCXdGBcwVDvgCRqFZ+b9F6E+1ehikZ+5igpRO1Q6VLSamASyDNSEtoMGkphBdynnF E2bbr76XArTl9nUzv12eOgSxUOYet8aghYZYncct3fJwwusmyJ9XJcQrlJpCPUEiVSj0 mk6nWKuAbiUJTlHXo5RK/ygW/Tw8xpiBeKDJD51Wxkw0hSwF8hJlpktHQaJuphHyLke2 fbnBfUx0Xi03ZNrhWpUqdlcpqoOPN7iPC0iehK3lpcEZsSBInjD2XlxmOvkDoxrR/OPb OWgophhuLNQ8v9TMe7r1kYLzqPLMGxFQ6Xdkzafh5RVe2y/pY9gaV0W5FnPnL8O38fe4 KDyw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=w56JfpoPTBvKDxhszdWrwSVInQHrFM79VlpDgl47F2s=; b=r/I2x3Q1xy1Hxuu51xfxohQIAjyhpUSf6PrB3wTQjCB0d54Rh41GQi1fQLlUivjNxU I38Qp/dP7d/vWWeGC7LaW+GfYg+7aELwE2nR9wVNlcRe4NMpLw2KkRfkhyk8TkGTXGOD fqH0nKKijLlaF6Gd/CjQJSMz4BdVI6Lb3PKEhhMZUxE2gtIMnG+Wb1EEGCLA2ebSJ8il I9XFW/7hmH7z+XgnOxjebes0ctL1C6rbyinBD5JFJ/DeRLMHj3cSk9BDkjikqtgKWB8g MJFQ4nRpHQMRujsGnDZTN/Clho2E5BvExaGdTIsdrvsTBN1/+BIw/eHS8AcR/BNvnoAm gg/A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=rRlYmlr8; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ae9-20020a17090725c900b007ae72fcabb3si24618333ejc.838.2023.01.13.14.06.12; Fri, 13 Jan 2023 14:06:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=rRlYmlr8; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229585AbjAMWCW (ORCPT + 99 others); Fri, 13 Jan 2023 17:02:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231335AbjAMWCJ (ORCPT ); Fri, 13 Jan 2023 17:02:09 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2058.outbound.protection.outlook.com [40.107.243.58]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B5C06DB81; Fri, 13 Jan 2023 14:02:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A3zqP1p+WPkYtp52y52CLYDAzmiph91f7gM6LHePNLGW7KfrUL3AbXecAI7IO48zZ5oE5IN4iE5FTGmXP5fIbjhW2yLhveUSC/FEp5EJLWC9oTqFg43zMCWnnF1DDOqzLZsp442uuE8pnBJrGJuNuW35n+237KAu0IcKkvt3fUnVL8Nki5uwlIC4Nq00TypMceFViHsLwzd2xURux773HoIXtHT6T56GGztHhMAyWU7kI0/3pMagGwDMlvkGUl5ph/8b/gMoc7ZUfEFKeQlmIsxdqh/Tb4QT2g82JiIQ9QSL5qOxos+17SQFEKdq4FvN7+PDRdMFSZ1u92bz2amLTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w56JfpoPTBvKDxhszdWrwSVInQHrFM79VlpDgl47F2s=; b=CP5agHwxAi2ZAKNuS2tzlQRIPlTpHXTjT1p7SHHu8Gr+8jnSEvSQuxKOqFMBt3BQux08kfsCjZUVlUP3lBvPTDARRX2pf6YHCoiogPry+rf8kqyfnWe1Dw8skKAWA48PQsUDywEIhiayoRH+L+ssloqPCyQq6lWPGLFiWLpTl0Q3woK+LeRhmU43ZKMzr0ETo9ENa9HWqan/Y5cufi3C4cPpsSnxhd/Iyfcy+O3ZFw7PcqPi7MH9Av2NMMt0pQOlElUH9PNfct6FIxZcafXMLSIvRKd9wqq8LMO6qP2Iptd/1CDPNsEh6GQ+yMrZ8V0EqC8BR9fQ1wmF8FOGoa6h/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w56JfpoPTBvKDxhszdWrwSVInQHrFM79VlpDgl47F2s=; b=rRlYmlr8I4AvrDLspS99hpg9c805urTn421BFnA0BSyrNI69wOi7m10cDyy6qbnTq5xrqR9w4dgcy385XD6SIyH6wDTVWUYOPy1V4Gaf4nKU6FZqPIJ2NroFTs6OBtCl+KMT6+1Wv1/0Z4NsDqfP6Jq0nLLMxTHizcW6LKg4LCdXJwqGpFrMAjheX3RQTB0yHuAVG+aHIgqmjy1I7eHBAuucBiGzfs/eqc1ivDfWjQR4MrdLbt7uMvfCM2sZs5ZB1hMUL/VM/dsJWh8+19s1wVNytpjfX8e8k/UjHDRie7lyrLEJ+yeb0t5BCvpVEArjZde4kgXsZdzlm3B3BTGRqA== Received: from DM6PR12CA0031.namprd12.prod.outlook.com (2603:10b6:5:1c0::44) by IA1PR12MB8287.namprd12.prod.outlook.com (2603:10b6:208:3f5::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Fri, 13 Jan 2023 22:02:06 +0000 Received: from DM6NAM11FT072.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1c0:cafe::a2) by DM6PR12CA0031.outlook.office365.com (2603:10b6:5:1c0::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.14 via Frontend Transport; Fri, 13 Jan 2023 22:02:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT072.mail.protection.outlook.com (10.13.173.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:02:06 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:54 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:01:54 -0800 From: Kechen Lu To: , , CC: , , , , Subject: [RFC PATCH v5 3/6] KVM: x86: Reject disabling of MWAIT interception when not allowed Date: Fri, 13 Jan 2023 22:01:11 +0000 Message-ID: <20230113220114.2437-4-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT072:EE_|IA1PR12MB8287:EE_ X-MS-Office365-Filtering-Correlation-Id: 51207a77-012e-4325-0a39-08daf5b1cfa8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vRZLwZbsxFRzXefC9pLMV/5JkhPFlvxpCP8PfeGI5qzQovW6z2Q7Zv0zQXh4qjjy/jf9lHyxvxFDr3CekEmcHu3h6d2mfFhk3cDCDmiaWxDLdx6xeDHntC1WXTaiif4+HH7SCZbegjgfIpEUuuELzTZ4B9hRjj6K6sl8i/jlTxyOZG9wc1BD7PaAgCuOYfUd5xHtqx+t5rVPgrJRCmN5JSLQe84xKXfNhxc/eQ+H3y8eMBzX0T1/I/LaoH2G5jhnxEOAQpnyjPJepeW/p9EUmUK24rcdy3eLMwEz9Z8sc7tIV5x9fhob3JbIRPRlzjf/nlLxwPJ2iWCns6pFKiFvm3fQlxB106XJpDGcINQISAHUTZV7mYr7PAoxxRqECYD9BrJ/lY+IEuUEBYNjgQPbGdHGFpHQly0KQaNbTg/EHmo+8Ls5UWS4pV0OMwNkyTW4YwzzFMd+TLKXLkYtkhsNC84po1tZbDC8vuu2FJdZA6MpmAK4PYxglhbbW872JyU2roojbEjMp4sTBcWpwxJ2jXHKT4ixgJgMY4dURrvDgwElzFT1WIkuD0+RefJwTMkHGbljRFgY262OKYBna/VSdRcBiStELYZbsNsfiEzxDNsHmiHyQD9wkDQYzuZ3+9C6kEvSk4S2DL91kb/siFyvZC+NJVilzqp7MSH+cY/3Tv8yAqDXqXXYgNcZ5N7muJ055nxd05Ad4/iiv/Z9uKwXVQ== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(376002)(39860400002)(136003)(346002)(451199015)(36840700001)(40470700004)(46966006)(47076005)(16526019)(40480700001)(6666004)(26005)(336012)(186003)(4326008)(2616005)(7696005)(70206006)(316002)(1076003)(478600001)(54906003)(70586007)(110136005)(356005)(82740400003)(7636003)(86362001)(36860700001)(426003)(83380400001)(40460700003)(82310400005)(41300700001)(8676002)(36756003)(5660300002)(2906002)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:02:06.6853 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51207a77-012e-4325-0a39-08daf5b1cfa8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT072.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8287 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946702107625808?= X-GMAIL-MSGID: =?utf-8?q?1754946702107625808?= From: Sean Christopherson Reject KVM_CAP_X86_DISABLE_EXITS if userspace attempts to disable MWAIT exits and KVM previously reported (via KVM_CHECK_EXTENSION) that MWAIT is not allowed in guest, e.g. because it's not supported or the CPU doesn't have an aways-running APIC timer. Fixes: 4d5422cea3b6 ("KVM: X86: Provide a capability to disable MWAIT intercepts") Signed-off-by: Sean Christopherson Signed-off-by: Kechen Lu --- arch/x86/kvm/x86.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9a77b55142c6..60caa3fd40e5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4326,6 +4326,16 @@ static inline bool kvm_can_mwait_in_guest(void) boot_cpu_has(X86_FEATURE_ARAT); } +static u64 kvm_get_allowed_disable_exits(void) +{ + u64 r = KVM_X86_DISABLE_VALID_EXITS; + + if (!kvm_can_mwait_in_guest()) + r &= ~KVM_X86_DISABLE_EXITS_MWAIT; + + return r; +} + static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 __user *cpuid_arg) { @@ -4448,10 +4458,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = KVM_CLOCK_VALID_FLAGS; break; case KVM_CAP_X86_DISABLE_EXITS: - r |= KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_PAUSE | - KVM_X86_DISABLE_EXITS_CSTATE; - if(kvm_can_mwait_in_guest()) - r |= KVM_X86_DISABLE_EXITS_MWAIT; + r |= kvm_get_allowed_disable_exits(); break; case KVM_CAP_X86_SMM: if (!IS_ENABLED(CONFIG_KVM_SMM)) @@ -6224,15 +6231,14 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, break; case KVM_CAP_X86_DISABLE_EXITS: r = -EINVAL; - if (cap->args[0] & ~KVM_X86_DISABLE_VALID_EXITS) + if (cap->args[0] & ~kvm_get_allowed_disable_exits()) break; mutex_lock(&kvm->lock); if (kvm->created_vcpus) goto disable_exits_unlock; - if ((cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) && - kvm_can_mwait_in_guest()) + if (cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) kvm->arch.mwait_in_guest = true; if (cap->args[0] & KVM_X86_DISABLE_EXITS_HLT) kvm->arch.hlt_in_guest = true; From patchwork Fri Jan 13 22:01:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43634 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3120wrn; Fri, 13 Jan 2023 14:10:23 -0800 (PST) X-Google-Smtp-Source: AMrXdXsOQeac9gP9TW6GjTwrXtjTWOhhndPRWdS0ki/DIFa6K/lS46yNtCwf98g4n4PKvG2Lrzi7 X-Received: by 2002:a17:906:6dd4:b0:836:e6f7:8138 with SMTP id j20-20020a1709066dd400b00836e6f78138mr84054520ejt.13.1673647822951; Fri, 13 Jan 2023 14:10:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647822; cv=pass; d=google.com; s=arc-20160816; b=FaK2/QIPGhWrrdbEssOqoSCZMrbOu5uR6ZR8dLfyyS1twu9fR+bPeYf7cqh8skILuH 0baQ/6019zp7Gpw5JhpSxc24lprTWJQSZ+1XdqK0dV3hfYNI5jz0NGfs/t6+rKFvuTJM RcTJCEMvJ3D6J3tZvMp5gsUDnI68FZuMUEA33amMaaCkTocKc9J1f6wwgsrUpJleWEbm wRzlTH2Fep8q74MttT0BXyWKvRh+0OakLTLlGUa/jnOuLoLu/5NE1Ve8G8k5DEXpZWXW vXn742ru3OrAad6ZJhc/1WOYBEC5qsun3yrMmUpvkq9jOXvUQprA33W3u4QNiu6MtLDO mVpA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MO0/hDqKGtucXNnwb5hqeLIqWTtw1nk7baueYJElB5Q=; b=Mf+6lvTLVReaonNsyR6iMYAFusilqkYs/hCQQeeAFZvthAmeU2lbghTLyT18uv+vHC qRMpViNVeVS+a2mf2nI3W1atmg86joHmtIk372jk3IZ5Dnp581yfVIkYN5iqD9HExnRh xv+yV8WE3WmUb6JuEmV5Jvg6doZLzPrhnv1wDSIfBdALOggclQSq1E/Xi+wd8Ob84Xr2 vHu4sz2aUPKUrpXVfpjqzl3sfKMCNeBo6fbsp7UFFeoXq0yP7DN85zZwLFsr/AcaY8M4 V5ZjfE1+kbqbgHK2ceqth45OsBBOYRp4fJb890DrzjmlGIAdxKahsicYdUY7gStBE1gm 73Eg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=fKJJwq5y; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xd9-20020a170907078900b0084cdfa26c19si20916061ejb.841.2023.01.13.14.09.36; Fri, 13 Jan 2023 14:10:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=fKJJwq5y; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231445AbjAMWDA (ORCPT + 99 others); Fri, 13 Jan 2023 17:03:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231395AbjAMWCe (ORCPT ); Fri, 13 Jan 2023 17:02:34 -0500 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2078.outbound.protection.outlook.com [40.107.237.78]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B927275D33; Fri, 13 Jan 2023 14:02:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SkiQ9CZmK1Gz0/uYi7v93jlSQyTDLrnQ9jj2J6uLvmpEG4XExy42G5r5ijOHl0Eq96rAqUJQTDZ/HxIY2CYyeimBovrsya0TpBKWARvwsVRPumugfDx8zPcpZYoeKDKpP2y/UJ/hElNu52Q9smP9Z6OA380UUHKalhiybasGZXbQ5sbeFgg02sHLu7MYNGxpykv5VQ+DAp6V1olwRONYRDvLefPPOK/1T2AfhT2V7iElhd3q5DooOv6TDnjyTuiidM5lIa3v7drw2maSelDEYMsJQRa/8MJp5PnJdymoBj8cQ0NRIznz7eFAEMZ9u4tmhpqsrGHCJZDWPsotMQgjYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MO0/hDqKGtucXNnwb5hqeLIqWTtw1nk7baueYJElB5Q=; b=B3E/Ui8fkhsMbYUlyonvDqbIYbe7EJ5Ou+JMVhUOJkEHqWF0Ar8VrQd79WVZvztovUOBVSXHTiWDWes+Wlr+0B7KMEIwnwn1SgHVnPK6PllX/Ka7Wgeaw+/icbUmgkkrxRElJXO5AiCMxEXE9wY9WY+1SjnFxhjXLPUGFzg3rd+14CYanNQc9LQUfCe7Rag2iwwQ7+KnfCRuWIqdDo130+VH/glHw/FRi4jrq/sZ53hKC2zCMeHkpDCF0lx/JBtsz7EADbhX3j2oRSHoMLNIoAG+IP12k7FM0a3Lt9X0IY0Jud9EDqnd9F0e9kBDkdaN9lXO41UyAY2ZJNpPjJj6IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MO0/hDqKGtucXNnwb5hqeLIqWTtw1nk7baueYJElB5Q=; b=fKJJwq5yeEMgTrBoxKFNkX1KXBle5rmQgYHJFiHc8CLcvfZq/UzB1/QlvOYOuGHN8anVEmNt0VybwLqtbf0i6jdis3tSwjwqLwCkbsxAviPTytv+QoQ6fUUeyde9fncTB7omjyBsM/vwZgw37G68tb+niJNAHxh1cfiR6bLP0D4iqkEBgInmgFqDa/XHYhAQFHa6KLU/mQqLLORov8/PUI1KvjUHIy6sEqAmWUafkhen4aAn22l0OnxeNBlrgBwtLOWKMkPHFt2C1ukAO9t3Rii8x4cnVuC1H+uosE2MYig0j/QZFgRZABc7pBsOOoSeERGo9gL8KXfRBDyBbSUKvA== Received: from DM6PR02CA0061.namprd02.prod.outlook.com (2603:10b6:5:177::38) by BN9PR12MB5242.namprd12.prod.outlook.com (2603:10b6:408:11f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13; Fri, 13 Jan 2023 22:02:18 +0000 Received: from DM6NAM11FT049.eop-nam11.prod.protection.outlook.com (2603:10b6:5:177:cafe::88) by DM6PR02CA0061.outlook.office365.com (2603:10b6:5:177::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.16 via Frontend Transport; Fri, 13 Jan 2023 22:02:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT049.mail.protection.outlook.com (10.13.172.188) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:02:16 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:02 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:01 -0800 From: Kechen Lu To: , , CC: , , , , Subject: [RFC PATCH v5 4/6] KVM: x86: Let userspace re-enable previously disabled exits Date: Fri, 13 Jan 2023 22:01:12 +0000 Message-ID: <20230113220114.2437-5-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT049:EE_|BN9PR12MB5242:EE_ X-MS-Office365-Filtering-Correlation-Id: b27e4642-03cb-4d8f-cdc6-08daf5b1d5bf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KzWGL0tiMrsW55Cew9unYpdBT5LdxMnWrEcuSjXL1MCZurrr3I4nFtz5M2rrWejgiX9YNa3XyGtHorQbSRKSC06/2vCjmk90UphoUCeqa9RMqsst0JWm6BIfkTLvKKgcM2e7M/UIp/f+CmA05aa1lkiUFP8yZZOLVmYb6rkgi4IkMKoCjbK5wMmjyg7cL1J91kcz3TRwnrSssFe3rcH1jRLOj1xre1B+uJy+IoxFdoc+ID1+ZlGaJDcN2PofS9IoyHdsujKBAZg01yptKtM0SmNcpFuy/4h9sRpuadZV6n5zOFSj5LsK9ENU3SAXCiJcQGpia1DRRBZHVccRHPwIgX2pd/anLijjuDa2wThzHANCFB+uY1mCrTyeaDMKx1OMEszmw0hcPYJtpp99378LdQnu9F4Y2VreYIJngywVqQQU5vwnffAk/NY2DyhLX15YaXETm/EEJPsXdV+WXiJIq7xUys1zfKpIh2sgsbdbUB/vuD43mth2gqS2FCvyxmEipUFhuhLVfaoSfEwtgfBN1xMgW+ZRHKHNayPgQykjbAz1MKd82XKbb4qeVp/Fz1dht+Ah8TnXXt6b87VlS7FdWWALrgnD/CRMdCUrQQ9RT54KZ0scgTkVMmul2gN/pbiiBnN8tzK8gZm1cDrzkjpc8y6PI0ngYDYDb/z6uBLMYTsV1p0dkYJCGoG46mFP1KJkxkuP1x3WPllRBGmkri0QDw== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(376002)(136003)(396003)(39860400002)(346002)(451199015)(36840700001)(40470700004)(46966006)(426003)(478600001)(82740400003)(7636003)(41300700001)(356005)(47076005)(1076003)(316002)(54906003)(110136005)(40460700003)(2616005)(86362001)(16526019)(26005)(336012)(70586007)(40480700001)(186003)(7696005)(70206006)(82310400005)(5660300002)(36860700001)(36756003)(2906002)(6666004)(83380400001)(8676002)(4326008)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:02:16.8917 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b27e4642-03cb-4d8f-cdc6-08daf5b1d5bf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT049.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5242 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946939672489673?= X-GMAIL-MSGID: =?utf-8?q?1754946939672489673?= From: Sean Christopherson Add an OVERRIDE flag to KVM_CAP_X86_DISABLE_EXITS allow userspace to re-enable exits and/or override previous settings. There's no real use case for the the per-VM ioctl, but a future per-vCPU variant wants to let userspace toggle interception while the vCPU is running; add the OVERRIDE functionality now to provide consistent between between the per-VM and per-vCPU variants. Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/api.rst | 5 +++++ arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++-------- include/uapi/linux/kvm.h | 4 +++- 3 files changed, 32 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index fb0fcc566d5a..3850202942d0 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7095,6 +7095,7 @@ Valid bits in args[0] are:: #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) + #define KVM_X86_DISABLE_EXITS_OVERRIDE (1ull << 63) Enabling this capability on a VM provides userspace with a way to no longer intercept some instructions for improved latency in some @@ -7103,6 +7104,10 @@ physical CPUs. More bits can be added in the future; userspace can just pass the KVM_CHECK_EXTENSION result to KVM_ENABLE_CAP to disable all such vmexits. +By default, this capability only disables exits. To re-enable an exit, or to +override previous settings, userspace can set KVM_X86_DISABLE_EXITS_OVERRIDE, +in which case KVM will enable/disable according to the mask (a '1' == disable). + Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. 7.14 KVM_CAP_S390_HPAGE_1M diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 60caa3fd40e5..3ea5f12536a0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5484,6 +5484,28 @@ static int kvm_vcpu_ioctl_device_attr(struct kvm_vcpu *vcpu, return r; } + +#define kvm_ioctl_disable_exits(a, mask) \ +({ \ + if (!kvm_can_mwait_in_guest()) \ + (mask) &= KVM_X86_DISABLE_EXITS_MWAIT; \ + if ((mask) & KVM_X86_DISABLE_EXITS_OVERRIDE) { \ + (a).mwait_in_guest = (mask) & KVM_X86_DISABLE_EXITS_MWAIT; \ + (a).hlt_in_guest = (mask) & KVM_X86_DISABLE_EXITS_HLT; \ + (a).pause_in_guest = (mask) & KVM_X86_DISABLE_EXITS_PAUSE; \ + (a).cstate_in_guest = (mask) & KVM_X86_DISABLE_EXITS_CSTATE; \ + } else { \ + if ((mask) & KVM_X86_DISABLE_EXITS_MWAIT) \ + (a).mwait_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_HLT) \ + (a).hlt_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_PAUSE) \ + (a).pause_in_guest = true; \ + if ((mask) & KVM_X86_DISABLE_EXITS_CSTATE) \ + (a).cstate_in_guest = true; \ + } \ +}) + static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, struct kvm_enable_cap *cap) { @@ -6238,14 +6260,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, if (kvm->created_vcpus) goto disable_exits_unlock; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_MWAIT) - kvm->arch.mwait_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_HLT) - kvm->arch.hlt_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_PAUSE) - kvm->arch.pause_in_guest = true; - if (cap->args[0] & KVM_X86_DISABLE_EXITS_CSTATE) - kvm->arch.cstate_in_guest = true; + kvm_ioctl_disable_exits(kvm->arch, cap->args[0]); + r = 0; disable_exits_unlock: mutex_unlock(&kvm->lock); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 55155e262646..12ea7dd80471 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -823,10 +823,12 @@ struct kvm_ioeventfd { #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_OVERRIDE (1ull << 63) #define KVM_X86_DISABLE_VALID_EXITS (KVM_X86_DISABLE_EXITS_MWAIT | \ KVM_X86_DISABLE_EXITS_HLT | \ KVM_X86_DISABLE_EXITS_PAUSE | \ - KVM_X86_DISABLE_EXITS_CSTATE) + KVM_X86_DISABLE_EXITS_CSTATE | \ + KVM_X86_DISABLE_EXITS_OVERRIDE) /* for KVM_ENABLE_CAP */ struct kvm_enable_cap { From patchwork Fri Jan 13 22:01:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43630 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2003wrn; Fri, 13 Jan 2023 14:07:20 -0800 (PST) X-Google-Smtp-Source: AMrXdXslDSR5YBddY9ut5MMDdCjbraLuJybyQVAbLkNmbE7dgExCA7P4nY8UMMi2Agn5GLccmw+/ X-Received: by 2002:a17:906:158c:b0:7c4:f752:e959 with SMTP id k12-20020a170906158c00b007c4f752e959mr65543327ejd.33.1673647640541; Fri, 13 Jan 2023 14:07:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647640; cv=pass; d=google.com; s=arc-20160816; b=KZ1mh0ScsDNnkECiCqyWEJDnqCMUcuNExnhXRBQLmwy+8zigCQrzjkW62UITGjnNmV jMZF/ph1TlLDDfqViWYC+DGN/YP4r/2VUNORe0vlX2Ds0Qlv/Q+R+HBriW5UNoOelCPK jqV/5Jm43vLHmwgQ2lPLmpfQRYZ26xK9tgQtjaWEQC5s6DD084GkrzBnkilvI3RiWj12 qZY9KCED8XghIonWSjVGYiA0lGlktJ03OFw7e1xwFWa29HX9b6PPFrWshgFh1ypjEloc IxQIjiEEC8CKAtdQJIEJNq8dW6NEaIWmBFca/FrJCV7mX+66Y3J9hfZUu3iOYcadjlf6 appw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pCVwnfon+D+exnG4W9ehny4BDM7lJpQ2S3wz9rg4rFw=; b=I1oebNtEOPYSFrbMymsXiCHw/6TfqAZZP+lVih6zKp0YKf96Gkh4Po1p4D8amJ2y12 RqCS3MeyFm5HN3bi8+1xZxv/2jVk0jD2YIPX+nEw900hAICR3nt0+7/E8DC4VV1YsJ4c 5UJGjZbgFoT9I5qmZIVCpz7L7eY7xPVUeo4qmKksjKMVnkLWy/ObP6bq2V4lFCyv/evP afkI38ZQQTwDOpY5D/yKsMCesjcPJSyOTTYtI/7d3QU1DOnSXHR8pc4g3nZW6n3XxKAg c4bBpBpyxqIUTU5w/XjzBRPqCWeXY8W2w2sXDOEOtlMyEisXDI5ozXyokVbJhxXYlLsN tUXg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=nHbg6a2N; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xg8-20020a170907320800b00836ecced351si22992855ejb.380.2023.01.13.14.06.56; Fri, 13 Jan 2023 14:07:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=nHbg6a2N; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229509AbjAMWCk (ORCPT + 99 others); Fri, 13 Jan 2023 17:02:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230268AbjAMWCd (ORCPT ); Fri, 13 Jan 2023 17:02:33 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2087.outbound.protection.outlook.com [40.107.94.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6977475776; Fri, 13 Jan 2023 14:02:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gRh1gJTE0H/Y6gYtBfOzMZxDj8FB2Gwz8GyD1aYIonts2wPOjJIR9UB9wMJPyvq1oiTSdffDqsU3+qzRFkDsnOBkfqWA6cnE9/Bmz9sPbV9mnWn6L/ZVBrkXLogCPXJhJi6GSk5O2E/XwhneSj8sAdWtvMImx3bUZyqKhy0bcs0SZuyZvfzwy2EYyYmtXXqqsa1Jtj0prx7IGSXhgtJ1Q66JYFUKv7gH+bXommWt7zF3eRaYatVMDzY0C2EcxTGRQOndsNPPjd+vSs/WfCc2SQXEPU+JV51gcHBpOT+a8SkUunzZmyQkfoulxGXMFFW0e2PXpRE9CMb9hF/zlTLwrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pCVwnfon+D+exnG4W9ehny4BDM7lJpQ2S3wz9rg4rFw=; b=bT2xIgRK8lC9RhkQJGRqAKgaoYFesfF4W4PDNvBqr2mYT02BPQ8gmesCZP6Zv9q+5PDb2XnHQp+qPdMpw8x0KctFZhtfdj95+ap5EYpogjaR4kOmWFhUqXEm2RmkQC6xYzHHx5R5i91Le0YB9Op3/seBO6BZ7bqMKVP66Uz/we0Tw6lFS3yKj2HQlCpDOG1gSpnWUny/Ia2ieb4Eljf4XK2o1XacxEAeEYpPxXYDrBlFrrwURlQ/ZGZOwxCSmrkUCX5S2ja16UGQ4RWOT8WRKwgQkbnkAGGtOwrTKB4wHAxEEsYulePWuBsL+ln36zGTklN37ftqA+otmdj2N5TZ0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pCVwnfon+D+exnG4W9ehny4BDM7lJpQ2S3wz9rg4rFw=; b=nHbg6a2Nn+Of0AgtHGfYGlAt0q3nExcD8UycgFup/phU88qpo1ol38gu87/mY/lMKuyetxZX6NO7pTDt7JXw8dN9i53qm46vkqKaH8oI0+zG/rYK0zixsv0IlQ1j1hb4si3swaYAqeE97XesTALE8th8WlN9/c3cnaHLPCSp1GDS79EIFiQDBcTC9BP9dClIe4sApiD4DYvUpz4NPRIVmevCfg3Lnw3dmgQXiUxDtvP/ypSq0uCHlcwSB4qDCEJeA9EHluqm91OgFdW69oI5qnuM7NphHK/ABrRw9U00QTam4iexcJXvDkFVhrlOobSIaVXg007HOPmJw5e5gCBolw== Received: from MW4P222CA0023.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::28) by SA1PR12MB8144.namprd12.prod.outlook.com (2603:10b6:806:337::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Fri, 13 Jan 2023 22:02:17 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:303:114:cafe::d3) by MW4P222CA0023.outlook.office365.com (2603:10b6:303:114::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.16 via Frontend Transport; Fri, 13 Jan 2023 22:02:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:02:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:04 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:03 -0800 From: Kechen Lu To: , , CC: , , , , Subject: [RFC PATCH v5 5/6] KVM: x86: add vCPU scoped toggling for disabled exits Date: Fri, 13 Jan 2023 22:01:13 +0000 Message-ID: <20230113220114.2437-6-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT065:EE_|SA1PR12MB8144:EE_ X-MS-Office365-Filtering-Correlation-Id: 0cd70f0d-e14c-4aa9-8fe1-08daf5b1d62e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: soPD7O1ivDgTW5w9do9YWUcdeD06x/7kmAz0tDsdTSBl0RFgCNM0AgJOPW1ATVEgngpPBW0hAbq69llUeIzv+ckcCKLUfJOlQPRrp+mSed4JSn4HTQktPfumJe+nlsWB/usCot0aKUK9uAXkTzqRYbnNyE3gfpZOrB3imvdgJGWQTPDyv+U2ZUTFq9wzfm0kpFgOT//Mvt0Vk9GqTgZIiK7XwPaBAT5+nJxATupESahxfpMNl9RB2x1XcT4hN8KhVCDbYshR/EgzKUVrJVuBpS6OK2s+VldjyFthkEvpjLokqVV2ZQZhBpcUiADHYmboRJuu/anc4zQAmPos+xINA1eaZvxucmBF9Hz5w3hY7n1far8G3GyFzhl89qL7XDIB/ogJllYdh27IXXrM/Ha0Un4/L+sD7cRLl2kwu+y2nFX6svusTyWjLsQV5paPwUMYNYGpWt+DL+MYHaDN4pHHwEXYNT9+IA9JODsm6Sexnbu1rIwHKtEKqMuFWF7GIh1nN/mKVFI3yy5fT4H8NEwOTZotj71GfV3zyCi5xoj7MdJR0/ikNIzcjpSe23Mg0zd1L5V6oPVtI/aBJin4QTd/vuUy7x5v+7j6dWmhtFRRLLtuhO8T7X2x5SYDsTh8403aBWu3lctsSgcg/0W9f26NAjupfb/AllClf51i4mKpVfxoFHzvBi3++kKczrtbJEsEMZnbSXgUZ2dnFJqMKRZXuA== X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(396003)(376002)(39860400002)(136003)(346002)(451199015)(36840700001)(40470700004)(46966006)(47076005)(16526019)(40480700001)(6666004)(26005)(336012)(186003)(4326008)(2616005)(7696005)(70206006)(316002)(1076003)(478600001)(54906003)(70586007)(110136005)(356005)(82740400003)(7636003)(86362001)(36860700001)(426003)(83380400001)(40460700003)(82310400005)(41300700001)(8676002)(36756003)(5660300002)(2906002)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:02:17.6801 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0cd70f0d-e14c-4aa9-8fe1-08daf5b1d62e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8144 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946747950121843?= X-GMAIL-MSGID: =?utf-8?q?1754946747950121843?= Introduce support of vCPU-scoped ioctl with KVM_CAP_X86_DISABLE_EXITS cap for disabling exits to enable finer-grained VM exits disabling on per vCPU scales instead of whole guest. This patch enables the vCPU-scoped exits control toggling, but keeps the VM-scoped exits control behaviors restriction as before. In use cases like Windows guest running heavy CPU-bound workloads, disabling HLT VM-exits could mitigate host sched ctx switch overhead. Simply HLT disabling on all vCPUs could bring performance benefits, but if no pCPUs reserved for host threads, could happened to the forced preemption as host does not know the time to do the schedule for other host threads want to run. With this patch, we could only disable part of vCPUs HLT exits for one guest, this still keeps performance benefits, and also shows resiliency to host stressing workload running at the same time. In the host stressing workload experiment with Windows guest heavy CPU-bound workloads, it shows good resiliency and having the ~3% performance improvement. E.g. Passmark running in a Windows guest with this patch disabling HLT exits on only half of vCPUs still showing 2.4% higher main score v/s baseline. Suggested-by: Sean Christopherson Suggested-by: Chao Gao Signed-off-by: Kechen Lu --- Documentation/virt/kvm/api.rst | 2 +- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 30 ++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 37 ++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 7 ++++++ 6 files changed, 78 insertions(+), 1 deletion(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 3850202942d0..698f476d36dd 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7102,7 +7102,7 @@ longer intercept some instructions for improved latency in some workloads, and is suggested when vCPUs are associated to dedicated physical CPUs. More bits can be added in the future; userspace can just pass the KVM_CHECK_EXTENSION result to KVM_ENABLE_CAP to disable -all such vmexits. +all such vmexits. VM scoped and vCPU scoped capability are both supported. By default, this capability only disables exits. To re-enable an exit, or to override previous settings, userspace can set KVM_X86_DISABLE_EXITS_OVERRIDE, diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index abccd51dcfca..534322c21168 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP(update_disabled_exits) #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 41b998234a04..e21e5d452b5d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1711,6 +1711,8 @@ struct kvm_x86_ops { * Returns vCPU specific APICv inhibit reasons */ unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); + + void (*update_disabled_exits)(struct kvm_vcpu *vcpu); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index dc7176605e01..81c387dfa46c 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4680,6 +4680,33 @@ static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) sev_vcpu_deliver_sipi_vector(vcpu, vector); } +static void svm_update_disabled_exits(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + struct vmcb_control_area *control = &svm->vmcb->control; + + if (kvm_hlt_in_guest(vcpu)) + svm_clr_intercept(svm, INTERCEPT_HLT); + else + svm_set_intercept(svm, INTERCEPT_HLT); + + if (kvm_mwait_in_guest(vcpu)) { + svm_clr_intercept(svm, INTERCEPT_MONITOR); + svm_clr_intercept(svm, INTERCEPT_MWAIT); + } else { + svm_set_intercept(svm, INTERCEPT_MONITOR); + svm_set_intercept(svm, INTERCEPT_MWAIT); + } + + if (kvm_pause_in_guest(vcpu)) { + svm_clr_intercept(svm, INTERCEPT_PAUSE); + } else { + control->pause_filter_count = pause_filter_count; + if (pause_filter_thresh) + control->pause_filter_thresh = pause_filter_thresh; + } +} + static void svm_vm_destroy(struct kvm *kvm) { avic_vm_destroy(kvm); @@ -4825,7 +4852,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .complete_emulated_msr = svm_complete_emulated_msr, .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, + .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons, + + .update_disabled_exits = svm_update_disabled_exits, }; /* diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 019a20029878..f5137afdd424 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8070,6 +8070,41 @@ static void vmx_vm_destroy(struct kvm *kvm) free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm)); } +static void vmx_update_disabled_exits(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + if (kvm_hlt_in_guest(vcpu)) + exec_controls_clearbit(vmx, CPU_BASED_HLT_EXITING); + else + exec_controls_setbit(vmx, CPU_BASED_HLT_EXITING); + + if (kvm_mwait_in_guest(vcpu)) + exec_controls_clearbit(vmx, CPU_BASED_MWAIT_EXITING | + CPU_BASED_MONITOR_EXITING); + else + exec_controls_setbit(vmx, CPU_BASED_MWAIT_EXITING | + CPU_BASED_MONITOR_EXITING); + + if (!kvm_pause_in_guest(vcpu)) { + vmcs_write32(PLE_GAP, ple_gap); + vmx->ple_window = ple_window; + vmx->ple_window_dirty = true; + } + + if (kvm_cstate_in_guest(vcpu)) { + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); + vmx_disable_intercept_for_msr(vcpu, MSR_CORE_C7_RESIDENCY, MSR_TYPE_R); + } else { + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C1_RES, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C3_RESIDENCY, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C6_RESIDENCY, MSR_TYPE_R); + vmx_enable_intercept_for_msr(vcpu, MSR_CORE_C7_RESIDENCY, MSR_TYPE_R); + } +} + static struct kvm_x86_ops vmx_x86_ops __initdata = { .name = "kvm_intel", @@ -8207,6 +8242,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .complete_emulated_msr = kvm_complete_insn_gp, .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + + .update_disabled_exits = vmx_update_disabled_exits, }; static unsigned int vmx_handle_intel_pt_intr(void) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3ea5f12536a0..8c15292c6886 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5552,6 +5552,13 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, if (vcpu->arch.pv_cpuid.enforce) kvm_update_pv_runtime(vcpu); + return 0; + case KVM_CAP_X86_DISABLE_EXITS: + if (cap->args[0] & ~kvm_get_allowed_disable_exits()) + return -EINVAL; + + kvm_ioctl_disable_exits(vcpu->arch, cap->args[0]); + static_call(kvm_x86_update_disabled_exits)(vcpu); return 0; default: return -EINVAL; From patchwork Fri Jan 13 22:01:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kechen Lu X-Patchwork-Id: 43631 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2252wrn; Fri, 13 Jan 2023 14:08:04 -0800 (PST) X-Google-Smtp-Source: AMrXdXvf1u4y8ulE7PkA/NGPduNuBE/G/6QlYHy/3XEvlLsL8M8REAKljX0Vix60IezQyVVFDOIl X-Received: by 2002:aa7:cb88:0:b0:499:8849:5fb8 with SMTP id r8-20020aa7cb88000000b0049988495fb8mr17598313edt.30.1673647684153; Fri, 13 Jan 2023 14:08:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1673647684; cv=pass; d=google.com; s=arc-20160816; b=HS9hBfXq/qLcjDJPToy3VZJNi6ckQpT+1/DCxSzKh1kmE4Xl9nhrE+Twe2UHJ204IU m+8BBpvRuTjgQAG35VwW0FAzB2agllYuJu2gexn2j/Ic9z3gB9HXfEOljnbkF10/qOLD 89EsYi5I+apJOf9wofRdi5xLJMyAcuF+0mnScWafL3Eitbck+VUONOITXtIl1YRd7GXe h1dP5BEb5xKZMlxv2769n4cE0Hzagio33pWmQJqr+KDz6XwnCycAjYuLFXafv+je0i8F iWH7kYlzCmgfG5lNC+Vwmw1+51Ro+/kKx99bWeOcynBYc2yNVkNplYqElNfIqb2hLJaO ikQw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NKDycVnZfRlgX0CLjScE7vgCoOPktCmR0T6UvKo1v14=; b=UZfQQqdK2oZLsY0YrB4geTV6In0CC2w3aBPK/bbFrdpHfEEZNHfuW156zrIo8gqhuC ogimNK/ybSLp4RjHgMWOupztLPV9IQrJXCGU4c7nrMlv8uJEZUZe66HcMYdMBBEiaQwU +f9O1ER1V/jFDkXLNcxHkuzz6cTq7S0UVHQt6gewx5OwzigrL0pdmXiGDQfVXbIlLGHG ERb6lXtLdp81Ra8PK4w+tbKgOZEa8kP88kkZ4iJ92IYwd+LS/xOLbNO7gy9XDid22Ano 6hYC8tj4K8Yaa6DyvPeB/0BwuuWD7eSopSZi/suFzc0pdji88jFzz5VblfjLt0N7HNZ7 QXZQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=hfBMvjXm; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qb20-20020a1709077e9400b0084d15a1bc6fsi24229421ejc.418.2023.01.13.14.07.40; Fri, 13 Jan 2023 14:08:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=hfBMvjXm; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230268AbjAMWDG (ORCPT + 99 others); Fri, 13 Jan 2023 17:03:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231317AbjAMWCh (ORCPT ); Fri, 13 Jan 2023 17:02:37 -0500 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2046.outbound.protection.outlook.com [40.107.92.46]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96C2D76808; Fri, 13 Jan 2023 14:02:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BlNzB4mjp7l8dVUIS6+lyWLLnIesWTqjP3/2daDeegunms00NS0UPtWn01uWPg6JHx7r5wUff1qCRrvDe9uvA+kjHGNfhosY4BzV9L/3NNwFXC7vO6LDNUbBlycry2c36jWaycWf0v5S41HqptlIxUuM8xYA6VcFrmvdBVcRU2vbxM1vOK5c765slyjopNkXJFm1M+H/sQCpHjWHfa0GxvJUottxt3f/KFITs17HddRpQX7jgQJ3FgPoPMKDKjijkPvM8jjpc0sWnZ4xPFzOLRSdo7wyugxv/ZdhgjUxei4IbYcjeb823ItBtBy35/dJLxA4RRaO77YvmH45CBvv1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NKDycVnZfRlgX0CLjScE7vgCoOPktCmR0T6UvKo1v14=; b=KopY7AouAyGD8/cuvL6z6wtx7HWShrICVpTCXAiyTqvru3zZRgSuzmYuKTuF+eUevH8dpAZPzw+fNlyFRQ1m6aZBdKSdxsYLxzxLf4K1xs5KMrWaojMvvvYcQkNiAoRbAodS2Ul9q79djU2VscRL/lZDQMhb8VwiBs8g7g0ycEfXUsQuN3PmBKp2XDwWKLIgvaXvgN8O6WO8G74JUIWskPjkuj1ie386E7fztjIq6P1X9uKKUia8tP9P4nneIHeQo/Nj+K9fFQ7epkwAU8VrQuYI5VUCa7H3lqv6SsmnYWsyPMoYqTKAA32zML1LYgo6qHfd4fxQlZ07CGhrmLFoVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NKDycVnZfRlgX0CLjScE7vgCoOPktCmR0T6UvKo1v14=; b=hfBMvjXmX2ww+AbqKHg+WTEx2zIQyy+tWGE5pz/EcOEt4qh+SO5BpwVmVQBuCtUYLGDTaMH0Pg0oxdHn3s6d6sPkCAmxH1Uyy7a2m5P6cDrtkpE4UfrYEzOrKcQe2HhsuCqQkTmtqaOq/s9ACWIHU4+VZ8wUZSQiUAe8f2QViXkaR6bpAt7RKje7nXPZoE8t9DBwnifLop5XPvBarfy7xgFArmK7nS2U6pKPGNsxurK0YjyllVjRYUSq2IoSVBfdEoJ5juklmw1EF0xZVwuW+fsmjBZVMW8qLNoJWnp2whhAGXA8fIqNTF8qBEVTV2jiZLVmo6y35q+lm9d0COlscg== Received: from DS7PR03CA0227.namprd03.prod.outlook.com (2603:10b6:5:3ba::22) by BY5PR12MB4853.namprd12.prod.outlook.com (2603:10b6:a03:1da::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13; Fri, 13 Jan 2023 22:02:19 +0000 Received: from DM6NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3ba:cafe::c3) by DS7PR03CA0227.outlook.office365.com (2603:10b6:5:3ba::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.16 via Frontend Transport; Fri, 13 Jan 2023 22:02:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT016.mail.protection.outlook.com (10.13.173.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.13 via Frontend Transport; Fri, 13 Jan 2023 22:02:19 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:05 -0800 Received: from dvt1-1.nvidia.com (10.126.231.37) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Fri, 13 Jan 2023 14:02:05 -0800 From: Kechen Lu To: , , CC: , , , , Subject: [RFC PATCH v5 6/6] KVM: selftests: Add tests for VM and vCPU cap KVM_CAP_X86_DISABLE_EXITS Date: Fri, 13 Jan 2023 22:01:14 +0000 Message-ID: <20230113220114.2437-7-kechenl@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230113220114.2437-1-kechenl@nvidia.com> References: <20230113220114.2437-1-kechenl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.37] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT016:EE_|BY5PR12MB4853:EE_ X-MS-Office365-Filtering-Correlation-Id: a1cd7d0e-6113-4817-ef89-08daf5b1d719 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jdreHZGAvBfiWDs77OIr3LfZGRR2U9JePJr2zJih3G6YO7UkhGDEcpLDeufoZxPKU7kOMJwlYR1MRPLVjYe2tRWcCSJTFe3f7nCSaFXdac7GyNrdOzUgEDmBTh33mahnrX0HX0SXdnKl6m6xr/lW2ZSjeZAk0zjvDJ8FH7PN5zlIHkX/KVe8qkXIgNa/d3SjJmuhvD0c8lnHpD9wLrQggXZ6lXIvr+SWvEFtb5w0/mmh8a3hG82z4EOEG81X207hoHSIPejWQlllj1Fc7cvr/AsaC/SO1nVdADoRfUmJgesr7KA6VtEl9OHQ9k+1OprxBOD1I2tsZXWwqc9YAH8SGizBCVTOhMQBzo0jZoZrWvrubynA5WkVBNOk/qD+TdjBuoe9cMIdBXx+WnOrimSED0De82yc4Xwu8tQpNOSfzFn8d0+6mAk7XYPWSMmZFAtf7j8QCbWZkHPUKfJ/EDeKoTTmhYZjd/RKuJrhrDA33n/8VzaxyO5a/TTcMBaQxB/TsXYIrDiyXELY9joPXMuCbJ/NQ+RA6/M0fSo0zqzWuvC5KsQQfd+mipWyW+rA7/z/2YJDgdQes2nDPbEf5F+BSnxxhn0IH20zoF4zGZ0L92tDJi3XxQ8ra9vottt/0oVRSzgIQGLimja4i4A/Fch4KjtfC9pzl8oEEPexSdeBkVC8eIIxA/LyhqNRlaBX4adZD4RLmEW5tDDMfzb6wI+k5g== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230022)(4636009)(376002)(346002)(136003)(39860400002)(396003)(451199015)(36840700001)(46966006)(40470700004)(7636003)(82740400003)(356005)(40480700001)(36756003)(40460700003)(82310400005)(86362001)(2616005)(110136005)(54906003)(316002)(26005)(1076003)(336012)(478600001)(186003)(6666004)(16526019)(7696005)(30864003)(36860700001)(2906002)(83380400001)(4326008)(41300700001)(8676002)(70206006)(70586007)(47076005)(426003)(8936002)(5660300002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2023 22:02:19.1601 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a1cd7d0e-6113-4817-ef89-08daf5b1d719 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4853 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754946794002564152?= X-GMAIL-MSGID: =?utf-8?q?1754946794002564152?= Add selftests for KVM cap KVM_CAP_X86_DISABLE_EXITS overriding flags in VM and vCPU scope both works as expected. Suggested-by: Chao Gao Suggested-by: Shaoqin Huang Signed-off-by: Kechen Lu --- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/x86_64/disable_exits_test.c | 457 ++++++++++++++++++ 2 files changed, 458 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/disable_exits_test.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 1750f91dd936..eeeba35e2536 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -114,6 +114,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/sev_migrate_tests TEST_GEN_PROGS_x86_64 += x86_64/amx_test TEST_GEN_PROGS_x86_64 += x86_64/max_vcpuid_cap_test TEST_GEN_PROGS_x86_64 += x86_64/triple_fault_event_test +TEST_GEN_PROGS_x86_64 += x86_64/disable_exits_test TEST_GEN_PROGS_x86_64 += access_tracking_perf_test TEST_GEN_PROGS_x86_64 += demand_paging_test TEST_GEN_PROGS_x86_64 += dirty_log_test diff --git a/tools/testing/selftests/kvm/x86_64/disable_exits_test.c b/tools/testing/selftests/kvm/x86_64/disable_exits_test.c new file mode 100644 index 000000000000..dceba3bcef5f --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/disable_exits_test.c @@ -0,0 +1,457 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Test per-VM and per-vCPU disable exits cap + * 1) Per-VM scope + * 2) Per-vCPU scope + * + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include +#include +#include +#include +#include + +#include "test_util.h" +#include "kvm_util.h" +#include "svm_util.h" +#include "vmx.h" +#include "processor.h" +#include "asm/kvm.h" +#include "linux/kvm.h" + +/* Arbitary chosen IPI vector value from sender to halter vCPU */ +#define IPI_VECTOR 0xa5 +/* Number of HLTs halter vCPU thread executes */ +#define COUNT_HLT_EXITS 10 + +struct guest_stats { + uint32_t halter_apic_id; + volatile uint64_t hlt_count; + volatile uint64_t wake_count; +}; + +static u64 read_vcpu_stats_halt_exits(struct kvm_vcpu *vcpu) +{ + int i; + struct kvm_stats_header header; + u64 *stats_data; + u64 ret = 0; + struct kvm_stats_desc *stats_desc; + struct kvm_stats_desc *pdesc; + int stats_fd = vcpu_get_stats_fd(vcpu); + + read_stats_header(stats_fd, &header); + if (header.num_desc == 0) { + fprintf(stderr, + "Cannot read halt exits since no KVM stats defined\n"); + return ret; + } + + stats_desc = read_stats_descriptors(stats_fd, &header); + for (i = 0; i < header.num_desc; ++i) { + pdesc = get_stats_descriptor(stats_desc, i, &header); + if (!strncmp(pdesc->name, "halt_exits", 10)) { + stats_data = malloc(pdesc->size * sizeof(*stats_data)); + read_stat_data(stats_fd, &header, pdesc, stats_data, + pdesc->size); + ret = *stats_data; + free(stats_data); + break; + } + } + free(stats_desc); + return ret; +} + +/* HLT multiple times in one vCPU */ +static void halter_guest_code(struct guest_stats *data) +{ + xapic_enable(); + data->halter_apic_id = GET_APIC_ID_FIELD(xapic_read_reg(APIC_ID)); + + for (;;) { + data->hlt_count++; + asm volatile("sti; hlt; cli"); + data->wake_count++; + } +} + +static void halter_waiting_guest_code(struct guest_stats *data) +{ + uint64_t tsc_start = rdtsc(); + + xapic_enable(); + data->halter_apic_id = GET_APIC_ID_FIELD(xapic_read_reg(APIC_ID)); + + for (;;) { + data->hlt_count++; + asm volatile("sti; hlt; cli"); + data->wake_count++; + /* Wait for ~0.5sec for each HLT execution */ + tsc_start = rdtsc(); + while (rdtsc() - tsc_start < 2000000000); + } +} + +/* Runs on halter vCPU when IPI arrives */ +static void guest_ipi_handler(struct ex_regs *regs) +{ + xapic_write_reg(APIC_EOI, 11); +} + +/* Sender vCPU waits for ~1sec to assume HLT executed */ +static void sender_wait_loop(struct guest_stats *data, uint64_t old_hlt_count, + uint64_t old_wake_count) +{ + uint64_t tsc_start = rdtsc(); + while (rdtsc() - tsc_start < 4000000000); + GUEST_ASSERT((data->wake_count != old_wake_count) && + (data->hlt_count != old_hlt_count)); +} + +/* Sender vCPU loops sending IPI to halter vCPU every ~1sec */ +static void sender_guest_code(struct guest_stats *data) +{ + uint32_t icr_val; + uint32_t icr2_val; + uint64_t old_hlt_count = 0; + uint64_t old_wake_count = 0; + + xapic_enable(); + /* Init interrupt command register for sending IPIs */ + icr_val = (APIC_DEST_PHYSICAL | APIC_DM_FIXED | IPI_VECTOR); + icr2_val = SET_APIC_DEST_FIELD(data->halter_apic_id); + + for (;;) { + /* + * Send IPI to halted vCPU + * First IPI sends here as already waited before sender vCPU + * thread creation + */ + xapic_write_reg(APIC_ICR2, icr2_val); + xapic_write_reg(APIC_ICR, icr_val); + sender_wait_loop(data, old_hlt_count, old_wake_count); + GUEST_ASSERT((data->wake_count != old_wake_count) && + (data->hlt_count != old_hlt_count)); + old_wake_count = data->wake_count; + old_hlt_count = data->hlt_count; + } +} + +static void *vcpu_thread(void *arg) +{ + struct kvm_vcpu *vcpu = (struct kvm_vcpu *)arg; + int old; + int r; + + r = pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old); + TEST_ASSERT(r == 0, + "pthread_setcanceltype failed on vcpu_id=%u with errno=%d", + vcpu->id, r); + fprintf(stderr, "vCPU thread running vCPU %u\n", vcpu->id); + vcpu_run(vcpu); + return NULL; +} + +static void cancel_join_vcpu_thread(pthread_t thread, struct kvm_vcpu *vcpu) +{ + void *retval; + int r; + + r = pthread_cancel(thread); + TEST_ASSERT(r == 0, + "pthread_cancel on vcpu_id=%d failed with errno=%d", + vcpu->id, r); + + r = pthread_join(thread, &retval); + TEST_ASSERT(r == 0, + "pthread_join on vcpu_id=%d failed with errno=%d", + vcpu->id, r); +} + +/* + * Test case 1: + * Normal VM running with one vCPU keeps executing HLTs, + * another vCPU sending IPIs to wake it up, should expect + * all HLTs exiting to host + */ +static void test_vm_without_disable_exits_cap(void) +{ + int r; + int wait_secs; + const int first_halter_wait = 10; + uint64_t kvm_halt_exits; + struct kvm_vm *vm; + struct kvm_vcpu *halter_vcpu; + struct kvm_vcpu *sender_vcpu; + struct guest_stats *data; + vm_vaddr_t guest_stats_page_vaddr; + pthread_t threads[2]; + + /* Create VM */ + vm = vm_create(2); + + /* Add vCPU with loops halting */ + halter_vcpu = vm_vcpu_add(vm, 0, halter_guest_code); + + vm_init_descriptor_tables(vm); + vcpu_init_descriptor_tables(halter_vcpu); + vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler); + virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA); + + /* Add vCPU with IPIs waking up halter vCPU */ + sender_vcpu = vm_vcpu_add(vm, 1, sender_guest_code); + + guest_stats_page_vaddr = vm_vaddr_alloc_page(vm); + data = addr_gva2hva(vm, guest_stats_page_vaddr); + memset(data, 0, sizeof(*data)); + + vcpu_args_set(halter_vcpu, 1, guest_stats_page_vaddr); + vcpu_args_set(sender_vcpu, 1, guest_stats_page_vaddr); + + /* Start halter vCPU thread and wait for it to execute first HLT. */ + r = pthread_create(&threads[0], NULL, vcpu_thread, halter_vcpu); + TEST_ASSERT(r == 0, + "pthread_create halter failed errno=%d", errno); + fprintf(stderr, "Halter vCPU thread started\n"); + + wait_secs = 0; + while ((wait_secs < first_halter_wait) && !data->hlt_count) { + sleep(1); + wait_secs++; + } + TEST_ASSERT(data->hlt_count, + "Halter vCPU did not execute first HLT within %d seconds", + first_halter_wait); + fprintf(stderr, + "Halter vCPU thread reported its first HLT executed " + "after %d seconds.\n", + wait_secs); + + /* + * After guest halter vCPU executed first HLT, start the sender + * vCPU thread to wakeup halter vCPU + */ + r = pthread_create(&threads[1], NULL, vcpu_thread, sender_vcpu); + TEST_ASSERT(r == 0, "pthread_create sender failed errno=%d", errno); + + while (data->hlt_count < COUNT_HLT_EXITS); + + cancel_join_vcpu_thread(threads[0], halter_vcpu); + cancel_join_vcpu_thread(threads[1], sender_vcpu); + + kvm_halt_exits = read_vcpu_stats_halt_exits(halter_vcpu); + TEST_ASSERT(kvm_halt_exits == data->hlt_count, + "Halter vCPU had unmatched %lu halt exits - %lu HLTs " + "executed, when not disabling VM halt exits\n", + kvm_halt_exits, data->hlt_count); + fprintf(stderr, "Halter vCPU had %lu halt exits\n", + kvm_halt_exits); + fprintf(stderr, "Guest records %lu HLTs executed, " + "waked %lu times\n", + data->hlt_count, data->wake_count); + + kvm_vm_free(vm); +} + +/* + * Test case 2: + * VM scoped exits disabling, HLT instructions + * stay inside guest without exits + */ +static void test_vm_disable_exits_cap(void) +{ + int r; + uint64_t kvm_halt_exits; + struct kvm_vm *vm; + struct kvm_vcpu *halter_vcpu; + struct guest_stats *data; + vm_vaddr_t guest_stats_page_vaddr; + pthread_t halter_thread; + + /* Create VM */ + vm = vm_create(1); + + /* + * Before adding any vCPUs, enable the KVM_X86_DISABLE_EXITS cap + * with flag KVM_X86_DISABLE_EXITS_HLT + */ + vm_enable_cap(vm, KVM_CAP_X86_DISABLE_EXITS, + KVM_X86_DISABLE_EXITS_HLT); + + /* Add vCPU with loops halting */ + halter_vcpu = vm_vcpu_add(vm, 0, halter_waiting_guest_code); + + vm_init_descriptor_tables(vm); + vcpu_init_descriptor_tables(halter_vcpu); + vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler); + virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA); + + guest_stats_page_vaddr = vm_vaddr_alloc_page(vm); + data = addr_gva2hva(vm, guest_stats_page_vaddr); + memset(data, 0, sizeof(*data)); + vcpu_args_set(halter_vcpu, 1, guest_stats_page_vaddr); + + /* Start halter vCPU thread and execute HLTs immediately */ + r = pthread_create(&halter_thread, NULL, vcpu_thread, halter_vcpu); + TEST_ASSERT(r == 0, + "pthread_create halter failed errno=%d", errno); + fprintf(stderr, "Halter vCPU thread started\n"); + + while (data->hlt_count < COUNT_HLT_EXITS); + + cancel_join_vcpu_thread(halter_thread, halter_vcpu); + + kvm_halt_exits = read_vcpu_stats_halt_exits(halter_vcpu); + TEST_ASSERT(kvm_halt_exits == 0, + "Halter vCPU had unexpected halt exits occuring after " + "disabling VM-scoped halt exits cap\n"); + fprintf(stderr, "Halter vCPU had %lu HLT exits\n", + kvm_halt_exits); + fprintf(stderr, "Guest records %lu HLTs executed\n", + data->hlt_count); + + kvm_vm_free(vm); +} + +/* + * Test case 3: + * VM overrides exits disable flags after vCPU created, + * which is not allowed + */ +static void test_vm_disable_exits_cap_with_vcpu_created(void) +{ + int r; + struct kvm_vm *vm; + struct kvm_enable_cap cap = { + .cap = KVM_CAP_X86_DISABLE_EXITS, + .args[0] = KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_OVERRIDE, + }; + + /* Create VM */ + vm = vm_create(1); + /* Add vCPU with loops halting */ + vm_vcpu_add(vm, 0, halter_waiting_guest_code); + + /* + * After creating vCPU, the current VM-scoped ABI should + * discard the cap enable of KVM_CAP_X86_DISABLE_EXITS + * and return non-zero. Since vm_enabled_cap() not able + * to assert the return value, so use the __vm_ioctl() + */ + r = __vm_ioctl(vm, KVM_ENABLE_CAP, &cap); + + TEST_ASSERT(r != 0, + "Setting VM-scoped KVM_CAP_X86_DISABLE_EXITS after " + "vCPUs created is not allowed, but it succeeds here\n"); +} + +/* + * Test case 4: + * vCPU scoped halt exits disabling and enabling tests, + * verify overides are working after vCPU created + */ +static void test_vcpu_toggling_disable_exits_cap(void) +{ + int r; + uint64_t kvm_halt_exits; + struct kvm_vm *vm; + struct kvm_vcpu *halter_vcpu; + struct kvm_vcpu *sender_vcpu; + struct guest_stats *data; + vm_vaddr_t guest_stats_page_vaddr; + pthread_t threads[2]; + + /* Create VM */ + vm = vm_create(2); + + /* Add vCPU with loops halting */ + halter_vcpu = vm_vcpu_add(vm, 0, halter_waiting_guest_code); + /* Set KVM_CAP_X86_DISABLE_EXITS_HLT for halter vCPU */ + vcpu_enable_cap(halter_vcpu, KVM_CAP_X86_DISABLE_EXITS, + KVM_X86_DISABLE_EXITS_HLT | KVM_X86_DISABLE_EXITS_OVERRIDE); + + vm_init_descriptor_tables(vm); + vcpu_init_descriptor_tables(halter_vcpu); + vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler); + + virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA); + + /* Add vCPU with IPIs waking up halter vCPU */ + sender_vcpu = vm_vcpu_add(vm, 1, sender_guest_code); + + guest_stats_page_vaddr = vm_vaddr_alloc_page(vm); + data = addr_gva2hva(vm, guest_stats_page_vaddr); + memset(data, 0, sizeof(*data)); + + vcpu_args_set(halter_vcpu, 1, guest_stats_page_vaddr); + vcpu_args_set(sender_vcpu, 1, guest_stats_page_vaddr); + + r = pthread_create(&threads[0], NULL, vcpu_thread, halter_vcpu); + TEST_ASSERT(r == 0, + "pthread_create halter failed errno=%d", errno); + fprintf(stderr, "Halter vCPU thread started with halt exits" + "disabled\n"); + + /* + * For the first phase of the running, halt exits + * are disabled, halter vCPU executes HLT instruction + * but never exits to host + */ + while (data->hlt_count < (COUNT_HLT_EXITS / 2)); + + cancel_join_vcpu_thread(threads[0], halter_vcpu); + /* + * Override and clean KVM_CAP_X86_DISABLE_EXITS flags + * for halter vCPU. Expect to see halt exits occurs then. + */ + vcpu_enable_cap(halter_vcpu, KVM_CAP_X86_DISABLE_EXITS, + KVM_X86_DISABLE_EXITS_OVERRIDE); + + r = pthread_create(&threads[0], NULL, vcpu_thread, halter_vcpu); + TEST_ASSERT(r == 0, + "pthread_create halter failed errno=%d", errno); + fprintf(stderr, "Halter vCPU thread restarted and cleared " + "halt exits flag\n"); + + sleep(1); + /* + * Second phase of the test, after guest halter vCPU + * reenabled halt exits, start the sender + * vCPU thread to wakeup halter vCPU + */ + r = pthread_create(&threads[1], NULL, vcpu_thread, sender_vcpu); + TEST_ASSERT(r == 0, "pthread_create sender failed errno=%d", errno); + + while (data->hlt_count < COUNT_HLT_EXITS); + + cancel_join_vcpu_thread(threads[0], halter_vcpu); + cancel_join_vcpu_thread(threads[1], sender_vcpu); + + kvm_halt_exits = read_vcpu_stats_halt_exits(halter_vcpu); + TEST_ASSERT(kvm_halt_exits == (COUNT_HLT_EXITS / 2), + "Halter vCPU had unexpected %lu halt exits, " + "there should be %d halt exits while " + "not disabling VM halt exits\n", + kvm_halt_exits, COUNT_HLT_EXITS / 2); + fprintf(stderr, "Halter vCPU had %lu halt exits\n", + kvm_halt_exits); + fprintf(stderr, "Guest records %lu HLTs executed, " + "waked %lu times\n", + data->hlt_count, data->wake_count); + + kvm_vm_free(vm); +} + +int main(int argc, char *argv[]) +{ + fprintf(stderr, "VM-scoped tests start\n"); + test_vm_without_disable_exits_cap(); + test_vm_disable_exits_cap(); + test_vm_disable_exits_cap_with_vcpu_created(); + fprintf(stderr, "vCPU-scoped test starts\n"); + test_vcpu_toggling_disable_exits_cap(); + return 0; +}