From patchwork Wed Oct 19 16:56:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 366 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp432151wrs; Wed, 19 Oct 2022 09:58:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM75BOwPXYnk2o9sZSUK2ZJnJk5bFoMlLPD56fQ3HuTqayoLNbLU32XjC6IlPspBad5xgChQ X-Received: by 2002:a05:6402:154c:b0:459:b29:da88 with SMTP id p12-20020a056402154c00b004590b29da88mr8580554edx.296.1666198731372; Wed, 19 Oct 2022 09:58:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666198731; cv=none; d=google.com; s=arc-20160816; b=QmecW8cCL7H3EZYi3RIXdvfHzuJKDTzzfPyzZ7UUWuThVrBw/M5iGg1Ak0EkSClgAA OXejk2wb2pA/PVhe9e82Aq+FLuWKiD9cM0IzRGl5Tg6MQ74aHkuriIab1wIFu3ixXW1D xL4/c0v3lM3vQBN/UZDdBx7VSA1QkgksjtytBFHWkpuHPaKIOHDV5uqgY9HYepIKrILu I3MVhB79EBSfATxMUebUGwWHr5wBfrpA3TTWsCD9+4LbYXGZnu7WFNoa6dkVa3sVSpVy piSSCdyKGA0mgx79ZxnKELLm1zS50xQPb6miCIvHUkSaxg5lrLwqsMCjdglDWcoUXCNG b29Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :reply-to:dkim-signature; bh=oY8XEU2fkYE98MWfcMi47y+6Tzv6AnBcVCW4WAVAxZs=; b=bHcrYnBvtIv2HjyHLI72qj/l/NQWgrQ3Z50lLajoCrpJTMUtqnC/VUGICqdB9JFS2b wFYbu+DzYwXs7WJ8Uo5ApNXYzVZLbbLBjYnIiLma/uT/FJt4ndTRl6+Tdyl6bzGBrj9r DPq9M/7MvCwzSV1IZk5cOKgOgOhKP1MGj1oDRmumQ0D8HUO37RjfwyNQcJVeSV66pde/ prIN3d4x2oEXx+cT08+OeB0fAEnZdAFX0elhtdlTlRIo0Dqq+5cdEFU4HZWIfeKGnX2V x6/Wk9c5SVWUOlvw8Afyz1fAk2pvsZTecjbctqi1aOspyvkvlAfDBDkobtWE42kDd3/j EwwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=fVWJHQO1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c22-20020aa7d616000000b00451a9bc10casi12680705edr.196.2022.10.19.09.58.25; Wed, 19 Oct 2022 09:58:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=fVWJHQO1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231460AbiJSQ5A (ORCPT + 99 others); Wed, 19 Oct 2022 12:57:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231137AbiJSQ4g (ORCPT ); Wed, 19 Oct 2022 12:56:36 -0400 Received: from mail-oo1-xc49.google.com (mail-oo1-xc49.google.com [IPv6:2607:f8b0:4864:20::c49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08E23159A22 for ; Wed, 19 Oct 2022 09:56:33 -0700 (PDT) Received: by mail-oo1-xc49.google.com with SMTP id w22-20020a4a9d16000000b00480e74f0addso1329188ooj.4 for ; Wed, 19 Oct 2022 09:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:reply-to:from:to:cc :subject:date:message-id:reply-to; bh=oY8XEU2fkYE98MWfcMi47y+6Tzv6AnBcVCW4WAVAxZs=; b=fVWJHQO1iTUYtO4q3pmDdPVGWF8C/VkhPKaSCksdqw87DP+QbVYQ/D1uq5s9E8NTyM dTa/z2un1OlcetlYIE/0vuJ9bmf4ZSHOqS4EhSIukjXDon/1k2yigUL7qePGTrrn0v7h zpVk1ki0ET/8ZBMiJGz97h1Bpis+t3+V/1FEEWdhJAM1GKu5aV0ylCHq9UIfYTzmTSqm rmWLLInstHjp94OdB5019J/+pnIwEXlcNcJOiLLTcMBSQmbzgPt3a4DEr2HmW6W4OcUn jBIhNHhi+fU16/EWupEOzUQWwoZY7RijHxcm7pgbVuK/09IiULzqaR352cCzp7W2LNrJ bQlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:reply-to :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oY8XEU2fkYE98MWfcMi47y+6Tzv6AnBcVCW4WAVAxZs=; b=Ucn8RMt+IkYVGTV8B7WA56aOAehEhdcloQdsIgP2o3neluoNnr5wCHo6eSkh+0tBaw YxEkPf0GQlALYgAADF4GusmmlHAZpc/vra+vVFTxjVeNeAKI2dKuS8ZqTGC2Kg9ZdNxg 1CADLXPepPtGwEaPFTOP/wXzAxpWlBC1T3KRc2vpk7F+XXeMufBFEPvCdqaXG2SnSzUj IQJ0q56yBZ4gmNMQWWm61h6fxXeKlZMjl+aSnlTx7YoPCwr2PF9OzwxwaCot/t6SPPJU 3oDybCeH76gYdWH+xjRkw0p5MKeZFwiCUZOQv3OPJmsOdY/UbtFphkccP+7evEfjMRqy pBFg== X-Gm-Message-State: ACrzQf2ml0QVMAf5Gp0PRrT/EfVUWfwyCFADlAdtq6cdycQsB6mVoTBW yKgU7nvWMwNdlUd4KVi9CLG9ljJswsU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:90b:3505:b0:20d:ba2e:994b with SMTP id ls5-20020a17090b350500b0020dba2e994bmr34572645pjb.46.1666198582055; Wed, 19 Oct 2022 09:56:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 19 Oct 2022 16:56:10 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221019165618.927057-1-seanjc@google.com> Subject: [PATCH v6 0/8] KVM: x86: Apply NX mitigation more precisely From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mingwei Zhang , David Matlack , Yan Zhao , Ben Gardon X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747136000923645805?= X-GMAIL-MSGID: =?utf-8?q?1747136000923645805?= David and Ben, I kept your Reviewed-by for patch 7 even though it's now a fairly different patch than what you reviewed, as the code generation is identical. Holler if you want your review dropped. Precisely track (via kvm_mmu_page) if a non-huge page is being forced and use that info to avoid unnecessarily forcing smaller page sizes in disallowed_hugepage_adjust(). KVM incorrectly assumes that the NX huge page mitigation is the only scenario where KVM will create a non-leaf page instead of a huge page. As a result, if the original source of huge page incompatibility goes away, the NX mitigation is enabled, and KVM encounters an present shadow page when attempting to install a huge page, KVM will force a smaller page regardless of whether or not a smaller page is actually necessary to satisfy the NX huge page mitigation. Unnecessarily forcing small pages can result in degraded guest performance, especially on larger VMs. The bug was originally discovered when testing dirty log performance, as KVM would leave small pages lying around when zapping collapsible SPTEs. That case was indadvertantly fixed by commit 5ba7c4c6d1c7 ("KVM: x86/MMU: Zap non-leaf SPTEs when disabling dirty logging"), but other scenarios are still affected, e.g. KVM will not rebuild a huge page if the mmu_notifier zaps a range of PTEs because the primary MMU is creating a huge page. v6: - Fold the nx_huge_page_disallowed check in disallowed_hugepage_adjust() into the existing if-statement. [Yan] - Add a patch to extend the accounting sanity check in recovery to the TDP MMU. [Yan] v5: - Drop boneheaded KVM_BUG_ON() GFN aliasing. [Vitaly] - Drop incorrect barrier documentation. [Yan] v4: - https://lore.kernel.org/all/20220830235537.4004585-1-seanjc@google.com - Collect reviews. [Mingwei] - Add comment to document possible_nx_huge_pages. [Mingwei] - Drop extra memory barriers. [Paolo] - Document ordering providing by TDP SPTE helpers. [Paolo] v3: - https://lore.kernel.org/all/20220805230513.148869-1-seanjc@google.com - Bug the VM if KVM attempts to double account a shadow page that disallows a NX huge page. [David] - Split the rename to separate patch. [Paolo] - Rename more NX huge page variables/functions. [David] - Combine and tweak the comments about enforcing the NX huge page mitigation for non-paging MMUs. [Paolo, David] - Call out that the shadow MMU holds mmu_lock for write and doesn't need to manual handle memory ordering when accounting NX huge pages. [David] - Add a smp_rmb() when unlinking shadow pages in the TDP MMU. - Rename spte_to_sp() to spte_to_child_sp(). [David] - Collect reviews. [David] - Tweak the changelog for the final patch to call out that precise accounting addresses real world performance bugs. [Paolo] - Reword the changelog for the patch to (almost) always tag disallowed NX huge pages, and call out that it doesn't fix the TDP MMU. [David] v2: Rebase, tweak a changelog accordingly. v1: https://lore.kernel.org/all/20220409003847.819686-1-seanjc@google.com Mingwei Zhang (1): KVM: x86/mmu: explicitly check nx_hugepage in disallowed_hugepage_adjust() Sean Christopherson (7): KVM: x86/mmu: Tag disallowed NX huge pages even if they're not tracked KVM: x86/mmu: Rename NX huge pages fields/functions for consistency KVM: x86/mmu: Properly account NX huge page workaround for nonpaging MMUs KVM: x86/mmu: Set disallowed_nx_huge_page in TDP MMU before setting SPTE KVM: x86/mmu: Track the number of TDP MMU pages, but not the actual pages KVM: x86/mmu: Add helper to convert SPTE value to its shadow page KVM: x86/mmu: WARN if TDP MMU SP disallows hugepage after being zapped arch/x86/include/asm/kvm_host.h | 30 ++++---- arch/x86/kvm/mmu/mmu.c | 122 ++++++++++++++++++++------------ arch/x86/kvm/mmu/mmu_internal.h | 33 ++++----- arch/x86/kvm/mmu/paging_tmpl.h | 6 +- arch/x86/kvm/mmu/spte.c | 12 ++++ arch/x86/kvm/mmu/spte.h | 17 +++++ arch/x86/kvm/mmu/tdp_mmu.c | 43 ++++++----- arch/x86/kvm/mmu/tdp_mmu.h | 2 + 8 files changed, 170 insertions(+), 95 deletions(-) base-commit: e18d6152ff0f41b7f01f9817372022df04e0d354