From patchwork Fri Feb 3 19:28:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vipin Sharma X-Patchwork-Id: 52619 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1020509wrn; Fri, 3 Feb 2023 11:29:38 -0800 (PST) X-Google-Smtp-Source: AK7set9S9Q0PJlpMa+VNvRwj27NV+2zUL5EPrzaCfVXO1YSqV07+T6ekDJA35YBkzg+28dDmadZd X-Received: by 2002:a17:906:a88e:b0:87a:3aff:cffa with SMTP id ha14-20020a170906a88e00b0087a3affcffamr10833840ejb.27.1675452578174; Fri, 03 Feb 2023 11:29:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675452578; cv=none; d=google.com; s=arc-20160816; b=qq0MygsNV3zfWnP4to/SCRFD8AXDHhW/QXME5IgW0ZT4qPsVm2UOOW6huHTf9lV5k2 mAhY2JeDwWPg5pLyIXlutB7aeFQwKJioQlQnZiWHSqjMpkzt3e4Aglbby1qUZOiyr3tJ hDBuAGZhfUafoWYRoaZYK4P/DnNkgIzwIuh7sMSVVtNel8pdXPB1dbTEU6ld3vPkrb61 Zz4hituX9TYzwgj7sLInKipdF4GIAJiP7r4o0YaRzhOCUuH7ILacnno2QHBc6qa4JR5P BSZJ/eMD5itcd8Oc60hsoJc3fWCyw2x8D5jfcdAxMlZnjSR8MM+fTGe4NuNsuWOx8tJF WHZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=QvoxEP7Nplva4MzlDA7W4XNJEUEXXd+N/4Z4GM8dC2Q=; b=bA0dKhQ/BUIBWsemKIKxQhZFRvqeaU/Uxx6X9IeA1ljsKGDVFPW7S9dR+YyK8SaKlZ gIGB/TDLRKVyaU5TYnR8YQ14LNRA6arXfa/b1VBsccksPKYm8FWnCTIScStbPKc1GoBG LNBnBR5u1okGigKB/DcGKrsMCdrf1xqIvU7tkbySl6I3/KQ6LWI4W7YcMiQC5wyRmkku coe9zFDnt7au1si2+/OxBdIjdFXeAIhnOIYqbkhXsC2WbGtxcOtrsnv4hGRz5vYYBChi KLlS5z7jxHf1z0KHYlb/N+hUW99/zJ8wURCIFtVtfOo14LQw7dQtgOJGx0P+AK2WJwrz f6XQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=dq4cgVqD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c15-20020a17090620cf00b0088ba2cea06asi4836414ejc.774.2023.02.03.11.29.14; Fri, 03 Feb 2023 11:29:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=dq4cgVqD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233106AbjBCT2a (ORCPT + 99 others); Fri, 3 Feb 2023 14:28:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233008AbjBCT22 (ORCPT ); Fri, 3 Feb 2023 14:28:28 -0500 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99E171C594 for ; Fri, 3 Feb 2023 11:28:27 -0800 (PST) Received: by mail-pl1-x64a.google.com with SMTP id p10-20020a170902eaca00b00198dd431ec1so1535354pld.3 for ; Fri, 03 Feb 2023 11:28:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QvoxEP7Nplva4MzlDA7W4XNJEUEXXd+N/4Z4GM8dC2Q=; b=dq4cgVqDREWhgjpIoJfDa+A3p6xxbzcLyVJGtvLa/V7CoJW6H9Cu6rQ98CK0vPae0U cD8L4ox0jevG14e4DnZWH8fm72ypGb30L0bmwZCi5nd0Pjn7naE+mKfHmiBB1EK2DAgN cQA8mpsPbslgoxlbZQ2vEPI8PSg/yQOWMSRyLXfgbtWb7nSCKt34OwvNGnq/8qNhW5yt WF4+dgOP4BgUwaUSkwyNdmRkXEHUCoOiqzz1rLjOz8bWLJMiV5WRtQ5jrlkvgHTRVhpV 4q6/7JYef1DTp2adDwnJC/ljaN+vuPvWhnKoRt9yVTxX7bsoAIIQAcLhL+avml5alamo Zhgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QvoxEP7Nplva4MzlDA7W4XNJEUEXXd+N/4Z4GM8dC2Q=; b=tYIroD7J/MV75AYPfS0Oio/2F57+YIOKSItK6PaYVoEOWIDrJnuhgiMgmSbSkta6PP WNGOdSLF2qERDjeNDLdhrSe0Ke+zr+WzI20C6uCk7oIVO6z3cBvW+wpyMQ6WsD3GAZa7 Kz1IzQLfT+j58bH+rx/lVvj7gnMVpyrYTQ0e5BrWlgzZ0XNgDREr90LSL3GjWRdip3KI 8iWLuoijbbLsnnr4s2vA5SLIX17XbF1dX2obKGSfuViaRmQkcaOfNrjnsy1XZneNg8kC fQSUeU/C5hBBY+tO96xNSHgmC9G9k07EbLL7H5V1g8xPbjL5rUGRQFlQFy2tTuMx6AJ1 AWNQ== X-Gm-Message-State: AO0yUKVB1uL8Zre30QlxPRoffBaJi4Uz7qMKx1FZjf3TJmwLqUV8zI3b DWyEr5uuYwcKLmbQfuS5ilVgPfmRQfEr X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a17:902:6b0a:b0:196:7cf5:ba64 with SMTP id o10-20020a1709026b0a00b001967cf5ba64mr2638442plk.14.1675452507082; Fri, 03 Feb 2023 11:28:27 -0800 (PST) Date: Fri, 3 Feb 2023 11:28:18 -0800 In-Reply-To: <20230203192822.106773-1-vipinsh@google.com> Mime-Version: 1.0 References: <20230203192822.106773-1-vipinsh@google.com> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog Message-ID: <20230203192822.106773-2-vipinsh@google.com> Subject: [Patch v2 1/5] KVM: x86/mmu: Make separate function to check for SPTEs atomic write conditions From: Vipin Sharma To: seanjc@google.com, pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756839362562017887?= X-GMAIL-MSGID: =?utf-8?q?1756839362562017887?= Move condition checks in kvm_tdp_mmu_write_spte() for writing spte atomically in a separate function. New function will be used in future commits to clear bits in SPTE. Signed-off-by: Vipin Sharma Reviewed-by: Ben Gardon Reviewed-by: David Matlack --- arch/x86/kvm/mmu/tdp_iter.h | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index f0af385c56e0..30a52e5e68de 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -29,11 +29,10 @@ static inline void __kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 new_spte) WRITE_ONCE(*rcu_dereference(sptep), new_spte); } -static inline u64 kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 old_spte, - u64 new_spte, int level) +static inline bool kvm_tdp_mmu_spte_has_volatile_bits(u64 old_spte, int level) { /* - * Atomically write the SPTE if it is a shadow-present, leaf SPTE with + * Atomically write SPTEs if it is a shadow-present, leaf SPTE with * volatile bits, i.e. has bits that can be set outside of mmu_lock. * The Writable bit can be set by KVM's fast page fault handler, and * Accessed and Dirty bits can be set by the CPU. @@ -44,8 +43,15 @@ static inline u64 kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 old_spte, * logic needs to be reassessed if KVM were to use non-leaf Accessed * bits, e.g. to skip stepping down into child SPTEs when aging SPTEs. */ - if (is_shadow_present_pte(old_spte) && is_last_spte(old_spte, level) && - spte_has_volatile_bits(old_spte)) + return is_shadow_present_pte(old_spte) && + is_last_spte(old_spte, level) && + spte_has_volatile_bits(old_spte); +} + +static inline u64 kvm_tdp_mmu_write_spte(tdp_ptep_t sptep, u64 old_spte, + u64 new_spte, int level) +{ + if (kvm_tdp_mmu_spte_has_volatile_bits(old_spte, level)) return kvm_tdp_mmu_write_spte_atomic(sptep, new_spte); __kvm_tdp_mmu_write_spte(sptep, new_spte); From patchwork Fri Feb 3 19:28:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vipin Sharma X-Patchwork-Id: 52620 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1020578wrn; Fri, 3 Feb 2023 11:29:52 -0800 (PST) X-Google-Smtp-Source: AK7set9ZBqcHXj3Fa0LnzB16pBAyQ3ZuQwGLQG9+vJzk7LrrZzb5HJc1Y4W9HswP7LEfjfp0/n81 X-Received: by 2002:a05:6402:4511:b0:49e:16fc:b525 with SMTP id ez17-20020a056402451100b0049e16fcb525mr11457738edb.41.1675452592516; Fri, 03 Feb 2023 11:29:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675452592; cv=none; d=google.com; s=arc-20160816; b=K3fNU2Zh91Pn7bUu9L6edejW0+pEfkCeqCgAzOL37iP0OVymY3aXRGg0scHPRP29Au lV9hBXARfwRj/KgR8nS7fasnrwk0qHECD4A1Yl4JpXAyXwOnIXgGQC0vAn3DL77eNaGb R/wEsx4QFUSz7qPIl0PRVKUpiUZ09dfaDdmy4Zkuw196TX6TJ75coXTN5Q+l8h4h7DPG ViIs6WJRTJJMm5HonyHOHhmrPc6Q3ISCEin2hMoHO4hpGQd/Q2UN6xqvT4kOFx+HVQDt 21jCVKmVpOBolnvaMUZiXu0GRN0A5BJ3RgmxFAsyReIdesMneUMjzcFCJQ8pTsAdDcZe 20qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=ak5zcwBMhePTPGhWw+9Y0sgu3ygPb+KvfrYr09hJ3Yk=; b=F4lLKpkzQfPa2pjAF1vsNHqEp8xgsW1drOG8eo1dRDMt5rYjou8Jog7cosmE4iTggL JxoCReU1aYiy4CI0HOIX0X6mQofDNlwLFWZpf1+Xe/v3h/5e17E1tuZBTwDlzPnuq2W8 NZByk5tUXoFOTNBoyIsuTn672MCuGEikyfrKUg10qPiKQyBJmSBP/fVg8sWKRoqvfAaE CzAwee1Lq33TbpdQ2H5UEkjpIOZT/mp8KbpQfOn9GEXrFnIxn8EHggsaveT1Tr72VNTE E7lv6gKWgOYbl7blJZYaaJkpr57lpBoVhR4vCT5ldNRcdCdBrf/3oTQAiE+JgTdJtc/L Mxng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=qZfkHXGG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e7-20020aa7d7c7000000b004a165d44f62si3215454eds.580.2023.02.03.11.29.28; Fri, 03 Feb 2023 11:29:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=qZfkHXGG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233557AbjBCT2i (ORCPT + 99 others); Fri, 3 Feb 2023 14:28:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233119AbjBCT2b (ORCPT ); Fri, 3 Feb 2023 14:28:31 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B18DDA8A08 for ; Fri, 3 Feb 2023 11:28:29 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id c8-20020a17090a674800b0022cb9c81fb0so5050906pjm.2 for ; Fri, 03 Feb 2023 11:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ak5zcwBMhePTPGhWw+9Y0sgu3ygPb+KvfrYr09hJ3Yk=; b=qZfkHXGGr1qHpXZJvKaEUSahiehmsi/2hWFtvV03HTTdOPrfyNEML/o9tHSOHWGqJj HTDm7EQ+xPnvX4pp24MFsSYtGLuEAcJxCrY7l0CY11mqiV/AfHXTU2MHaKfoNy3eu+9f TcEGcpXdv9TUsfcLjALKCicgVFuTyqyWg2ZHcSLaAGXOFBugjpmPo5o5LISp/NZW5/dW /chBexL1SIqKTJWFn80Nx3IcK6ziBiW1bt4oz/1FO3Z/X6PJpA57To9fYyinnQiBChXG 5EzN/m151xiuEbOLOjHA5W5a8CD8OXzA4WtwkemGddRtywIQSDPADkBldJuwJHGHwm4r uZ2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ak5zcwBMhePTPGhWw+9Y0sgu3ygPb+KvfrYr09hJ3Yk=; b=EmWicoe0atE8vHVnD/Sf2cp9+U8sLOLcZKbTNMqN/v6TIWYank1p3weCwuNfFzkbZa ddyqM58gBPleUw+sqHVWa4IRltM9EUKjhYWBAri7lkxrTg4nGP2ps/HwUOSj1AbIczsw GKvEEtQDm3KAHVd+iXv22Mg7CHPo6OYHKgxReH74GQ38uDKUxaKKkfpvSB/oRuOEqaSA X7Dtyduv6NtRvnUnELFY/jFQnNY8Z7W3iZJKV9qPxbRdwKH50OowEll+FQuKS9vbj3uO 0xvLW4rBP6CGMPfvODfqW+TE1KV4LNUKkT+OvkoebwwPrI7MRaHFxL1z1JJ8FVdxccBH IL6Q== X-Gm-Message-State: AO0yUKUBjqgczIBoXoDM3rCdUvH3YDlWh3POaeV5xwpURzIVzH7HiShx zJBA8dFAhZ4VlsXY3CiFT6T+IfcBA21t X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a17:90a:eb12:b0:225:eaa2:3f5d with SMTP id j18-20020a17090aeb1200b00225eaa23f5dmr3601pjz.2.1675452508924; Fri, 03 Feb 2023 11:28:28 -0800 (PST) Date: Fri, 3 Feb 2023 11:28:19 -0800 In-Reply-To: <20230203192822.106773-1-vipinsh@google.com> Mime-Version: 1.0 References: <20230203192822.106773-1-vipinsh@google.com> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog Message-ID: <20230203192822.106773-3-vipinsh@google.com> Subject: [Patch v2 2/5] KVM: x86/mmu: Optimize SPTE change flow for clear-dirty-log From: Vipin Sharma To: seanjc@google.com, pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756839377770521210?= X-GMAIL-MSGID: =?utf-8?q?1756839377770521210?= No need to check all of the conditions in __handle_changed_spte() as clearing dirty log only involves resetting dirty or writable bit. Make atomic change to dirty or writable bit and mark pfn dirty. Tested on 160 VCPU-160 GB VM and found that performance of clear dirty log stage improved by ~38% in dirty_log_perf_test Before optimization: -------------------- Test iterations: 3 Testing guest mode: PA-bits:ANY, VA-bits:48, 4K pages guest physical test memory: [0x3fd7c0000000, 0x3fffc0000000) Populate memory time: 6.298459671s Setting dirty log mode took : 0.000000052s Enabling dirty logging time: 0.003815691s Iteration 1 dirty memory time: 0.185538848s Iteration 1 get dirty log time: 0.002562641s Iteration 1 clear dirty log time: 3.638543593s Iteration 2 dirty memory time: 0.192226071s Iteration 2 get dirty log time: 0.001558446s Iteration 2 clear dirty log time: 3.145032742s Iteration 3 dirty memory time: 0.193606295s Iteration 3 get dirty log time: 0.001559425s Iteration 3 clear dirty log time: 3.142340358s Disabling dirty logging time: 3.002873664s Get dirty log over 3 iterations took 0.005680512s. (Avg 0.001893504s/iteration) Clear dirty log over 3 iterations took 9.925916693s. (Avg 3.308638897s/iteration) After optimization: ------------------- Test iterations: 3 Testing guest mode: PA-bits:ANY, VA-bits:48, 4K pages guest physical test memory: [0x3fd7c0000000, 0x3fffc0000000) Populate memory time: 6.581448437s Setting dirty log mode took : 0.000000058s Enabling dirty logging time: 0.003981283s Iteration 1 dirty memory time: 0.285693420s Iteration 1 get dirty log time: 0.002743004s Iteration 1 clear dirty log time: 2.384343157s Iteration 2 dirty memory time: 0.290414476s Iteration 2 get dirty log time: 0.001720445s Iteration 2 clear dirty log time: 1.882770288s Iteration 3 dirty memory time: 0.289965965s Iteration 3 get dirty log time: 0.001728232s Iteration 3 clear dirty log time: 1.881043086s Disabling dirty logging time: 2.930387523s Get dirty log over 3 iterations took 0.006191681s. (Avg 0.002063893s/iteration) Clear dirty log over 3 iterations took 6.148156531s. (Avg 2.049385510s/iteration) Signed-off-by: Vipin Sharma --- arch/x86/kvm/mmu/tdp_iter.h | 13 ++++++++++ arch/x86/kvm/mmu/tdp_mmu.c | 51 +++++++++++++++---------------------- 2 files changed, 34 insertions(+), 30 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index 30a52e5e68de..21046b34f94e 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -121,4 +121,17 @@ void tdp_iter_start(struct tdp_iter *iter, struct kvm_mmu_page *root, void tdp_iter_next(struct tdp_iter *iter); void tdp_iter_restart(struct tdp_iter *iter); +static inline u64 kvm_tdp_mmu_clear_spte_bit(struct tdp_iter *iter, u64 mask) +{ + atomic64_t *sptep; + + if (kvm_tdp_mmu_spte_has_volatile_bits(iter->old_spte, iter->level)) { + sptep = (atomic64_t *)rcu_dereference(iter->sptep); + return (u64)atomic64_fetch_and(~mask, sptep); + } + + __kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte & ~mask); + return iter->old_spte; +} + #endif /* __KVM_X86_MMU_TDP_ITER_H */ diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index bba33aea0fb0..83f15052aa6c 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -710,18 +710,13 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, * notifier for access tracking. Leaving record_acc_track * unset in that case prevents page accesses from being * double counted. - * @record_dirty_log: Record the page as dirty in the dirty bitmap if - * appropriate for the change being made. Should be set - * unless performing certain dirty logging operations. - * Leaving record_dirty_log unset in that case prevents page - * writes from being double counted. * * Returns the old SPTE value, which _may_ be different than @old_spte if the * SPTE had voldatile bits. */ static u64 __tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, u64 old_spte, u64 new_spte, gfn_t gfn, int level, - bool record_acc_track, bool record_dirty_log) + bool record_acc_track) { lockdep_assert_held_write(&kvm->mmu_lock); @@ -740,42 +735,34 @@ static u64 __tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, if (record_acc_track) handle_changed_spte_acc_track(old_spte, new_spte, level); - if (record_dirty_log) - handle_changed_spte_dirty_log(kvm, as_id, gfn, old_spte, - new_spte, level); + + handle_changed_spte_dirty_log(kvm, as_id, gfn, old_spte, new_spte, + level); return old_spte; } static inline void _tdp_mmu_set_spte(struct kvm *kvm, struct tdp_iter *iter, - u64 new_spte, bool record_acc_track, - bool record_dirty_log) + u64 new_spte, bool record_acc_track) { WARN_ON_ONCE(iter->yielded); iter->old_spte = __tdp_mmu_set_spte(kvm, iter->as_id, iter->sptep, iter->old_spte, new_spte, iter->gfn, iter->level, - record_acc_track, record_dirty_log); + record_acc_track); } static inline void tdp_mmu_set_spte(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte) { - _tdp_mmu_set_spte(kvm, iter, new_spte, true, true); + _tdp_mmu_set_spte(kvm, iter, new_spte, true); } static inline void tdp_mmu_set_spte_no_acc_track(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte) { - _tdp_mmu_set_spte(kvm, iter, new_spte, false, true); -} - -static inline void tdp_mmu_set_spte_no_dirty_log(struct kvm *kvm, - struct tdp_iter *iter, - u64 new_spte) -{ - _tdp_mmu_set_spte(kvm, iter, new_spte, true, false); + _tdp_mmu_set_spte(kvm, iter, new_spte, false); } #define tdp_root_for_each_pte(_iter, _root, _start, _end) \ @@ -925,7 +912,7 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp) return false; __tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, 0, - sp->gfn, sp->role.level + 1, true, true); + sp->gfn, sp->role.level + 1, true); return true; } @@ -1678,7 +1665,7 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, gfn_t gfn, unsigned long mask, bool wrprot) { struct tdp_iter iter; - u64 new_spte; + u64 clear_bits; rcu_read_lock(); @@ -1694,18 +1681,22 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, mask &= ~(1UL << (iter.gfn - gfn)); if (wrprot || spte_ad_need_write_protect(iter.old_spte)) { - if (is_writable_pte(iter.old_spte)) - new_spte = iter.old_spte & ~PT_WRITABLE_MASK; - else + if (!is_writable_pte(iter.old_spte)) continue; + + clear_bits = PT_WRITABLE_MASK; } else { - if (iter.old_spte & shadow_dirty_mask) - new_spte = iter.old_spte & ~shadow_dirty_mask; - else + if (!(iter.old_spte & shadow_dirty_mask)) continue; + + clear_bits = shadow_dirty_mask; } - tdp_mmu_set_spte_no_dirty_log(kvm, &iter, new_spte); + iter.old_spte = kvm_tdp_mmu_clear_spte_bit(&iter, clear_bits); + trace_kvm_tdp_mmu_spte_changed(iter.as_id, iter.gfn, iter.level, + iter.old_spte, + iter.old_spte & ~clear_bits); + kvm_set_pfn_dirty(spte_to_pfn(iter.old_spte)); } rcu_read_unlock(); From patchwork Fri Feb 3 19:28:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vipin Sharma X-Patchwork-Id: 52624 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1031851wrn; Fri, 3 Feb 2023 12:00:35 -0800 (PST) X-Google-Smtp-Source: AK7set/oJq58fKQvqcN7mqHAqM8uBFw8Vf7/XJcDEv+QnuAyh+Fv6NPrmcRdsP84cNVeeXGwXHr8 X-Received: by 2002:a17:902:e3d2:b0:196:33dc:98ab with SMTP id r18-20020a170902e3d200b0019633dc98abmr8985328ple.20.1675454434999; Fri, 03 Feb 2023 12:00:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675454434; cv=none; d=google.com; s=arc-20160816; b=l5to96gVZSENfKld14nu1ykIkmBjgc+IdmfUL+1vN1tSwc3OIzB1h2rTqc+2py6TTA rswQEbjss55VkXWZRviRGgldtPfKEHOYlrm3X8zXVqmHcilC5riWr+rPcs3TKaacFrB1 ZjdYCqvMke5ysNeW3wCk4HNJh0NrFRcmU2O06N4wcVF3QcNmTE3mQSnJgqRWL5sekjjG J7Rp1uHLZgZ/mqG6EUjAEYOy+Y7DwKd4x96Bnfnho2eqnUppubo5E6Uv1uI9NqacQykf KsI5RExbislQmi04WO1QIwJYtvNcI5TKIuGJutdrXHp3YVvWU0unEpqkHS2mNpXKJ0M0 dPcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=JnRekf2u9xlodJ6kWv+4sMoTYGQkhV0MHFbmGB0K31Y=; b=wrJrBroHqaT24UEb/+n147I3M2y7pXz9Szxuc5O2cZRqmzGXIN86XLlPvhr9yjbaXG nwJAMiJThBQwAnVn33KNMFn3DBpyoRa1VL7vwe6XPzxnQ8A1gMH+/Y5k5ra9SIP8A6Dq Ws8NT/jDM/1EH8Kyn69AItFas5lYY34PKpc6snjomGEcj08/wMK8OIz641bpBlSAQ3BO A2adpW89s6gV5M8Y0FjqqcaTFTG1l9J0T3lOs9TwU5P4O+U49kUxOG7LVZXSHPG1coqx 2vDYD33alcax61Ks6sTrdKNc/8HNwVQ7SbOaQ8VKqB2I8IZZpYD/Ll2p3sP7PgtmV0+e vk+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Zi6H4ARQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id be3-20020a170902aa0300b0019648e516dfsi3068487plb.352.2023.02.03.12.00.17; Fri, 03 Feb 2023 12:00:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Zi6H4ARQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233143AbjBCT2k (ORCPT + 99 others); Fri, 3 Feb 2023 14:28:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233294AbjBCT2g (ORCPT ); Fri, 3 Feb 2023 14:28:36 -0500 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4482DA913D for ; Fri, 3 Feb 2023 11:28:31 -0800 (PST) Received: by mail-pl1-x649.google.com with SMTP id y9-20020a1709027c8900b00195e237dc8bso3009683pll.13 for ; Fri, 03 Feb 2023 11:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JnRekf2u9xlodJ6kWv+4sMoTYGQkhV0MHFbmGB0K31Y=; b=Zi6H4ARQow0EtXg0mUHG9UGNsiViGbeZO/GDNJqScT1LjoaHQG/12s0iu9hLuYReeH t9C1ND+dlehha7qxQlRIP3e3fttV6FiwBMPowHok+6ipPXZJMeqYAk3AS0hnbk34FRk4 GRrc3WZIrvEOJAtBfEjk3pbqjV7T/JZXj4XsXMpgodFXBxvGflCKR88pCDoLsqjU9Hx/ rfdTnpdHymBjyFU2lVEsP8hzxnLaRnQo2GLObrDxfiv76bQJlDJkKVrHOsVRhEQ1xoLC bwd49pgcBDx5UlmZRnmSn4IJ/BG8CEufoOwm2BZvpbAUTAPsciIep4wD/BTUu7bR74p4 2ASA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JnRekf2u9xlodJ6kWv+4sMoTYGQkhV0MHFbmGB0K31Y=; b=XO1iRNwgZzuz63VdqXG2q+jEjPuOLuoGrQ5SyW3KXWvhSHLPJqwBLIOtLb7cXK2Mv9 mV2fPC+45OWBZt+lLi7L7oYQDs0TXlECXcGzUVedyFTgLtywRcWYCxtKkIUKa4/NvSqA 9P/bwk0d77HPi3lDDn5U00YTl8cpgXrdye5Y27V2J6XUohPQqM4b50dee/ZR1WgA3jGc P2xigKpacApam955kPZLtP7cOdjYmJre/LFsskvzibAjZe1lq4l9T+r1JxyZfGktvodG 9i9/IA2oh7zaZOFGYh5t4K68I7OXWkvI4v1pkw16QF4tplxLmn0LyTBrchFLm1KuxlUP RziA== X-Gm-Message-State: AO0yUKV+HoCrUDGtv+VHNSX08bf8FgzkZ283gcl46d0dPMizA2z4mfwX wyB7yKsJ1uP3nQmqun2Pir9KBUnEUYdR X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a17:90a:b10f:b0:22c:6d9e:10f3 with SMTP id z15-20020a17090ab10f00b0022c6d9e10f3mr1496364pjq.57.1675452510817; Fri, 03 Feb 2023 11:28:30 -0800 (PST) Date: Fri, 3 Feb 2023 11:28:20 -0800 In-Reply-To: <20230203192822.106773-1-vipinsh@google.com> Mime-Version: 1.0 References: <20230203192822.106773-1-vipinsh@google.com> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog Message-ID: <20230203192822.106773-4-vipinsh@google.com> Subject: [Patch v2 3/5] KVM: x86/mmu: Optimize SPTE change for aging gfn range From: Vipin Sharma To: seanjc@google.com, pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756841309513175160?= X-GMAIL-MSGID: =?utf-8?q?1756841309513175160?= No need to check all of the conditions in __handle_changed_spte(). Aging a gfn range implies resetting access bit or marking spte for access tracking. Use atomic operation to only reset those bits. This avoids checking many conditions in __handle_changed_spte() API. Also, clean up code by removing dead code and API parameters. Signed-off-by: Vipin Sharma --- arch/x86/kvm/mmu/tdp_mmu.c | 68 ++++++++++++++------------------------ 1 file changed, 25 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 83f15052aa6c..18630a06fa1f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -697,7 +697,7 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, /* - * __tdp_mmu_set_spte - Set a TDP MMU SPTE and handle the associated bookkeeping + * _tdp_mmu_set_spte - Set a TDP MMU SPTE and handle the associated bookkeeping * @kvm: KVM instance * @as_id: Address space ID, i.e. regular vs. SMM * @sptep: Pointer to the SPTE @@ -705,18 +705,12 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, * @new_spte: The new value that will be set for the SPTE * @gfn: The base GFN that was (or will be) mapped by the SPTE * @level: The level _containing_ the SPTE (its parent PT's level) - * @record_acc_track: Notify the MM subsystem of changes to the accessed state - * of the page. Should be set unless handling an MMU - * notifier for access tracking. Leaving record_acc_track - * unset in that case prevents page accesses from being - * double counted. * * Returns the old SPTE value, which _may_ be different than @old_spte if the * SPTE had voldatile bits. */ -static u64 __tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, - u64 old_spte, u64 new_spte, gfn_t gfn, int level, - bool record_acc_track) +static u64 _tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, + u64 old_spte, u64 new_spte, gfn_t gfn, int level) { lockdep_assert_held_write(&kvm->mmu_lock); @@ -732,37 +726,20 @@ static u64 __tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, old_spte = kvm_tdp_mmu_write_spte(sptep, old_spte, new_spte, level); __handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, false); - - if (record_acc_track) - handle_changed_spte_acc_track(old_spte, new_spte, level); - + handle_changed_spte_acc_track(old_spte, new_spte, level); handle_changed_spte_dirty_log(kvm, as_id, gfn, old_spte, new_spte, level); return old_spte; } -static inline void _tdp_mmu_set_spte(struct kvm *kvm, struct tdp_iter *iter, - u64 new_spte, bool record_acc_track) -{ - WARN_ON_ONCE(iter->yielded); - - iter->old_spte = __tdp_mmu_set_spte(kvm, iter->as_id, iter->sptep, - iter->old_spte, new_spte, - iter->gfn, iter->level, - record_acc_track); -} - static inline void tdp_mmu_set_spte(struct kvm *kvm, struct tdp_iter *iter, u64 new_spte) { - _tdp_mmu_set_spte(kvm, iter, new_spte, true); -} + WARN_ON_ONCE(iter->yielded); -static inline void tdp_mmu_set_spte_no_acc_track(struct kvm *kvm, - struct tdp_iter *iter, - u64 new_spte) -{ - _tdp_mmu_set_spte(kvm, iter, new_spte, false); + iter->old_spte = _tdp_mmu_set_spte(kvm, iter->as_id, iter->sptep, + iter->old_spte, new_spte, + iter->gfn, iter->level); } #define tdp_root_for_each_pte(_iter, _root, _start, _end) \ @@ -911,8 +888,8 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp) if (WARN_ON_ONCE(!is_shadow_present_pte(old_spte))) return false; - __tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, 0, - sp->gfn, sp->role.level + 1, true); + _tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, 0, + sp->gfn, sp->role.level + 1); return true; } @@ -1251,32 +1228,37 @@ static __always_inline bool kvm_tdp_mmu_handle_gfn(struct kvm *kvm, /* * Mark the SPTEs range of GFNs [start, end) unaccessed and return non-zero * if any of the GFNs in the range have been accessed. + * + * No need to mark corresponding PFN as accessed as this call is coming from + * MMU notifier for that page via HVA. */ static bool age_gfn_range(struct kvm *kvm, struct tdp_iter *iter, struct kvm_gfn_range *range) { - u64 new_spte = 0; + u64 new_spte; /* If we have a non-accessed entry we don't need to change the pte. */ if (!is_accessed_spte(iter->old_spte)) return false; - new_spte = iter->old_spte; - - if (spte_ad_enabled(new_spte)) { - new_spte &= ~shadow_accessed_mask; + if (spte_ad_enabled(iter->old_spte)) { + iter->old_spte = kvm_tdp_mmu_clear_spte_bit(iter, + shadow_accessed_mask); + new_spte = iter->old_spte & ~shadow_accessed_mask; } else { + new_spte = mark_spte_for_access_track(iter->old_spte); + iter->old_spte = kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte, + new_spte, iter->level); /* * Capture the dirty status of the page, so that it doesn't get * lost when the SPTE is marked for access tracking. */ - if (is_writable_pte(new_spte)) - kvm_set_pfn_dirty(spte_to_pfn(new_spte)); - - new_spte = mark_spte_for_access_track(new_spte); + if (is_writable_pte(iter->old_spte)) + kvm_set_pfn_dirty(spte_to_pfn(iter->old_spte)); } - tdp_mmu_set_spte_no_acc_track(kvm, iter, new_spte); + trace_kvm_tdp_mmu_spte_changed(iter->as_id, iter->gfn, iter->level, + iter->old_spte, new_spte); return true; } From patchwork Fri Feb 3 19:28:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vipin Sharma X-Patchwork-Id: 52621 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1026046wrn; Fri, 3 Feb 2023 11:43:46 -0800 (PST) X-Google-Smtp-Source: AK7set80adhWtIitSUWt1duCQR2/64QUf1xdtHubNf8f9SiAo18wmb36/W7tv5mH3a1RYIW99ueF X-Received: by 2002:a17:90b:4a51:b0:230:76b1:8db2 with SMTP id lb17-20020a17090b4a5100b0023076b18db2mr2716412pjb.31.1675453426614; Fri, 03 Feb 2023 11:43:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675453426; cv=none; d=google.com; s=arc-20160816; b=hc5Ayl0sgmtWsic+j7mtcMYgTfrGQ8ZZ7ukLDrSM0jIX5yQpTVsqSmIdfnwwHw4M9O rKvqx7a6MfnMY2vzad8284sbY4p52vs5vrIDREuFVY8hl6vNAtlcER7UbZSWqAEPLU75 /sYtrNoEdo9ssvYydPx5dSVQBcxo4QCAre4982+Rc8KCF0cyQHLoaecoFwqf5TzUSY77 Z8Wxxv0EaU+9NPpw6zOmOsrnfQf9yuT3P4Axwsvv4R4mKoBs50OnQMUQHP4D/8oq4s1M 7wy6RK3G4mZfTar6HJDFWNV3Gz1X/t0IiY/EaAQJUNwL6sCwggNYYyg4KQB4U5cdn1po RZuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=AcI5nZSY78YItFhCkcPCsgYpVHXo4G2xX3PARLK9PNY=; b=RgR49feeLq8Jfh82HeoegYY+YIhew20yODkBaY73BNYN+2prl1WhAD80uC6j7XDSEu nbb3CNDBrAAM7/s1NQnnT203UEkqVHiVK1gQFz8x3TITb6mQHxeDDgc8TctXuND0hbDK dMaFpvL+vqVTN36OQkznEYREPDoWYoLDzPzmfAqTK/IZ2KcajIJ3uOALbk1Ms7ReWU1C 2jbrD4N9sz6ONv69pI6GNS/taxTcpuOukmBK7/51ecYVoXrFv6xb5UPU5Iuy8SJO01sG GCZVlkWf0Mj2mc1QHppcqWgwGbMFi6SR6f69a7PoknzIrTIFs2F4tXxBlhn+W1/KAHd0 LjXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=jnIgJE60; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lp7-20020a17090b4a8700b0022c34ca3e99si4387591pjb.159.2023.02.03.11.43.30; Fri, 03 Feb 2023 11:43:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=jnIgJE60; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233721AbjBCT2n (ORCPT + 99 others); Fri, 3 Feb 2023 14:28:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233142AbjBCT2h (ORCPT ); Fri, 3 Feb 2023 14:28:37 -0500 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E43B6A8A3D for ; Fri, 3 Feb 2023 11:28:32 -0800 (PST) Received: by mail-pf1-x44a.google.com with SMTP id i15-20020aa787cf000000b00593addd14a5so3169681pfo.15 for ; Fri, 03 Feb 2023 11:28:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AcI5nZSY78YItFhCkcPCsgYpVHXo4G2xX3PARLK9PNY=; b=jnIgJE609c2sptSoHjjd3x7jZtT7KKK/MLWWeUqiS8WA2JDk4FeRouptRJICfrxIiL ltXurNvSD6sG3igKDh2jIDvC139/r0ohRfJJG6pQnedSojgn6p86/PzyD7KtkRoozEMs kQk/oF3N032gLlif8xUP5eDhlBX1h9Xgbe6TauMdaNiztnykutoA6logtvV9fpbOhff/ n6fE5UFqDNwzbhaTz5ygCEMGiXIgDXG7fGIjHp4wkMPYAh1YXKf21kdIrLMzALj8a3u4 tudd1dArJcrxBZaQnyHLlr0CksaMvJ3/TD4Fet8r0s0LPkO4YK/ew1sp3r98Mpuf0rDv KtMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AcI5nZSY78YItFhCkcPCsgYpVHXo4G2xX3PARLK9PNY=; b=1WJ1VK4vNj1LvqXBiJXCHJ2ktYu+u0uctmEvScXBSaRJynf+bX74++Ri0V6y9mVJ/P Ugtze2YucY+gW0Eep/vVIfNc0Q3m066ic0GKH5fDFvaqVeKIyTN6+mMgMCoWx4Lg5c6f IzrK2JaUrhrm31OmpYnIx9/IkYeLRT5+T40CNrLyJ9GiaITt5CcUSDJ+2XPl7kWkcGO1 7SMsqgYgjGLesdJnTemFsOg53cjdqOR3DDRZBFWb7xc2P7/hQt0axlZfQ+x1tIStHbD9 nDNGoEgiwk+s012MzBbrR5KJ0NGjDIW6yWkowaxEfGSS5PICNoudUFVEVxKHo6c93DDR UNTg== X-Gm-Message-State: AO0yUKVgBh95NG4+Dqd3zEe2asG1K0QNpnjIDBWcWzAHuV2pcXkDiJRZ kYKFvpoSqFOIIDhI8WYfRfMoIHWsBykP X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a17:902:ea03:b0:196:6b0d:752b with SMTP id s3-20020a170902ea0300b001966b0d752bmr2649815plg.19.1675452512414; Fri, 03 Feb 2023 11:28:32 -0800 (PST) Date: Fri, 3 Feb 2023 11:28:21 -0800 In-Reply-To: <20230203192822.106773-1-vipinsh@google.com> Mime-Version: 1.0 References: <20230203192822.106773-1-vipinsh@google.com> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog Message-ID: <20230203192822.106773-5-vipinsh@google.com> Subject: [Patch v2 4/5] KVM: x86/mmu: Remove handle_changed_spte_dirty_log() From: Vipin Sharma To: seanjc@google.com, pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756840252122303619?= X-GMAIL-MSGID: =?utf-8?q?1756840252122303619?= Remove handle_changed_spte_dirty_log() as there is no code flow which sets leaf SPTE writable and hit this path. Signed-off-by: Vipin Sharma --- arch/x86/kvm/mmu/tdp_mmu.c | 22 ---------------------- 1 file changed, 22 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 18630a06fa1f..afe0dcb1859e 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -345,24 +345,6 @@ static void handle_changed_spte_acc_track(u64 old_spte, u64 new_spte, int level) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } -static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, - u64 old_spte, u64 new_spte, int level) -{ - bool pfn_changed; - struct kvm_memory_slot *slot; - - if (level > PG_LEVEL_4K) - return; - - pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte); - - if ((!is_writable_pte(old_spte) || pfn_changed) && - is_writable_pte(new_spte)) { - slot = __gfn_to_memslot(__kvm_memslots(kvm, as_id), gfn); - mark_page_dirty_in_slot(kvm, slot, gfn); - } -} - static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) { kvm_account_pgtable_pages((void *)sp->spt, +1); @@ -614,8 +596,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, __handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, shared); handle_changed_spte_acc_track(old_spte, new_spte, level); - handle_changed_spte_dirty_log(kvm, as_id, gfn, old_spte, - new_spte, level); } /* @@ -727,8 +707,6 @@ static u64 _tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, __handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, false); handle_changed_spte_acc_track(old_spte, new_spte, level); - handle_changed_spte_dirty_log(kvm, as_id, gfn, old_spte, new_spte, - level); return old_spte; } From patchwork Fri Feb 3 19:28:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vipin Sharma X-Patchwork-Id: 52627 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1033270wrn; Fri, 3 Feb 2023 12:03:01 -0800 (PST) X-Google-Smtp-Source: AK7set9zXmWXVufhGoFQoDqxS+xrpgl5ly6TIn92KdQR5dx1wm2fZAPkgZ7CwkDcUJV7TK4q7giK X-Received: by 2002:a17:90b:3909:b0:22b:f2d6:a336 with SMTP id ob9-20020a17090b390900b0022bf2d6a336mr12369048pjb.8.1675454581186; Fri, 03 Feb 2023 12:03:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675454581; cv=none; d=google.com; s=arc-20160816; b=kJvd4lXBvpehsjQB4wj8Y1J6QGsmOIAL48Ju+Pujujk1mnpkgXH5DAhKrErgMtp+RZ RtpBW9UsIaWcQEYALlGuLL4jltX2IsY64fOYQ3FQr8zfZ3Sxbfz6pcyTL59uXZJfLJPw k3Ei3c0zr8Y8wEccF/qEmulM3H7mwZWHBPvxBtETfrrX9Sca4009GNzsqYrerYHlBQZX EM7k3saFXns0CLE9v8EEhdIeAJ5/NXDHc1v/yfZqrs23N4nUlUjmyrS5wZCNQ0t4Yf1p pZRzcDvsz1ZxgPz/jzfcQryM5TgE5565JTu2rlPgayJiQWNEqnqnvt1j91Y9gzhfy52T kRgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=SZBOzrPag9qrBF2hWUY6jbrMh9ikRhKVuoUM3SUVBlw=; b=0hhuHlpQom3j9J0J5u2lCcSdkkiq/q/jsSG+9U/qgmIsrP+fBkWgYRjMGmVDcpN8A0 eliQEIrgz9/tQHvWkgS9L2pdMUgOAcr/+1+Gmc2YbHrVUFiROo4keJ5PyvdN47Pb4kNS U3Ic3RLR9zDiKTHXFM7RlIKPvGsEpHNX8dtgqarQy4WJ+XUhvXK+UGItVtxeEHn0gliB 2p8rbuYg2qpNz2c0U9K7/OP4pEG2Wyd3NMP5WahHKzKsotSwSB5Q6cwwlO1JFcVpqn7r KculFw01T8fIVNZwQ9HVJNmknIlu3urJMlCj3gdbd/dLGc8rxy8F38OiKMC8JLRP0Ho4 wGzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=pJq3E8Hx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j15-20020a63b60f000000b00473c36e3b75si3397179pgf.608.2023.02.03.12.02.47; Fri, 03 Feb 2023 12:03:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=pJq3E8Hx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233465AbjBCT2u (ORCPT + 99 others); Fri, 3 Feb 2023 14:28:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233618AbjBCT2i (ORCPT ); Fri, 3 Feb 2023 14:28:38 -0500 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB896A9111 for ; Fri, 3 Feb 2023 11:28:34 -0800 (PST) Received: by mail-pl1-x64a.google.com with SMTP id y8-20020a170902b48800b00192a600df83so3006534plr.15 for ; Fri, 03 Feb 2023 11:28:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SZBOzrPag9qrBF2hWUY6jbrMh9ikRhKVuoUM3SUVBlw=; b=pJq3E8HxuEK4dyqvrQfPM3ldp+gHJEW5GZjifURYqJ01MpDIVkBLWa20dx4quPs/sZ 1ZUgW86Qrb0XAtmmgbtsUN3EfbGwRGdh491O5khsZH4C5htGRlm7IaO5OU/7ViPc2DEc ICQpGLr0Xr/56wm7CFTxlfob8bjIcM8StLtysDeewJRgBwjiXoz9EROyyswUUNnlVmL1 c3oR075IT+ksuaTfm1gJjZ3VqvuZcL0V7DSyrW6LZv3GRhc8i4lFNS3QD05JwJHm+HTZ M8MGZRFd8uoyUJ3BrQe81xgQpwD8qPJEl9bS7KD5u3wkEsXyHu6yR2Usv771+y13Mqhh fDhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SZBOzrPag9qrBF2hWUY6jbrMh9ikRhKVuoUM3SUVBlw=; b=3NIf3HAgS5/3uW0+N8s3sDeD2hIWqlRfNiU2QoRHnQHsxcewpY/Aiji3opl/aYQqP0 ycTiM3iOzL8BopfhVj469eh/3fBlaCZ+JyRKvvjRmlSCE0CDRUIr7bRaNcPD3MGdRwY/ dlJFKZIiJhFb4AR9sVDvVm7aIZbbdZY8EvxEPv5eN1f6Eu0pO+YbFLz7gHQgqwUg+rh8 GoH9f/JZR7LzVZW8feAYu9GEPG5gm92q03pN9JA/CmxYHUW8W/J5q9Uwsvr2vPAvWIrz N9qvo2AqaSedGXdeGEQ3kg2n2Q8IFibHf6IFialBL0oZYXdrpV3aaAImeOeVBl1oPFH7 fbCg== X-Gm-Message-State: AO0yUKV97PXRkJFT55Pcyzwil+Nvp/9lm7aKnjtViTlAJiAA7shwg3R+ +bT0ULwtQEVEsHnVVcIUyGq0NQHt3Owa X-Received: from vipin.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:479f]) (user=vipinsh job=sendgmr) by 2002:a05:6a00:705:b0:594:2b99:77ad with SMTP id 5-20020a056a00070500b005942b9977admr1968810pfl.4.1675452514299; Fri, 03 Feb 2023 11:28:34 -0800 (PST) Date: Fri, 3 Feb 2023 11:28:22 -0800 In-Reply-To: <20230203192822.106773-1-vipinsh@google.com> Mime-Version: 1.0 References: <20230203192822.106773-1-vipinsh@google.com> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog Message-ID: <20230203192822.106773-6-vipinsh@google.com> Subject: [Patch v2 5/5] KVM: x86/mmu: Merge all handle_changed_pte* functions. From: Vipin Sharma To: seanjc@google.com, pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Vipin Sharma X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756841462771293334?= X-GMAIL-MSGID: =?utf-8?q?1756841462771293334?= __handle_changed_pte() and handle_changed_spte_acc_track() are always used together. Merge these two functions and name the new function handle_changed_pte(). Remove the existing handle_changed_pte() function which just calls __handle_changed_pte and handle_changed_spte_acc_track(). This converges SPTEs change handling code to a single place. Signed-off-by: Vipin Sharma Reviewed-by: Ben Gardon --- arch/x86/kvm/mmu/tdp_mmu.c | 42 +++++++++++--------------------------- 1 file changed, 12 insertions(+), 30 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index afe0dcb1859e..9b0c81a28f97 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -334,17 +334,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, u64 old_spte, u64 new_spte, int level, bool shared); -static void handle_changed_spte_acc_track(u64 old_spte, u64 new_spte, int level) -{ - if (!is_shadow_present_pte(old_spte) || !is_last_spte(old_spte, level)) - return; - - if (is_accessed_spte(old_spte) && - (!is_shadow_present_pte(new_spte) || !is_accessed_spte(new_spte) || - spte_to_pfn(old_spte) != spte_to_pfn(new_spte))) - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); -} - static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) { kvm_account_pgtable_pages((void *)sp->spt, +1); @@ -487,7 +476,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared) } /** - * __handle_changed_spte - handle bookkeeping associated with an SPTE change + * handle_changed_spte - handle bookkeeping associated with an SPTE change * @kvm: kvm instance * @as_id: the address space of the paging structure the SPTE was a part of * @gfn: the base GFN that was mapped by the SPTE @@ -501,9 +490,9 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared) * Handle bookkeeping that might result from the modification of a SPTE. * This function must be called for all TDP SPTE modifications. */ -static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, - u64 old_spte, u64 new_spte, int level, - bool shared) +static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, + u64 old_spte, u64 new_spte, int level, + bool shared) { bool was_present = is_shadow_present_pte(old_spte); bool is_present = is_shadow_present_pte(new_spte); @@ -587,15 +576,10 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (was_present && !was_leaf && (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); -} -static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, - u64 old_spte, u64 new_spte, int level, - bool shared) -{ - __handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, - shared); - handle_changed_spte_acc_track(old_spte, new_spte, level); + if (was_leaf && is_accessed_spte(old_spte) && + (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) + kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } /* @@ -638,9 +622,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, if (!try_cmpxchg64(sptep, &iter->old_spte, new_spte)) return -EBUSY; - __handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte, - new_spte, iter->level, true); - handle_changed_spte_acc_track(iter->old_spte, new_spte, iter->level); + handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte, + new_spte, iter->level, true); return 0; } @@ -705,8 +688,7 @@ static u64 _tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, old_spte = kvm_tdp_mmu_write_spte(sptep, old_spte, new_spte, level); - __handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, false); - handle_changed_spte_acc_track(old_spte, new_spte, level); + handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, false); return old_spte; } @@ -1273,7 +1255,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter, * Note, when changing a read-only SPTE, it's not strictly necessary to * zero the SPTE before setting the new PFN, but doing so preserves the * invariant that the PFN of a present * leaf SPTE can never change. - * See __handle_changed_spte(). + * See handle_changed_spte(). */ tdp_mmu_set_spte(kvm, iter, 0); @@ -1298,7 +1280,7 @@ bool kvm_tdp_mmu_set_spte_gfn(struct kvm *kvm, struct kvm_gfn_range *range) /* * No need to handle the remote TLB flush under RCU protection, the * target SPTE _must_ be a leaf SPTE, i.e. cannot result in freeing a - * shadow page. See the WARN on pfn_changed in __handle_changed_spte(). + * shadow page. See the WARN on pfn_changed in handle_changed_spte(). */ return kvm_tdp_mmu_handle_gfn(kvm, range, set_spte_gfn); }