From patchwork Fri Dec 22 23:52:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 182812 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp1394682dyi; Fri, 22 Dec 2023 15:54:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IFaWA27w1VbAzjyQ0EBqP1RuFKUVJ0l1xfzrOeVY9fpMkxQ6LmKrY6Ebnapynu2a2IstuqV X-Received: by 2002:a2e:3006:0:b0:2cc:9f12:365f with SMTP id w6-20020a2e3006000000b002cc9f12365fmr1196417ljw.63.1703289291391; Fri, 22 Dec 2023 15:54:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703289291; cv=none; d=google.com; s=arc-20160816; b=fcM8B1AuU96vboW7ZNgPqRoLSpAHWu2bEs1F0sYulP/1ruHr6i1sSaAttysnA//xLs jZtR9yQxNtc2/zbvb0i9k41N4ur7FbIRGx6vPmibuO4RnibX7rWVAuktzSlHlvgaUueK prQe1ypo9l9e6jMJlk6PvXibs+wp6K+z3+5mIzYriPcH0gv5HV1OTfCpWL0wHEK8MX9x XNGU5tZ9j9Nbf15mhwmBasfJUnnt2FSLRMB6k8oJ3fYBVSNU5xTwoJpamQBzrCX7LGrP Gmtz4Bg//xaTDUhSkCXR/3Ct5MNgygl5B/UVQ3zvoUZQKp5/drjxOy0ZPrL874DwbB6u FA8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nRb9odYj0zuUgjn5mN2pRzKe6eite6Cnh5+z/pgqCLI=; fh=OvJRnOqsMTm9XoNmEwebcqh9Ud7yh1CTeKAP84ols98=; b=b3S6StCwsjEPw0nhC842qRC01heWe4UtPenIxU5FH2Ie7jrZDMmvMyFFqjEAgyq3EA R2UTy9mAZLfvc6f9VtGWDhecWgxqglF+MLK8AbYN7QhO3sqJtlvp6eJD94w1s3KHlWru 0l8rrfLs2GZRCZb7rRBMLKkrAmPVXX57vOqD5ESzq+1I5iPyLGtjMjrhxR4ZJZDmYBye nO63F8H40NHgFxjWYsJmTyynvrvLnIEJlb+VYgmcL9FGy+A/Lnb8qRe5zgfEhq6GwFcK LpOl9LvZHHsSjfmPY+F30MqOxnpp0itghcHiKoR2YyYF83mR9A8BQpA34amMgfo0sMbn 2z2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=dDsZjT6T; spf=pass (google.com: domain of linux-kernel+bounces-10134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-10134-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id f17-20020a50a6d1000000b00553bd8ccd7asi2293603edc.241.2023.12.22.15.54.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Dec 2023 15:54:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-10134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=dDsZjT6T; spf=pass (google.com: domain of linux-kernel+bounces-10134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-10134-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 03B761F2368E for ; Fri, 22 Dec 2023 23:54:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9FCC44B5AD; Fri, 22 Dec 2023 23:52:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dDsZjT6T" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A962D3B796 for ; Fri, 22 Dec 2023 23:52:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703289148; x=1734825148; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7MgS2tz+eXb3t+LyzYLE/xkq/zAhJTq68EvRtEFLASY=; b=dDsZjT6TgC0iQHee3fQ3xec2LujX+LzK/UyK4DNuorUAqxuO9V/Vs336 ETGSZr2y6qOB7uhzDPYgaNQ9mWBg8JtFvvhK4aNH1TOKg+4HhQDTKz1R5 zeibiGvQxuAQeX8wqPN6BI9UI6PZhnTX5zCP1xq3T0w9jnSz7/KFtKfbx QC8AKBZQSDzcxLXVubqt8cyPI9/2AxaZyMLfjmg7CjtQ659qQee96ud5b XxkpAK3h/81dt3UhTAJfTm4gNsj6QFMILWvpl+1AxSISCNwpq3ZQZnPZz /9ZMoWYxTSG4UFI4+KaZ2Rbij0fR5+c7eQn0HPkK7qDk/reOwzRBJSgNp A==; X-IronPort-AV: E=McAfee;i="6600,9927,10932"; a="395063311" X-IronPort-AV: E=Sophos;i="6.04,297,1695711600"; d="scan'208";a="395063311" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Dec 2023 15:52:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10932"; a="900623010" X-IronPort-AV: E=Sophos;i="6.04,297,1695711600"; d="scan'208";a="900623010" Received: from jeroenke-mobl.ger.corp.intel.com (HELO box.shutemov.name) ([10.249.35.180]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Dec 2023 15:52:22 -0800 Received: by box.shutemov.name (Postfix, from userid 1000) id 2DC7510A491; Sat, 23 Dec 2023 02:52:12 +0300 (+03) From: "Kirill A. Shutemov" To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org Cc: "Rafael J. Wysocki" , Peter Zijlstra , Adrian Hunter , Kuppuswamy Sathyanarayanan , Elena Reshetova , Jun Nakajima , Rick Edgecombe , Tom Lendacky , "Kalra, Ashish" , Sean Christopherson , "Huang, Kai" , Baoquan He , kexec@lists.infradead.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv5 08/16] x86/mm: Return correct level from lookup_address() if pte is none Date: Sat, 23 Dec 2023 02:52:00 +0300 Message-ID: <20231222235209.32143-9-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231222235209.32143-1-kirill.shutemov@linux.intel.com> References: <20231222235209.32143-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786028271894663324 X-GMAIL-MSGID: 1786028271894663324 lookup_address() only returns correct page table level for the entry if the entry is not none. Make the helper to always return correct 'level'. It allows to implement iterator over kernel page tables using lookup_address(). Add one more entry into enum pg_level to indicate size of VA covered by one PGD entry in 5-level paging mode. Signed-off-by: Kirill A. Shutemov Reviewed-by: Rick Edgecombe --- arch/x86/include/asm/pgtable_types.h | 1 + arch/x86/mm/pat/set_memory.c | 8 ++++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h index 0b748ee16b3d..3f648ffdfbe5 100644 --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -548,6 +548,7 @@ enum pg_level { PG_LEVEL_2M, PG_LEVEL_1G, PG_LEVEL_512G, + PG_LEVEL_256T, PG_LEVEL_NUM }; diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 6fbf22d5fa56..01f827eb8e80 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -666,32 +666,32 @@ pte_t *lookup_address_in_pgd(pgd_t *pgd, unsigned long address, pud_t *pud; pmd_t *pmd; - *level = PG_LEVEL_NONE; + *level = PG_LEVEL_256T; if (pgd_none(*pgd)) return NULL; + *level = PG_LEVEL_512G; p4d = p4d_offset(pgd, address); if (p4d_none(*p4d)) return NULL; - *level = PG_LEVEL_512G; if (p4d_large(*p4d) || !p4d_present(*p4d)) return (pte_t *)p4d; + *level = PG_LEVEL_1G; pud = pud_offset(p4d, address); if (pud_none(*pud)) return NULL; - *level = PG_LEVEL_1G; if (pud_large(*pud) || !pud_present(*pud)) return (pte_t *)pud; + *level = PG_LEVEL_2M; pmd = pmd_offset(pud, address); if (pmd_none(*pmd)) return NULL; - *level = PG_LEVEL_2M; if (pmd_large(*pmd) || !pmd_present(*pmd)) return (pte_t *)pmd;