From patchwork Mon Dec 4 17:26:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 173471 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp2916578vqy; Mon, 4 Dec 2023 09:27:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IGoUtnAXTcgJpatpR42hWhzt8/e0JJiMnpmtQ3zb2cnvEKtbSQH55iuQWOmUNE59hwplBgb X-Received: by 2002:a05:6a00:2d03:b0:6ce:4627:eca9 with SMTP id fa3-20020a056a002d0300b006ce4627eca9mr1626173pfb.64.1701710838843; Mon, 04 Dec 2023 09:27:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701710838; cv=none; d=google.com; s=arc-20160816; b=LnNrTRFj/0E+hpshVzJh5yQCi6V4oyfPVxAUaKMgUrwdrlDIl2yiJH8KHV/Mlj6U9q n/v2NLKJdlVq2eSH9Eh2eElB8PV/2aKI76X1B2May3cR4Qvg2vPVmQfSst+Ehgp2ZBUj V/uK5kEbIQxilce5s372odI7EOupfA6Bm+IEiKocF1ZXzF/sYeEO96S8RNVWyg0QFtrK AFQ96mpHIunsMc4DMtlssKGl1/Vnagq3UpElFbq9ry85yfl43iLsGUpR1Jr0YHlnDAxm QS5eXtVJmpBA0WgNd37PJcb4UqdJzAphRJg/jwdYWn79E5T1APN8GrhJd7k/Zcv5O/Eo YLgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=WhY7K21fTuleJoRqinTH8FQOk8orwGh5uZOpWymHf1Q=; fh=hE0xNwJ0u2DWIBBk0CilG3wWGmUr4MUdljkjokxIi6w=; b=OZuQXwJLtEg0ciRPMoNzSDzCiqPMRFd6Q2xk1anwDw7E1gSZaZiQuQPqX70huQ0cI8 5BcEt4T4PcW8t30DEOPtYcdlmhOG98Dk/oDSS88Va0KFytz+mG98KM2Oyfc7T8fMrzPd cWhxdmkRF7XcLJmr/lVLHPDW/TqRC7X/nrmfW+pUUc27+wL7qiVpQmAGPjzltoAWZbAK JZ11iWH2N5f6FCVy+58xg69klygqLcfuDdT/SmLSN/lUsOwqaCh9VjEhECbTXKM2F4NR GpFn7JtmRnPcWe9iFQP8UsDRDvkKc2+SICslNsnI4cfucRwq43sNDbR2C94MfrfKkWt0 0sZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=WzGYea8n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id eb4-20020a056a004c8400b006ce52bb9996si1892453pfb.301.2023.12.04.09.27.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 09:27:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=WzGYea8n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 9DF65805B9DA; Mon, 4 Dec 2023 09:27:14 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229485AbjLDR1F (ORCPT + 99 others); Mon, 4 Dec 2023 12:27:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231737AbjLDR1D (ORCPT ); Mon, 4 Dec 2023 12:27:03 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CDBCFA for ; Mon, 4 Dec 2023 09:27:09 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5d3f951af5aso51937347b3.0 for ; Mon, 04 Dec 2023 09:27:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701710829; x=1702315629; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WhY7K21fTuleJoRqinTH8FQOk8orwGh5uZOpWymHf1Q=; b=WzGYea8n8gKWhOwUN/QBnPZuZ64gGSovVSoWBoiu4lbkEjmhix1LpnW/02s19SFZrU QMVFdWKaxmPZunHSypbzY+U7yR2JTPyA41Cbhk3LyKYudi8ww5ybiC3hT+8dLtwBDuQq Qg74t1rFE9TL1mv3NkTfGz2vK3NybbXMukI32VwczFC2vBgCRWOjUSHMYJs+emm/e3nO xJFhfR5/0+lQmbwE/Uo0mGt4fp6L7vE/lQzYwMNzQTTLWdUS1e4TAi4krkOISbwofcZg mKe0cT4CAIz81vMLSxpK8K8AUvk5aMUMSlTkMo+6raEgYH+8Z5D8apajVYlCQjqmzIDZ 9+Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701710829; x=1702315629; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WhY7K21fTuleJoRqinTH8FQOk8orwGh5uZOpWymHf1Q=; b=gRPyqFiFL/MxV/JjRMpo7MxeOT5auYFD5roBdq2tOlXx2A2NiTEAzuDD7Yizdg7/1c B4gyLvFJil2qn6PSudch586cMuDk1Ub6F7Z8ppxQMtOds1hEEX4M1dOQwqjoCrRTii9o v26hxz0Zi/CfcH/S2xPJUm1ZhrJdRMAU9rVJ/5kVY6mk+B1KYIblKJt3jJ9qpQ3aI+PH jLmruS9adutV96JJGhRknkXXKpFlO1GYQBw9qOoXo/UX0aXlRVfZxO0V33IEFGozhudm 4RKzOpgFgteEWhwkbzBkliNc1utIX7udem0tqEq9yJny/ndRno7SgaKc0khjd6vQzobo TZ0g== X-Gm-Message-State: AOJu0Yy77gQP8aJvSeM/Rd3sGPPzcQIbYQ/AWZWX40RiJXg9+L9T3n7Z X1b8oQmhju/2uXrgCElTxnF3p0JctX67X3o0 X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a81:b61d:0:b0:5d3:84d4:eb35 with SMTP id u29-20020a81b61d000000b005d384d4eb35mr358241ywh.3.1701710828830; Mon, 04 Dec 2023 09:27:08 -0800 (PST) Date: Mon, 4 Dec 2023 17:26:45 +0000 In-Reply-To: <20231204172646.2541916-1-jthoughton@google.com> Mime-Version: 1.0 References: <20231204172646.2541916-1-jthoughton@google.com> X-Mailer: git-send-email 2.43.0.rc2.451.g8631bc7472-goog Message-ID: <20231204172646.2541916-2-jthoughton@google.com> Subject: [PATCH 1/2] arm64: hugetlb: Distinguish between hw and sw dirtiness in __cont_access_flags_changed From: James Houghton To: Steve Capper , Will Deacon , Andrew Morton Cc: Mike Kravetz , Muchun Song , Anshuman Khandual , Catalin Marinas , Ryan Roberts , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton , stable@vger.kernel.org X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 04 Dec 2023 09:27:14 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784373144585858138 X-GMAIL-MSGID: 1784373144585858138 __cont_access_flags_changed was originally introduced to avoid making unnecessary changes to the PTEs. Consider the following case: all the PTEs in the contiguous group have PTE_DIRTY | PTE_RDONLY | PTE_WRITE, and we are running on a system without HAFDBS. When writing via these PTEs, we will get a page fault, and hugetlb_fault will (rightly) attempt to update the PTEs with PTE_DIRTY | PTE_WRITE, but, as both the original PTEs and the new PTEs are pte_dirty(), __cont_access_flags_changed prevents the pgprot update from occurring. To avoid the page fault loop that we get ourselves into, distinguish between hardware-dirty and software-dirty for this check. Non-contiguous PTEs aren't broken in the same way, as we will always write a new PTE unless the new PTE is exactly equal to the old one. Fixes: 031e6e6b4e12 ("arm64: hugetlb: Avoid unnecessary clearing in huge_ptep_set_access_flags") Signed-off-by: James Houghton Cc: diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index f5aae342632c..87a9564976fa 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -437,7 +437,10 @@ static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) for (i = 0; i < ncontig; i++) { pte_t orig_pte = ptep_get(ptep + i); - if (pte_dirty(pte) != pte_dirty(orig_pte)) + if (pte_sw_dirty(pte) != pte_sw_dirty(orig_pte)) + return 1; + + if (pte_hw_dirty(pte) != pte_hw_dirty(orig_pte)) return 1; if (pte_young(pte) != pte_young(orig_pte)) From patchwork Mon Dec 4 17:26:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 173472 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp2916683vqy; Mon, 4 Dec 2023 09:27:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IFKgqRjPl1B/BO8w+jG20laJtZVjf4sMpmou4KPIIjKgUzExLWknt0iXiWZO3sWBtEiOKSa X-Received: by 2002:a17:903:22c1:b0:1cf:e9b5:90ee with SMTP id y1-20020a17090322c100b001cfe9b590eemr79135plg.24.1701710849375; Mon, 04 Dec 2023 09:27:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701710849; cv=none; d=google.com; s=arc-20160816; b=wuQY+Hzvjyiknghal8bDhORxzI90ZALvQyIdALE+/YcSmCUIEb9JqJA4+zKbx5d0Wv pQG7XfDi/dLWVXjLSzaAvAMVR+b1EggF132nAzkig7n3Lrur6baubEEtw7RTufH+3uKs 5dQfYSD9OEHVsW9rrVWMrtfFtsQHehko5BgbP2fFynF2H1MopMUOlvqM06o4v+wK4O9I XLDCm9WKr1AR/iz7NN4pPqefL4zWSogBKGG1w5RgIWHdwTCh/v+SGb429dUaQI/FJex3 A7Z1rTw2ro5EasQIChIN96xPdBUe5G3Qq8Hk9GmrtESJOsWKWlUZ0OeA5XkqHbNI9n7u c0BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=BIP1VBSrHfriGjh5Ogk9INZtbtmoj9c5sEQfKrkJBSE=; fh=MJxrAMNk1xt5eTwPYlce6LT4l1O5DiRP7H6bsCVP2JY=; b=WgdEiEHKaG45JmoJihDZAqQDp8ieLmVHdk9Ktc7LMB4ftNcd0EAJL1IN02pUbk6nk9 p17uUbuUwM/ML7VMu8UunlHRrQrymboMfWf0kLNWPtNU6NvLoiVahrkXzPwEJpRWEpsy U4ak9vdlGUWIh0SLRGzj8FCYskVO/7ncV+HQkXQRsRKiOFpKOUuL/u+Bzi2Ll4WVPISJ ekBG+jwVg+jiBgd/plNqIhyu3E+OJ3vNXR4u1TP0rrkE3cAUbGWhX7nxZJGLwqZQ/Hx6 kTUS+X2Jd/p6MZkbDVYlPu21XCOHs1izEOUWzdrJaAVo/cXIHX2tBpaimtkNZLCTWvL0 z/CA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b="m4uH/7hz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id h1-20020a170902eec100b001d050ef0078si6214821plb.21.2023.12.04.09.27.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 09:27:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b="m4uH/7hz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E8F02804C21A; Mon, 4 Dec 2023 09:27:26 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231777AbjLDR1I (ORCPT + 99 others); Mon, 4 Dec 2023 12:27:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231655AbjLDR1G (ORCPT ); Mon, 4 Dec 2023 12:27:06 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DA43C0 for ; Mon, 4 Dec 2023 09:27:12 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5d0c4ba7081so78366597b3.0 for ; Mon, 04 Dec 2023 09:27:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701710832; x=1702315632; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BIP1VBSrHfriGjh5Ogk9INZtbtmoj9c5sEQfKrkJBSE=; b=m4uH/7hzVu5zigT1huoKtvcvFkWnSAug3GAlEda3+9FUh0K+crF/J5hHQzUHGAt4rl ej3lyThtJQiZunyJaNC8od6dZxLx6rwkT40wY/fwMY+B7TLGEa/fVu4xO6M/KNGnm+sD rBNbNpbOGcoFRpglZ4viyH0DMU1/KrJnD7N2wzTTr5YcHzI+FPQG3SXhraLaBbNdyQNQ 6c2Wxx+98ueDomh7lThK9UM52Ewh0/BMzWkxtC0A/6QZGdUF82kkmgvxSpknTvVf8aUW T1trWAXZclpo1OP63LF7Yaacq9pxr//sppS719lkVKiTquvM2eTH2zy1ZqXzdQBXjlsO I7BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701710832; x=1702315632; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BIP1VBSrHfriGjh5Ogk9INZtbtmoj9c5sEQfKrkJBSE=; b=pQCpQ7NQLtJm+nr9S9imziSyZKEqLU0xcPaptpFgPVdOixN7kcOlg5y4/x+jhSp3aB pcsz//4yXFwmteaeucEqeC/RHmTz6Cq3Bwojf4RH+7Q1wqrrmSLvbblWsgKwL8xRhGCG hfJCJQbLiM/EwHD+iD6dIb8gth87eyzIXznvvpqjMVARLOwtCNp3mj4ARez5uPtk7l46 6YqtpE58zFSHRVEVTUuiATkI86Pl9G958zsl+UYeBzAz4O3uvaDj13CLrQOntfP5Dl7U R9jcBWCl6a2M7ndfWSJaBkKrsM9uLWD6RxK3aWY10N7zJzuvmxM7N1bzxUyCgB9uqPGm V7aQ== X-Gm-Message-State: AOJu0YxJt9qkbLmh7nZCFzH+UrJBwpAB+OnaWWIRw+33mNk1b/BWvFzv iJuEXdCBI8AKSK+Zapt6wzZiylhGLoXUEAY9 X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a05:690c:2710:b0:5d5:5183:ebd7 with SMTP id dy16-20020a05690c271000b005d55183ebd7mr222984ywb.7.1701710831851; Mon, 04 Dec 2023 09:27:11 -0800 (PST) Date: Mon, 4 Dec 2023 17:26:46 +0000 In-Reply-To: <20231204172646.2541916-1-jthoughton@google.com> Mime-Version: 1.0 References: <20231204172646.2541916-1-jthoughton@google.com> X-Mailer: git-send-email 2.43.0.rc2.451.g8631bc7472-goog Message-ID: <20231204172646.2541916-3-jthoughton@google.com> Subject: [PATCH 2/2] arm64: mm: Always make sw-dirty PTEs hw-dirty in pte_modify From: James Houghton To: Steve Capper , Will Deacon , Andrew Morton Cc: Mike Kravetz , Muchun Song , Anshuman Khandual , Catalin Marinas , Ryan Roberts , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 04 Dec 2023 09:27:27 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784373155283179718 X-GMAIL-MSGID: 1784373155283179718 Make it impossible to create a sw-dirty, hw-clean PTE with pte_modify. Such a PTE should be impossible to create, and there may be places that assume that pte_dirty() implies pte_hw_dirty(). Signed-off-by: James Houghton Reviewed-by: Ryan Roberts diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b19a8aee684c..79ce70fbb751 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -834,6 +834,12 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) pte = set_pte_bit(pte, __pgprot(PTE_DIRTY)); pte_val(pte) = (pte_val(pte) & ~mask) | (pgprot_val(newprot) & mask); + /* + * If we end up clearing hw dirtiness for a sw-dirty PTE, set hardware + * dirtiness again. + */ + if (pte_sw_dirty(pte)) + pte = pte_mkdirty(pte); return pte; }