From patchwork Thu Feb 15 12:17:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 20426 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:b825:b0:106:860b:bbdd with SMTP id da37csp355677dyb; Thu, 15 Feb 2024 04:18:36 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWPhnJqknqxZ+BeBvmIQwVQeoitdP3ym3EmnAdlcNqxr9vKy0OF2WyVoN3jTIxmhIJfD8b/P/H7AWGpoat2GnqAIveCsg== X-Google-Smtp-Source: AGHT+IHR97SAqNK4F60yfLYrU/2EM11R232Q/nZsJhS97xPD0nsfIMY/FebCWiNMYpbAicvf6EuD X-Received: by 2002:a05:6808:309d:b0:3c1:395b:9c4e with SMTP id bl29-20020a056808309d00b003c1395b9c4emr733632oib.2.1707999516404; Thu, 15 Feb 2024 04:18:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707999516; cv=pass; d=google.com; s=arc-20160816; b=IEMnKP2nkLaOEDDJBWuKXHZAqhTiyIZfDvlu5LoJKZa5eKxIP5xqFwf11LPMva0rau PmWhGt332sB2YsmRteq1Pw3WUsr+n/Ef0Vzw8UI90WApxKDaw7+VvrBRTnbBS07JzZ4k ZA2+PRVy7YD9Kd6bORRZ03dvPEGB+YBncJvkdkWURu+EkQipdACUhdgk+s8fNYBMNYOS 1MYX7N1iaktddPKg6d7Re+3FRdsGNRafBl1M3qeFsLA2w/ufWn9hF8Aw9l3oFkqchSRS D8CCoPj32PEcgLZW5lc47Scp893sZBoUmmkrHlxhKQvAlmOz2K2ycGIRLcDkMkNHDzrw wHRg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from; bh=EgATNENApWy8csVmwonky1cJjqyqybzBOEU21JHol90=; fh=2FwJMiZjh7JNs0IOfHdUsjppsJNoMkV8ot/xk2oRsPU=; b=QQLUPbJrzLgQGEBko57wdN1LAt645X0/U9dSIv0JgbP8+PWWDfASWg/tqCHIQAkEu+ 34z0n1RsnchJSXArelVcZFcTGv+mEODQgF+sLN7GnlRH3m1lPeShkTT9Py4oo39Sz659 5uAwmiOgQEFgJ76s7Wc/3W78ARkXU1hSgoUVwYNKlPXpGxhUFGDfI5UEw4TQabL0uE91 vOTPdYqVQKWm+oZXLdPB0AQdR7e+9ZKZEU1OJW9V6UkV4JzF1CiGaOvmAT07SuMqvqWs 3CBzq6u2h6UOUfKarOwvnQhPQkRx3Od2EbBfB42sJczk1O4w67ByFl7I0gzEmSrydObM 4Khw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-66872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-66872-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id ay12-20020a05622a228c00b0042c6bf58b4fsi1309476qtb.616.2024.02.15.04.18.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Feb 2024 04:18:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-66872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-66872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-66872-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3287F1C21B18 for ; Thu, 15 Feb 2024 12:18:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 921BD12C819; Thu, 15 Feb 2024 12:18:10 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A612E12BEAF for ; Thu, 15 Feb 2024 12:18:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707999489; cv=none; b=Fk8l0vunljLVs2ehqDhvsqAkmbPaHt9z87k+Im7kx3kpyxg0FpODvEqljQtSKKGOtGig5cO99IsrgFjEDEUNG1VCrXx0WUnm/4y3aoLgTr5H2tMdHRPxiYYgEB6Fq4p4JYQxvh7lWYNHwe0r2CSKmoh3TPskk8g4wcwkucb/tCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707999489; c=relaxed/simple; bh=x9FtnrleqpJuZy6f30x2br9cQeME+hU2HTs978RoTLA=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=qdG81AQy9t1duLggBJlhR1MNxAUWDG8LCPXd2GZfVlO8Utjn/YDNJTYOvaSWQhQ3U8n13Zokyw/i4EAjjPkhJm+5LivQtv6qAnnbaPcRI4piGa+Z7DjxeNYThtBi6ckB4ycDXQ+JI4GFyhFQLeDC88KE9i+tO5BzzOqumBXNSrs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C309F1FB; Thu, 15 Feb 2024 04:18:46 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 256853F766; Thu, 15 Feb 2024 04:18:04 -0800 (PST) From: Ryan Roberts To: David Hildenbrand , Mark Rutland , Catalin Marinas , Will Deacon , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Andrew Morton , Muchun Song Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH v1 0/4] Reduce cost of ptep_get_lockless on arm64 Date: Thu, 15 Feb 2024 12:17:52 +0000 Message-Id: <20240215121756.2734131-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790967300964610853 X-GMAIL-MSGID: 1790967300964610853 This is an RFC for a series that aims to reduce the cost and complexity of ptep_get_lockless() for arm64 when supporting transparent contpte mappings [1]. The approach came from discussion with Mark and David [2]. It introduces a new helper, ptep_get_lockless_norecency(), which allows the access and dirty bits in the returned pte to be incorrect. This relaxation permits arm64's implementation to just read the single target pte, and avoids having to iterate over the full contpte block to gather the access and dirty bits, for the contpte case. It turns out that none of the call sites using ptep_get_lockless() require accurate access and dirty bit information, so we can also convert those sites. Although a couple of places need care (see patches 2 and 3). Arguably patch 3 is a bit fragile, given the wide accessibility of vmf->orig_pte. So it might make sense to drop this patch and stick to using ptep_get_lockless() in the page fault path. I'm keen to hear opinions. I've chosen the name "recency" because it's shortish and somewhat descriptive, and is alredy used in a couple of places to mean similar things (see mglru and damon). I'm open to other names if anyone has better ideas. If concensus is that this approach is generally acceptable, I intend to create a series in future to do a similar thing with ptep_get() -> ptep_get_norecency(). --- This series applies on top of [1]. [1] https://lore.kernel.org/linux-mm/20240215103205.2607016-1-ryan.roberts@arm.com/ [2] https://lore.kernel.org/linux-mm/a91cfe1c-289e-4828-8cfc-be34eb69a71b@redhat.com/ Thanks, Ryan Ryan Roberts (4): mm: Introduce ptep_get_lockless_norecency() mm/gup: Use ptep_get_lockless_norecency() mm/memory: Use ptep_get_lockless_norecency() for orig_pte arm64/mm: Override ptep_get_lockless_norecency() arch/arm64/include/asm/pgtable.h | 6 ++++ include/linux/pgtable.h | 55 ++++++++++++++++++++++++++++-- kernel/events/core.c | 2 +- mm/gup.c | 7 ++-- mm/hugetlb.c | 2 +- mm/khugepaged.c | 2 +- mm/memory.c | 57 ++++++++++++++++++++------------ mm/swap_state.c | 2 +- mm/swapfile.c | 2 +- 9 files changed, 102 insertions(+), 33 deletions(-) -- 2.25.1