Message ID | 20231228084642.1765-3-jszhang@kernel.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp1879682dyb; Thu, 28 Dec 2023 01:00:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IHQaXocXJjCOKGKMhHzaLA0+sG0BYvx71AT8li/19oiRQM5fuGpEIX3l2V4TSjNGjIwsSxp X-Received: by 2002:a05:6214:d4f:b0:67f:47a8:84cd with SMTP id 15-20020a0562140d4f00b0067f47a884cdmr17893139qvr.36.1703754007848; Thu, 28 Dec 2023 01:00:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703754007; cv=none; d=google.com; s=arc-20160816; b=i6kwSUaZb2T5i6ff8U6i+s0vXmRbhpzmtyo61T/UyrTYkTIEJ7ENAjygGN/BO7S0Zl xwYp7/m66mHMO7SmG4vSaftYuAA+Hanvx5IKHm/3eZG8na5bPGBEcKn4fFaXyQrHqTRB QYqGzX/i5lQ8gRwXol1HDm0IH+noxfytFV3Z9fyyAJeBA3LCW5/tPWk+5jw36N6EJr17 Qf7/Zs0nfhAeBqY40hF7IPYmyAcH4z0cQ+l7YY6sSN+6g+BekK/Q8CEtzQ3qGdCcKOAQ eNClXav5e0+qU1iZkcYmoIP4mlgKl+evEqJOItNigRFm1mXItwRB12HNm5HduBAsj/2C wCZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=U0bHBCfybk8M/ty/5LuxJpVChy9zxALoGJGRJoUUtYY=; fh=7yoSLDYtBiATajkf0jny2nUhpLBvg39awZKJqL/P6xI=; b=XypYeHmK1FhRC2G3HwDI5KcpMoVq4CxAkX09jW0FcRQ/fStH1SEI2O7qFXRZEqCvhl Ehq8yxGFtO8jPfZJMKBrShbbz0hQn8vbFENm3RDvbGS3Gm7h4UImmP2gfPhxg46jm2lj FU2IyHp35m2YMAxRpde4906KgnC4w7MBVXiV03+hK1hJR1wVBkuMJgDIaYNf7av5ypjK lgnHFc8Duu7NAJx8RqbdwrO3Xbg6h18D9wCwMYyT6Bsr6zHye9fCKF6z8GuXtWyPNE23 kAHPLxPeEGuR+gJ/x7n8pAVoiKP/O7kDc+w+gz25x3J6Cewt4N7JduD69EVHZzCwAr82 T42A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TKBsmvO8; spf=pass (google.com: domain of linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id k4-20020a0cf584000000b0067f9474ef53si14094291qvm.368.2023.12.28.01.00.07 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 01:00:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TKBsmvO8; spf=pass (google.com: domain of linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12528-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9F6031C227C5 for <ouuuleilei@gmail.com>; Thu, 28 Dec 2023 09:00:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D34D37487; Thu, 28 Dec 2023 08:59:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TKBsmvO8" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46E2E6FCB; Thu, 28 Dec 2023 08:59:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5389DC433CC; Thu, 28 Dec 2023 08:59:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703753970; bh=4x9t1f04s06EEvjvtQ7yC2/ljPKJqNDBrcj2L/hT3Js=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TKBsmvO8700bopUsWROGXLmoCWFXsTW+dDVnNGSxOkaL/USi53RKl8TdsixW0+meq ZCWxQdbp33ikg/G9ezCtq7NkctrZnj90/OzQ0n9iUxG+YKDW4DQ0nPcXFY42wiBv+r 997dRhgKccetlwPkyjqAZY+f5mx70kEYFtXj+CoiB8mr5EJbZvQA7ejpm/TS62Kx8Z MVCzbTzHlBLytfAxGrORGDpTnrYATlbt8PPwaCWGIg2WfXt/VlliMYXrTExd8VnHSR LxHkWdaNYJGBiXawHPjLTvQ6a2cSRLkXc9Y/wtpnv3MN3QHCLVBpveSUN4P0ShLMkB BBgaja1UR4WBA== From: Jisheng Zhang <jszhang@kernel.org> To: Will Deacon <will@kernel.org>, "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Nick Piggin <npiggin@gmail.com>, Peter Zijlstra <peterz@infradead.org>, Catalin Marinas <catalin.marinas@arm.com>, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, Arnd Bergmann <arnd@arndb.de> Cc: linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [PATCH 2/2] riscv: tlb: avoid tlb flushing if fullmm == 1 Date: Thu, 28 Dec 2023 16:46:42 +0800 Message-Id: <20231228084642.1765-3-jszhang@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20231228084642.1765-1-jszhang@kernel.org> References: <20231228084642.1765-1-jszhang@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786515562530322800 X-GMAIL-MSGID: 1786515562530322800 |
Series |
riscv: tlb: avoid tlb flushing on exit & execve
|
|
Commit Message
Jisheng Zhang
Dec. 28, 2023, 8:46 a.m. UTC
The mmu_gather code sets fullmm=1 when tearing down the entire address
space for an mm_struct on exit or execve. So if the underlying platform
supports ASID, the tlb flushing can be avoided because the ASID
allocator will never re-allocate a dirty ASID.
Use the performance of Process creation in unixbench on T-HEAD TH1520
platform is improved by about 4%.
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
---
arch/riscv/include/asm/tlb.h | 9 +++++++++
1 file changed, 9 insertions(+)
Comments
Hi Jisheng, On 28/12/2023 09:46, Jisheng Zhang wrote: > The mmu_gather code sets fullmm=1 when tearing down the entire address > space for an mm_struct on exit or execve. So if the underlying platform > supports ASID, the tlb flushing can be avoided because the ASID > allocator will never re-allocate a dirty ASID. > > Use the performance of Process creation in unixbench on T-HEAD TH1520 > platform is improved by about 4%. > > Signed-off-by: Jisheng Zhang <jszhang@kernel.org> > --- > arch/riscv/include/asm/tlb.h | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h > index 1eb5682b2af6..35f3c214332e 100644 > --- a/arch/riscv/include/asm/tlb.h > +++ b/arch/riscv/include/asm/tlb.h > @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb); > > #define tlb_flush tlb_flush > #include <asm-generic/tlb.h> > +#include <asm/mmu_context.h> > > static inline void tlb_flush(struct mmu_gather *tlb) > { > #ifdef CONFIG_MMU > + /* > + * If ASID is supported, the ASID allocator will either invalidate the > + * ASID or mark it as used. So we can avoid TLB invalidation when > + * pulling down a full mm. > + */ Given the number of bits are limited for the ASID, at some point we'll reuse previously allocated ASID so the ASID allocator must make sure to invalidate the entries when reusing an ASID: can you point where this is done? Thanks, Alex > + if (static_branch_likely(&use_asid_allocator) && tlb->fullmm) > + return; > + > if (tlb->fullmm || tlb->need_flush_all) > flush_tlb_mm(tlb->mm); > else
On Sat, Dec 30, 2023 at 07:26:11PM +0100, Alexandre Ghiti wrote: > Hi Jisheng, Hi Alex, > > On 28/12/2023 09:46, Jisheng Zhang wrote: > > The mmu_gather code sets fullmm=1 when tearing down the entire address > > space for an mm_struct on exit or execve. So if the underlying platform > > supports ASID, the tlb flushing can be avoided because the ASID > > allocator will never re-allocate a dirty ASID. > > > > Use the performance of Process creation in unixbench on T-HEAD TH1520 > > platform is improved by about 4%. > > > > Signed-off-by: Jisheng Zhang <jszhang@kernel.org> > > --- > > arch/riscv/include/asm/tlb.h | 9 +++++++++ > > 1 file changed, 9 insertions(+) > > > > diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h > > index 1eb5682b2af6..35f3c214332e 100644 > > --- a/arch/riscv/include/asm/tlb.h > > +++ b/arch/riscv/include/asm/tlb.h > > @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb); > > #define tlb_flush tlb_flush > > #include <asm-generic/tlb.h> > > +#include <asm/mmu_context.h> > > static inline void tlb_flush(struct mmu_gather *tlb) > > { > > #ifdef CONFIG_MMU > > + /* > > + * If ASID is supported, the ASID allocator will either invalidate the > > + * ASID or mark it as used. So we can avoid TLB invalidation when > > + * pulling down a full mm. > > + */ > > > Given the number of bits are limited for the ASID, at some point we'll reuse > previously allocated ASID so the ASID allocator must make sure to invalidate > the entries when reusing an ASID: can you point where this is done? Per my understanding of the code, the path would be set_mm_asid() __new_context() __flush_context() // set context_tlb_flush_pending if (need_flush_tlb) local_flush_tlb_all() Thanks > > > + if (static_branch_likely(&use_asid_allocator) && tlb->fullmm) > > + return; > > + > > if (tlb->fullmm || tlb->need_flush_all) > > flush_tlb_mm(tlb->mm); > > else
On 02/01/2024 04:12, Jisheng Zhang wrote: > On Sat, Dec 30, 2023 at 07:26:11PM +0100, Alexandre Ghiti wrote: >> Hi Jisheng, > Hi Alex, > >> On 28/12/2023 09:46, Jisheng Zhang wrote: >>> The mmu_gather code sets fullmm=1 when tearing down the entire address >>> space for an mm_struct on exit or execve. So if the underlying platform >>> supports ASID, the tlb flushing can be avoided because the ASID >>> allocator will never re-allocate a dirty ASID. >>> >>> Use the performance of Process creation in unixbench on T-HEAD TH1520 >>> platform is improved by about 4%. >>> >>> Signed-off-by: Jisheng Zhang <jszhang@kernel.org> >>> --- >>> arch/riscv/include/asm/tlb.h | 9 +++++++++ >>> 1 file changed, 9 insertions(+) >>> >>> diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h >>> index 1eb5682b2af6..35f3c214332e 100644 >>> --- a/arch/riscv/include/asm/tlb.h >>> +++ b/arch/riscv/include/asm/tlb.h >>> @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb); >>> #define tlb_flush tlb_flush >>> #include <asm-generic/tlb.h> >>> +#include <asm/mmu_context.h> >>> static inline void tlb_flush(struct mmu_gather *tlb) >>> { >>> #ifdef CONFIG_MMU >>> + /* >>> + * If ASID is supported, the ASID allocator will either invalidate the >>> + * ASID or mark it as used. So we can avoid TLB invalidation when >>> + * pulling down a full mm. >>> + */ >> >> Given the number of bits are limited for the ASID, at some point we'll reuse >> previously allocated ASID so the ASID allocator must make sure to invalidate >> the entries when reusing an ASID: can you point where this is done? > Per my understanding of the code, the path would be > set_mm_asid() > __new_context() > __flush_context() // set context_tlb_flush_pending > if (need_flush_tlb) > local_flush_tlb_all() Ok thanks, so feel free to add: Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Thanks! Alex > > Thanks > >>> + if (static_branch_likely(&use_asid_allocator) && tlb->fullmm) >>> + return; >>> + >>> if (tlb->fullmm || tlb->need_flush_all) >>> flush_tlb_mm(tlb->mm); >>> else > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv
diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h index 1eb5682b2af6..35f3c214332e 100644 --- a/arch/riscv/include/asm/tlb.h +++ b/arch/riscv/include/asm/tlb.h @@ -12,10 +12,19 @@ static void tlb_flush(struct mmu_gather *tlb); #define tlb_flush tlb_flush #include <asm-generic/tlb.h> +#include <asm/mmu_context.h> static inline void tlb_flush(struct mmu_gather *tlb) { #ifdef CONFIG_MMU + /* + * If ASID is supported, the ASID allocator will either invalidate the + * ASID or mark it as used. So we can avoid TLB invalidation when + * pulling down a full mm. + */ + if (static_branch_likely(&use_asid_allocator) && tlb->fullmm) + return; + if (tlb->fullmm || tlb->need_flush_all) flush_tlb_mm(tlb->mm); else