From patchwork Sat Apr 8 10:45:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 81179 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp841027vqo; Sat, 8 Apr 2023 05:21:02 -0700 (PDT) X-Google-Smtp-Source: AKy350b18+vvHKGCLdTnWybkiB4dCQ0Yl8wjdPqjUMPvaeWZw6I3nIRwXIZY6sjZDQhqFpgJa7G/ X-Received: by 2002:a17:902:dccc:b0:1a2:626b:ba1d with SMTP id t12-20020a170902dccc00b001a2626bba1dmr1626991pll.39.1680956462599; Sat, 08 Apr 2023 05:21:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680956462; cv=none; d=google.com; s=arc-20160816; b=Y8yoxTu73Y/W164i+L7GcE3tIxze/CX+EXgxxbDLtyECvApkxXDhjmJzRLlSVy265k Y8P6dxOJuSa2e4y3G3LkPOCKVIs829rlD4g43aC1VHyAv9pAV0EF6lHNih9LR6BDrKUi tSK8F/qeuEBufTk3tgUIceevpHqexa7H3Qk37IHT0MpR9iCOY9mVbKfS0wraQNqPuCMD F1+I+1RkwBIWWMEDPsVlEvoCWMgycTPFP90xgyI982JKtPZCawRVjzAfJ++rtaYU/AaQ l5EAMnHgzmcTdEa0EHzSHJTBsRv/HkoYW596K052EfCj+P4wJwhxgBpbr4Anp0Km8zsG hmKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=ayGvbRTCswSwC99PgN1n1uhvyXBASTcVcAl7M3jFjhY=; b=Etm3YGqf4+hvQAODYeIMwHjUkcWdRnF82fvGZflS1RByExiEs1oZOUptz7+tAtrRmb YZbwSKXsamwm8QYkOKJclaVVp9tr1ULyRL+SLbCAGuodJ7KMzyOnP906dXTvODSrq5v7 pUjpYxFZwU+iTy6bzHbFpWSDWt6ETIRUn7Db0M1OPLTshwAEGGiIJarz8PxwocRYAWrf dgkOJJ/h6y4fFRghpkfOfqKSIDlaDPaY9KnBszx2oMByBG9G2D84Hd7MZETaE+jPyy5y SlsLFPO91HfR5TFShKoOLTrFromyJnP8nQWRGxADRQhMoqFYgiCurynkFAiWlUQXemiv 7GwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=dqOCOgY+; dkim=neutral (no key) header.i=@linutronix.de header.b=sY1JdN1x; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q17-20020a170902789100b001a5145cf5d5si3591153pll.167.2023.04.08.05.20.20; Sat, 08 Apr 2023 05:21:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=dqOCOgY+; dkim=neutral (no key) header.i=@linutronix.de header.b=sY1JdN1x; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229686AbjDHLAV (ORCPT + 99 others); Sat, 8 Apr 2023 07:00:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229570AbjDHLAU (ORCPT ); Sat, 8 Apr 2023 07:00:20 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA102E41 for ; Sat, 8 Apr 2023 04:00:18 -0700 (PDT) Date: Sat, 08 Apr 2023 10:45:17 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1680950717; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ayGvbRTCswSwC99PgN1n1uhvyXBASTcVcAl7M3jFjhY=; b=dqOCOgY+SzsabsKwIwTfy+srULg7dvMuJ02SVVcjM6A6CgU/pHXLTb0OYpqxvtFxlabNfS ybCKreGVJjKr78ZSiK1xdluQJSCbj6eKmMG8i15Z0QVMkGXpbWw1cnUsEJFWxg+Vzu/V2Y Ywury98V7oeq8lg1JPARwyakX3wsmOOhVG09nfLWafuKfnFMtVgSVT2yN/XFeCNbmJHCSy +k5BQm18yc5YnxDQWa6F9MXJK63X8WKOAtX55lBACxeWQv0A+zfgDyzVVO7/WT8TN7xzMB zMZOyT6Qj0ZAR/D9QzhvYoJpRTQY32TkmNXROdcvGmXjw1BSZqCXZu0ukWmmcQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1680950717; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ayGvbRTCswSwC99PgN1n1uhvyXBASTcVcAl7M3jFjhY=; b=sY1JdN1xC7uE+7an1UeNDBYTRkAzdn5r8K0T90BM4u8McfUeRFRZ9k+ysz/Vp4VlLZavga u0izMoMCQV9FqnCg== From: "irqchip-bot for Anup Patel" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-kernel@vger.kernel.org Subject: [irqchip: irq/irqchip-next] RISC-V: Use IPIs for remote TLB flush when possible Cc: Anup Patel , Atish Patra , Palmer Dabbelt , Marc Zyngier , tglx@linutronix.de In-Reply-To: <20230328035223.1480939-6-apatel@ventanamicro.com> References: <20230328035223.1480939-6-apatel@ventanamicro.com> MIME-Version: 1.0 Message-ID: <168095071741.404.18009750298069050198.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1761584996806147145?= X-GMAIL-MSGID: =?utf-8?q?1762610603520635441?= The following commit has been merged into the irq/irqchip-next branch of irqchip: Commit-ID: 18d2199d81054f44e6d2a51177cc80566f43bf23 Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/18d2199d81054f44e6d2a51177cc80566f43bf23 Author: Anup Patel AuthorDate: Tue, 28 Mar 2023 09:22:21 +05:30 Committer: Marc Zyngier CommitterDate: Sat, 08 Apr 2023 11:26:24 +01:00 RISC-V: Use IPIs for remote TLB flush when possible If we have specialized interrupt controller (such as AIA IMSIC) which allows supervisor mode to directly inject IPIs without any assistance from M-mode or HS-mode then using such specialized interrupt controller, we can do remote TLB flushes directly from supervisor mode instead of using the SBI RFENCE calls. This patch extends remote TLB flush functions to use supervisor mode IPIs whenever direct supervisor mode IPIs.are supported by interrupt controller. Signed-off-by: Anup Patel Reviewed-by: Atish Patra Acked-by: Palmer Dabbelt Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/20230328035223.1480939-6-apatel@ventanamicro.com --- arch/riscv/mm/tlbflush.c | 93 ++++++++++++++++++++++++++++++++------- 1 file changed, 78 insertions(+), 15 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index ef701fa..77be59a 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -23,14 +23,62 @@ static inline void local_flush_tlb_page_asid(unsigned long addr, : "memory"); } +static inline void local_flush_tlb_range(unsigned long start, + unsigned long size, unsigned long stride) +{ + if (size <= stride) + local_flush_tlb_page(start); + else + local_flush_tlb_all(); +} + +static inline void local_flush_tlb_range_asid(unsigned long start, + unsigned long size, unsigned long stride, unsigned long asid) +{ + if (size <= stride) + local_flush_tlb_page_asid(start, asid); + else + local_flush_tlb_all_asid(asid); +} + +static void __ipi_flush_tlb_all(void *info) +{ + local_flush_tlb_all(); +} + void flush_tlb_all(void) { - sbi_remote_sfence_vma(NULL, 0, -1); + if (riscv_use_ipi_for_rfence()) + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); + else + sbi_remote_sfence_vma(NULL, 0, -1); +} + +struct flush_tlb_range_data { + unsigned long asid; + unsigned long start; + unsigned long size; + unsigned long stride; +}; + +static void __ipi_flush_tlb_range_asid(void *info) +{ + struct flush_tlb_range_data *d = info; + + local_flush_tlb_range_asid(d->start, d->size, d->stride, d->asid); +} + +static void __ipi_flush_tlb_range(void *info) +{ + struct flush_tlb_range_data *d = info; + + local_flush_tlb_range(d->start, d->size, d->stride); } -static void __sbi_tlb_flush_range(struct mm_struct *mm, unsigned long start, - unsigned long size, unsigned long stride) +static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, + unsigned long size, unsigned long stride) { + struct flush_tlb_range_data ftd; struct cpumask *cmask = mm_cpumask(mm); unsigned int cpuid; bool broadcast; @@ -45,19 +93,34 @@ static void __sbi_tlb_flush_range(struct mm_struct *mm, unsigned long start, unsigned long asid = atomic_long_read(&mm->context.id) & asid_mask; if (broadcast) { - sbi_remote_sfence_vma_asid(cmask, start, size, asid); - } else if (size <= stride) { - local_flush_tlb_page_asid(start, asid); + if (riscv_use_ipi_for_rfence()) { + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, + __ipi_flush_tlb_range_asid, + &ftd, 1); + } else + sbi_remote_sfence_vma_asid(cmask, + start, size, asid); } else { - local_flush_tlb_all_asid(asid); + local_flush_tlb_range_asid(start, size, stride, asid); } } else { if (broadcast) { - sbi_remote_sfence_vma(cmask, start, size); - } else if (size <= stride) { - local_flush_tlb_page(start); + if (riscv_use_ipi_for_rfence()) { + ftd.asid = 0; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, + __ipi_flush_tlb_range, + &ftd, 1); + } else + sbi_remote_sfence_vma(cmask, start, size); } else { - local_flush_tlb_all(); + local_flush_tlb_range(start, size, stride); } } @@ -66,23 +129,23 @@ static void __sbi_tlb_flush_range(struct mm_struct *mm, unsigned long start, void flush_tlb_mm(struct mm_struct *mm) { - __sbi_tlb_flush_range(mm, 0, -1, PAGE_SIZE); + __flush_tlb_range(mm, 0, -1, PAGE_SIZE); } void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr) { - __sbi_tlb_flush_range(vma->vm_mm, addr, PAGE_SIZE, PAGE_SIZE); + __flush_tlb_range(vma->vm_mm, addr, PAGE_SIZE, PAGE_SIZE); } void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - __sbi_tlb_flush_range(vma->vm_mm, start, end - start, PAGE_SIZE); + __flush_tlb_range(vma->vm_mm, start, end - start, PAGE_SIZE); } #ifdef CONFIG_TRANSPARENT_HUGEPAGE void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - __sbi_tlb_flush_range(vma->vm_mm, start, end - start, PMD_SIZE); + __flush_tlb_range(vma->vm_mm, start, end - start, PMD_SIZE); } #endif