From patchwork Tue Nov 29 14:24:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 27280 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp377005wrr; Tue, 29 Nov 2022 06:37:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf7Q1YEtHPqVji+zj/1JZyhLVly+crlEUvVLng4WGO3iMUfYnALvYseslma8II3sYXVbPirW X-Received: by 2002:a63:4f63:0:b0:476:df56:b35e with SMTP id p35-20020a634f63000000b00476df56b35emr32931987pgl.449.1669732666748; Tue, 29 Nov 2022 06:37:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669732666; cv=none; d=google.com; s=arc-20160816; b=Z4wbZIZgqUbZ1nhzX6KXwVIERILN+q429fZt9AciNA4RzUep2dwbvF8zrm6BZ28XJq ytS1llb0e2j7Ej4ZJGepYY5fd7Masns+dfpGwP3ykQMsKSll6UapjCy21RwGNIoew9rT Ipsg0Cba1of04aAWwEeYDKmOxo0/0ab0zIWcP9m9wJJZtct4vtiz3HT0mxTI2RJJBQYv 8qhZ2JDby71VEPipf+dQQjUI3RC5yEVXZwOczylU+S/PU4UUNbI4ShPw796kbvYVjELm gT/IF8YvlDrScpRW24yozgohCICWAz/Tl1nVYAAX+JQuIQu2/Lnnptht03KNHlHviobJ k0eg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hrIW2y5CKd+1oUTF027uNozTIj8Vreh5isz2dDkBkRM=; b=dJ1tSMCuPXBhprLM5UEq65y8PkcMvLCX6N8zY2JrFtuj/9qRxAcgmTlx7fJqsLLLnF dGHUavHIVxhWk0kz4RTkvFhMz1j6Q2PDu/sJQXlVVfVH7lCBWYNXXbEANRGZGML/R+7Q rpx6NOuY0QrT1fmpcthQeFdKNxF1mQNObgx+ys4wnB6gPkFeJholJfRIiJFIjmaPyNET DoUv5npiqYf1huf1ah1SIt+3+XC9R/vkt7WHyJks28GUaTtRA9ib4vk402GZviswcKIu YriL4psp1+iYKE1P8ZGN9T8ppjtLw5BxT3ZMKoiRd5i5hv1opExuSU3dDK2EYH7L4KaK aCUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ventanamicro.com header.s=google header.b=Fjw60jDZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x5-20020a654145000000b0042b807728e3si13363756pgp.762.2022.11.29.06.37.33; Tue, 29 Nov 2022 06:37:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@ventanamicro.com header.s=google header.b=Fjw60jDZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235017AbiK2OZ2 (ORCPT + 99 others); Tue, 29 Nov 2022 09:25:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234237AbiK2OZM (ORCPT ); Tue, 29 Nov 2022 09:25:12 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F44F2BFC for ; Tue, 29 Nov 2022 06:25:11 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id ns17so6003426pjb.1 for ; Tue, 29 Nov 2022 06:25:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hrIW2y5CKd+1oUTF027uNozTIj8Vreh5isz2dDkBkRM=; b=Fjw60jDZBLrGczONi7wzRHpEr86UANHlQ4vrYoDzTuWL4WVwJ/lz2MeuBhY6lbasha pPUVUh+k68hO7zQeO06/R3SVQxsZsBdSzuIQewNmM3mofwX5y2CsfmCzHDV//Dz8hh5S xpZWbk2hzwk7yt/CYIlyt9xB7pw0QImrtauNwq/nvc2x7Vz04dmMusUegHnR9GRaUgS7 xnzUm338rz3gagoGW1nty/tvhMHcMJ1XWKTu5Z4crgs+EqtFQ7OglU5u2qYpax4kfnPM iBZ6kDmm6aLY0tpDlJK7M4SHmuNieapHeUUUCEyqPWr6Zm8tlOfD50UBMLi5UegTISfF Vljg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hrIW2y5CKd+1oUTF027uNozTIj8Vreh5isz2dDkBkRM=; b=eEu34ixnOuEjYQgc3UkXnaMEKTPxOyOXrFIRCHaK0u8TrmI31wBT/X0Ty33pDt5VFS jMtOedt0Ug1EADXg1mhmMBNYkvhjPw8W8gkfEmAP1v+aXgGNfeTqCcPXS7yNP6/T3F/y RTs7EH3vaEP6QMXf2oSSGUWuRw9zubRjF7rUYSUkFDIvRw87z13BKoSPIvqQwB6epjV5 ULRE0/uvJPhm8ebbXlDb9BmYfVApwnRFNfKJ7Ep3YWpKLdl9aUj9RjVuI8fbfdoD4SYY 26VD/CbxC7EkZd8QxIsYPzCI6DTfzn0M3/LA8LAsyGadLOl+U1rKH1oKKQfas5MRikA8 TarA== X-Gm-Message-State: ANoB5pnocUsuD93wb+MKg+KNFv9EWHZ5Dn30+Q5VneYBm6uaVCm5QB2S /jzLhyFsZVbJD28Qkqn7fULvuPlIOKjteA== X-Received: by 2002:a17:90a:460b:b0:218:8a84:aeca with SMTP id w11-20020a17090a460b00b002188a84aecamr53596216pjg.63.1669731910901; Tue, 29 Nov 2022 06:25:10 -0800 (PST) Received: from anup-ubuntu-vm.localdomain ([171.76.84.98]) by smtp.gmail.com with ESMTPSA id l12-20020a170903120c00b00176a2d23d1asm11039076plh.56.2022.11.29.06.25.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Nov 2022 06:25:10 -0800 (PST) From: Anup Patel To: Palmer Dabbelt , Paul Walmsley , Thomas Gleixner , Marc Zyngier , Daniel Lezcano Cc: Atish Patra , Alistair Francis , Anup Patel , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Anup Patel Subject: [PATCH v13 3/7] genirq: Add mechanism to multiplex a single HW IPI Date: Tue, 29 Nov 2022 19:54:45 +0530 Message-Id: <20221129142449.886518-4-apatel@ventanamicro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221129142449.886518-1-apatel@ventanamicro.com> References: <20221129142449.886518-1-apatel@ventanamicro.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750841600379331453?= X-GMAIL-MSGID: =?utf-8?q?1750841600379331453?= All RISC-V platforms have a single HW IPI provided by the INTC local interrupt controller. The HW method to trigger INTC IPI can be through external irqchip (e.g. RISC-V AIA), through platform specific device (e.g. SiFive CLINT timer), or through firmware (e.g. SBI IPI call). To support multiple IPIs on RISC-V, we add a generic IPI multiplexing mechanism which help us create multiple virtual IPIs using a single HW IPI. This generic IPI multiplexing is inspired from the Apple AIC irqchip driver and it is shared by various RISC-V irqchip drivers. Signed-off-by: Anup Patel --- include/linux/irq.h | 3 + kernel/irq/Kconfig | 5 ++ kernel/irq/Makefile | 1 + kernel/irq/ipi-mux.c | 193 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 202 insertions(+) create mode 100644 kernel/irq/ipi-mux.c diff --git a/include/linux/irq.h b/include/linux/irq.h index c3eb89606c2b..b1b28affb32a 100644 --- a/include/linux/irq.h +++ b/include/linux/irq.h @@ -1266,6 +1266,9 @@ int __ipi_send_mask(struct irq_desc *desc, const struct cpumask *dest); int ipi_send_single(unsigned int virq, unsigned int cpu); int ipi_send_mask(unsigned int virq, const struct cpumask *dest); +void ipi_mux_process(void); +int ipi_mux_create(unsigned int nr_ipi, void (*mux_send)(unsigned int cpu)); + #ifdef CONFIG_GENERIC_IRQ_MULTI_HANDLER /* * Registers a generic IRQ handling function as the top-level IRQ handler in diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig index db3d174c53d4..df17dbc54b02 100644 --- a/kernel/irq/Kconfig +++ b/kernel/irq/Kconfig @@ -86,6 +86,11 @@ config GENERIC_IRQ_IPI depends on SMP select IRQ_DOMAIN_HIERARCHY +# Generic IRQ IPI Mux support +config GENERIC_IRQ_IPI_MUX + bool + depends on SMP + # Generic MSI interrupt support config GENERIC_MSI_IRQ bool diff --git a/kernel/irq/Makefile b/kernel/irq/Makefile index b4f53717d143..f19d3080bf11 100644 --- a/kernel/irq/Makefile +++ b/kernel/irq/Makefile @@ -15,6 +15,7 @@ obj-$(CONFIG_GENERIC_IRQ_MIGRATION) += cpuhotplug.o obj-$(CONFIG_PM_SLEEP) += pm.o obj-$(CONFIG_GENERIC_MSI_IRQ) += msi.o obj-$(CONFIG_GENERIC_IRQ_IPI) += ipi.o +obj-$(CONFIG_GENERIC_IRQ_IPI_MUX) += ipi-mux.o obj-$(CONFIG_SMP) += affinity.o obj-$(CONFIG_GENERIC_IRQ_DEBUGFS) += debugfs.o obj-$(CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR) += matrix.o diff --git a/kernel/irq/ipi-mux.c b/kernel/irq/ipi-mux.c new file mode 100644 index 000000000000..626c564233e9 --- /dev/null +++ b/kernel/irq/ipi-mux.c @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Multiplex several virtual IPIs over a single HW IPI. + * + * Copyright The Asahi Linux Contributors + * Copyright (c) 2022 Ventana Micro Systems Inc. + */ + +#define pr_fmt(fmt) "ipi-mux: " fmt +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct ipi_mux_cpu { + atomic_t enable; + atomic_t bits; +}; + +static struct ipi_mux_cpu __percpu *ipi_mux_pcpu; +static struct irq_domain *ipi_mux_domain; +static void (*ipi_mux_send)(unsigned int cpu); + +static void ipi_mux_mask(struct irq_data *d) +{ + struct ipi_mux_cpu *icpu = this_cpu_ptr(ipi_mux_pcpu); + + atomic_andnot(BIT(irqd_to_hwirq(d)), &icpu->enable); +} + +static void ipi_mux_unmask(struct irq_data *d) +{ + u32 ibit = BIT(irqd_to_hwirq(d)); + struct ipi_mux_cpu *icpu = this_cpu_ptr(ipi_mux_pcpu); + + atomic_or(ibit, &icpu->enable); + + /* + * The atomic_or() above must complete before the atomic_read() + * below to avoid racing ipi_mux_send_mask(). + */ + smp_mb__after_atomic(); + + /* If a pending IPI was unmasked, raise a parent IPI immediately. */ + if (atomic_read(&icpu->bits) & ibit) + ipi_mux_send(smp_processor_id()); +} + +static void ipi_mux_send_mask(struct irq_data *d, const struct cpumask *mask) +{ + u32 ibit = BIT(irqd_to_hwirq(d)); + struct ipi_mux_cpu *icpu = this_cpu_ptr(ipi_mux_pcpu); + unsigned long pending; + int cpu; + + for_each_cpu(cpu, mask) { + icpu = per_cpu_ptr(ipi_mux_pcpu, cpu); + pending = atomic_fetch_or_release(ibit, &icpu->bits); + + /* + * The atomic_fetch_or_release() above must complete + * before the atomic_read() below to avoid racing with + * ipi_mux_unmask(). + */ + smp_mb__after_atomic(); + + if (!(pending & ibit) && (atomic_read(&icpu->enable) & ibit)) + ipi_mux_send(cpu); + } +} + +static const struct irq_chip ipi_mux_chip = { + .name = "IPI Mux", + .irq_mask = ipi_mux_mask, + .irq_unmask = ipi_mux_unmask, + .ipi_send_mask = ipi_mux_send_mask, +}; + +static int ipi_mux_domain_alloc(struct irq_domain *d, unsigned int virq, + unsigned int nr_irqs, void *arg) +{ + int i; + + for (i = 0; i < nr_irqs; i++) { + irq_set_percpu_devid(virq + i); + irq_domain_set_info(d, virq + i, i, &ipi_mux_chip, NULL, + handle_percpu_devid_irq, NULL, NULL); + } + + return 0; +} + +static const struct irq_domain_ops ipi_mux_domain_ops = { + .alloc = ipi_mux_domain_alloc, + .free = irq_domain_free_irqs_top, +}; + +/** + * ipi_mux_process - Process multiplexed virtual IPIs + */ +void ipi_mux_process(void) +{ + struct ipi_mux_cpu *icpu = this_cpu_ptr(ipi_mux_pcpu); + irq_hw_number_t hwirq; + unsigned long ipis; + unsigned int en; + + /* + * Reading enable mask does not need to be ordered as long as + * this function called from interrupt handler because only + * the CPU itself can change it's own enable mask. + */ + en = atomic_read(&icpu->enable); + + /* + * Clear the IPIs we are about to handle. This pairs with the + * atomic_fetch_or_release() in ipi_mux_send_mask(). + */ + ipis = atomic_fetch_andnot(en, &icpu->bits) & en; + + for_each_set_bit(hwirq, &ipis, BITS_PER_TYPE(int)) + generic_handle_domain_irq(ipi_mux_domain, hwirq); +} + +/** + * ipi_mux_create - Create virtual IPIs multiplexed on top of a single + * parent IPI. + * @nr_ipi: number of virtual IPIs to create. This should + * be <= BITS_PER_TYPE(int) + * @mux_send: callback to trigger parent IPI for a particular CPU + * + * Returns first virq of the newly created virtual IPIs upon success + * or <=0 upon failure + */ +int ipi_mux_create(unsigned int nr_ipi, void (*mux_send)(unsigned int cpu)) +{ + struct fwnode_handle *fwnode; + struct irq_domain *domain; + int rc; + + if (ipi_mux_domain) + return -EEXIST; + + if (BITS_PER_TYPE(int) < nr_ipi || !mux_send) + return -EINVAL; + + ipi_mux_pcpu = alloc_percpu(typeof(*ipi_mux_pcpu)); + if (!ipi_mux_pcpu) + return -ENOMEM; + + fwnode = irq_domain_alloc_named_fwnode("IPI-Mux"); + if (!fwnode) { + pr_err("unable to create IPI Mux fwnode\n"); + rc = -ENOMEM; + goto fail_free_cpu; + } + + domain = irq_domain_create_simple(fwnode, nr_ipi, 0, + &ipi_mux_domain_ops, NULL); + if (!domain) { + pr_err("unable to add IPI Mux domain\n"); + rc = -ENOMEM; + goto fail_free_fwnode; + } + + domain->flags |= IRQ_DOMAIN_FLAG_IPI_SINGLE; + irq_domain_update_bus_token(domain, DOMAIN_BUS_IPI); + + rc = __irq_domain_alloc_irqs(domain, -1, nr_ipi, + NUMA_NO_NODE, NULL, false, NULL); + if (rc <= 0) { + pr_err("unable to alloc IRQs from IPI Mux domain\n"); + goto fail_free_domain; + } + + ipi_mux_domain = domain; + ipi_mux_send = mux_send; + + return rc; + +fail_free_domain: + irq_domain_remove(domain); +fail_free_fwnode: + irq_domain_free_fwnode(fwnode); +fail_free_cpu: + free_percpu(ipi_mux_pcpu); + return rc; +}