From patchwork Mon Jan 16 13:50:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johan Hovold X-Patchwork-Id: 44164 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1202815wrn; Mon, 16 Jan 2023 05:54:17 -0800 (PST) X-Google-Smtp-Source: AMrXdXtvxqVfJfk5hzwID+QUHVeLL4+6i0xI/yrBA6C+MOusc24vZ9b9pANAKOfx18WR0fmPza77 X-Received: by 2002:a17:906:124c:b0:7c0:d60b:2883 with SMTP id u12-20020a170906124c00b007c0d60b2883mr61739902eja.50.1673877257181; Mon, 16 Jan 2023 05:54:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673877257; cv=none; d=google.com; s=arc-20160816; b=vQtQsF9VUhVkiUyMkv9GVl76rxaGFQgha15nJlCjJZomHZ02Om30Nh705rIqL34NBv 58MGNda3t17ozCjeEiLIG85JtAfNLldq51drmYn4vaySRyJCsoCwuu0RTMcIFsG5BJ76 JtdKE3nUmA9g8LlVh8PEyhPiW5c/l7wyk+MoIxhRq/qhwDRI9fc8NAg61S/RPZ86x+lB 0xOu+ne/oRWp/t8o/SEWSpl6Fp/3YtmhiTcFxd+MPGEuOVjzBzDSWVWVbAetI3pIKPGX t+oed438pPhXJHFjmA9+GG8FJ3ADeVpkYVIDxMSRHwqtAjo4rwYDuMT3+T1tayUzxOCB 6JkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+0y6gW5mG7sbBcQ8pHRfwB+ejxB0MOL4pwqjyeqdQWc=; b=QFB/x9G/9EiEECPMnweOlwwgjsVkaNtnqiA8OensK+H1Ab5pWxr0ZbMQ1gzUBkFadN sUbH7TRLq/Di+3/9vxwqiKcOoR2hNEZJKdUpuh6ETM5W/cdWScNUuGVSKJomr7a77YwL 0bA9LFlOFwPa0JNMHeFT4sXkzwSUrLop57LdY+bNiusMydtc5TbfZPegfaLmEpXorXlW VGQ5Cn8EgpKZ8KaToyeC72tDreOSMQf28LvaB9Bvmrtz6hIj5sFgu4zTbYNhw8wV/TUL kvlfZJsDN5Lcp+B23u/6Tz8bC9klZof2fT4g5VZcB3EyWZQ9GuLwQ2ThpE64R0IewHgR KMog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=pziiGpNm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xb13-20020a170907070d00b0078d3b940ec5si32346506ejb.373.2023.01.16.05.53.53; Mon, 16 Jan 2023 05:54:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=pziiGpNm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231184AbjAPNvh (ORCPT + 99 others); Mon, 16 Jan 2023 08:51:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231454AbjAPNup (ORCPT ); Mon, 16 Jan 2023 08:50:45 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAA0D21A0F; Mon, 16 Jan 2023 05:50:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5F0A260FE6; Mon, 16 Jan 2023 13:50:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0ED5CC4339C; Mon, 16 Jan 2023 13:50:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1673877042; bh=CHsjHqlh8sOEcnUziGGttJ51j1NClOatkbqwQ5xcKxw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pziiGpNmWYMO4ElKa8U9ULDlRUp1vp4lHnZX5r/lNxx/KiKH9Wy41kNgwAKRtcCSL 7ny5y7OxduGaV53lUbxVVEqDndg3CA5kvNTbdG8R7/fqZ1RH3JWhaSB5pQEUvLKZ1p pFh/QRvN4mB8IzPwrBu0TPLqueTZ9VRhnfwgnrWJ1oOkBg2ctpaxYk4240jd9QOY9B 2xAP7w9YgBgGVvsq2Rt1v86wq+1V8J4aoIuNZkfC6Dp/FoM0mKLQQQFPb9fbPb9OWl ivYM0K5gSiX8CGG+yMMgJRPp+CJqoO5Uo33hdUlnkgzygxKqAcjUyLlmu9GIuad3ia O3RrDt/z+CbUQ== Received: from johan by xi.lan with local (Exim 4.94.2) (envelope-from ) id 1pHPt7-0003va-5v; Mon, 16 Jan 2023 14:51:01 +0100 From: Johan Hovold To: Marc Zyngier , Thomas Gleixner Cc: x86@kernel.org, platform-driver-x86@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-kernel@vger.kernel.org, Johan Hovold , Hsin-Yi Wang , Mark-PK Tsai Subject: [PATCH v4 19/19] irqdomain: Switch to per-domain locking Date: Mon, 16 Jan 2023 14:50:44 +0100 Message-Id: <20230116135044.14998-20-johan+linaro@kernel.org> X-Mailer: git-send-email 2.38.2 In-Reply-To: <20230116135044.14998-1-johan+linaro@kernel.org> References: <20230116135044.14998-1-johan+linaro@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755187518829020702?= X-GMAIL-MSGID: =?utf-8?q?1755187518829020702?= The IRQ domain structures are currently protected by the global irq_domain_mutex. Switch to using more fine-grained per-domain locking, which may potentially speed up parallel probing somewhat. Note that the domain lock of the root domain (innermost domain) must be used for hierarchical domains. For non-hierarchical domain (as for root domains), the new root pointer is set to the domain itself so that domain->root->mutex can be used in shared code paths. Also note that hierarchical domains should be constructed using irq_domain_create_hierarchy() (or irq_domain_add_hierarchy()) to avoid poking at irqdomain internals. As a safeguard, the lockdep assertion in irq_domain_set_mapping() will catch any offenders that fail to set the root domain pointer. Tested-by: Hsin-Yi Wang Tested-by: Mark-PK Tsai Signed-off-by: Johan Hovold --- include/linux/irqdomain.h | 4 +++ kernel/irq/irqdomain.c | 61 +++++++++++++++++++++++++-------------- 2 files changed, 44 insertions(+), 21 deletions(-) diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h index 7fd3939328c2..b1b06d75d31a 100644 --- a/include/linux/irqdomain.h +++ b/include/linux/irqdomain.h @@ -125,6 +125,8 @@ struct irq_domain_chip_generic; * core code. * @flags: Per irq_domain flags * @mapcount: The number of mapped interrupts + * @mutex: Domain lock, hierarhical domains use root domain's lock + * @root: Pointer to root domain, or containing structure if non-hierarchical * * Optional elements: * @fwnode: Pointer to firmware node associated with the irq_domain. Pretty easy @@ -152,6 +154,8 @@ struct irq_domain { void *host_data; unsigned int flags; unsigned int mapcount; + struct mutex mutex; + struct irq_domain *root; /* Optional data */ struct fwnode_handle *fwnode; diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c index 6f2b8a1248e1..77c31b89740d 100644 --- a/kernel/irq/irqdomain.c +++ b/kernel/irq/irqdomain.c @@ -226,6 +226,17 @@ struct irq_domain *__irq_domain_add(struct fwnode_handle *fwnode, unsigned int s domain->revmap_size = size; + /* + * Hierarchical domains use the domain lock of the root domain + * (innermost domain). + * + * For non-hierarchical domains (as for root domains), the root + * pointer is set to the domain itself so that domain->root->mutex + * can be used in shared code paths. + */ + mutex_init(&domain->mutex); + domain->root = domain; + irq_domain_check_hierarchy(domain); mutex_lock(&irq_domain_mutex); @@ -503,7 +514,7 @@ static bool irq_domain_is_nomap(struct irq_domain *domain) static void irq_domain_clear_mapping(struct irq_domain *domain, irq_hw_number_t hwirq) { - lockdep_assert_held(&irq_domain_mutex); + lockdep_assert_held(&domain->root->mutex); if (irq_domain_is_nomap(domain)) return; @@ -518,7 +529,11 @@ static void irq_domain_set_mapping(struct irq_domain *domain, irq_hw_number_t hwirq, struct irq_data *irq_data) { - lockdep_assert_held(&irq_domain_mutex); + /* + * This also makes sure that all domains point to the same root when + * called from irq_domain_insert_irq() for each domain in a hierarchy. + */ + lockdep_assert_held(&domain->root->mutex); if (irq_domain_is_nomap(domain)) return; @@ -540,7 +555,7 @@ static void irq_domain_disassociate(struct irq_domain *domain, unsigned int irq) hwirq = irq_data->hwirq; - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->mutex); irq_set_status_flags(irq, IRQ_NOREQUEST); @@ -562,7 +577,7 @@ static void irq_domain_disassociate(struct irq_domain *domain, unsigned int irq) /* Clear reverse map for this hwirq */ irq_domain_clear_mapping(domain, hwirq); - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->mutex); } static int __irq_domain_associate(struct irq_domain *domain, unsigned int virq, @@ -612,9 +627,9 @@ int irq_domain_associate(struct irq_domain *domain, unsigned int virq, { int ret; - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->mutex); ret = __irq_domain_associate(domain, virq, hwirq); - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->mutex); return ret; } @@ -731,7 +746,7 @@ unsigned int irq_create_mapping_affinity(struct irq_domain *domain, return 0; } - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->mutex); /* Check if mapping already exists */ virq = irq_find_mapping(domain, hwirq); @@ -742,7 +757,7 @@ unsigned int irq_create_mapping_affinity(struct irq_domain *domain, virq = __irq_create_mapping_affinity(domain, hwirq, affinity); out: - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->mutex); return virq; } @@ -811,7 +826,7 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec) if (WARN_ON(type & ~IRQ_TYPE_SENSE_MASK)) type &= IRQ_TYPE_SENSE_MASK; - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->root->mutex); /* * If we've already configured this interrupt, @@ -864,11 +879,11 @@ unsigned int irq_create_fwspec_mapping(struct irq_fwspec *fwspec) /* Store trigger type */ irqd_set_trigger_type(irq_data, type); out: - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->root->mutex); return virq; err: - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->root->mutex); return 0; } @@ -1132,6 +1147,7 @@ struct irq_domain *irq_domain_create_hierarchy(struct irq_domain *parent, else domain = irq_domain_create_tree(fwnode, ops, host_data); if (domain) { + domain->root = parent->root; domain->parent = parent; domain->flags |= flags; } @@ -1528,10 +1544,10 @@ int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base, return -EINVAL; } - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->root->mutex); ret = ___irq_domain_alloc_irqs(domain, irq_base, nr_irqs, node, arg, realloc, affinity); - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->root->mutex); return ret; } @@ -1542,7 +1558,7 @@ static void irq_domain_fix_revmap(struct irq_data *d) { void __rcu **slot; - lockdep_assert_held(&irq_domain_mutex); + lockdep_assert_held(&d->domain->root->mutex); if (irq_domain_is_nomap(d->domain)) return; @@ -1608,7 +1624,7 @@ int irq_domain_push_irq(struct irq_domain *domain, int virq, void *arg) if (!parent_irq_data) return -ENOMEM; - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->root->mutex); /* Copy the original irq_data. */ *parent_irq_data = *irq_data; @@ -1636,7 +1652,7 @@ int irq_domain_push_irq(struct irq_domain *domain, int virq, void *arg) irq_domain_fix_revmap(parent_irq_data); irq_domain_set_mapping(domain, irq_data->hwirq, irq_data); error: - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->root->mutex); return rv; } @@ -1691,7 +1707,7 @@ int irq_domain_pop_irq(struct irq_domain *domain, int virq) if (WARN_ON(!parent_irq_data)) return -EINVAL; - mutex_lock(&irq_domain_mutex); + mutex_lock(&domain->root->mutex); irq_data->parent_data = NULL; @@ -1703,7 +1719,7 @@ int irq_domain_pop_irq(struct irq_domain *domain, int virq) irq_domain_fix_revmap(irq_data); - mutex_unlock(&irq_domain_mutex); + mutex_unlock(&domain->root->mutex); kfree(parent_irq_data); @@ -1719,17 +1735,20 @@ EXPORT_SYMBOL_GPL(irq_domain_pop_irq); void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs) { struct irq_data *data = irq_get_irq_data(virq); + struct irq_domain *domain; int i; if (WARN(!data || !data->domain || !data->domain->ops->free, "NULL pointer, cannot free irq\n")) return; - mutex_lock(&irq_domain_mutex); + domain = data->domain; + + mutex_lock(&domain->root->mutex); for (i = 0; i < nr_irqs; i++) irq_domain_remove_irq(virq + i); - irq_domain_free_irqs_hierarchy(data->domain, virq, nr_irqs); - mutex_unlock(&irq_domain_mutex); + irq_domain_free_irqs_hierarchy(domain, virq, nr_irqs); + mutex_unlock(&domain->root->mutex); irq_domain_free_irq_data(virq, nr_irqs); irq_free_descs(virq, nr_irqs);