From patchwork Mon Nov 21 14:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 23799 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1626897wrr; Mon, 21 Nov 2022 06:38:29 -0800 (PST) X-Google-Smtp-Source: AA0mqf6DM2PDiva/CCss9NEdmkYx1gx29HGUAAztEgCo3Wlajz9jDi/xSNNYVqL49ZprFrJV/lP5 X-Received: by 2002:a17:90a:760f:b0:218:bf69:a7b2 with SMTP id s15-20020a17090a760f00b00218bf69a7b2mr2595494pjk.106.1669041509025; Mon, 21 Nov 2022 06:38:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669041509; cv=none; d=google.com; s=arc-20160816; b=guFqw3TicIC/8NAjb+RpTurnHkt+T9W3wsGHqPlz+p6ZxTT/sJIpQfMTdcyWLShDjM JaW0yUB6AI7aAlG6se9lDBxgVYeTqwAU8L1j/RTdWwJO+mWjKe+oqAh9jsn1QR1VfD/u uBb4DTyJ7p12oRcT+QIk+ficO5xn98kT72d4iHRpwjoSvM025cxNTiWGE+so0HjY/qZU /Nih9RZaw7osAyB8ibgchWnq4Qwq/UZ4gtn35Dc4C70iY8JqLjGnE1+UC6ylUdPkAx0a HMCeBNtDEMwbnJzW8F2+aSqxvKDLkwinXHgVclMoRay+LM1NbZ7ONkfuWT/IgXiZJlRL 24jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=FZrl2FVP1hN+AFQ07RZcCo5g9uO23ic8F3HA5Y+tWWI=; b=gSprzHhwtRj9fb+GzXogedU1EUnXwDDBFFzC2xZJ6IUgkCgcqtWFAOhSk3gYBJ8RoB c4B+wBq/j+wdG0lXeyJ/f0Nq/V3mmOIx4d62d+6JZHNLXiXtI4rWku6642syk7O+cR4b P/I9cs2EGWSMtikku2tEEoBhll/f1cLl6GiDH+fGqyZUheW5ZJF6bDMEGsOiM3DLUYMm OwjXZhb+45mRUPjqb+Ax2WAssc1iAc9XJi/FhxrFkuf74wqifkbEDusqfZCEpEM6Q5gj 7PoOBnyLfOM1SNrW5D4fAHE0SwLuX7+klDCkwlvZoUCuda1GgT6nDp1zo7PT/xGSsZV3 sIBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=hPqYNF2D; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h11-20020a056a00230b00b00562617987a1si11079827pfh.264.2022.11.21.06.38.12; Mon, 21 Nov 2022 06:38:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=hPqYNF2D; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231597AbiKUOhj (ORCPT + 99 others); Mon, 21 Nov 2022 09:37:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230286AbiKUOgl (ORCPT ); Mon, 21 Nov 2022 09:36:41 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 356DFC6D21; Mon, 21 Nov 2022 06:36:31 -0800 (PST) Message-ID: <20221121083326.002153609@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669041389; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FZrl2FVP1hN+AFQ07RZcCo5g9uO23ic8F3HA5Y+tWWI=; b=hPqYNF2DsT62d8jFpqufSgUXB2xq4BHYK46LQXLs8LvhhMRZmJxDY1OoVVTvaVjoqWpZ0Y ThYxzYkV8YhufbGeDBZffz4iA2L12l5gXZPwYrmsxoxohz4UUm8geUVp9ioqOxZih0RIbT mvmwSmS7Ny4e22DTMKGbYJd2ZrHDYKb00OZgUogfOGyZPJSsoyVnjLtFvS48AjwAGCn4RY I8bV2OLSVL0UBv1sbVQKa+F4e9uthLGMV1eWASwZ8p6OV/vb0vGMSHly5fy2J7TF/LRf2o 6LUKpPaCIRZ/K9eiUV+IE+R+1CrU6lsEFjJQIimVhgJcmq+0m46oyb5i2wAl6g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669041389; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=FZrl2FVP1hN+AFQ07RZcCo5g9uO23ic8F3HA5Y+tWWI=; b=o0e/ebYAoMgZUK81TTG6lP8jMxcUsK/2dHJpOIeH/cIYMCTmlye3uklyty7gZi7+LyjxqI Be4IPoUPOZGCGKDA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V2 09/21] genirq/msi: Make MSI descriptor iterators device domain aware References: <20221121083210.309161925@linutronix.de> MIME-Version: 1.0 Date: Mon, 21 Nov 2022 15:36:29 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750116869214756286?= X-GMAIL-MSGID: =?utf-8?q?1750116869214756286?= To support multiple MSI interrupt domains per device it is necessary to segment the xarray MSI descriptor storage. Each domain gets up to MSI_MAX_INDEX entries. Change the iterators so they operate with domain ids and take the domain offsets into account. The publicly available iterators which are mostly used in legacy implementations and the PCI/MSI core default to MSI_DEFAULT_DOMAIN (0) which is the id for the existing "global" domains. No functional change. Signed-off-by: Thomas Gleixner --- V2: Fix the off by one so the index space is including MSI_MAX_INDEX (Kevin) --- include/linux/msi.h | 45 +++++++++++++++++++++++++++++++++++++++++---- kernel/irq/msi.c | 43 +++++++++++++++++++++++++++++++++++-------- 2 files changed, 76 insertions(+), 12 deletions(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -181,6 +181,7 @@ enum msi_desc_filter { * @mutex: Mutex protecting the MSI descriptor store * @__store: Xarray for storing MSI descriptor pointers * @__iter_idx: Index to search the next entry for iterators + * @__iter_max: Index to limit the search * @__irqdomains: Per device interrupt domains */ struct msi_device_data { @@ -189,6 +190,7 @@ struct msi_device_data { struct mutex mutex; struct xarray __store; unsigned long __iter_idx; + unsigned long __iter_max; struct irq_domain *__irqdomains[MSI_MAX_DEVICE_IRQDOMAINS]; }; @@ -197,14 +199,34 @@ int msi_setup_device_data(struct device void msi_lock_descs(struct device *dev); void msi_unlock_descs(struct device *dev); -struct msi_desc *msi_first_desc(struct device *dev, enum msi_desc_filter filter); +struct msi_desc *msi_domain_first_desc(struct device *dev, unsigned int domid, + enum msi_desc_filter filter); + +/** + * msi_first_desc - Get the first MSI descriptor of the default irqdomain + * @dev: Device to operate on + * @filter: Descriptor state filter + * + * Must be called with the MSI descriptor mutex held, i.e. msi_lock_descs() + * must be invoked before the call. + * + * Return: Pointer to the first MSI descriptor matching the search + * criteria, NULL if none found. + */ +static inline struct msi_desc *msi_first_desc(struct device *dev, + enum msi_desc_filter filter) +{ + return msi_domain_first_desc(dev, MSI_DEFAULT_DOMAIN, filter); +} + struct msi_desc *msi_next_desc(struct device *dev, enum msi_desc_filter filter); /** - * msi_for_each_desc - Iterate the MSI descriptors + * msi_domain_for_each_desc - Iterate the MSI descriptors in a specific domain * * @desc: struct msi_desc pointer used as iterator * @dev: struct device pointer - device to iterate + * @domid: The id of the interrupt domain which should be walked. * @filter: Filter for descriptor selection * * Notes: @@ -212,10 +234,25 @@ struct msi_desc *msi_next_desc(struct de * pair. * - It is safe to remove a retrieved MSI descriptor in the loop. */ -#define msi_for_each_desc(desc, dev, filter) \ - for ((desc) = msi_first_desc((dev), (filter)); (desc); \ +#define msi_domain_for_each_desc(desc, dev, domid, filter) \ + for ((desc) = msi_domain_first_desc((dev), (domid), (filter)); (desc); \ (desc) = msi_next_desc((dev), (filter))) +/** + * msi_for_each_desc - Iterate the MSI descriptors in the default irqdomain + * + * @desc: struct msi_desc pointer used as iterator + * @dev: struct device pointer - device to iterate + * @filter: Filter for descriptor selection + * + * Notes: + * - The loop must be protected with a msi_lock_descs()/msi_unlock_descs() + * pair. + * - It is safe to remove a retrieved MSI descriptor in the loop. + */ +#define msi_for_each_desc(desc, dev, filter) \ + msi_domain_for_each_desc((desc), (dev), MSI_DEFAULT_DOMAIN, (filter)) + #define msi_desc_to_dev(desc) ((desc)->dev) #ifdef CONFIG_IRQ_MSI_IOMMU --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -21,6 +21,10 @@ static inline int msi_sysfs_create_group(struct device *dev); +/* Invalid XA index which is outside of any searchable range */ +#define MSI_XA_MAX_INDEX (ULONG_MAX - 1) +#define MSI_XA_DOMAIN_SIZE (MSI_MAX_INDEX + 1) + static inline void msi_setup_default_irqdomain(struct device *dev, struct msi_device_data *md) { if (!dev->msi.domain) @@ -33,6 +37,20 @@ static inline void msi_setup_default_irq md->__irqdomains[MSI_DEFAULT_DOMAIN] = dev->msi.domain; } +static int msi_get_domain_base_index(struct device *dev, unsigned int domid) +{ + lockdep_assert_held(&dev->msi.data->mutex); + + if (WARN_ON_ONCE(domid >= MSI_MAX_DEVICE_IRQDOMAINS)) + return -ENODEV; + + if (WARN_ON_ONCE(!dev->msi.data->__irqdomains[domid])) + return -ENODEV; + + return domid * MSI_XA_DOMAIN_SIZE; +} + + /** * msi_alloc_desc - Allocate an initialized msi_desc * @dev: Pointer to the device for which this is allocated @@ -229,6 +247,7 @@ int msi_setup_device_data(struct device xa_init(&md->__store); mutex_init(&md->mutex); + md->__iter_idx = MSI_XA_MAX_INDEX; dev->msi.data = md; devres_add(dev, md); return 0; @@ -251,7 +270,7 @@ EXPORT_SYMBOL_GPL(msi_lock_descs); void msi_unlock_descs(struct device *dev) { /* Invalidate the index wich was cached by the iterator */ - dev->msi.data->__iter_idx = MSI_MAX_INDEX; + dev->msi.data->__iter_idx = MSI_XA_MAX_INDEX; mutex_unlock(&dev->msi.data->mutex); } EXPORT_SYMBOL_GPL(msi_unlock_descs); @@ -260,17 +279,18 @@ static struct msi_desc *msi_find_desc(st { struct msi_desc *desc; - xa_for_each_start(&md->__store, md->__iter_idx, desc, md->__iter_idx) { + xa_for_each_range(&md->__store, md->__iter_idx, desc, md->__iter_idx, md->__iter_max) { if (msi_desc_match(desc, filter)) return desc; } - md->__iter_idx = MSI_MAX_INDEX; + md->__iter_idx = MSI_XA_MAX_INDEX; return NULL; } /** - * msi_first_desc - Get the first MSI descriptor of a device + * msi_domain_first_desc - Get the first MSI descriptor of an irqdomain associated to a device * @dev: Device to operate on + * @domid: The id of the interrupt domain which should be walked. * @filter: Descriptor state filter * * Must be called with the MSI descriptor mutex held, i.e. msi_lock_descs() @@ -279,19 +299,26 @@ static struct msi_desc *msi_find_desc(st * Return: Pointer to the first MSI descriptor matching the search * criteria, NULL if none found. */ -struct msi_desc *msi_first_desc(struct device *dev, enum msi_desc_filter filter) +struct msi_desc *msi_domain_first_desc(struct device *dev, unsigned int domid, + enum msi_desc_filter filter) { struct msi_device_data *md = dev->msi.data; + int baseidx; if (WARN_ON_ONCE(!md)) return NULL; lockdep_assert_held(&md->mutex); - md->__iter_idx = 0; + baseidx = msi_get_domain_base_index(dev, domid); + if (baseidx < 0) + return NULL; + + md->__iter_idx = baseidx; + md->__iter_max = baseidx + MSI_MAX_INDEX; return msi_find_desc(md, filter); } -EXPORT_SYMBOL_GPL(msi_first_desc); +EXPORT_SYMBOL_GPL(msi_domain_first_desc); /** * msi_next_desc - Get the next MSI descriptor of a device @@ -315,7 +342,7 @@ struct msi_desc *msi_next_desc(struct de lockdep_assert_held(&md->mutex); - if (md->__iter_idx >= (unsigned long)MSI_MAX_INDEX) + if (md->__iter_idx >= md->__iter_max) return NULL; md->__iter_idx++;