From patchwork Thu Nov 24 23:25:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25742 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3660494wrr; Thu, 24 Nov 2022 15:28:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf5HgeLTLSkocpcODzYn0YaqMmIo72oX3AW8UOZpE668ieaE7j4i0UvI2nVdRN2DjqWERVCG X-Received: by 2002:a05:6402:248e:b0:461:e2ab:912d with SMTP id q14-20020a056402248e00b00461e2ab912dmr15977369eda.93.1669332495421; Thu, 24 Nov 2022 15:28:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332495; cv=none; d=google.com; s=arc-20160816; b=Zqofpypbp4GJ984uXs8W1F9+y/mXN68WwWg1ZxXZdrHg6owzuFlVCah/mBfAO/mA0O 9g8RLg/1zWxnASkBDULU40G1Lvyw/9wtTeokH7woSesgf/rIJHBdzV9KPhove2YRIohD FzTMtahXqJmuuHTm0yABmXlAxBc/o9J4pz3hLYm4oAxxQntRLY39mZP/fmB7i1KAAl/c fXSDFsh/XJ0Tf0EdCxFQ26gcMlOmZh71bicf7FPqLvzgn3c3cyMz8h9HE6Jg0aUi/RzV xlFNIFsE30F9mmLmMQNIdNIVOvUchLrd1bTrgiZVZwqZoJCY5iPvxH6DWZB6H+B5BLJH mrqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=+NyMkWp2GrUOU3BDiTzPL4SzyzA5986NyW2OtHWo5NY=; b=KWKKz7el1jFo1LFH832b44JbkG4Flx9uyfFmmk9xpkzmEA8Lai4Z+pJoqlVr5WH6bv 6LDY3W4jyC0NXZvnNBht2LD6aR2sURfvt9ry4FnZwqunlHd5ReL/C7MSQa5owbftvhF7 c9GxQPqGt8x0v9qYMI1QvFJiUtAaFPbjbM5GZQyuB8Ei7Yp/vixyMQZtFDb9h01YZ42C 5C6M84KKzFKpAgdvZdCglWlJRGupZ/ZjhLhlElGUdIZ2wfE7k3rp4jbz1wjtSpNlimDp fRf3NeWDuhJVFYHjaP7xdy8sKFApeZRRYapeRXjYNH7M+8xQw4iHa49xHd2b5oRsjzCP FXZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=wrPZ5arX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id di20-20020a170906731400b007adb2862222si1602162ejc.828.2022.11.24.15.27.52; Thu, 24 Nov 2022 15:28:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=wrPZ5arX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229487AbiKXX1Q (ORCPT + 99 others); Thu, 24 Nov 2022 18:27:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229910AbiKXX07 (ORCPT ); Thu, 24 Nov 2022 18:26:59 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96A9DC6D1E; Thu, 24 Nov 2022 15:25:49 -0800 (PST) Message-ID: <20221124232325.322714918@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=+NyMkWp2GrUOU3BDiTzPL4SzyzA5986NyW2OtHWo5NY=; b=wrPZ5arXlEuMY3dc+oI+StHT4KdlWJKlCvS3aFKiaaafmyjqdWa8s67KwfOU4A6TNXY/mN Uupdh56kZux2e6Msz+cNT1zMAppfiMea1cDLszyduul4m/glJImW++N37CVo6Ym7ZUGMHR +NMszj6yOTvGHR5k/vHllq1eqfxF+rscjSL92mxTUaOZzZJz0jN23aPNrsfz8Xa8QRxCMI 09DW7Q6DVXrYxbDJbynQLmTGzvB9pDkSEG9hUqwNnJTd6fNqu3ORSDXdi8ecLjxwDyFZVh Ui8mIspY3J1GP4TfGkO6qGCnGqzaVtd3X3jvlWN8ZZ3LBbpcadDledPxEGd3Rg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332347; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=+NyMkWp2GrUOU3BDiTzPL4SzyzA5986NyW2OtHWo5NY=; b=3If8dJ4/lkxNmnocohjV1Y3E1AJKtKtRh1sYRx2I5xdpITvzwfmgemEuM32+O4bMazHIhf nG65hDHwUJpLhQBQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , Jason Gunthorpe Subject: [patch V3 01/33] genirq/msi: Rearrange MSI domain flags References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:46 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750421991163870589?= X-GMAIL-MSGID: =?utf-8?q?1750421991163870589?= These flags got added as necessary and have no obvious structure. For feature support checks and masking it's convenient to have two blocks of flags: 1) Flags to control the internal behaviour like allocating/freeing MSI descriptors. Those flags do not need any support from the underlying MSI parent domain. They are mostly under the control of the outermost domain which implements the actual MSI support. 2) Flags to expose features, e.g. PCI multi-MSI or requirements which can depend on a underlying domain. No functional change. Signed-off-by: Thomas Gleixner Reviewed-by: Jason Gunthorpe --- include/linux/msi.h | 49 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 15 deletions(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -24,6 +24,8 @@ #include #include #include +#include + #include /* Dummy shadow structures if an architecture does not define them */ @@ -440,7 +442,16 @@ struct msi_domain_info { void *data; }; -/* Flags for msi_domain_info */ +/* + * Flags for msi_domain_info + * + * Bit 0-15: Generic MSI functionality which is not subject to restriction + * by parent domains + * + * Bit 16-31: Functionality which depends on the underlying parent domain and + * can be masked out by msi_parent_ops::init_dev_msi_info() when + * a device MSI domain is initialized. + */ enum { /* * Init non implemented ops callbacks with default MSI domain @@ -452,33 +463,41 @@ enum { * callbacks. */ MSI_FLAG_USE_DEF_CHIP_OPS = (1 << 1), - /* Support multiple PCI MSI interrupts */ - MSI_FLAG_MULTI_PCI_MSI = (1 << 2), - /* Support PCI MSIX interrupts */ - MSI_FLAG_PCI_MSIX = (1 << 3), /* Needs early activate, required for PCI */ - MSI_FLAG_ACTIVATE_EARLY = (1 << 4), + MSI_FLAG_ACTIVATE_EARLY = (1 << 2), /* * Must reactivate when irq is started even when * MSI_FLAG_ACTIVATE_EARLY has been set. */ - MSI_FLAG_MUST_REACTIVATE = (1 << 5), - /* Is level-triggered capable, using two messages */ - MSI_FLAG_LEVEL_CAPABLE = (1 << 6), + MSI_FLAG_MUST_REACTIVATE = (1 << 3), /* Populate sysfs on alloc() and destroy it on free() */ - MSI_FLAG_DEV_SYSFS = (1 << 7), - /* MSI-X entries must be contiguous */ - MSI_FLAG_MSIX_CONTIGUOUS = (1 << 8), + MSI_FLAG_DEV_SYSFS = (1 << 4), /* Allocate simple MSI descriptors */ - MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS = (1 << 9), + MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS = (1 << 5), /* Free MSI descriptors */ - MSI_FLAG_FREE_MSI_DESCS = (1 << 10), + MSI_FLAG_FREE_MSI_DESCS = (1 << 6), /* * Quirk to handle MSI implementations which do not provide * masking. Currently known to affect x86, but has to be partially * handled in the core MSI code. */ - MSI_FLAG_NOMASK_QUIRK = (1 << 11), + MSI_FLAG_NOMASK_QUIRK = (1 << 7), + + /* Mask for the generic functionality */ + MSI_GENERIC_FLAGS_MASK = GENMASK(15, 0), + + /* Mask for the domain specific functionality */ + MSI_DOMAIN_FLAGS_MASK = GENMASK(31, 16), + + /* Support multiple PCI MSI interrupts */ + MSI_FLAG_MULTI_PCI_MSI = (1 << 16), + /* Support PCI MSIX interrupts */ + MSI_FLAG_PCI_MSIX = (1 << 17), + /* Is level-triggered capable, using two messages */ + MSI_FLAG_LEVEL_CAPABLE = (1 << 18), + /* MSI-X entries must be contiguous */ + MSI_FLAG_MSIX_CONTIGUOUS = (1 << 19), + }; int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask, From patchwork Thu Nov 24 23:25:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25739 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3660455wrr; Thu, 24 Nov 2022 15:28:10 -0800 (PST) X-Google-Smtp-Source: AA0mqf5niZ3HYtuZ2FdFc1WoFXx05mxcdoxcAlDN4ByjgCIjDmHAaYDC1ETmifDDXnWS7cVpqAkR X-Received: by 2002:a17:90b:3d8a:b0:20c:4f5b:f6cb with SMTP id pq10-20020a17090b3d8a00b0020c4f5bf6cbmr43740948pjb.153.1669332490573; Thu, 24 Nov 2022 15:28:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332490; cv=none; d=google.com; s=arc-20160816; b=Kh/jVTMgwvWNps8WlhSbUXkjaZw7DPxVFnBVAgr/muxeASdanJNfcdpMm3i+Z0vi3T y/CwcEWrjMq1ZFsvTZ2wHZ/iPGOIsOLiProkht74OWMRgWouMRb+OzCbDCBe8Bc881fT wfQTGD6A8Zdv2xctsgDdn1KCPpNfLupNNfsCN9K70TJTnNwcy1Lmhqgod0UqcfEjwtGP UEAt0WFRG0yy4p4NDrmNJBgsjEb39mjVaW6Gv58XKK3myQRmOiQOM432gtptCpofRk3V 0/456NMvtQF8EBmzu5Xh/ZDIx3RcnWIVUwXf9u4MUDPJSSvc/AlG9J4OsWv70RwbQej2 TcTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=LpT5bWEZO8pdMpxaAOZXaaWsLza6Gnp607G1pMhReoc=; b=jRjR54w//dg/aEom8hZy6zmzkt+dUdEx9sGAXh3j4GEEHiIiVXa1bf6Cb7O443zP5g 8fjG6b7oq5XczPPcQXqY72zuxUShmnz3phwFBFiRP+UNYv5xPZLN7R0nT/npcvS8YPQb LxgnyhYAOPI1wjNLsUmopGTScJjPRMDJuCspk2XoDIdkFWfNH0Fupl39JZ1s324W7wyA rE5Hu8vuJMzrOmtK906pw3A3hcUwnaYy8KK1rnKxz1Wt+zKb5KIMQNXT0+u2/3ocCxww rPvuKyV2VZj7BejiDFpyNX7QjBQ2NL5qq3XIMpn9tj/RXoacqtnxnfFhpeyk6NhI4/a5 hL/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=0MH4NHb6; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a37-20020a631a65000000b0046141525a73si1909810pgm.464.2022.11.24.15.27.57; Thu, 24 Nov 2022 15:28:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=0MH4NHb6; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229916AbiKXX1T (ORCPT + 99 others); Thu, 24 Nov 2022 18:27:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229875AbiKXX1B (ORCPT ); Thu, 24 Nov 2022 18:27:01 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE316C6961; Thu, 24 Nov 2022 15:25:51 -0800 (PST) Message-ID: <20221124232325.382485843@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=LpT5bWEZO8pdMpxaAOZXaaWsLza6Gnp607G1pMhReoc=; b=0MH4NHb6VIKYBiir3GhyqPzfF6YTH4g0U3Bv7+hXm0Aa/VCHHzSoNKmN1gru5H9XsAq1M9 y4sQo2/0VoTey1MzXhUumLjoj0sN+DVmnmnfRJ5cy9op3P/aofDx5Rx492E+ySpnCnSHOz w9VGgP+x0jvub7l+cZY1xWeovnUxJPOZ0ZM0sHRxHEX/VRVLBRMCgIREEVq10BD/IGj67G NP5l049cYFy9guiI7zPZMVgTEPlkw1yaJicKyL3gQxyOadRi9nyumSwl8uqGElMzwR0djq FM1TqEVdf4ldjacsepXtBHBrBL8qjOFR1Ds9Eitoux9yDrpuqhcGM8NX/ErAlA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=LpT5bWEZO8pdMpxaAOZXaaWsLza6Gnp607G1pMhReoc=; b=fXn8YaoJHN3fPzAHifqZIeCwtmyM9g+FF7JclrWARpdUgCDf5vudSvYFSwi98Q2eVs1LuJ hZv9P0rZKQeV+LAQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 02/33] genirq/msi: Provide struct msi_parent_ops References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:48 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750421985780164092?= X-GMAIL-MSGID: =?utf-8?q?1750421985780164092?= MSI parent domains must have some control over the MSI domains which are built on top. On domain creation they need to fill in e.g. architecture specific chip callbacks or msi domain ops to make the outermost domain parent agnostic which is obviously required for architecture independence etc. The structure contains: 1) A bitfield which exposes the supported functional features. This allows to check for features and is also used in the initialization callback to mask out unsupported features when the actual domain implementation requests a broader range, e.g. on x86 PCI multi-MSI is only supported by remapping domains but not by the underlying vector domain. The PCI/MSI code can then always request multi-MSI support, but the resulting feature set after creation might not have it set. 2) An optional string prefix which is put in front of domain and chip names during creation of the MSI domain. That allows to keep the naming schemes e.g. on x86 where PCI-MSI domains have a IR- prefix when interrupt remapping is enabled. 3) An initialization callback to sanity check the domain info of the to be created MSI domain, to restrict features and to apply changes in MSI ops and interrupt chip callbacks to accomodate to the particular MSI parent implementation and/or the underlying hierarchy. Add a conveniance function to delegate the initialization from the MSI parent domain to an underlying domain in the hierarchy. Signed-off-by: Thomas Gleixner --- V2: Renamed arguments and updated comments (Jason) --- include/linux/irqdomain.h | 5 +++++ include/linux/msi.h | 21 +++++++++++++++++++++ kernel/irq/msi.c | 41 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+) --- a/include/linux/irqdomain.h +++ b/include/linux/irqdomain.h @@ -46,6 +46,7 @@ struct irq_desc; struct cpumask; struct seq_file; struct irq_affinity_desc; +struct msi_parent_ops; #define IRQ_DOMAIN_IRQ_SPEC_PARAMS 16 @@ -134,6 +135,7 @@ struct irq_domain_chip_generic; * @pm_dev: Pointer to a device that can be utilized for power management * purposes related to the irq domain. * @parent: Pointer to parent irq_domain to support hierarchy irq_domains + * @msi_parent_ops: Pointer to MSI parent domain methods for per device domain init * * Revmap data, used internally by the irq domain code: * @revmap_size: Size of the linear map table @revmap[] @@ -157,6 +159,9 @@ struct irq_domain { #ifdef CONFIG_IRQ_DOMAIN_HIERARCHY struct irq_domain *parent; #endif +#ifdef CONFIG_GENERIC_MSI_IRQ + const struct msi_parent_ops *msi_parent_ops; +#endif /* reverse map data. The linear map gets appended to the irq_domain */ irq_hw_number_t hwirq_max; --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -500,6 +500,27 @@ enum { }; +/** + * struct msi_parent_ops - MSI parent domain callbacks and configuration info + * + * @supported_flags: Required: The supported MSI flags of the parent domain + * @prefix: Optional: Prefix for the domain and chip name + * @init_dev_msi_info: Required: Callback for MSI parent domains to setup parent + * domain specific domain flags, domain ops and interrupt chip + * callbacks when a per device domain is created. + */ +struct msi_parent_ops { + u32 supported_flags; + const char *prefix; + bool (*init_dev_msi_info)(struct device *dev, struct irq_domain *domain, + struct irq_domain *msi_parent_domain, + struct msi_domain_info *msi_child_info); +}; + +bool msi_parent_init_dev_msi_info(struct device *dev, struct irq_domain *domain, + struct irq_domain *msi_parent_domain, + struct msi_domain_info *msi_child_info); + int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask, bool force); --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -789,6 +789,47 @@ struct irq_domain *msi_create_irq_domain return domain; } +/** + * msi_parent_init_dev_msi_info - Delegate initialization of device MSI info down + * in the domain hierarchy + * @dev: The device for which the domain should be created + * @domain: The domain in the hierarchy this op is being called on + * @msi_parent_domain: The IRQ_DOMAIN_FLAG_MSI_PARENT domain for the child to + * be created + * @msi_child_info: The MSI domain info of the IRQ_DOMAIN_FLAG_MSI_DEVICE + * domain to be created + * + * Return: true on success, false otherwise + * + * This is the most complex problem of per device MSI domains and the + * underlying interrupt domain hierarchy: + * + * The device domain to be initialized requests the broadest feature set + * possible and the underlying domain hierarchy puts restrictions on it. + * + * That's trivial for a simple parent->child relationship, but it gets + * interesting with an intermediate domain: root->parent->child. The + * intermediate 'parent' can expand the capabilities which the 'root' + * domain is providing. So that creates a classic hen and egg problem: + * Which entity is doing the restrictions/expansions? + * + * One solution is to let the root domain handle the initialization that's + * why there is the @domain and the @msi_parent_domain pointer. + */ +bool msi_parent_init_dev_msi_info(struct device *dev, struct irq_domain *domain, + struct irq_domain *msi_parent_domain, + struct msi_domain_info *msi_child_info) +{ + struct irq_domain *parent = domain->parent; + + if (WARN_ON_ONCE(!parent || !parent->msi_parent_ops || + !parent->msi_parent_ops->init_dev_msi_info)) + return false; + + return parent->msi_parent_ops->init_dev_msi_info(dev, parent, msi_parent_domain, + msi_child_info); +} + int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) { From patchwork Thu Nov 24 23:25:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25746 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661334wrr; Thu, 24 Nov 2022 15:30:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf6K+yWJzOAoNhOClg+bxvVy2j1a45vQMqVJFCJEGqjPvNgUVT+JN+csL+07Jo4A11ZTdSKh X-Received: by 2002:a05:6402:5305:b0:467:69e3:c25b with SMTP id eo5-20020a056402530500b0046769e3c25bmr31804152edb.3.1669332636347; Thu, 24 Nov 2022 15:30:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332636; cv=none; d=google.com; s=arc-20160816; b=b8Nnz8/Bzt89JhWcs8AwxSYR/puSPamigf4VWqWkDogeq0nRxEYqYY0TmHM1ceV7PL Ib7io1gGmlnV1CsgyNDEVbryZa5p9DBQZtt3/QZmImvpytmB0DFZLleoODbN+PVX5z4k iQQFkN7tX51xaEX6sal7gFrcdKtn6JbCQ3EhlO0ch8TEQ2/ua/admYSRAePvI3/7xLpG oKrf8dNT88AAjicRDBDqvQ04+VMS4URKfTaOrWiczFcZzOHGVhB22qkpYbhi2N+A+ZfH 4ObRGu7Dv51s+5rYomI2r93HD9dpAEb7TSpjpur1Qpqwj4amJZaRhdkMghryqZGuUeDv IkgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=qV//gzPmOqH3W5duAbtHBZqaagM7sPrAqRRWld0OnjU=; b=sAB1krvzBbJIIDTI20tbBIAWj8JUMHdzbhJrjdF6qBUxx3Wd/w2Ah/vEp8hIepaDWR /7Wt9CYhORILl4QvgzKcG84BXI8Ssn29G5OhyTft6Eg2u8v7fjMm3b3aMUXRWRViUnyO 2qb5xvLcEK+jkJRvbwhwPHtTNFgREmRuhNUYPpcCQcBMNBvBSFwESHXLK6nSt1LlaCwG 5shgX3JOiHZ9TiDw5k97I+Wm1Gg2Dziw+2uziD0/rJyZ9Ajir9VpwhaGx4mcq6FUM+hk MQy1sx8aLU+cNHAQ0P0yewR7lawCMgLX7t4dA9QZwak9uC4lhndPafgxJFDMIG7xopio JEEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=HEs+D51n; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=oMDZPkI4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s25-20020aa7c559000000b00469634340b4si1902370edr.285.2022.11.24.15.30.12; Thu, 24 Nov 2022 15:30:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=HEs+D51n; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=oMDZPkI4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229802AbiKXX1v (ORCPT + 99 others); Thu, 24 Nov 2022 18:27:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229918AbiKXX1W (ORCPT ); Thu, 24 Nov 2022 18:27:22 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8A55CFA6A; Thu, 24 Nov 2022 15:26:13 -0800 (PST) Message-ID: <20221124232325.442499757@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=qV//gzPmOqH3W5duAbtHBZqaagM7sPrAqRRWld0OnjU=; b=HEs+D51n1rtP2vHE8zTlVoE+bOEuK2V6v930W5rI9u4JP69P6fXmSVDnDQ1OHvh8M447Sn HIlQFnvYDrdPoJcHKUF9zWPOzTFdotIaDJeP4PmQt47rpPs5JkVCmNFfPamCp6ElalPaoe GsMsWwhrkK0Xk94tKAGIT7UlJ+i4IEFczaZR3txMYlivHdJ431STot8EbTkltcYS0wEGao et9k4iKua0uRgP7tqps8ulngmt5DqTRm38bPI+asJObtPINw0VLpRZKqWInlIQ4nji5CYU fUFGNYViSoRtRnRwtLtY2osg/SfBArVb8hmj3MCn8fuKZIukKgQmUAyDMQfRlg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=qV//gzPmOqH3W5duAbtHBZqaagM7sPrAqRRWld0OnjU=; b=oMDZPkI4sNS0Y6FEh3qMJ6jAioLVyTTklUXpF0S2QnDYthHjgNU62jXoYHjPYaGX3cqY2M Bz+7VnU1G1xZI4AQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 03/33] genirq/msi: Provide data structs for per device domains References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:49 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422138572383567?= X-GMAIL-MSGID: =?utf-8?q?1750422138572383567?= Provide struct msi_domain_template which contains a bundle of struct irq_chip, struct msi_domain_ops and struct msi_domain_info and a name field. This template is used by MSI device domain implementations to provide the domain specific functionality, feature bits etc. When a MSI domain is created the template is duplicated in the core code so that it can be modified per instance. That means templates can be marked const at the MSI device domain code. The template is a bundle to avoid several allocations and duplications of the involved structures. The name field is used to construct the final domain and chip name via: $PREFIX$NAME-$DEVNAME where prefix is the optional prefix of the MSI parent domain, $NAME is the provided name in template::chip and the device name so that the domain is properly identified. On x86 this results for PCI/MSI in: PCI-MSI-0000:3d:00.1 or IR-PCI-MSIX-0000:3d:00.1 depending on the domain type and the availability of remapping. Signed-off-by: Thomas Gleixner --- V3: Correct changelog (Kevin) --- include/linux/msi.h | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -74,7 +75,6 @@ struct msi_msg { extern int pci_msi_ignore_mask; /* Helper functions */ -struct irq_data; struct msi_desc; struct pci_dev; struct platform_msi_priv_data; @@ -442,6 +442,20 @@ struct msi_domain_info { void *data; }; +/** + * struct msi_domain_template - Template for MSI device domains + * @name: Storage for the resulting name. Filled in by the core. + * @chip: Interrupt chip for this domain + * @ops: MSI domain ops + * @info: MSI domain info data + */ +struct msi_domain_template { + char name[48]; + struct irq_chip chip; + struct msi_domain_ops ops; + struct msi_domain_info info; +}; + /* * Flags for msi_domain_info * From patchwork Thu Nov 24 23:25:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25748 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661391wrr; Thu, 24 Nov 2022 15:30:43 -0800 (PST) X-Google-Smtp-Source: AA0mqf5IAeTP3T4P8uPUXG6/9gBM7CmyAg4T2JjO+cahm70eaBsyOKN0OtBH1BAiCFX4P3uWXSxk X-Received: by 2002:a17:906:e083:b0:78d:c155:56b2 with SMTP id gh3-20020a170906e08300b0078dc15556b2mr21009609ejb.291.1669332643533; Thu, 24 Nov 2022 15:30:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332643; cv=none; d=google.com; s=arc-20160816; b=ILRAUbtdVgGdm130m6gDpA34S87suyQOMOxCORpLDThzoarh715lo8F8WDcrb7tF1u J3XxnSJja01hih6SlvCgJ+71m5ouYEZrj5yFQDmAF3X71r/okNwc4WcldlHi3dS0npPu 0Yj83m3WN+uzyLCP2aUwLao2asVIlvSP5f73gMykenFd2Y26Pd/YB3QONo9pFYZaIgPe QrBDuESOvbAFFagsgTxp802e5w30dFS+cDlVCjK9v1YToIZOQVWYbt5yZbdZeoVeLzSR nuDksc0CxmPsg0oxNaORT6ggBYBNL8GqMZNSTa3JthKsOSc9PcLRpEE8NnuszVbB6p/A HVpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=KKwu6EVH8ruKNrQNXAsaHWUDYRK2+tT7uRCo5fxd1Vs=; b=1JOhOZkccOX6PWI+U7XxX9iH7slGJt0JwwJCfvGpLm9b3tMgrzxsgHd3j45riXRu1p Ij2CtQU9F//u3wvuX7SpLBVoQu2Z86On5t2hcZENGefORCWlt2AQhrHnLAqfyA8r/Te+ jdiboR5nDMQMEmeWdzsE5iBkYZZH/n3oxr2Ri+kmqaXOXYPH21L/4JyybDtBKKhXRHl+ zKYimEMJ8LCP+sm4Hdwmfqr81711B63HMsWd/CBp+TcmCc+dUkG59XIw3gg+/o2xPyWP ZvvVu3mLL+Ik0R08/vus61dB7h2a0nS1qwwdx0b8909Jg6AlipT1IXtPkGY3Vt/2DxCK VFjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="zBLb/Ox7"; dkim=neutral (no key) header.i=@linutronix.de header.b=U714pILZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a21-20020aa7cf15000000b00459dc4e4b7fsi1006421edy.143.2022.11.24.15.30.19; Thu, 24 Nov 2022 15:30:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="zBLb/Ox7"; dkim=neutral (no key) header.i=@linutronix.de header.b=U714pILZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229983AbiKXX2D (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229689AbiKXX1Y (ORCPT ); Thu, 24 Nov 2022 18:27:24 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F00818B11A; Thu, 24 Nov 2022 15:26:16 -0800 (PST) Message-ID: <20221124232325.501144862@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=KKwu6EVH8ruKNrQNXAsaHWUDYRK2+tT7uRCo5fxd1Vs=; b=zBLb/Ox7OJ0VV/3WNc7cfyfHZRZ9kH0LXxBr9yvermW/Cdpi8j9vMj4pORxOMop+qv+3ul M3AvejZVya/4s3APJ5vXtUlaJcaOCznMcq98LQlTnrztPkg7iPWCmwtABH316vm/9aC1fC ks8b27Ofl/d9YGQMDbfUXGZvyGv4kNz+92aLF6Pufvcw1JP+GlRb+eLDaGiyy5/XlBY9UX zLb7mUcRS+p6qPgYkrHpBqO9htQGC889H1qiM8jIbPYu4WH2G9ouMebAgnAwc4evmStG79 Ykn//ds95V5V1BUEPoKLhHNj65WBzT9Zd4Losv9sn6OxOZipDmatJuJbWy+N5A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=KKwu6EVH8ruKNrQNXAsaHWUDYRK2+tT7uRCo5fxd1Vs=; b=U714pILZM40FuHR0FKCS8MjRY15jpHxDIzBby9AK0wRMzB+f2fhZeyenVKgAS965j0wNDK FLef1Gx9MlMtcsBA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 04/33] genirq/msi: Add size info to struct msi_domain_info References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:51 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422146339260647?= X-GMAIL-MSGID: =?utf-8?q?1750422146339260647?= To allow proper range checking especially for dynamic allocations add a size field to struct msi_domain_info. If the field is 0 then the size is unknown or unlimited (up to MSI_MAX_INDEX) to provide backwards compability. Signed-off-by: Thomas Gleixner --- V3: Move the initialization into the common domain creation code --- include/linux/msi.h | 5 +++++ kernel/irq/msi.c | 11 +++++++++++ 2 files changed, 16 insertions(+) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -422,6 +422,10 @@ struct msi_domain_ops { * struct msi_domain_info - MSI interrupt domain data * @flags: Flags to decribe features and capabilities * @bus_token: The domain bus token + * @hwsize: The hardware table size or the software index limit. + * If 0 then the size is considered unlimited and + * gets initialized to the maximum software index limit + * by the domain creation code. * @ops: The callback data structure * @chip: Optional: associated interrupt chip * @chip_data: Optional: associated interrupt chip data @@ -433,6 +437,7 @@ struct msi_domain_ops { struct msi_domain_info { u32 flags; enum irq_domain_bus_token bus_token; + unsigned int hwsize; struct msi_domain_ops *ops; struct irq_chip *chip; void *chip_data; --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -773,6 +773,17 @@ struct irq_domain *msi_create_irq_domain { struct irq_domain *domain; + if (info->hwsize > MSI_XA_DOMAIN_SIZE) + return NULL; + + /* + * Hardware size 0 is valid for backwards compatibility and for + * domains which are not backed by a hardware table. Grant the + * maximum index space. + */ + if (!info->hwsize) + info->hwsize = MSI_XA_DOMAIN_SIZE; + msi_domain_update_dom_ops(info); if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS) msi_domain_update_chip_ops(info); From patchwork Thu Nov 24 23:25:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25747 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661361wrr; Thu, 24 Nov 2022 15:30:39 -0800 (PST) X-Google-Smtp-Source: AA0mqf7tQzuLGgPazf9t+w2JmbGme7P0NRLdZ9Ge/fHoi5h1I6hoJjuZcu8ahhL3ju1jNbII7h+l X-Received: by 2002:a17:906:925a:b0:7ad:8bc6:48e1 with SMTP id c26-20020a170906925a00b007ad8bc648e1mr30130203ejx.673.1669332639813; Thu, 24 Nov 2022 15:30:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332639; cv=none; d=google.com; s=arc-20160816; b=uygqSMbJYOCIkcz+uoTmVdq3HCoFKObCsEyFD8Mp2/ZRJpfSaOzikCxx2p/5Gpanq2 UluVRyMYtW10Ve65P+zYCm07u6JnJ6ybbPwoH1m0UpQsCPbFnkoDZK7Y1h6tguEqBYz3 8mWI44NALrztU5apfarnlMaCESNi0pPDh36QibD/VSsc74MODAY8dC7Y9nXLu+KE53/x 1nL6gih+RrqkVG82Qy8UPIQbM8AHmm7MVeFwIaRD4StNux84kp6MuzYruo23WtOgEkE6 cXxM59QVfxgb6ZYU4LWrzn3E30aVkjJxd2M05u98weQbDkC/5X8AYnR8nKMcDKTKXpf2 GoSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=NNfDCQRDz9DltLDIG/7m5g8s58pcCqqYMsewa1N3sZE=; b=P/zdY4dVkKZE1UnTiRqhPj3SCjPzQSJAc/RTZk9HZGElHYpwZ/NjWbtJrO4jCcYpzO 2ii2x+LO2+nWa7xPPT6T84YMsy7084g2lLxF4bGzEBKRWZAGXRcUCs69zaFRV7nk5w1s hCmRRyOETHbro8dFk+eqxYrGUJUGjrD5YMvGTw/tSRKU7M1WgOqihMU0fEeRYN5QMSg+ dfn4hc9khyH/+757egbn0htfod6Sq/QX0SE0bh5grqkvAZmRRfdQRaQEcH2cYoDT0ck1 p5nO08yFx0lfWvKIU9QpQk/BHVoWuysbucnKAS7H0xRjVVIi4BSgR8OHATyv0xTudJ1/ zV2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=afGnU5wX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q21-20020aa7cc15000000b00461a7bb34c2si1753134edt.473.2022.11.24.15.30.15; Thu, 24 Nov 2022 15:30:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=afGnU5wX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229884AbiKXX2A (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229732AbiKXX1X (ORCPT ); Thu, 24 Nov 2022 18:27:23 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5112A976DF; Thu, 24 Nov 2022 15:26:16 -0800 (PST) Message-ID: <20221124232325.559086358@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332353; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=NNfDCQRDz9DltLDIG/7m5g8s58pcCqqYMsewa1N3sZE=; b=afGnU5wXi26xEDzhSpDaPnEag0oZF2TFyPqjsgHSNRixpkbaGf7N5ZxDRRpJ3K+kr4VdO+ U6Oz2WCjDUPpJ3xXt2d12nuPkGrdmzJ8nNCcM1F5WvMtIl2Pb6lThpuIDMWhzxCZaO8Ji6 3ZZ87Y7YioBTLHbyM8dFmw4DZdVq8pPmSUX8DNxgO7ocoJrPHEyItI8fsx0TymMXyiJ3PM kiE5TnW49uMz6ZeAAhSM8kz6iL+uKOMfA9VfjaEHQGJzTiVC2279X+1EkWWlfauRQObaL5 vNaKIjAHvjf5EeIPB1kVnTYUXBygL1qjFXBVBWYtc+YZx+l3VIfBXWyB0uRTOA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332353; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=NNfDCQRDz9DltLDIG/7m5g8s58pcCqqYMsewa1N3sZE=; b=Ojr368Qg/5Wa/X97FxSO9JlwdMYyVi5CrQmizkHWeZAoQ2OCJ5O/Sj4JHpk+mfgQcVfZax Zo/+IoxXwkpwzLDw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 05/33] genirq/msi: Split msi_create_irq_domain() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:52 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422142441834999?= X-GMAIL-MSGID: =?utf-8?q?1750422142441834999?= Split the functionality of msi_create_irq_domain() so it can be reused for creating per device irq domains. No functional change. Signed-off-by: Thomas Gleixner --- kernel/irq/msi.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-) --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -759,17 +759,10 @@ static void msi_domain_update_chip_ops(s chip->irq_set_affinity = msi_domain_set_affinity; } -/** - * msi_create_irq_domain - Create an MSI interrupt domain - * @fwnode: Optional fwnode of the interrupt controller - * @info: MSI domain info - * @parent: Parent irq domain - * - * Return: pointer to the created &struct irq_domain or %NULL on failure - */ -struct irq_domain *msi_create_irq_domain(struct fwnode_handle *fwnode, - struct msi_domain_info *info, - struct irq_domain *parent) +static struct irq_domain *__msi_create_irq_domain(struct fwnode_handle *fwnode, + struct msi_domain_info *info, + unsigned int flags, + struct irq_domain *parent) { struct irq_domain *domain; @@ -788,7 +781,7 @@ struct irq_domain *msi_create_irq_domain if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS) msi_domain_update_chip_ops(info); - domain = irq_domain_create_hierarchy(parent, IRQ_DOMAIN_FLAG_MSI, 0, + domain = irq_domain_create_hierarchy(parent, flags | IRQ_DOMAIN_FLAG_MSI, 0, fwnode, &msi_domain_ops, info); if (domain) { @@ -801,6 +794,21 @@ struct irq_domain *msi_create_irq_domain } /** + * msi_create_irq_domain - Create an MSI interrupt domain + * @fwnode: Optional fwnode of the interrupt controller + * @info: MSI domain info + * @parent: Parent irq domain + * + * Return: pointer to the created &struct irq_domain or %NULL on failure + */ +struct irq_domain *msi_create_irq_domain(struct fwnode_handle *fwnode, + struct msi_domain_info *info, + struct irq_domain *parent) +{ + return __msi_create_irq_domain(fwnode, info, 0, parent); +} + +/** * msi_parent_init_dev_msi_info - Delegate initialization of device MSI info down * in the domain hierarchy * @dev: The device for which the domain should be created From patchwork Thu Nov 24 23:25:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25744 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3660859wrr; Thu, 24 Nov 2022 15:29:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf7FG6uLzFbdRylbt1IbvV5DfJ3V/C7NIHMRqe3KDB/986Y63baguqjLfyHzMpnLZffyLIWr X-Received: by 2002:a17:906:9f02:b0:7b5:f5c9:b450 with SMTP id fy2-20020a1709069f0200b007b5f5c9b450mr18648724ejc.65.1669332572263; Thu, 24 Nov 2022 15:29:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332572; cv=none; d=google.com; s=arc-20160816; b=mXOKcKh6VtnbbWLYzNLbrI0ryyuEcbCKiQ4UCMZwY/mPCS3N8AYW3kTBzJWr2MaY2f Jgstj5Y1FEeNNARenkV3Jkotdsl2dHqM0G359wAX6vAf+invunRk1wdPed7QcFhsmqDx CRGhmlV195ptELwd5aRb/NzNAZhBA73IBQUOqlMvaclwK9QyPSNBvSUK7g9HIBHd2/sm mp12ld26vcBRCs1wLQlsfSNZRl35GmVLAxDMTEpPOp1tneXdUiymf70CcZcciityx6sb SUUjLNuUfFx40bXBybZzEYONREuRsjgNlDO7jSnFyo+tQoci8h4JXvYCSYdwjvmk4sUO LZKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=xdXQN8DRlFx/ul31qn4JSIb+1+AWkLgJWGGnxBYyOqY=; b=iyXFo5ZAa7sEaOHRWiigT0WPA1illMdnJec7B/Wit5w16WdnNhTOB6dcCNRLbZSypp j8oIsJUMiV6CHqAS1tbz+eaRI8u4/xkfip5pPDQj9wck2IvjUTrGZ7UtuAhN5KGnc2mZ Mo21SOA4OvCE0pbkuR3w7zA+UHxPGllGPq2983yfyxG0K8oWLFaZwFPpSQvJ6EMYxKwk cqC8i6P5sz4LHaVOCt68DGB6+XhyJsrrpdc0+2oujKughkTYGeVpSnSvu3EX71Ac5cJk RQlaqgSfUqqrvhXfvSvXP3dkNisy/Ym3HpppO6tUxheoD8v2SBgDh6TZa6/O03/nR+Hy SzBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=Vlm0wQdG; dkim=neutral (no key) header.i=@linutronix.de header.b=AVFno4In; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hd8-20020a170907968800b007ae86742c3bsi1775062ejc.1009.2022.11.24.15.29.09; Thu, 24 Nov 2022 15:29:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=Vlm0wQdG; dkim=neutral (no key) header.i=@linutronix.de header.b=AVFno4In; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229804AbiKXX1z (ORCPT + 99 others); Thu, 24 Nov 2022 18:27:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229728AbiKXX1W (ORCPT ); Thu, 24 Nov 2022 18:27:22 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 319A5CFA6C; Thu, 24 Nov 2022 15:26:15 -0800 (PST) Message-ID: <20221124232325.618807601@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=xdXQN8DRlFx/ul31qn4JSIb+1+AWkLgJWGGnxBYyOqY=; b=Vlm0wQdGH0cvpPDw9AF5vvmxmHq927pTmvBgDIh99qQEiq4jdu45fdvTx1D3Tt1fiEpHs4 yBKGesX7U/LcOMMDk560CEY4/+3YlGzH18XsNFqHPKJBTL8tenQ6rpdd9nJQFly+geH9/b FZPnfCKKqrLNjCqS0yNKP0vsht/dzlVVxMBs8mPoG3H9krJ8RyGg6S3WJuMSIZP8KIQoQ2 Ot1aa4guG2NkztvZUKwJMaWDK1/mzwa1qIcaIPkSniBe5F0yCYotBEYxtkiEoq8w3Tlgyr zpRy7fhJ/hs9cZjEl7nPk5HxayStXhsuB8cLUBHaMDFt5AewPu7o/7f2C99pbg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=xdXQN8DRlFx/ul31qn4JSIb+1+AWkLgJWGGnxBYyOqY=; b=AVFno4In/9G6BVel7cWwEfGpIFeOHiNWGDWUDht/TIbuCqDY3yB+KyZvAACsUcKa/1xU0S Rcjkwulq8PRkUHDQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 06/33] genirq/irqdomain: Add irq_domain::dev for per device MSI domains References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:54 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422071463681902?= X-GMAIL-MSGID: =?utf-8?q?1750422071463681902?= Per device domains require the device pointer of the device which instantiated the domain for some purposes. Add the pointer to struct irq_domain. It will be used in the next step which provides the infrastructure to create per device MSI domains. Signed-off-by: Thomas Gleixner --- include/linux/irqdomain.h | 4 ++++ 1 file changed, 4 insertions(+) --- a/include/linux/irqdomain.h +++ b/include/linux/irqdomain.h @@ -132,6 +132,9 @@ struct irq_domain_chip_generic; * @gc: Pointer to a list of generic chips. There is a helper function for * setting up one or more generic chips for interrupt controllers * drivers using the generic chip library which uses this pointer. + * @dev: Pointer to the device which instantiated the irqdomain + * With per device irq domains this is not necessarily the same + * as @pm_dev. * @pm_dev: Pointer to a device that can be utilized for power management * purposes related to the irq domain. * @parent: Pointer to parent irq_domain to support hierarchy irq_domains @@ -155,6 +158,7 @@ struct irq_domain { struct fwnode_handle *fwnode; enum irq_domain_bus_token bus_token; struct irq_domain_chip_generic *gc; + struct device *dev; struct device *pm_dev; #ifdef CONFIG_IRQ_DOMAIN_HIERARCHY struct irq_domain *parent; From patchwork Thu Nov 24 23:25:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25745 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3660912wrr; Thu, 24 Nov 2022 15:29:41 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ltWNdRkf3YODjdTL3iM6PLbyyrP0P7dz60Kh0OzttthaTfxYjt9hByBNgNtLb5cwfM5JR X-Received: by 2002:a17:906:8e0a:b0:7b9:bef6:3eea with SMTP id rx10-20020a1709068e0a00b007b9bef63eeamr8656002ejc.487.1669332581019; Thu, 24 Nov 2022 15:29:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332581; cv=none; d=google.com; s=arc-20160816; b=CZRsrRs5rVm/EXY7YtmZKNJE/neCda2uVfjE4N9Wtedd3y6UxO91xYQpIGqo0D1mZl iJ7+IBDh16fJrdVXSyUMfPwtustZ/S8koBfE4NHgu+AWy8LIgSCLLaAXOCNTasHsEcLD m3amqO1n5Y3UOmgS5WMgW63qc3hy+wU3PjDTxmylOnAcWbfK3VWnxQTxAqdeh5hPcMo6 tWh+JeIXZMgFREeSUMxMX/vCX0zf9JMk3NtXwBJtZiODToCqUiDhdTleynMYkD3NdoKA VBLAtdekhkrTqDvrsAPLysWRx3+uqb/g2TFVPJ7ooB4l9MeaaOlkACXJT0ji0be/0erm L1fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=Rptg2jhZEWRDJFBrlepwa1/xZvdteCw3yYQE94xTOcY=; b=EpyQ9gG89if7lLxXo6BxUTBlAY+7rMF42q5IuI0nah9v1tXl0LF4fbcS7hG8CpF0vm 3BG8wtfEudTqEunXtpkbxArfQP5LqrXMkyqp+lUagRK8MooBev1r/frNIfivd+SjJRBk EamqleJ4rlMUXprvTRzVwWF+NmX3GVRfrqk9lZz5HXUO5xLILUQIetSKL0cT2e+cKtgr kjKYyhw2rLMb77TPSDja9D3f/FvCMg2rjwQOirzzykUHtPnYLjfp6CB/b8po8KYulKLQ 2f1Jcb8xdhwyqdzrNykibBYfQTfZcwtmWwerEjp1okkPWZ6sh7S9uDAyohNzkpNGQZhm cazQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=fW2DsyCM; dkim=neutral (no key) header.i=@linutronix.de header.b=L4uU5SYa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ji9-20020a170907980900b007a46fa50b26si2309590ejc.517.2022.11.24.15.29.18; Thu, 24 Nov 2022 15:29:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=fW2DsyCM; dkim=neutral (no key) header.i=@linutronix.de header.b=L4uU5SYa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229909AbiKXX2R (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229868AbiKXX1m (ORCPT ); Thu, 24 Nov 2022 18:27:42 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D953CFE94; Thu, 24 Nov 2022 15:26:27 -0800 (PST) Message-ID: <20221124232325.678838546@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rptg2jhZEWRDJFBrlepwa1/xZvdteCw3yYQE94xTOcY=; b=fW2DsyCMmwC/fWggszLgz8x8UiAskhOBvhZHehVRbyM687CPHaUbhefSmCGGaxpuwnDa5P 7NsJ0AHxQDCBWxDwBMT92EMgSCalRB3f5CV6RTBOFIsCDDLPx5KwRprr5HfR1UGK9C54gq dFTo2tF3WuxOJkhfEFs69fiY7Ln+aLsXntX/Iig77MMARArntCn/n5FhrrViaKNylxjRPk GY0yeyOzK/CXqXPPxWCg/fLbrJGRZBiMzfb8I5x5alDD1oUwGDPEeMNRCzu5Cof8YAw4Vw vho8a+FCZDrraaLA4CTxAiWXXciZYN2/PZFHMO126y3QrLwbhs6qdWsOIlVo5A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rptg2jhZEWRDJFBrlepwa1/xZvdteCw3yYQE94xTOcY=; b=L4uU5SYaFhw2E0gZeFX/eZrNo/oW66U+Mb9vA8sLvLtkUdgJ65TnsiZgKisP9jPzm6bnz9 947zQTU2sBQUtoBA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 07/33] genirq/msi: Provide msi_create/free_device_irq_domain() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:56 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422080345168464?= X-GMAIL-MSGID: =?utf-8?q?1750422080345168464?= Now that all prerequsites are in place, provide the actual interfaces for creating and removing per device interrupt domains. MSI device interrupt domains are created from the provided msi_domain_template which is duplicated so that it can be modified for the particular device. The name of the domain and the name of the interrupt chip are composed by "$(PREFIX)$(CHIPNAME)-$(DEVNAME)" $PREFIX: The optional prefix provided by the underlying MSI parent domain via msi_parent_ops::prefix. $CHIPNAME: The name of the irq_chip in the template $DEVNAME: The name of the device The domain is further initialized through a MSI parent domain callback which fills in the required functionality for the parent domain or domains further down the hierarchy. This initialization can fail, e.g. when the requested feature or MSI domain type cannot be supported. The domain pointer is stored in the pointer array inside of msi_device_data which is attached to the domain. The domain can be removed via the API or left for disposal via devres when the device is torn down. The API removal is useful e.g. for PCI to have seperate domains for MSI and MSI-X, which are mutually exclusive and always occupy the default domain id slot. Signed-off-by: Thomas Gleixner --- V3: Remove unused argument and adopt the hwsize init (Kevin) Adopt to the xarray and domain storage split --- include/linux/msi.h | 6 ++ kernel/irq/msi.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 144 insertions(+) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -547,6 +547,12 @@ struct irq_domain *msi_create_irq_domain struct msi_domain_info *info, struct irq_domain *parent); +bool msi_create_device_irq_domain(struct device *dev, unsigned int domid, + const struct msi_domain_template *template, + unsigned int hwsize, void *domain_data, + void *chip_data); +void msi_remove_device_irq_domain(struct device *dev, unsigned int domid); + int msi_domain_alloc_irqs_range_locked(struct device *dev, unsigned int domid, unsigned int first, unsigned int last); int msi_domain_alloc_irqs_range(struct device *dev, unsigned int domid, --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -240,6 +240,7 @@ static void msi_device_data_release(stru int i; for (i = 0; i < MSI_MAX_DEVICE_IRQDOMAINS; i++) { + msi_remove_device_irq_domain(dev, i); WARN_ON_ONCE(!xa_empty(&md->__domains[i].store)); xa_destroy(&md->__domains[i].store); } @@ -849,6 +850,143 @@ bool msi_parent_init_dev_msi_info(struct msi_child_info); } +/** + * msi_create_device_irq_domain - Create a device MSI interrupt domain + * @dev: Pointer to the device + * @domid: Domain id + * @template: MSI domain info bundle used as template + * @hwsize: Maximum number of MSI table entries (0 if unknown or unlimited) + * @domain_data: Optional pointer to domain specific data which is set in + * msi_domain_info::data + * @chip_data: Optional pointer to chip specific data which is set in + * msi_domain_info::chip_data + * + * Return: True on success, false otherwise + * + * There is no firmware node required for this interface because the per + * device domains are software constructs which are actually closer to the + * hardware reality than any firmware can describe them. + * + * The domain name and the irq chip name for a MSI device domain are + * composed by: "$(PREFIX)$(CHIPNAME)-$(DEVNAME)" + * + * $PREFIX: Optional prefix provided by the underlying MSI parent domain + * via msi_parent_ops::prefix. If that pointer is NULL the prefix + * is empty. + * $CHIPNAME: The name of the irq_chip in @template + * $DEVNAME: The name of the device + * + * This results in understandable chip names and hardware interrupt numbers + * in e.g. /proc/interrupts + * + * PCI-MSI-0000:00:1c.0 0-edge Parent domain has no prefix + * IR-PCI-MSI-0000:00:1c.4 0-edge Same with interrupt remapping prefix 'IR-' + * + * IR-PCI-MSIX-0000:3d:00.0 0-edge Hardware interrupt numbers reflect + * IR-PCI-MSIX-0000:3d:00.0 1-edge the real MSI-X index on that device + * IR-PCI-MSIX-0000:3d:00.0 2-edge + * + * On IMS domains the hardware interrupt number is either a table entry + * index or a purely software managed index but it is guaranteed to be + * unique. + * + * The domain pointer is stored in @dev::msi::data::__irqdomains[]. All + * subsequent operations on the domain depend on the domain id. + * + * The domain is automatically freed when the device is removed via devres + * in the context of @dev::msi::data freeing, but it can also be + * independently removed via @msi_remove_device_irq_domain(). + */ +bool msi_create_device_irq_domain(struct device *dev, unsigned int domid, + const struct msi_domain_template *template, + unsigned int hwsize, void *domain_data, + void *chip_data) +{ + struct irq_domain *domain, *parent = dev->msi.domain; + const struct msi_parent_ops *pops; + struct msi_domain_template *bundle; + struct fwnode_handle *fwnode; + + if (!irq_domain_is_msi_parent(parent)) + return false; + + if (domid >= MSI_MAX_DEVICE_IRQDOMAINS) + return false; + + bundle = kmemdup(template, sizeof(*bundle), GFP_KERNEL); + if (!bundle) + return false; + + bundle->info.hwsize = hwsize; + bundle->info.chip = &bundle->chip; + bundle->info.ops = &bundle->ops; + bundle->info.data = domain_data; + bundle->info.chip_data = chip_data; + + pops = parent->msi_parent_ops; + snprintf(bundle->name, sizeof(bundle->name), "%s%s-%s", + pops->prefix ? : "", bundle->chip.name, dev_name(dev)); + bundle->chip.name = bundle->name; + + fwnode = irq_domain_alloc_named_fwnode(bundle->name); + if (!fwnode) + goto free_bundle; + + if (msi_setup_device_data(dev)) + goto free_fwnode; + + msi_lock_descs(dev); + + if (WARN_ON_ONCE(msi_get_device_domain(dev, domid))) + goto fail; + + if (!pops->init_dev_msi_info(dev, parent, parent, &bundle->info)) + goto fail; + + domain = __msi_create_irq_domain(fwnode, &bundle->info, IRQ_DOMAIN_FLAG_MSI_DEVICE, parent); + if (!domain) + goto fail; + + domain->dev = dev; + dev->msi.data->__domains[domid].domain = domain; + msi_unlock_descs(dev); + return true; + +fail: + msi_unlock_descs(dev); +free_fwnode: + kfree(fwnode); +free_bundle: + kfree(bundle); + return false; +} + +/** + * msi_remove_device_irq_domain - Free a device MSI interrupt domain + * @dev: Pointer to the device + * @domid: Domain id + */ +void msi_remove_device_irq_domain(struct device *dev, unsigned int domid) +{ + struct msi_domain_info *info; + struct irq_domain *domain; + + msi_lock_descs(dev); + + domain = msi_get_device_domain(dev, domid); + + if (!domain || !irq_domain_is_msi_device(domain)) + goto unlock; + + dev->msi.data->__domains[domid].domain = NULL; + info = domain->host_data; + irq_domain_remove(domain); + kfree(container_of(info, struct msi_domain_template, info)); + +unlock: + msi_unlock_descs(dev); +} + int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) { From patchwork Thu Nov 24 23:25:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25764 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3680001wrr; Thu, 24 Nov 2022 16:20:33 -0800 (PST) X-Google-Smtp-Source: AA0mqf4BELc/zNi8S5Y4yYC8UJsXYTvDKuBQ9eNNxCoAd6YzWtuuiRXgMZegHaGPNh9CVJjqYI3a X-Received: by 2002:a17:906:1e83:b0:78d:addf:67c3 with SMTP id e3-20020a1709061e8300b0078daddf67c3mr20824606ejj.253.1669335633220; Thu, 24 Nov 2022 16:20:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669335633; cv=none; d=google.com; s=arc-20160816; b=YgFf/yu9PrWSoFQ1LAv+kQswPAvcyinFlmzEdhH0q8aywQwGwMYT7KMuzlMsxAwWDr glT9E9qWuuI4IzVHCz1E+1fGZjrHwU6o93oXPC/gCqixDIsOBEandLaqkEF6t7ZLibW8 WH0nhBK081paKpaa7p7xRg4Bz3ya8FWcgo4Dj7uHveXRhD8mpjspnmAvb6KwakesEsSr KFQodLd9JAY2KMaMga1WgrTo/GmxZCpVCKRVUm6ip2+dJL5h0eUX5unLLEQHD2765iCn 5nxS2l4L7fRLZlCOyeSwoa/nw4S38Kzsk/gzGw//UI+wDSt0ENNfrwQNNfdlUFAsitSf TjLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=MxT52QfeG/NT2BA2XnRso0NWmZQGQnhH9yEiG1ZbsIM=; b=zhjtGUJNw6k8cx0ym3ajU6p3nNxiTd70GjIg6Z9vTKdmJ8K3mLnGzR71pNUS1ITacj 19xPlRbvnSiG4g9PDDY2x+2/ujROAnmyN5pdX+hFBdJJdw2gXgJPBSnYkaw0zK85lNFL ezJPwp/pHZhr6Zq/B6+sZ4oZh1He3CSnjl58dK3gBawiOv77EowWvjk1JtRZ9jFp3qsy kXyYhnAcg4F0Uh1hkYa8XN3UBJeya//5kJz3elC8QfSEeZpk+gikApHMoE1F87YVnRUN q1FE8hKlO2DlWoGhpIYbVvVVIzILaGEYyc1+60xzrkhMfmwKpt/xKGClskrCpoQjDIy3 OAdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="rzNt/s64"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f19-20020a50a6d3000000b0046a223ca073si2142126edc.182.2022.11.24.16.20.01; Thu, 24 Nov 2022 16:20:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="rzNt/s64"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229668AbiKXXfk (ORCPT + 99 others); Thu, 24 Nov 2022 18:35:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230095AbiKXXf0 (ORCPT ); Thu, 24 Nov 2022 18:35:26 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24514A2883; Thu, 24 Nov 2022 15:31:46 -0800 (PST) Message-ID: <20221124232325.738047902@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=MxT52QfeG/NT2BA2XnRso0NWmZQGQnhH9yEiG1ZbsIM=; b=rzNt/s64LZ9a80Jio7t/Pir3vg0mIG5LsgimCWVrmkl7G/4ehoDgsvRxrzObYSSf3LSIGM yAJUTQKnsTxQZ1WF+nBpy98jUVIAg5b23gQs3ChumbfI/MV8zQh9j+RzIMIcKgPUPbwKhy MHEObd2TPTduz84DwpW3Dolo/j1oBJxz3CpxIcXUU8kpWj1yz7g743tVCS/cwsL9if4ZxX PaaGODwRhzM9l3lX8Ekjfp9EPdtui3KHKG1PgNUNwMjuL005h8krACYm9CMqbK1urEavZM OJWgTdmkPvXCww4Z4lUGJVYh9wuc+rlmiaDxQFbfdPIr5bZfuTnhHGARLH3goA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=MxT52QfeG/NT2BA2XnRso0NWmZQGQnhH9yEiG1ZbsIM=; b=dPMpS7gRpstwKeMvYNv/vsLOAMsakSmDEPVkSaUgvmOTMm0QcW/fOQPttnSESjCHWwCrsN 2l6GTCIB+RAkL9CA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 08/33] genirq/msi: Provide msi_match_device_domain() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:57 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750425281135312839?= X-GMAIL-MSGID: =?utf-8?q?1750425281135312839?= Provide an interface to match a per device domain bus token. This allows to query which type of domain is installed for a particular domain id. Will be used for PCI to avoid frequent create/remove cycles for the MSI resp. MSI-X domains. Signed-off-by: Thomas Gleixner --- include/linux/msi.h | 3 +++ kernel/irq/msi.c | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -553,6 +553,9 @@ bool msi_create_device_irq_domain(struct void *chip_data); void msi_remove_device_irq_domain(struct device *dev, unsigned int domid); +bool msi_match_device_irq_domain(struct device *dev, unsigned int domid, + enum irq_domain_bus_token bus_token); + int msi_domain_alloc_irqs_range_locked(struct device *dev, unsigned int domid, unsigned int first, unsigned int last); int msi_domain_alloc_irqs_range(struct device *dev, unsigned int domid, --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -987,6 +987,31 @@ void msi_remove_device_irq_domain(struct msi_unlock_descs(dev); } +/** + * msi_match_device_irq_domain - Match a device irq domain against a bus token + * @dev: Pointer to the device + * @domid: Domain id + * @bus_token: Bus token to match against the domain bus token + * + * Return: True if device domain exists and bus tokens match. + */ +bool msi_match_device_irq_domain(struct device *dev, unsigned int domid, + enum irq_domain_bus_token bus_token) +{ + struct msi_domain_info *info; + struct irq_domain *domain; + bool ret = false; + + msi_lock_descs(dev); + domain = msi_get_device_domain(dev, domid); + if (domain && irq_domain_is_msi_device(domain)) { + info = domain->host_data; + ret = info->bus_token == bus_token; + } + msi_unlock_descs(dev); + return ret; +} + int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) { From patchwork Thu Nov 24 23:25:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25749 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661699wrr; Thu, 24 Nov 2022 15:31:23 -0800 (PST) X-Google-Smtp-Source: AA0mqf6Zas/G5EvUgsCXRFtr05UGQzRSz5JMpPIUBM6gC2Z3SVEJEKZap5etUm/lgAHGFfyPtwUa X-Received: by 2002:a05:6402:5:b0:459:cdb:92ad with SMTP id d5-20020a056402000500b004590cdb92admr15380467edu.77.1669332683559; Thu, 24 Nov 2022 15:31:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332683; cv=none; d=google.com; s=arc-20160816; b=kIke9vl0P51Hq4tGw9I3zFktwIGzQfBbBHe9zr5Ii+57Ub31zD6m+JM0gaF1fN3DED k/JhW6JV4vA6twsSHx5nj+vPppQtN2WN/iXpQSwQVjKhi4ZIAkodxliKzXYiufCaCytV +1AVtLS1jrfMyPFO2FOPYmfeRHgpekL+ATqFSrLrgNz4HCWexAs92z0qtHNqNyTtll4f vlyGgZJErzc+AA/T8CI02i1Kcivvwnx7YR9QkH5NbmZabgLDTKSSuWV6PJNfOp/KrIOe jiduio3B7Rjw82X+3V7XzizJSA1vpWgHvMKEYr/RXcmOvnyK6A+5NNZyA6CR8u1bE02X UZow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=O52hKM/eyXA5495ofBmhJYgeFgFMKz1bKsf4ukXppYY=; b=kPQUUaBY6mJkaffS7xE1RehK0HFNCirRrSOXEOFXsQqQQcRWmTRGRcaRoe/cmhqRaF 0AypGlgVS9PaVSw8+5stHkCAhsHzeB2XaJKPj7kYCNO0PMOKkFG/4hg011SATZ74DG1c +6CSJPwv4Sz2xJSbJ2Yg1ZwiQJWwHpy3hFx8g+HsWgo/dCa/tSgBOROyDB6IiwSNSYbE tfNBIz3kS06IvBXUmnlFIVXDDE4SA2Kus8GKEwEmTEMByFOv4PvaXW6NYHCLV4wuPb1P eJkQ7wh5IyXO8Cggs/jTQVoSdz595pNOirAE0GKS7R4kYotGP6crs1VbdfYzTHuBkczL hChQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=dopcgaOB; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v18-20020aa7d812000000b00460faf7d2a3si1985400edq.277.2022.11.24.15.31.00; Thu, 24 Nov 2022 15:31:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=dopcgaOB; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229635AbiKXX2p (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229966AbiKXX2a (ORCPT ); Thu, 24 Nov 2022 18:28:30 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EE43D33BF; Thu, 24 Nov 2022 15:26:49 -0800 (PST) Message-ID: <20221124232325.798556374@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=O52hKM/eyXA5495ofBmhJYgeFgFMKz1bKsf4ukXppYY=; b=dopcgaOBeuvvrVBBjM0IZ0Z4kvik/8Xxm+j9VeWrZCxYVZSjAt7u7Wu3jFdqxEDwUs6Dpe ZRigkklTSw5Qbi46/l+53EZbtiawpGi1awSivW/h0qEf3eKvOkyLD+9JzcvGduXRbZsEGB xs+n1tn5OojF8ZuNu/AJLkCszv9eTIlOAJDJJ4Nn3dJCCo9O6gaS0qRvESCs5Cgfnt9YEK T5yZg75EvqwFYUGAgCoP5qqPeaZgIhea5xAsHXj5ZIWpkryKhPu39X2YnTr2E7kId+rjog RVerFXwJ+rt+WWZzl8R8LpOv8Uy2HKj//8IxlIJWp8zyBglmainSmPymXLBx3A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=O52hKM/eyXA5495ofBmhJYgeFgFMKz1bKsf4ukXppYY=; b=gHSz9L5yPdDRuaLfE8LQ98i2E3jVTmq3Z5/qFrAeRScCdHPL18H1Wxeq8YXUhiN2rmB9LF +zoMdTSXWUgufbDA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 09/33] genirq/msi: Add range checking to msi_insert_desc() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:25:59 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422187804583032?= X-GMAIL-MSGID: =?utf-8?q?1750422187804583032?= Per device domains provide the real domain size to the core code. This allows range checking on insertion of MSI descriptors and also paves the way for dynamic index allocations which are required e.g. for IMS. This avoids external mechanisms like bitmaps on the device side and just utilizes the core internal MSI descriptor storxe for it. Signed-off-by: Thomas Gleixner --- V3: Adopt to the new info->hwsize handling and to the new xarray split --- kernel/irq/msi.c | 58 ++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 11 deletions(-) --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -40,6 +40,7 @@ struct msi_ctrl { #define MSI_XA_DOMAIN_SIZE (MSI_MAX_INDEX + 1) static void msi_domain_free_locked(struct device *dev, struct msi_ctrl *ctrl); +static unsigned int msi_domain_get_hwsize(struct device *dev, unsigned int domid); static inline int msi_sysfs_create_group(struct device *dev); @@ -80,16 +81,28 @@ static void msi_free_desc(struct msi_des kfree(desc); } -static int msi_insert_desc(struct msi_device_data *md, struct msi_desc *desc, +static int msi_insert_desc(struct device *dev, struct msi_desc *desc, unsigned int domid, unsigned int index) { + struct msi_device_data *md = dev->msi.data; struct xarray *xa = &md->__domains[domid].store; + unsigned int hwsize; int ret; + hwsize = msi_domain_get_hwsize(dev, domid); + if (index >= hwsize) { + ret = -ERANGE; + goto fail; + } + desc->msi_index = index; ret = xa_insert(xa, index, desc, GFP_KERNEL); if (ret) - msi_free_desc(desc); + goto fail; + return 0; + +fail: + msi_free_desc(desc); return ret; } @@ -117,7 +130,7 @@ int msi_domain_insert_msi_desc(struct de /* Copy type specific data to the new descriptor. */ desc->pci = init_desc->pci; - return msi_insert_desc(dev->msi.data, desc, domid, init_desc->msi_index); + return msi_insert_desc(dev, desc, domid, init_desc->msi_index); } static bool msi_desc_match(struct msi_desc *desc, enum msi_desc_filter filter) @@ -136,11 +149,16 @@ static bool msi_desc_match(struct msi_de static bool msi_ctrl_valid(struct device *dev, struct msi_ctrl *ctrl) { + unsigned int hwsize; + if (WARN_ON_ONCE(ctrl->domid >= MSI_MAX_DEVICE_IRQDOMAINS || - !dev->msi.data->__domains[ctrl->domid].domain || - ctrl->first > ctrl->last || - ctrl->first > MSI_MAX_INDEX || - ctrl->last > MSI_MAX_INDEX)) + !dev->msi.data->__domains[ctrl->domid].domain)) + return false; + + hwsize = msi_domain_get_hwsize(dev, ctrl->domid); + if (WARN_ON_ONCE(ctrl->first > ctrl->last || + ctrl->first >= hwsize || + ctrl->last >= hwsize)) return false; return true; } @@ -208,7 +226,7 @@ static int msi_domain_add_simple_msi_des desc = msi_alloc_desc(dev, 1, NULL); if (!desc) goto fail_mem; - ret = msi_insert_desc(dev->msi.data, desc, ctrl->domid, idx); + ret = msi_insert_desc(dev, desc, ctrl->domid, idx); if (ret) goto fail; } @@ -407,7 +425,10 @@ unsigned int msi_domain_get_virq(struct if (!dev->msi.data) return 0; - if (WARN_ON_ONCE(index > MSI_MAX_INDEX || domid >= MSI_MAX_DEVICE_IRQDOMAINS)) + if (WARN_ON_ONCE(domid >= MSI_MAX_DEVICE_IRQDOMAINS)) + return 0; + + if (WARN_ON_ONCE(index >= msi_domain_get_hwsize(dev, domid))) return 0; /* This check is only valid for the PCI default MSI domain */ @@ -569,6 +590,20 @@ static struct irq_domain *msi_get_device return domain; } +static unsigned int msi_domain_get_hwsize(struct device *dev, unsigned int domid) +{ + struct msi_domain_info *info; + struct irq_domain *domain; + + domain = msi_get_device_domain(dev, domid); + if (domain) { + info = domain->host_data; + return info->hwsize; + } + /* No domain, no size... */ + return 0; +} + static inline void irq_chip_write_msi_msg(struct irq_data *data, struct msi_msg *msg) { @@ -1359,7 +1394,7 @@ int msi_domain_alloc_irqs_all_locked(str struct msi_ctrl ctrl = { .domid = domid, .first = 0, - .last = MSI_MAX_INDEX, + .last = msi_domain_get_hwsize(dev, domid) - 1, .nirqs = nirqs, }; @@ -1473,7 +1508,8 @@ void msi_domain_free_irqs_range(struct d */ void msi_domain_free_irqs_all_locked(struct device *dev, unsigned int domid) { - msi_domain_free_irqs_range_locked(dev, domid, 0, MSI_MAX_INDEX); + msi_domain_free_irqs_range_locked(dev, domid, 0, + msi_domain_get_hwsize(dev, domid) - 1); } /** From patchwork Thu Nov 24 23:26:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25752 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3662016wrr; Thu, 24 Nov 2022 15:32:07 -0800 (PST) X-Google-Smtp-Source: AA0mqf4+sKcPOiTOXee7efuF8L+CEKckbSqv4odwnVACH1fXB3HN9HXSkrwh587ndcyeNCYXBjSM X-Received: by 2002:a05:6402:370e:b0:464:fa1:9dc3 with SMTP id ek14-20020a056402370e00b004640fa19dc3mr17466404edb.343.1669332727185; Thu, 24 Nov 2022 15:32:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332727; cv=none; d=google.com; s=arc-20160816; b=NEs91P45PIAiTMkjmTPkT8xQVFPOfFnjtZv41voE4joLWgqc2JxD0PMCo3QuBnL1/Y tJ3iC5ETTVcKfl4Iv01SGxLfjgqmDGfZztw9NFpnSjhiObgGT9GtOfA4YXnktyBG093r USjreUKsb4IxaFFPuLKZw1zx4d9lXwKE5oIQjdRyjTc/7DEg/bQjSyu+ax5dVqQT9g4R B630zjCRqHdQrXLMSSTfQNM4oVAEOgWGxizQE1IFyoVobToEJg+IBXz38MeNVEHn0edL cYRBGfI7fR+iVB8rxJ/BeAR8XOV1+LGkEJnIGEJTj4aZKXJ4Qup+Ppg3P33x5QvGJKJl s2SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=osB1ROcuLYeqTjSSCjR+4n5EoRccoICYxGf0yGM+gI4=; b=DixBq/gW8/KA8eZpzDh7ggkb5LWkTogZYUkRJuimp0BwbIcJ38wBBq8ghqiqUbGF/x 4CervHm0Q2LheK5lAaTVkQdMeWcbc6Ii1iczKMhJx3h4CMrjtnKRfFRt2tRJ2rBv4fDS iluaiEQq5EM/shgeXkJ1MsrCIIocG1h+W2CYJUiFgP/55oywVckDLh97gDAr/KBLug/c NeqnmJNVcqm7mLEzy4CnY0r6NmmH3z2xPewqRuazip5vy81Tw4aJsuN2mocVBBuDM7ZU tX2fQdTHyxwQIL6ABtSAB/F78aG6dAlYlxWhfBHq/bSNB4OrJ/kCaBFzdHQQpiZuPFRK gxpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=ABaIW7MV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=9C5N+ZHv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n9-20020aa7db49000000b00469a5b7de5esi1503321edt.567.2022.11.24.15.31.42; Thu, 24 Nov 2022 15:32:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=ABaIW7MV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=9C5N+ZHv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229639AbiKXX2s (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229968AbiKXX2a (ORCPT ); Thu, 24 Nov 2022 18:28:30 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EEEAD39D7; Thu, 24 Nov 2022 15:26:49 -0800 (PST) Message-ID: <20221124232325.857982142@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=osB1ROcuLYeqTjSSCjR+4n5EoRccoICYxGf0yGM+gI4=; b=ABaIW7MV8O3R2mRIwO7XbR954a66E6czAFyHzAy1nqq+3Go8dnsOQrvE9HTCyZ5TUSR8GJ SkEDDe1FnLWq8QYG9vIDYGLGuyfQjYB8sTAZXRHfwSwt9Atwg7i9WikjEgxECyGJRagXsG munrndXOg5nsbq3O9UZxoMRauswae1SnyuIo+djGBCG8R2ctFvcE0w3RW/kg3KLxid+oXd x1Zpe/t0GEwwuh6LN5BQtsIxtqVfE/Om6ubID8cuKr1U8UZ1bJD+oZClqJN/i1Yc3+O1MJ IrnynicLqy4ZxrS+JEU3JmVDJo4tpo76cTuxTUvg0DErzz508yCTDkpTysd0eg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=osB1ROcuLYeqTjSSCjR+4n5EoRccoICYxGf0yGM+gI4=; b=9C5N+ZHv6PSbAktC6PZccDrlqHU363uVBq/cMK/CZrvzFFxFLzZOb1195PdSrGa0TK5YFB YAt7DEbWz78jWgCQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , "Ahmed S. Darwish" Subject: [patch V3 10/33] PCI/MSI: Split __pci_write_msi_msg() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:00 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422233783376828?= X-GMAIL-MSGID: =?utf-8?q?1750422233783376828?= The upcoming per device MSI domains will create different domains for MSI and MSI-X. Split the write message function into MSI and MSI-X helpers so they can be used by those new domain functions seperately. Signed-off-by: Ahmed S. Darwish Signed-off-by: Thomas Gleixner Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas --- drivers/pci/msi/msi.c | 104 +++++++++++++++++++++++++------------------------- 1 file changed, 54 insertions(+), 50 deletions(-) --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -180,6 +180,58 @@ void __pci_read_msi_msg(struct msi_desc } } +static inline void pci_write_msg_msi(struct pci_dev *dev, struct msi_desc *desc, + struct msi_msg *msg) +{ + int pos = dev->msi_cap; + u16 msgctl; + + pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl); + msgctl &= ~PCI_MSI_FLAGS_QSIZE; + msgctl |= desc->pci.msi_attrib.multiple << 4; + pci_write_config_word(dev, pos + PCI_MSI_FLAGS, msgctl); + + pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_LO, msg->address_lo); + if (desc->pci.msi_attrib.is_64) { + pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_HI, msg->address_hi); + pci_write_config_word(dev, pos + PCI_MSI_DATA_64, msg->data); + } else { + pci_write_config_word(dev, pos + PCI_MSI_DATA_32, msg->data); + } + /* Ensure that the writes are visible in the device */ + pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl); +} + +static inline void pci_write_msg_msix(struct msi_desc *desc, struct msi_msg *msg) +{ + void __iomem *base = pci_msix_desc_addr(desc); + u32 ctrl = desc->pci.msix_ctrl; + bool unmasked = !(ctrl & PCI_MSIX_ENTRY_CTRL_MASKBIT); + + if (desc->pci.msi_attrib.is_virtual) + return; + /* + * The specification mandates that the entry is masked + * when the message is modified: + * + * "If software changes the Address or Data value of an + * entry while the entry is unmasked, the result is + * undefined." + */ + if (unmasked) + pci_msix_write_vector_ctrl(desc, ctrl | PCI_MSIX_ENTRY_CTRL_MASKBIT); + + writel(msg->address_lo, base + PCI_MSIX_ENTRY_LOWER_ADDR); + writel(msg->address_hi, base + PCI_MSIX_ENTRY_UPPER_ADDR); + writel(msg->data, base + PCI_MSIX_ENTRY_DATA); + + if (unmasked) + pci_msix_write_vector_ctrl(desc, ctrl); + + /* Ensure that the writes are visible in the device */ + readl(base + PCI_MSIX_ENTRY_DATA); +} + void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg) { struct pci_dev *dev = msi_desc_to_pci_dev(entry); @@ -187,63 +239,15 @@ void __pci_write_msi_msg(struct msi_desc if (dev->current_state != PCI_D0 || pci_dev_is_disconnected(dev)) { /* Don't touch the hardware now */ } else if (entry->pci.msi_attrib.is_msix) { - void __iomem *base = pci_msix_desc_addr(entry); - u32 ctrl = entry->pci.msix_ctrl; - bool unmasked = !(ctrl & PCI_MSIX_ENTRY_CTRL_MASKBIT); - - if (entry->pci.msi_attrib.is_virtual) - goto skip; - - /* - * The specification mandates that the entry is masked - * when the message is modified: - * - * "If software changes the Address or Data value of an - * entry while the entry is unmasked, the result is - * undefined." - */ - if (unmasked) - pci_msix_write_vector_ctrl(entry, ctrl | PCI_MSIX_ENTRY_CTRL_MASKBIT); - - writel(msg->address_lo, base + PCI_MSIX_ENTRY_LOWER_ADDR); - writel(msg->address_hi, base + PCI_MSIX_ENTRY_UPPER_ADDR); - writel(msg->data, base + PCI_MSIX_ENTRY_DATA); - - if (unmasked) - pci_msix_write_vector_ctrl(entry, ctrl); - - /* Ensure that the writes are visible in the device */ - readl(base + PCI_MSIX_ENTRY_DATA); + pci_write_msg_msix(entry, msg); } else { - int pos = dev->msi_cap; - u16 msgctl; - - pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl); - msgctl &= ~PCI_MSI_FLAGS_QSIZE; - msgctl |= entry->pci.msi_attrib.multiple << 4; - pci_write_config_word(dev, pos + PCI_MSI_FLAGS, msgctl); - - pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_LO, - msg->address_lo); - if (entry->pci.msi_attrib.is_64) { - pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_HI, - msg->address_hi); - pci_write_config_word(dev, pos + PCI_MSI_DATA_64, - msg->data); - } else { - pci_write_config_word(dev, pos + PCI_MSI_DATA_32, - msg->data); - } - /* Ensure that the writes are visible in the device */ - pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl); + pci_write_msg_msi(dev, entry, msg); } -skip: entry->msg = *msg; if (entry->write_msi_msg) entry->write_msi_msg(entry, entry->write_msi_msg_data); - } void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg) From patchwork Thu Nov 24 23:26:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25761 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3664868wrr; Thu, 24 Nov 2022 15:40:44 -0800 (PST) X-Google-Smtp-Source: AA0mqf5vBcuLVW6HdVvaw8dFEFgTuW1kaN0LcU1KoxKv7jx3NRYbGF1gntHQuBoITKKZTtI8heEw X-Received: by 2002:a17:906:1b51:b0:78d:a30f:3f3a with SMTP id p17-20020a1709061b5100b0078da30f3f3amr29657093ejg.386.1669333244761; Thu, 24 Nov 2022 15:40:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669333244; cv=none; d=google.com; s=arc-20160816; b=TM0QLyPZgE1lPK668KKKvOPDS6pbsZ9Pit2VTlUCcgVTtYz6Ucnu6R31wx5OgmSrju fnfdpf3lz79UrbMqELxph8SGUOK9xMx+YmNCWOf68rZY5HALUPnttval0RskO+7+qa1N WwdOLbtcOJXTInKB5h/HxZGYz2o/gdySQ6Zd/3FN0akkmY9vBZ4PCpLigqynii2Aumig U/HHK9dV46nPv37LEmnqLP5u0qTlTNj22PxLvsDbtPHsTnCfk/N5e9vS3BqAK9nX2htC PcfKUnNuQPsMbqeu1ZsCgVZVQL+iwzdow3jMKiouX7p0QnBSfpbSCqSXtHlgcNyStZmq k6AA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=/ACIVj3Kux3YVP+JKXyXfLBMvNMP5EOMeimbPlaFLJQ=; b=UXlLFaxlHvu+wA2LK6KHXTp1wBbTkHRPnYR75b6Mjoiu4bshD4VjE94sYItdTw4Stn aM8J24AUVFpBJYHKxO/NpNhVihCqk1htaGncXnvDhQpU1epYslL/Re5JTbxqRk6fj8QK mDMdcffC2P7cry32rqXtEC5m3izSL845voCt3OPJxL+Jn0FhvnLkpbpTh9HJPhXiZkAD L5ZZFSWYZlCrhnyfDxQkzC2ewUqwRPTR1b679GMoL53ABdc5WFQsS4l0kxyNyMCS3JUe Vi2fvdGhvjKGk+NAE1otsMm7+/VaZnSyiAoGpQKOKpPZt7w1kf81rYlsaT9jbt9axIGf 4IBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=BsKbsW1S; dkim=neutral (no key) header.i=@linutronix.de header.b=LpEqpWKs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nd27-20020a170907629b00b00769d94690fesi2204121ejc.326.2022.11.24.15.40.21; Thu, 24 Nov 2022 15:40:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=BsKbsW1S; dkim=neutral (no key) header.i=@linutronix.de header.b=LpEqpWKs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229948AbiKXXgQ (ORCPT + 99 others); Thu, 24 Nov 2022 18:36:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbiKXXgC (ORCPT ); Thu, 24 Nov 2022 18:36:02 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3CDC116AA8; Thu, 24 Nov 2022 15:32:07 -0800 (PST) Message-ID: <20221124232325.917219885@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/ACIVj3Kux3YVP+JKXyXfLBMvNMP5EOMeimbPlaFLJQ=; b=BsKbsW1SxIOI3uIIk1XAIrdhEqrqbHncgrocGX9Uh2YWGWVfMlj2O+PN8VYUYNIq0qZ2Ay A9eUTO7Y79lWaVpR//G51TZ+CebF9CLLQuwFq12Y4ZjtHbFlI8FfU/PS1vLDzw/vr8k2yK R25WIFL3ItTAodR/56Sbqi5K8lBnIbuFwfcGzQKlLVEbmtKc/shnbHyruPAjOxQyEYfFNW aEKuw2uz9cpTeDcnyyScSx3V7GUezDsnMAf25sbvSpOt7WbVWcMTcw7JrQcGRI3xE0MfQ/ e34GT/Wn8HrKOWDLB5KP53Qpyk9dsYEtnV9fRqv6oFUj4suIw+fmJ6hPg9gIsw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/ACIVj3Kux3YVP+JKXyXfLBMvNMP5EOMeimbPlaFLJQ=; b=LpEqpWKsx5hfed/pW/S8H+NTS72hnkS7dFww6SXUe5jHOBr+9zaeOuaPLondJsu/P79pqN hKCH+lWyvnEmReDg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 11/33] genirq/msi: Provide BUS_DEVICE_PCI_MSI[X] References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:02 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422776581462430?= X-GMAIL-MSGID: =?utf-8?q?1750422776581462430?= Provide new bus tokens for the upcoming per device PCI/MSI and PCI/MSIX interrupt domains. Signed-off-by: Thomas Gleixner --- include/linux/irqdomain_defs.h | 2 ++ kernel/irq/msi.c | 4 ++++ 2 files changed, 6 insertions(+) --- a/include/linux/irqdomain_defs.h +++ b/include/linux/irqdomain_defs.h @@ -21,6 +21,8 @@ enum irq_domain_bus_token { DOMAIN_BUS_TI_SCI_INTA_MSI, DOMAIN_BUS_WAKEUP, DOMAIN_BUS_VMD_MSI, + DOMAIN_BUS_PCI_DEVICE_MSI, + DOMAIN_BUS_PCI_DEVICE_MSIX, }; #endif /* _LINUX_IRQDOMAIN_DEFS_H */ --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -1122,6 +1122,8 @@ static bool msi_check_reservation_mode(s switch(domain->bus_token) { case DOMAIN_BUS_PCI_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSIX: case DOMAIN_BUS_VMD_MSI: break; default: @@ -1147,6 +1149,8 @@ static int msi_handle_pci_fail(struct ir { switch(domain->bus_token) { case DOMAIN_BUS_PCI_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSIX: case DOMAIN_BUS_VMD_MSI: if (IS_ENABLED(CONFIG_PCI_MSI)) break; From patchwork Thu Nov 24 23:26:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25770 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3685904wrr; Thu, 24 Nov 2022 16:38:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf6DK9ApZul9c/I0BhAnBI2WvnrKVXgyDG9Ppq6P3IYN6rRwDLR6baJIqFI2FkuF2ECzULNG X-Received: by 2002:a17:902:d592:b0:186:8930:20e6 with SMTP id k18-20020a170902d59200b00186893020e6mr16576459plh.64.1669336694777; Thu, 24 Nov 2022 16:38:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669336694; cv=none; d=google.com; s=arc-20160816; b=s9GBFGzIQw8OY7EUPzrdHCH/RAMkp/ZHmDq+cHLAc3NgEUK12717t7mmVfcxa8YwFO mrDHQvRIEO++ldGLy/A4pcD2lyUGPHThXEnNLicgwCUbuijaUfhJj+YtkCeyhXujpKn9 RN4Ez2xNhgD3xQUMRvOJUX4ccRk0hfhuFGfiaWmOanF1aMMJTktRM4oeSvC0BcyFzBc7 rdMJg2R9CTqAqugvh1lP+Q2ib6U5Bdza6oc76dQ+065ezIX14tJlLbvXzzLWxc1WAwFC BEGAcfqCceIz2cIgWgiaa25FPVf9uur5Stx52yaFlP8YXx8uQk0+OIMWFaBptSE9rwWc 5ZXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=gr/0xB7L9vVdyKa8sOQHHY/5znBVwp0MmRkXe2JkU+M=; b=ivW4cv/dCqSKcVaot/TWXPQB6CIl7ey1WKZPIPwjJ4x7zLFX1oCWf9tmJfoCdWqSEZ 8gJNiP76T7tyuMhRgychL2m+JZ0rp1zzCd4i8rkbf/OeVObRRkr4VCGVmdTO3UsN5Yy8 ++6yYmQxH6uHtk0TbnjbJwRqGTGdpLLWHIF12M054dq6dgY/sFAIOkZQre6MeSc97/90 JLFYmjxT/X2HT/qMjRS9tlF2+WNkpQuvB7W/OJaeP6ujF0b1KELjXLZeseA+gu8v5e3Z 1AZ/p2oa693NKva+uMKhWe/E3AJRoWmZJLcfMNzk962HHgb+BFizM1+eJlI21WCJUCxp qDow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=vaC95fD4; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mw4-20020a17090b4d0400b00218890f4874si6386435pjb.116.2022.11.24.16.38.02; Thu, 24 Nov 2022 16:38:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=vaC95fD4; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230113AbiKXXgZ (ORCPT + 99 others); Thu, 24 Nov 2022 18:36:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229581AbiKXXgJ (ORCPT ); Thu, 24 Nov 2022 18:36:09 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76FE7EC787; Thu, 24 Nov 2022 15:32:13 -0800 (PST) Message-ID: <20221124232325.975388241@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332364; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=gr/0xB7L9vVdyKa8sOQHHY/5znBVwp0MmRkXe2JkU+M=; b=vaC95fD4qt6wyQ1/p/FVHf8WGrWybpIHeARNzGuu8fZw8zdiH8dcq3J09xFc8kn7E0cK0N QUpe6ClPNMEDVLjKDzoSsIkbAo97jA4FTT8TA6K9Urmpjp9CAjIJbOxkPmEkyZMmZ5hYZI 1uIvVCalZ4UdyGVOirEFkSFD0gw+qXCgXzAYGRWllFYJyZgy97kgKvvijdMsUyLuTjKmPq FqF2jbwEMHAmwYy65MKj3xcBAVhYzD6qbtn+cfQKX+ct18Cq6UWfX14m3600IG1l/aa0ki Hvpvz/DSUHThW6Gp6S4fiKrDc2itZJ9myCxa1jtU75YC8B3XWDGDEecioBiZ6w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332364; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=gr/0xB7L9vVdyKa8sOQHHY/5znBVwp0MmRkXe2JkU+M=; b=KAW2Vg51rhd0RxsbVTjF6iRnRs8lL7XbTbte7iQY+PEGx5CLJ546DHcqq/fMs9gnvVO/kQ gayhyKFBzF6w6+Dg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , "Ahmed S. Darwish" Subject: [patch V3 12/33] PCI/MSI: Add support for per device MSI[X] domains References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:04 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426393779021085?= X-GMAIL-MSGID: =?utf-8?q?1750426393779021085?= Provide a template and the necessary callbacks to create PCI/MSI and PCI/MSI-X domains. The domains are created when MSI or MSI-X is enabled. The domain's lifetime is either the device lifetime or in case that e.g. MSI-X was tried first and failed, then the MSI-X domain is removed and a MSI domain is created as both are mutually exclusive and reside in the default domain ID slot of the per device domain pointer array. Also expand pci_msi_domain_supports() to handle feature checks correctly even in the case that the per device domain was not yet created by checking the features supported by the MSI parent. Add the necessary setup calls into the MSI and MSI-X enable code path. These setup calls are backwards compatible. They return success when there is no parent domain found, which means the existing global domains or the legacy allocation path keep just working. Co-developed-by: Ahmed S. Darwish Signed-off-by: Ahmed S. Darwish Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas --- V3: Rename the chip callbacks and fix the check in the MSIX domain creation path (Kevin) --- drivers/pci/msi/irqdomain.c | 188 +++++++++++++++++++++++++++++++++++++++++++- drivers/pci/msi/msi.c | 16 +++ drivers/pci/msi/msi.h | 2 3 files changed, 201 insertions(+), 5 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -139,6 +139,170 @@ struct irq_domain *pci_msi_create_irq_do } EXPORT_SYMBOL_GPL(pci_msi_create_irq_domain); +/* + * Per device MSI[-X] domain functionality + */ +static void pci_device_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc) +{ + arg->desc = desc; + arg->hwirq = desc->msi_index; +} + +static void pci_irq_mask_msi(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + + pci_msi_mask(desc, BIT(data->irq - desc->irq)); +} + +static void pci_irq_unmask_msi(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + + pci_msi_unmask(desc, BIT(data->irq - desc->irq)); +} + +#ifdef CONFIG_GENERIC_IRQ_RESERVATION_MODE +# define MSI_REACTIVATE MSI_FLAG_MUST_REACTIVATE +#else +# define MSI_REACTIVATE 0 +#endif + +#define MSI_COMMON_FLAGS (MSI_FLAG_FREE_MSI_DESCS | \ + MSI_FLAG_ACTIVATE_EARLY | \ + MSI_FLAG_DEV_SYSFS | \ + MSI_REACTIVATE) + +static struct msi_domain_template pci_msi_template = { + .chip = { + .name = "PCI-MSI", + .irq_mask = pci_irq_mask_msi, + .irq_unmask = pci_irq_unmask_msi, + .irq_write_msi_msg = pci_msi_domain_write_msg, + .flags = IRQCHIP_ONESHOT_SAFE, + }, + + .ops = { + .set_desc = pci_device_domain_set_desc, + }, + + .info = { + .flags = MSI_COMMON_FLAGS | MSI_FLAG_MULTI_PCI_MSI, + .bus_token = DOMAIN_BUS_PCI_DEVICE_MSI, + }, +}; + +static void pci_irq_mask_msix(struct irq_data *data) +{ + pci_msix_mask(irq_data_get_msi_desc(data)); +} + +static void pci_irq_unmask_msix(struct irq_data *data) +{ + pci_msix_unmask(irq_data_get_msi_desc(data)); +} + +static struct msi_domain_template pci_msix_template = { + .chip = { + .name = "PCI-MSIX", + .irq_mask = pci_irq_mask_msix, + .irq_unmask = pci_irq_unmask_msix, + .irq_write_msi_msg = pci_msi_domain_write_msg, + .flags = IRQCHIP_ONESHOT_SAFE, + }, + + .ops = { + .set_desc = pci_device_domain_set_desc, + }, + + .info = { + .flags = MSI_COMMON_FLAGS | MSI_FLAG_PCI_MSIX, + .bus_token = DOMAIN_BUS_PCI_DEVICE_MSIX, + }, +}; + +static bool pci_match_device_domain(struct pci_dev *pdev, enum irq_domain_bus_token bus_token) +{ + return msi_match_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN, bus_token); +} + +static bool pci_create_device_domain(struct pci_dev *pdev, struct msi_domain_template *tmpl, + unsigned int hwsize) +{ + struct irq_domain *domain = dev_get_msi_domain(&pdev->dev); + + if (!domain || !irq_domain_is_msi_parent(domain)) + return true; + + return msi_create_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN, tmpl, + hwsize, NULL, NULL); +} + +/** + * pci_setup_msi_device_domain - Setup a device MSI interrupt domain + * @pdev: The PCI device to create the domain on + * + * Return: + * True when: + * - The device does not have a MSI parent irq domain associated, + * which keeps the legacy architecture specific and the global + * PCI/MSI domain models working + * - The MSI domain exists already + * - The MSI domain was successfully allocated + * False when: + * - MSI-X is enabled + * - The domain creation fails. + * + * The created MSI domain is preserved until: + * - The device is removed + * - MSI is disabled and a MSI-X domain is created + */ +bool pci_setup_msi_device_domain(struct pci_dev *pdev) +{ + if (WARN_ON_ONCE(pdev->msix_enabled)) + return false; + + if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSI)) + return true; + if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX)) + msi_remove_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN); + + return pci_create_device_domain(pdev, &pci_msi_template, 1); +} + +/** + * pci_setup_msix_device_domain - Setup a device MSI-X interrupt domain + * @pdev: The PCI device to create the domain on + * @hwsize: The size of the MSI-X vector table + * + * Return: + * True when: + * - The device does not have a MSI parent irq domain associated, + * which keeps the legacy architecture specific and the global + * PCI/MSI domain models working + * - The MSI-X domain exists already + * - The MSI-X domain was successfully allocated + * False when: + * - MSI is enabled + * - The domain creation fails. + * + * The created MSI-X domain is preserved until: + * - The device is removed + * - MSI-X is disabled and a MSI domain is created + */ +bool pci_setup_msix_device_domain(struct pci_dev *pdev, unsigned int hwsize) +{ + if (WARN_ON_ONCE(pdev->msi_enabled)) + return false; + + if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSIX)) + return true; + if (pci_match_device_domain(pdev, DOMAIN_BUS_PCI_DEVICE_MSI)) + msi_remove_device_irq_domain(&pdev->dev, MSI_DEFAULT_DOMAIN); + + return pci_create_device_domain(pdev, &pci_msix_template, hwsize); +} + /** * pci_msi_domain_supports - Check for support of a particular feature flag * @pdev: The PCI device to operate on @@ -152,13 +316,33 @@ bool pci_msi_domain_supports(struct pci_ { struct msi_domain_info *info; struct irq_domain *domain; + unsigned int supported; domain = dev_get_msi_domain(&pdev->dev); if (!domain || !irq_domain_is_hierarchy(domain)) return mode == ALLOW_LEGACY; - info = domain->host_data; - return (info->flags & feature_mask) == feature_mask; + + if (!irq_domain_is_msi_parent(domain)) { + /* + * For "global" PCI/MSI interrupt domains the associated + * msi_domain_info::flags is the authoritive source of + * information. + */ + info = domain->host_data; + supported = info->flags; + } else { + /* + * For MSI parent domains the supported feature set + * is avaliable in the parent ops. This makes checks + * possible before actually instantiating the + * per device domain because the parent is never + * expanding the PCI/MSI functionality. + */ + supported = domain->msi_parent_ops->supported_flags; + } + + return (supported & feature_mask) == feature_mask; } /* --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -436,6 +436,9 @@ int __pci_enable_msi_range(struct pci_de if (rc) return rc; + if (!pci_setup_msi_device_domain(dev)) + return -ENODEV; + for (;;) { if (affd) { nvec = irq_calc_affinity_vectors(minvec, nvec, affd); @@ -787,9 +790,13 @@ int __pci_enable_msix_range(struct pci_d if (!pci_msix_validate_entries(dev, entries, nvec, hwsize)) return -EINVAL; - /* PCI_IRQ_VIRTUAL is a horrible hack! */ - if (nvec > hwsize && !(flags & PCI_IRQ_VIRTUAL)) - nvec = hwsize; + if (hwsize < nvec) { + /* Keep the IRQ virtual hackery working */ + if (flags & PCI_IRQ_VIRTUAL) + hwsize = nvec; + else + nvec = hwsize; + } if (nvec < minvec) return -ENOSPC; @@ -798,6 +805,9 @@ int __pci_enable_msix_range(struct pci_d if (rc) return rc; + if (!pci_setup_msix_device_domain(dev, hwsize)) + return -ENODEV; + for (;;) { if (affd) { nvec = irq_calc_affinity_vectors(minvec, nvec, affd); --- a/drivers/pci/msi/msi.h +++ b/drivers/pci/msi/msi.h @@ -105,6 +105,8 @@ enum support_mode { }; bool pci_msi_domain_supports(struct pci_dev *dev, unsigned int feature_mask, enum support_mode mode); +bool pci_setup_msi_device_domain(struct pci_dev *pdev); +bool pci_setup_msix_device_domain(struct pci_dev *pdev, unsigned int hwsize); /* Legacy (!IRQDOMAIN) fallbacks */ From patchwork Thu Nov 24 23:26:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25758 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3663579wrr; Thu, 24 Nov 2022 15:36:29 -0800 (PST) X-Google-Smtp-Source: AA0mqf5lyzgVSFn8bHal8yHLcya6jm+pQBKcWRINyBhm8z0I0WFFyhptQl1NARRvlts51k8g43an X-Received: by 2002:a63:495e:0:b0:470:75a1:c6d7 with SMTP id y30-20020a63495e000000b0047075a1c6d7mr15424216pgk.120.1669332989507; Thu, 24 Nov 2022 15:36:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332989; cv=none; d=google.com; s=arc-20160816; b=BDPlu7JUwTqBzjsCOeBsRG85QoIBhKV9isEZc+6cvAzuCF2R2FRGvZCkM3HJVLOjH7 ByZfj5XA71jXKgfDnMSdmAnE4H0hgwJckHHVLC0mBgPPrbGgRRCFX7Do5nW7rakGqkB2 LOdJHKhYdQ7G0bLDEOc+ctFtm87TxEbE5rUMM3JG2X3ciRIc6+Ifdi9SjhTFFVaENM+q pxQ4rJ5or123/nbn+sMQb3akc0Tu34OCx7FPNzTJXrQCbcpLo2ILyueqNebwvLJOVrzU al6OZqplyuFS4wPPQ27HC/wKogZyrfy1j/WKCK6p0PaT3OGqI7hYrXr95wu0GzupvB2w gTeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=Bult+QOTyn3yk1J2BaE5BOxLAmykZwJ6i9DuwWbDSqo=; b=CrxAXZCQ3me4nZYG4pO05gAz6UF4eKPf6I1YUX48DTOI6P4+ypwrJzM61ZmSQUBucQ nQiKEv5pZmnO0eVcbM5gIW4DyfsHi939VHq53ADxq/PZ82mPi2hRnU+aM2v2P6Y/McT5 IIaeVIaLJ27uZ3Hjh1vGqHhElqDKsekdIZag3xNNhhVgk74S8z19OGNQdJgmP/vHOgvk Yyx21aPbWzqMyBkjWg6O72pOHQ3nIKfsM/P+I68iz/VbNVZvcvxccXb+6AUfup99Wlfx 7f4exPzAWDFgydN/7VdkhYg8IH+bOQLxwpsCxfxLSs/3jMo0uWyZ9LkGILZanJ59sD1V PuoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PlixV12p; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jd21-20020a170903261500b00178b5e035fcsi2128465plb.496.2022.11.24.15.36.16; Thu, 24 Nov 2022 15:36:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PlixV12p; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229979AbiKXX3g (ORCPT + 99 others); Thu, 24 Nov 2022 18:29:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230081AbiKXX3G (ORCPT ); Thu, 24 Nov 2022 18:29:06 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A361C7680; Thu, 24 Nov 2022 15:27:22 -0800 (PST) Message-ID: <20221124232326.034672592@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Bult+QOTyn3yk1J2BaE5BOxLAmykZwJ6i9DuwWbDSqo=; b=PlixV12pAnUfZPqsk7Ry0yvbXPzAibgac+BskTETQmgFkxzqfwnFLXb2+JPf6zBbP0MV0g R2w4KmWeDY5yCxUHghkfiX7tKcJFORTZVeXgqCC8FPAecJel26PnZLKNUzci0R9BNiGn91 gqEEkpq7CJpM12tpl54cxUwTfmAG0yOK0sMWvnCofHWdgSjpvroTc19+XsKDq1YqANH+dg EAcxO1jgOPBxNJKwoFER+Cd7AhqGySkHcafGoqQrtzbHND05AYCMMoGZkcV7jIQ0CNnM/T cpc4gsFHx1MMEIIzYUOQ6xhCtS/u9rGxhagXH3eEsZB+UeM+/Ju7OcGD0J5oOA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Bult+QOTyn3yk1J2BaE5BOxLAmykZwJ6i9DuwWbDSqo=; b=mfVtZJthVhuw1B9fPQbtyplPWPKvyhoo8ZZ2KmB8e4QuksPxZNscDEMNKhRW4lMbnvy/62 /rhIbFpq+tTEAeDA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 13/33] x86/apic/vector: Provide MSI parent domain References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:05 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422509191407268?= X-GMAIL-MSGID: =?utf-8?q?1750422509191407268?= Enable MSI parent domain support in the x86 vector domain and fixup the checks in the iommu implementations to check whether device::msi::domain is the default MSI parent domain. That keeps the existing logic to protect e.g. devices behind VMD working. The interrupt remap PCI/MSI code still works because the underlying vector domain still provides the same functionality. None of the other x86 PCI/MSI, e.g. XEN and HyperV, implementations are affected either. They still work the same way both at the low level and the PCI/MSI implementations they provide. Signed-off-by: Thomas Gleixner Tested-by: Kalle Valo --- V2: Fix kernel doc (robot) --- arch/x86/include/asm/msi.h | 6 + arch/x86/include/asm/pci.h | 1 arch/x86/kernel/apic/msi.c | 176 ++++++++++++++++++++++++++---------- drivers/iommu/amd/iommu.c | 2 drivers/iommu/intel/irq_remapping.c | 2 5 files changed, 138 insertions(+), 49 deletions(-) --- a/arch/x86/include/asm/msi.h +++ b/arch/x86/include/asm/msi.h @@ -62,4 +62,10 @@ typedef struct x86_msi_addr_hi { struct msi_msg; u32 x86_msi_msg_get_destid(struct msi_msg *msg, bool extid); +#define X86_VECTOR_MSI_FLAGS_SUPPORTED \ + (MSI_GENERIC_FLAGS_MASK | MSI_FLAG_PCI_MSIX) + +#define X86_VECTOR_MSI_FLAGS_REQUIRED \ + (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS) + #endif /* _ASM_X86_MSI_H */ --- a/arch/x86/include/asm/pci.h +++ b/arch/x86/include/asm/pci.h @@ -92,6 +92,7 @@ void pcibios_scan_root(int bus); struct irq_routing_table *pcibios_get_irq_routing_table(void); int pcibios_set_irq_routing(struct pci_dev *dev, int pin, int irq); +bool pci_dev_has_default_msi_parent_domain(struct pci_dev *dev); #define HAVE_PCI_MMAP #define arch_can_pci_mmap_wc() pat_enabled() --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -142,67 +142,131 @@ msi_set_affinity(struct irq_data *irqd, return ret; } -/* - * IRQ Chip for MSI PCI/PCI-X/PCI-Express Devices, - * which implement the MSI or MSI-X Capability Structure. +/** + * pci_dev_has_default_msi_parent_domain - Check whether the device has the default + * MSI parent domain associated + * @dev: Pointer to the PCI device */ -static struct irq_chip pci_msi_controller = { - .name = "PCI-MSI", - .irq_unmask = pci_msi_unmask_irq, - .irq_mask = pci_msi_mask_irq, - .irq_ack = irq_chip_ack_parent, - .irq_retrigger = irq_chip_retrigger_hierarchy, - .irq_set_affinity = msi_set_affinity, - .flags = IRQCHIP_SKIP_SET_WAKE | - IRQCHIP_AFFINITY_PRE_STARTUP, -}; +bool pci_dev_has_default_msi_parent_domain(struct pci_dev *dev) +{ + struct irq_domain *domain = dev_get_msi_domain(&dev->dev); -int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, - msi_alloc_info_t *arg) + if (!domain) + domain = dev_get_msi_domain(&dev->bus->dev); + if (!domain) + return false; + + return domain == x86_vector_domain; +} + +/** + * x86_msi_prepare - Setup of msi_alloc_info_t for allocations + * @domain: The domain for which this setup happens + * @dev: The device for which interrupts are allocated + * @nvec: The number of vectors to allocate + * @alloc: The allocation info structure to initialize + * + * This function is to be used for all types of MSI domains above the x86 + * vector domain and any intermediates. It is always invoked from the + * top level interrupt domain. The domain specific allocation + * functionality is determined via the @domain's bus token which allows to + * map the X86 specific allocation type. + */ +static int x86_msi_prepare(struct irq_domain *domain, struct device *dev, + int nvec, msi_alloc_info_t *alloc) { - init_irq_alloc_info(arg, NULL); - if (to_pci_dev(dev)->msix_enabled) - arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; - else - arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; + struct msi_domain_info *info = domain->host_data; - return 0; + init_irq_alloc_info(alloc, NULL); + + switch (info->bus_token) { + case DOMAIN_BUS_PCI_DEVICE_MSI: + alloc->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; + return 0; + case DOMAIN_BUS_PCI_DEVICE_MSIX: + alloc->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; + return 0; + default: + return -EINVAL; + } } -EXPORT_SYMBOL_GPL(pci_msi_prepare); -static struct msi_domain_ops pci_msi_domain_ops = { - .msi_prepare = pci_msi_prepare, -}; +/** + * x86_init_dev_msi_info - Domain info setup for MSI domains + * @dev: The device for which the domain should be created + * @domain: The (root) domain providing this callback + * @real_parent: The real parent domain of the to initialize domain + * @info: The domain info for the to initialize domain + * + * This function is to be used for all types of MSI domains above the x86 + * vector domain and any intermediates. The domain specific functionality + * is determined via the @real_parent. + */ +static bool x86_init_dev_msi_info(struct device *dev, struct irq_domain *domain, + struct irq_domain *real_parent, struct msi_domain_info *info) +{ + const struct msi_parent_ops *pops = real_parent->msi_parent_ops; + + /* MSI parent domain specific settings */ + switch (real_parent->bus_token) { + case DOMAIN_BUS_ANY: + /* Only the vector domain can have the ANY token */ + if (WARN_ON_ONCE(domain != real_parent)) + return false; + info->chip->irq_set_affinity = msi_set_affinity; + /* See msi_set_affinity() for the gory details */ + info->flags |= MSI_FLAG_NOMASK_QUIRK; + break; + default: + WARN_ON_ONCE(1); + return false; + } + + /* Is the target supported? */ + switch(info->bus_token) { + case DOMAIN_BUS_PCI_DEVICE_MSI: + case DOMAIN_BUS_PCI_DEVICE_MSIX: + break; + default: + WARN_ON_ONCE(1); + return false; + } + + /* + * Mask out the domain specific MSI feature flags which are not + * supported by the real parent. + */ + info->flags &= pops->supported_flags; + /* Enforce the required flags */ + info->flags |= X86_VECTOR_MSI_FLAGS_REQUIRED; + + /* This is always invoked from the top level MSI domain! */ + info->ops->msi_prepare = x86_msi_prepare; + + info->chip->irq_ack = irq_chip_ack_parent; + info->chip->irq_retrigger = irq_chip_retrigger_hierarchy; + info->chip->flags |= IRQCHIP_SKIP_SET_WAKE | + IRQCHIP_AFFINITY_PRE_STARTUP; -static struct msi_domain_info pci_msi_domain_info = { - .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_PCI_MSIX | MSI_FLAG_NOMASK_QUIRK, - - .ops = &pci_msi_domain_ops, - .chip = &pci_msi_controller, - .handler = handle_edge_irq, - .handler_name = "edge", + info->handler = handle_edge_irq; + info->handler_name = "edge"; + + return true; +} + +static const struct msi_parent_ops x86_vector_msi_parent_ops = { + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED, + .init_dev_msi_info = x86_init_dev_msi_info, }; struct irq_domain * __init native_create_pci_msi_domain(void) { - struct fwnode_handle *fn; - struct irq_domain *d; - if (disable_apic) return NULL; - fn = irq_domain_alloc_named_fwnode("PCI-MSI"); - if (!fn) - return NULL; - - d = pci_msi_create_irq_domain(fn, &pci_msi_domain_info, - x86_vector_domain); - if (!d) { - irq_domain_free_fwnode(fn); - pr_warn("Failed to initialize PCI-MSI irqdomain.\n"); - } - return d; + x86_vector_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT; + x86_vector_domain->msi_parent_ops = &x86_vector_msi_parent_ops; + return x86_vector_domain; } void __init x86_create_pci_msi_domain(void) @@ -210,7 +274,25 @@ void __init x86_create_pci_msi_domain(vo x86_pci_msi_default_domain = x86_init.irqs.create_pci_msi_domain(); } +/* Keep around for hyperV and the remap code below */ +int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, + msi_alloc_info_t *arg) +{ + init_irq_alloc_info(arg, NULL); + + if (to_pci_dev(dev)->msix_enabled) + arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; + else + arg->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; + return 0; +} +EXPORT_SYMBOL_GPL(pci_msi_prepare); + #ifdef CONFIG_IRQ_REMAP +static struct msi_domain_ops pci_msi_domain_ops = { + .msi_prepare = pci_msi_prepare, +}; + static struct irq_chip pci_msi_ir_controller = { .name = "IR-PCI-MSI", .irq_unmask = pci_msi_unmask_irq, --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -812,7 +812,7 @@ static void amd_iommu_set_pci_msi_domain(struct device *dev, struct amd_iommu *iommu) { if (!irq_remapping_enabled || !dev_is_pci(dev) || - pci_dev_has_special_msi_domain(to_pci_dev(dev))) + !pci_dev_has_default_msi_parent_domain(to_pci_dev(dev))) return; dev_set_msi_domain(dev, iommu->msi_domain); --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -1107,7 +1107,7 @@ static int reenable_irq_remapping(int ei */ void intel_irq_remap_add_device(struct dmar_pci_notify_info *info) { - if (!irq_remapping_enabled || pci_dev_has_special_msi_domain(info->dev)) + if (!irq_remapping_enabled || !pci_dev_has_default_msi_parent_domain(info->dev)) return; dev_set_msi_domain(&info->dev->dev, map_dev_to_ir(info->dev)); From patchwork Thu Nov 24 23:26:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3663105wrr; Thu, 24 Nov 2022 15:35:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf5DACFO64A/1vdy1TO5H/uXTygjhWwgXzzRfvZVN4uDM5UesuTOHfiC7EGyPY1S2/FlfNmf X-Received: by 2002:a17:906:c18c:b0:7b2:8a6e:c569 with SMTP id g12-20020a170906c18c00b007b28a6ec569mr29796744ejz.582.1669332901853; Thu, 24 Nov 2022 15:35:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332901; cv=none; d=google.com; s=arc-20160816; b=CGt5bpXePjwhu4XGvBnNtxOS8XZ4f9L0tMSN9SqC/CSBhFOksbMvTnK6zD5VXJm75A httFqVCDnkHnRcagrvOLVgWMdJL1+jWIU0Et6IWiE4TfOxycN16pydBJnywAZZFvsNle KHOK8ky0TP4lxi2vpu3TdC30VD8q8iQJY3N27UhbO9K1LXeQMwyJezZxp4aTqsDpQteu PyFpRwIdfDIMWfIk96XGxgvcP0xqwJnDd2puqL4vaIeINNv3UkfDwM/hup8pPodD/mFZ qcYkgeBK+Sh6q1mHD3kneUTrboSXNkdGoBkLbPsG9Vl1rs1aviVHH6zijN9QiSm8D50R GLCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=yBkAsYnt2I7B5+/On6wDlCIdm7qhkb9RizMNRpsl694=; b=hwV/HwrcbBsiJeyS6Ki26K97OkHaMEAjt1EfIadBthnQnzLPRH9Y0VxeqBEWClGT4A HMFbpzLCzAk/dxJbGqVaORKvYgC9DY2DbL7jxJCDuGyokuER+7Nm+Wfny3+90W7hPq27 QXdwZYR9vSaEZ86k0iCmpYyYu+d3mq/KUfgNpoDy0S9q8KvYvmb3FOy6KDnXo1LDOh5V x564ufbLpNSOTPMc3wIdPCG5FaR6ra21ZAe/5Nw0xTxK0epSOz8DiPaq98QZHH8aLGBb syVGQlpaVZYITRDK9Btj7546GS9Tj4p79w2Uwrc3MHzWC+Lcn+mlix415uxUSkMUaQr9 LiyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=HrvsEqLs; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=llw9Gy4L; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g22-20020a056402321600b0046150eb704csi1952329eda.532.2022.11.24.15.34.38; Thu, 24 Nov 2022 15:35:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=HrvsEqLs; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=llw9Gy4L; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229989AbiKXX3A (ORCPT + 99 others); Thu, 24 Nov 2022 18:29:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229890AbiKXX2e (ORCPT ); Thu, 24 Nov 2022 18:28:34 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7965A314C; Thu, 24 Nov 2022 15:26:52 -0800 (PST) Message-ID: <20221124232326.093093200@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yBkAsYnt2I7B5+/On6wDlCIdm7qhkb9RizMNRpsl694=; b=HrvsEqLsToIp55l3cj4SBWy8qS2MjQlZnu+8IatPQjHQbT5uVUsqrUHw9nH6PpX/P/PkWz TnGpDhRs1+2PE4R3ZHDZ1dRucY0efo1AYOp1P7WF0EEvbuXHGnnh6Yu41N5zq1dqKiTFGp vFFxvKBl6IKDjA5e9loUG+K23k4c4D2zEDkm9cenDZiLLkyYxQwEpkbVld2RId3hYK2eTF b2xKPBf4BemHdE9mVtPyjoYZEraUmFmNltveXAtSkoHCEldWQrYyf+ts7Ghify5lufLdsm ux+UqPSxSlKE87V+xdu/TBe9bXdUlCVlgqrMLBhYIqlieOz6rGJUOW/LU+t6AA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yBkAsYnt2I7B5+/On6wDlCIdm7qhkb9RizMNRpsl694=; b=llw9Gy4LLy7Jis6eY6x6b5HwgSNES0ZONTNs/hP8H+ArWsmIZZ/Q3JtM3eohnehUT/kcMN idqiyWXU1AMeo9Dw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 14/33] PCI/MSI: Remove unused pci_dev_has_special_msi_domain() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:07 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422416455956642?= X-GMAIL-MSGID: =?utf-8?q?1750422416455956642?= The check for special MSI domains like VMD which prevents the interrupt remapping code to overwrite device::msi::domain is not longer required and has been replaced by an x86 specific version which is aware of MSI parent domains. Remove it. Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas --- drivers/pci/msi/irqdomain.c | 21 --------------------- include/linux/msi.h | 1 - 2 files changed, 22 deletions(-) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -414,24 +414,3 @@ struct irq_domain *pci_msi_get_device_do DOMAIN_BUS_PCI_MSI); return dom; } - -/** - * pci_dev_has_special_msi_domain - Check whether the device is handled by - * a non-standard PCI-MSI domain - * @pdev: The PCI device to check. - * - * Returns: True if the device irqdomain or the bus irqdomain is - * non-standard PCI/MSI. - */ -bool pci_dev_has_special_msi_domain(struct pci_dev *pdev) -{ - struct irq_domain *dom = dev_get_msi_domain(&pdev->dev); - - if (!dom) - dom = dev_get_msi_domain(&pdev->bus->dev); - - if (!dom) - return true; - - return dom->bus_token != DOMAIN_BUS_PCI_MSI; -} --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -609,7 +609,6 @@ struct irq_domain *pci_msi_create_irq_do struct irq_domain *parent); u32 pci_msi_domain_get_msi_rid(struct irq_domain *domain, struct pci_dev *pdev); struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev); -bool pci_dev_has_special_msi_domain(struct pci_dev *pdev); #else /* CONFIG_PCI_MSI */ static inline struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev) { From patchwork Thu Nov 24 23:26:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25754 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3662592wrr; Thu, 24 Nov 2022 15:33:38 -0800 (PST) X-Google-Smtp-Source: AA0mqf7NEsj2Fd4M857oMAsSGQiHycJLS8AVsrNdqYnRLIQMnOqpfVBSwzz6/pAvTzQtGxApd0Kl X-Received: by 2002:a17:90a:1bc3:b0:218:fdd7:6ded with SMTP id r3-20020a17090a1bc300b00218fdd76dedmr4361534pjr.240.1669332817997; Thu, 24 Nov 2022 15:33:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332817; cv=none; d=google.com; s=arc-20160816; b=MMhLqiDkrNQwKHoA+lVFzCYsm60YOYcKms7+frukjNLcsoNwv0O3xeCG7SvOGuMe7U kwn1beScUqzAfaO/0EMhqnkzxne3Lk/0yD2emipxtA3uc/Bdnf3Oyx3gFmwwS7ac5N53 Gx3Pw6r2MxP5kWz9fVouU3bgGvCBzMDfccagv/H9qFbA+MT/yNrXgns9Hti9xjwN0bWV Vh7sZTxeY45USjhJGXkPL6wh1xjgCtVsjyRufyoKyzsJpkwOGknInqDRxaDGvwTNHoUc d5uy50XCaG2vm1i9ljAiVOrMvf7vYDKmO+zYMZCI3Ymk+P9tDZeCbAkWSbctmFzH/qT+ 8zyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=OWDR1HVKygNBsw9mLHuKFZNGcf87XagJ2m0BnhitI40=; b=V2CKB3a3LC4ZcqTofjSemG+A6HPpY0noIkVqHUTkyJfy8eR/3B7o1U8vshBvq7VvjX 0VYIB6d/WQS2qxFbeqwoBiMDZJKFBQ7LgU0LUO+D9+BfSyf+og8/tL/HEoYhToRdJAAE XkPSV1CFgNbx04IKCWxu1yQGqnXhpTuqL56dMdo9IVCV5ClSmsjvb8g3LyRwd+srbuZy kJ86OrmtuDzQczoScKKJwDYGe6aBREc5Z+/N/GXefQgsLnQuS2NUDJqyMmZIft8DpZoy NC/Uizv5pyXxsUvhxZ6iAYyHGTi6DSfVC/UNfIw7G9M8Fjk9+AKPaacXGWViOy9aq53x C16A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=k2Hm71vw; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cv5-20020a056a0044c500b00573ede1a4easi2139379pfb.88.2022.11.24.15.33.24; Thu, 24 Nov 2022 15:33:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=k2Hm71vw; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229825AbiKXX3U (ORCPT + 99 others); Thu, 24 Nov 2022 18:29:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229956AbiKXX2m (ORCPT ); Thu, 24 Nov 2022 18:28:42 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A08F288B4C; Thu, 24 Nov 2022 15:27:07 -0800 (PST) Message-ID: <20221124232326.151226317@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=OWDR1HVKygNBsw9mLHuKFZNGcf87XagJ2m0BnhitI40=; b=k2Hm71vwQsshTJDcXJRmzGfP1wzRiu4qghZe5twiFM/SUAzEK11Cr8Xf9dUIBkOSE0ZSWp aH33JI2dkSwy42CuU+YMSKxYQx+eWcOEFGuGkpUhHwIcbb263HiVPpgdlacPJREpuIpbLG P8Ky9bVtYbNb1m6GCWXB4YwYIqVfBsYldrxFrrZ4OoZiQWTn4ly9YSKcA0mvvTwebKCMeE IOPkBCwZZvtViI8ywxy9VJvHdkS45jL25yJ0XBP7HU/X7tbrpgXzd8PHRQKDGKKu5kDopN 9FzqT4nl/0xOuilEXPxFUOaUqLK8m9YcYm3200dD3NX2M/S+1QhLfkBGW88EHg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=OWDR1HVKygNBsw9mLHuKFZNGcf87XagJ2m0BnhitI40=; b=doPRitIv6rrL8P5Uy/AEb/D71FczzNKcoTl9PBsJmGGvst0mcO/7a/Heh5qVx62ECNDVzq svkOuNyxcxjn6XAQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 15/33] iommu/vt-d: Switch to MSI parent domains References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:08 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422329223804423?= X-GMAIL-MSGID: =?utf-8?q?1750422329223804423?= Remove the global PCI/MSI irqdomain implementation and provide the required MSI parent ops so the PCI/MSI code can detect the new parent and setup per device domains. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/apic/msi.c | 2 ++ drivers/iommu/intel/iommu.h | 1 - drivers/iommu/intel/irq_remapping.c | 27 ++++++++++++--------------- include/linux/irqdomain_defs.h | 1 + 4 files changed, 15 insertions(+), 16 deletions(-) --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -217,6 +217,8 @@ static bool x86_init_dev_msi_info(struct /* See msi_set_affinity() for the gory details */ info->flags |= MSI_FLAG_NOMASK_QUIRK; break; + case DOMAIN_BUS_DMAR: + break; default: WARN_ON_ONCE(1); return false; --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -600,7 +600,6 @@ struct intel_iommu { #ifdef CONFIG_IRQ_REMAP struct ir_table *ir_table; /* Interrupt remapping info */ struct irq_domain *ir_domain; - struct irq_domain *ir_msi_domain; #endif struct iommu_device iommu; /* IOMMU core code handle */ int node; --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -82,6 +82,7 @@ static const struct irq_domain_ops intel static void iommu_disable_irq_remapping(struct intel_iommu *iommu); static int __init parse_ioapics_under_ir(void); +static const struct msi_parent_ops dmar_msi_parent_ops; static bool ir_pre_enabled(struct intel_iommu *iommu) { @@ -230,7 +231,7 @@ static struct irq_domain *map_dev_to_ir( { struct dmar_drhd_unit *drhd = dmar_find_matched_drhd_unit(dev); - return drhd ? drhd->iommu->ir_msi_domain : NULL; + return drhd ? drhd->iommu->ir_domain : NULL; } static int clear_entries(struct irq_2_iommu *irq_iommu) @@ -573,10 +574,10 @@ static int intel_setup_irq_remapping(str pr_err("IR%d: failed to allocate irqdomain\n", iommu->seq_id); goto out_free_fwnode; } - iommu->ir_msi_domain = - arch_create_remap_msi_irq_domain(iommu->ir_domain, - "INTEL-IR-MSI", - iommu->seq_id); + + irq_domain_update_bus_token(iommu->ir_domain, DOMAIN_BUS_DMAR); + iommu->ir_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT; + iommu->ir_domain->msi_parent_ops = &dmar_msi_parent_ops; ir_table->base = page_address(pages); ir_table->bitmap = bitmap; @@ -620,9 +621,6 @@ static int intel_setup_irq_remapping(str return 0; out_free_ir_domain: - if (iommu->ir_msi_domain) - irq_domain_remove(iommu->ir_msi_domain); - iommu->ir_msi_domain = NULL; irq_domain_remove(iommu->ir_domain); iommu->ir_domain = NULL; out_free_fwnode: @@ -644,13 +642,6 @@ static void intel_teardown_irq_remapping struct fwnode_handle *fn; if (iommu && iommu->ir_table) { - if (iommu->ir_msi_domain) { - fn = iommu->ir_msi_domain->fwnode; - - irq_domain_remove(iommu->ir_msi_domain); - irq_domain_free_fwnode(fn); - iommu->ir_msi_domain = NULL; - } if (iommu->ir_domain) { fn = iommu->ir_domain->fwnode; @@ -1437,6 +1428,12 @@ static const struct irq_domain_ops intel .deactivate = intel_irq_remapping_deactivate, }; +static const struct msi_parent_ops dmar_msi_parent_ops = { + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | MSI_FLAG_MULTI_PCI_MSI, + .prefix = "IR-", + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; + /* * Support of Interrupt Remapping Unit Hotplug */ --- a/include/linux/irqdomain_defs.h +++ b/include/linux/irqdomain_defs.h @@ -23,6 +23,7 @@ enum irq_domain_bus_token { DOMAIN_BUS_VMD_MSI, DOMAIN_BUS_PCI_DEVICE_MSI, DOMAIN_BUS_PCI_DEVICE_MSIX, + DOMAIN_BUS_DMAR, }; #endif /* _LINUX_IRQDOMAIN_DEFS_H */ From patchwork Thu Nov 24 23:26:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25751 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661825wrr; Thu, 24 Nov 2022 15:31:39 -0800 (PST) X-Google-Smtp-Source: AA0mqf6PrhNxc5UjVZc8QzscnFNUP+H1dfkNn3UG2wYVR8YMfecDrW+uDV8q7oVf7+BRsgKCd/KF X-Received: by 2002:a17:907:a044:b0:7bc:27ab:6b2d with SMTP id gz4-20020a170907a04400b007bc27ab6b2dmr1552294ejc.750.1669332699633; Thu, 24 Nov 2022 15:31:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332699; cv=none; d=google.com; s=arc-20160816; b=ibnP+F9h+B7tiem4inFNQ4Dxzs+KSQN4pwLU9bO8GtamLFbcu7+CB89Bii7FIyYcht +xDxBUxONuuXrc3uEPL1nbwG4+RBLSwOFjguGTASayUq+/mjz/mVMgWxcpr8r5tYDvI+ zVdMxD/JAr4GQZdGefwaEd2PGcOxjsFsgtHcOkGJGTI6Y99kkL7WdYwz08MfJdgdc07u N9Wpw6MQarfduH3c6n6Bhfe22Vc07s1xUy62/AQ9VMlz5nc0s5qGLzMfGg4qBcQE6ekv 01QDj39//bOHFlXt7NF6aii5jTlmXg9jO3Dn3AiZnDQpHcRRueUkZ4nJL/D1CGA9XfX5 OIEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=5F8g3aU97Z6Kyag3K9E0SkLjFZeBlcTh6td6sXBxDio=; b=bq0JMaXf1M5Vrs7hkt2c3W5Gj5Rgu9Kn5JWOG03iSEBFfcYLx9/SVxu02WEi+y8MnX Xr7Eqr9B2hJFZ2jjKPOv8sJEqbRg3e9T7JN8Pfh0gzrBrtpBgAR1otNcIm5KQ+0E47cE ClOLYOpxR+VbRNyeQzA/h4lGYGJggW730hv1EFnWGE+no25NFRPRwVTwtjxW/YrAYzOF gUyyCxpL5p7S1Wef8clbYrb0Dah7ECoPqKQ05HnI5C6qP5ESpTnfAy63Xd1id2F7iWcC j+65gkEGb2A6rubGhCv/hhCfL6mtoe6GCeE1omwL+9LSDy+U1Km5qXZYuADfVuYRV592 nbBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=0QMWvruG; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j8-20020a50ed08000000b00461fa05b004si1717871eds.105.2022.11.24.15.31.16; Thu, 24 Nov 2022 15:31:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=0QMWvruG; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230023AbiKXX24 (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229925AbiKXX2g (ORCPT ); Thu, 24 Nov 2022 18:28:36 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EB0BA4647; Thu, 24 Nov 2022 15:26:54 -0800 (PST) Message-ID: <20221124232326.209212272@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5F8g3aU97Z6Kyag3K9E0SkLjFZeBlcTh6td6sXBxDio=; b=0QMWvruGb02j676y+kWV25htNvaj8TmHud0EjmSbeC/1Le4CRZC4eKuKyi2LcekxOhbpTQ qsZxoMGfFuGD3tdd3E8AdpdU9tROBwv/dRDavxeAAlV9Bk5YQkK3wH5lr6cBip285YMFtE QNqJzcEV+jr8AmC/lVxDvC9JEYGL4IhhFbGOxjGY9Z4DAJJStmfdt0DEuqCgCwnNrMYUJk I30R0A6wmSIjW9PEW9JCB4xquVWzv7LAgP4Xucsw/LYVkA8MIbkAjwFOOXjbqpFYMDYGmr gnNma2d3IrQu++q3SPqnzeEm8LuRq/046CTN99GKR5nENMDqIuZdJX6eqRrs3g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5F8g3aU97Z6Kyag3K9E0SkLjFZeBlcTh6td6sXBxDio=; b=ClTfWrEIaY4P9KjStHk0kUB1gM3OS1WdjBMYYFGyuGxQI38LBrOm11KiYhGmCnOAVorooc jCWQ/zZcLoaqqaCw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 16/33] iommu/amd: Switch to MSI base domains References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:10 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422204605415396?= X-GMAIL-MSGID: =?utf-8?q?1750422204605415396?= Remove the global PCI/MSI irqdomain implementation and provide the required MSI parent ops so the PCI/MSI code can detect the new parent and setup per device domains. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/apic/msi.c | 1 + drivers/iommu/amd/amd_iommu_types.h | 1 - drivers/iommu/amd/iommu.c | 19 +++++++++++++------ include/linux/irqdomain_defs.h | 1 + 4 files changed, 15 insertions(+), 7 deletions(-) --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -218,6 +218,7 @@ static bool x86_init_dev_msi_info(struct info->flags |= MSI_FLAG_NOMASK_QUIRK; break; case DOMAIN_BUS_DMAR: + case DOMAIN_BUS_AMDVI: break; default: WARN_ON_ONCE(1); --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -734,7 +734,6 @@ struct amd_iommu { u8 max_counters; #ifdef CONFIG_IRQ_REMAP struct irq_domain *ir_domain; - struct irq_domain *msi_domain; struct amd_irte_ops *irte_ops; #endif --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -815,7 +815,7 @@ amd_iommu_set_pci_msi_domain(struct devi !pci_dev_has_default_msi_parent_domain(to_pci_dev(dev))) return; - dev_set_msi_domain(dev, iommu->msi_domain); + dev_set_msi_domain(dev, iommu->ir_domain); } #else /* CONFIG_IRQ_REMAP */ @@ -3648,6 +3648,12 @@ static struct irq_chip amd_ir_chip = { .irq_compose_msi_msg = ir_compose_msi_msg, }; +static const struct msi_parent_ops amdvi_msi_parent_ops = { + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | MSI_FLAG_MULTI_PCI_MSI, + .prefix = "IR-", + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; + int amd_iommu_create_irq_domain(struct amd_iommu *iommu) { struct fwnode_handle *fn; @@ -3655,16 +3661,17 @@ int amd_iommu_create_irq_domain(struct a fn = irq_domain_alloc_named_id_fwnode("AMD-IR", iommu->index); if (!fn) return -ENOMEM; - iommu->ir_domain = irq_domain_create_tree(fn, &amd_ir_domain_ops, iommu); + iommu->ir_domain = irq_domain_create_hierarchy(arch_get_ir_parent_domain(), 0, 0, + fn, &amd_ir_domain_ops, iommu); if (!iommu->ir_domain) { irq_domain_free_fwnode(fn); return -ENOMEM; } - iommu->ir_domain->parent = arch_get_ir_parent_domain(); - iommu->msi_domain = arch_create_remap_msi_irq_domain(iommu->ir_domain, - "AMD-IR-MSI", - iommu->index); + irq_domain_update_bus_token(iommu->ir_domain, DOMAIN_BUS_AMDVI); + iommu->ir_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT; + iommu->ir_domain->msi_parent_ops = &amdvi_msi_parent_ops; + return 0; } --- a/include/linux/irqdomain_defs.h +++ b/include/linux/irqdomain_defs.h @@ -24,6 +24,7 @@ enum irq_domain_bus_token { DOMAIN_BUS_PCI_DEVICE_MSI, DOMAIN_BUS_PCI_DEVICE_MSIX, DOMAIN_BUS_DMAR, + DOMAIN_BUS_AMDVI, }; #endif /* _LINUX_IRQDOMAIN_DEFS_H */ From patchwork Thu Nov 24 23:26:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25753 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3662402wrr; Thu, 24 Nov 2022 15:33:05 -0800 (PST) X-Google-Smtp-Source: AA0mqf4ya9SSYZhdlVS52sXV0b2xhOBs7tyh+rqodfgiklpxzYDueqwQ0TtzLt0kaqXfMsXcBWY4 X-Received: by 2002:a63:5d58:0:b0:46f:f383:6faf with SMTP id o24-20020a635d58000000b0046ff3836fafmr31639224pgm.247.1669332784792; Thu, 24 Nov 2022 15:33:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332784; cv=none; d=google.com; s=arc-20160816; b=hye+ZTELv9GWZY4Q0vCfF3BpT0Bd/8LxC2W2LC0WPaGGakJUUZrJbl1lni4p4t/VLo MzsPXf5oa1igqRYb06pJ8SNYkkptERBcl3/Tlsc/jrGy/N/k6rpcDCkei01qvQkfi0ov koMD7fwYrybqzKwT977SVSNNv48+6MSVtyteL+m6R+TsFSSb1mtKRbw7BzGkHQrcdfcb yKbNLuv4IOspHOOb3sRXCVm5aLKbjOJTFCc8Cv+x+oqXUl3+M4P+KMcDhbEnZdLQhw/z SJ7cLQzzROZw4l5HxAk+AK4bepewuKuQhonfuHaJ3x5UxmppqEFK1jwSOhpovQvE8D+V /QTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=T4PJKhvy3BXQPG8m3pythcilM8fEjCtnrP9dFwguEN8=; b=soHeXor78ihlzHShRvO7bUXG/EZmhPI57UcdH1tAEiyMpqUAkgb4ATjKGottfWsdIH pFiQGk4gzk+GyukR/rdFQoTWf53Fx2+8c/OPjAjrQpoBKuQSA57EtUTfIVaF/nQC+U49 fBe8sh07j1Ys0YscWaepzttOtyqexfpp+rzUyTxjYgkGyUHIltGPh/S3sgJjDK6A3FAj MmiSQmPoVJU7wNeQmbl1etUG5Bv7XAdWUDvy6cjgGts+o/ROiEgT+WZFHPd0N/D6ETSv UNODdvN4ntDDJGcrRKhudrOi5N5M2gUHJvg8XpdNUTXm+6dDpLAmXMFOAuQawdkR46K9 Isyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tE5N3oRK; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=cNANfcEE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f14-20020a056a00228e00b0056dde9895e1si2390293pfe.46.2022.11.24.15.32.51; Thu, 24 Nov 2022 15:33:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tE5N3oRK; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=cNANfcEE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230032AbiKXX26 (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229820AbiKXX2h (ORCPT ); Thu, 24 Nov 2022 18:28:37 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 476FDBEB56; Thu, 24 Nov 2022 15:26:55 -0800 (PST) Message-ID: <20221124232326.267353814@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=T4PJKhvy3BXQPG8m3pythcilM8fEjCtnrP9dFwguEN8=; b=tE5N3oRKOjAHX3105MXBVaC6+dWNgZIv/gC3SLPLHGV9ku6tiH8UgVNUZUw4eUi0hNUsdF J4MDDtMgIfGtViejpi7MDiZj4o5R9Cun9n19TmVjFqsd2lSdjJ8U3ZE56rrCl1Mght7U2c deEGJKrWNzqvaHaMd+gQbQ7IZlslZK9UlFovTBdBWvbIw5Z/vxMPcsdAlXxgckkku25wK+ RBs2mjNbi8XYLRVY6Hxo+ptdaQlOttulx6RpewYFyQQqVoUEjxBpp/G35yIT7XMJ54MZwI RcrJpPbg7DrgvXme31A3Y8AOJN/GxqkIS4/jr+Bv7cb2p88X0Do4knWgM2djOw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=T4PJKhvy3BXQPG8m3pythcilM8fEjCtnrP9dFwguEN8=; b=cNANfcEEQE27ODvsgqRrQZDo0i8y9EQqNxujvg/ptmFNjv5/jDyNf8h1PL4mo3ZyDmEnbC AE6aJCfbovNKWNAg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 17/33] x86/apic/msi: Remove arch_create_remap_msi_irq_domain() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:12 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422294254216103?= X-GMAIL-MSGID: =?utf-8?q?1750422294254216103?= and related code which is not longer required now that the interrupt remap code has been converted to MSI parent domains. Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/irq_remapping.h | 4 --- arch/x86/kernel/apic/msi.c | 42 ----------------------------------- 2 files changed, 1 insertion(+), 45 deletions(-) --- a/arch/x86/include/asm/irq_remapping.h +++ b/arch/x86/include/asm/irq_remapping.h @@ -44,10 +44,6 @@ extern int irq_remapping_reenable(int); extern int irq_remap_enable_fault_handling(void); extern void panic_if_irq_remap(const char *msg); -/* Create PCI MSI/MSIx irqdomain, use @parent as the parent irqdomain. */ -extern struct irq_domain * -arch_create_remap_msi_irq_domain(struct irq_domain *par, const char *n, int id); - /* Get parent irqdomain for interrupt remapping irqdomain */ static inline struct irq_domain *arch_get_ir_parent_domain(void) { --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -277,7 +277,7 @@ void __init x86_create_pci_msi_domain(vo x86_pci_msi_default_domain = x86_init.irqs.create_pci_msi_domain(); } -/* Keep around for hyperV and the remap code below */ +/* Keep around for hyperV */ int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) { @@ -291,46 +291,6 @@ int pci_msi_prepare(struct irq_domain *d } EXPORT_SYMBOL_GPL(pci_msi_prepare); -#ifdef CONFIG_IRQ_REMAP -static struct msi_domain_ops pci_msi_domain_ops = { - .msi_prepare = pci_msi_prepare, -}; - -static struct irq_chip pci_msi_ir_controller = { - .name = "IR-PCI-MSI", - .irq_unmask = pci_msi_unmask_irq, - .irq_mask = pci_msi_mask_irq, - .irq_ack = irq_chip_ack_parent, - .irq_retrigger = irq_chip_retrigger_hierarchy, - .flags = IRQCHIP_SKIP_SET_WAKE | - IRQCHIP_AFFINITY_PRE_STARTUP, -}; - -static struct msi_domain_info pci_msi_ir_domain_info = { - .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, - .ops = &pci_msi_domain_ops, - .chip = &pci_msi_ir_controller, - .handler = handle_edge_irq, - .handler_name = "edge", -}; - -struct irq_domain *arch_create_remap_msi_irq_domain(struct irq_domain *parent, - const char *name, int id) -{ - struct fwnode_handle *fn; - struct irq_domain *d; - - fn = irq_domain_alloc_named_id_fwnode(name, id); - if (!fn) - return NULL; - d = pci_msi_create_irq_domain(fn, &pci_msi_ir_domain_info, parent); - if (!d) - irq_domain_free_fwnode(fn); - return d; -} -#endif - #ifdef CONFIG_DMAR_TABLE /* * The Intel IOMMU (ab)uses the high bits of the MSI address to contain the From patchwork Thu Nov 24 23:26:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25750 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3661804wrr; Thu, 24 Nov 2022 15:31:37 -0800 (PST) X-Google-Smtp-Source: AA0mqf73ozKdUXzXJ05QrF/M3F8TzFMlLyLrEVlkkcxd2pM61HhvNS83cT0i4pz8yWA5u5DiJ6Ty X-Received: by 2002:a05:6402:2024:b0:468:f633:9484 with SMTP id ay4-20020a056402202400b00468f6339484mr31573060edb.200.1669332697550; Thu, 24 Nov 2022 15:31:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332697; cv=none; d=google.com; s=arc-20160816; b=fVscUiKwSFp/VZQUottop716cX47mIzNWcxVDARj3S/59B26C6qG+ddnVopr5D0QEO JIzdkTZa0fEBAXvRCwUpYSiJShE4A+dw9cutz9rk3YqvS3F+K95/ceWsPRnW1varV7Ed cAIQgFdmYOPe6acw/oG9edVdlOyuojKw00hDeqeevcc3MldwNLCIpZekGGDO3Z15BsQI yEWQWvfmy0Ju+aKDwlWqQh40OTqitnxpAuJnfxse99OI6KkyKN0VVKS7kbFeKANY0ix+ iw5woBKqYJMxDsef7gt1uQkDa1K9uQQAVXy8bwx/q2/Ht3thLC6vIHVIsm9fEIc9y3N+ YAlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=cFx6dbaS3ynUGuWTq2v8hXwBFa/HBDy+17s0E2mkPoM=; b=efk6n75d/Rc0QiA5tv0jAldAci3TRRHro4j6RbanMl/chjRRxRqQzupYZFbRIplD6e mL4AhtH9hvv2vwf05EhxQmNxXkT66gGkPmOx1V9DfcNIm3SqM3eLhv3oTTK8MuI0WjNn Rehynmn9UL8dGJRX2zFh09PbH3uKyBfB+sxcFj5VfFFmiCDVaG+dtI5skHrN/OOYmeRp ivS+A7pRIwtdJvMVZvnZs9a8duncx9gHQXbmuRq85tsNNytAZBZkyEcrhRMYjQNV2Rzh GV1R1iHBQ/n1Fj4t0Zoa9cibbsP3HfRbFlfISZJLiBJmccT9+9uVi7ATdAhZK5Z5uBzB lfdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=31K5q8uM; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ds11-20020a170907724b00b0073de493b83esi2192507ejc.147.2022.11.24.15.31.13; Thu, 24 Nov 2022 15:31:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=31K5q8uM; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229711AbiKXX2w (ORCPT + 99 others); Thu, 24 Nov 2022 18:28:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229899AbiKXX2f (ORCPT ); Thu, 24 Nov 2022 18:28:35 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60ACDBFF64; Thu, 24 Nov 2022 15:26:54 -0800 (PST) Message-ID: <20221124232326.326410494@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332374; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=cFx6dbaS3ynUGuWTq2v8hXwBFa/HBDy+17s0E2mkPoM=; b=31K5q8uMuBncINZnLxPkhytuZEEXrW9lk0LdW6v3qDdvlw2PUjFKemzgH5YU8uEMr6zOCz mQQgK/1O//+uOU9hKmvMuo3lRgaisQFZC/qyQVHnfUZA7nhR0aUxf2n5GdcJuRqlOa8Pzx HzF+S8MFzbZnZp4F7vket2pvkCZxlRcwHQDn9ENI9xi7hLjgL5rfXBKJz0qqVoffFhPYYH BeJVbFixw9uMF+IYOEhAk5da3hIkuoXvEICPY/q2cQGRG4SLuDRAervbnrlziMQ+8/1jfc KanIfjMThPO9/ciqXi7uVCWKveu2lzPaw+wKZ3rJxOTj/k6blJN8C3TWSpLf/Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332374; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=cFx6dbaS3ynUGuWTq2v8hXwBFa/HBDy+17s0E2mkPoM=; b=kicHVkBwhMOxAjBHtI5ZkgE0oos+EEIN5rtJ/aaOwuNNNa3ML2NdxyuZ+k0klKqWjbRLZd t0xNj4Vu7AN/BdAw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 18/33] genirq/msi: Provide struct msi_map References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:13 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422202677276125?= X-GMAIL-MSGID: =?utf-8?q?1750422202677276125?= A simple struct to hold a MSI index / Linux interrupt number pair. It will be returned from the dynamic vector allocation function and handed back to the corresponding free() function. Signed-off-by: Thomas Gleixner --- include/linux/msi_api.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/include/linux/msi_api.h +++ b/include/linux/msi_api.h @@ -18,6 +18,19 @@ enum msi_domain_ids { MSI_MAX_DEVICE_IRQDOMAINS, }; +/** + * msi_map - Mapping between MSI index and Linux interrupt number + * @index: The MSI index, e.g. slot in the MSI-X table or + * a software managed index if >= 0. If negative + * the allocation function failed and it contains + * the error code. + * @virq: The associated Linux interrupt number + */ +struct msi_map { + int index; + int virq; +}; + unsigned int msi_domain_get_virq(struct device *dev, unsigned int domid, unsigned int index); /** From patchwork Thu Nov 24 23:26:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25757 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3663384wrr; Thu, 24 Nov 2022 15:35:57 -0800 (PST) X-Google-Smtp-Source: AA0mqf7WEdkjQt2qZ1uPBGU9QM71J0x8iJeAnpCuxZz2jJ4Z5sqzELzTo+ppZ5cm7p9yptKDF3fH X-Received: by 2002:a17:90a:cb11:b0:218:d772:ec60 with SMTP id z17-20020a17090acb1100b00218d772ec60mr12798079pjt.143.1669332957415; Thu, 24 Nov 2022 15:35:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332957; cv=none; d=google.com; s=arc-20160816; b=xrJvDYjYdzSLFxMnzH+ObLxKbp3HZmWoFLioevFSs3DKdiLw8LDrHM51K8daOPDKl5 ZEzniqCvDXxuLM91jkktWfhxCdmdtagzWiFLm8/Zcdxs25KmeG0cfWAC7ZlzpAtcbQa4 ryF0Avp1Jw9+a+Kq/FG4kxIQFQ8hzuA7Q3dqN3ABoEutJnqCfdMXO6lnKiBf8j28GYC2 siQ1+hNvVioQKIBLcaW749DtOLE4CJGV/Q348D9yYZjVYebspCKLCFLa+AZCGUMEQ/aq ENtg/99XMEccfk78bmiAqDKfcbTB3Fg7TnAH4WRSG7H4/e31Piy3ZG1Lqfd5KzWUBuHo t05g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=EqT6Jp+en7WSY8cSjz97kH8nOGu2P/AZ035PFV/I8FU=; b=ZQD4NOoLcQV6s6tiMvhsMm5ZjFLvsfKXvOWutBB99UG8cR7CBl8GKvOv7pfzLNweev /ybBoAuw6w+U8Ay/0MIoCN/NzGvFZQU83S9/sU22FO1EX/CNck2WdHOZVlfF6vtoILdO CKwOZl5HErlGNBu9TysktPsT7IdRLlA16STlqZTVYRT9N9fN4JmtmbFv0CiIn6xJTtnm +K9is6wQfBagYl3l3+jOFrMD0La6gA3TjHHyDcH4q1rG9KIfQrPCLYL2Jszhge+8rnhM mKJWUSR45QKE08UyXEk8FCx0LU8n2GH2eLIuxyIWwg3wgVbWzxmLT1+sWiTdyHjCs+i0 sDpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="ybChU8/X"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=qRfykJij; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s18-20020a056a0008d200b00573f637f57asi1190477pfu.238.2022.11.24.15.35.44; Thu, 24 Nov 2022 15:35:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="ybChU8/X"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=qRfykJij; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229911AbiKXX3Z (ORCPT + 99 others); Thu, 24 Nov 2022 18:29:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229807AbiKXX2p (ORCPT ); Thu, 24 Nov 2022 18:28:45 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7812A6A0F; Thu, 24 Nov 2022 15:27:11 -0800 (PST) Message-ID: <20221124232326.385036043@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EqT6Jp+en7WSY8cSjz97kH8nOGu2P/AZ035PFV/I8FU=; b=ybChU8/XTp45uEtLOm0cjM7PJqC8y1syCV4cmMLPZ+y78SQu+5s4SbJ/zBrNJmpEUwA2z2 1WhtRA+LNHl6snqrjQ6wCYxf/i8VK5e73gcJBIhfnfqxvWuTafny+bBBcWBZAtx8PlUyva ZtpCLUGgwm9J16LlP6jijqVkJTo25t6sHInznYWmlE+Xn71fYLIxQ+QQoLBHSbHgntaoxP v+C1gBEfs/b3f6JWSZzki8Moz5CjxNeMKgT30DaS/60F83khY0/VKs9ZHQMZ00VClcBSw3 qYSySUtQWKWhDupHDI9MU+2RhtO22xucbaoDG7BYO+o5ZOYlhx+TzJ3kHBtOpQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EqT6Jp+en7WSY8cSjz97kH8nOGu2P/AZ035PFV/I8FU=; b=qRfykJijzo2B6VS+E4Lf1XhFh+X+lVExfqwUIw718z9n6snR4ObIQuwz3/vIaqnvJZWKgE R0q9vJjmTY7Gu1CQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 19/33] genirq/msi: Provide msi_desc::msi_data References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:15 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422475109964592?= X-GMAIL-MSGID: =?utf-8?q?1750422475109964592?= The upcoming support for PCI/IMS requires to store some information related to the message handling in the MSI descriptor, e.g. PASID or a pointer to a queue. Provide a generic storage struct which maps over the existing PCI specific storage which means the size of struct msi_desc is not getting bigger. This storage struct has two elements: 1) msi_domain_cookie 2) msi_instance_cookie The domain cookie is going to be used to store domain specific information, e.g. iobase pointer, data pointer. The instance cookie is going to be handed in when allocating an interrupt on an IMS domain so the irq chip callbacks of the IMS domain have the necessary per vector information available. It also comes in handy when cleaning up the platform MSI code for wire to MSI bridges which need to hand down the type information to the underlying interrupt domain. For the core code the cookies are opaque and meaningless. It just stores the instance cookie during an allocation through the upcoming interfaces for IMS and wire to MSI brigdes. Signed-off-by: Thomas Gleixner --- V2: Rename and split into domain/instance V3: Update stale changelog (Kevin) --- include/linux/msi.h | 38 +++++++++++++++++++++++++++++++++++++- include/linux/msi_api.h | 17 +++++++++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -125,6 +125,38 @@ struct pci_msi_desc { }; }; +/** + * union msi_domain_cookie - Opaque MSI domain specific data + * @value: u64 value store + * @ptr: Pointer to domain specific data + * @iobase: Domain specific IOmem pointer + * + * The content of this data is implementation defined and used by the MSI + * domain to store domain specific information which is requried for + * interrupt chip callbacks. + */ +union msi_domain_cookie { + u64 value; + void *ptr; + void __iomem *iobase; +}; + +/** + * struct msi_desc_data - Generic MSI descriptor data + * @dcookie: Cookie for MSI domain specific data which is required + * for irq_chip callbacks + * @icookie: Cookie for the MSI interrupt instance provided by + * the usage site to the allocation function + * + * The content of this data is implementation defined, e.g. PCI/IMS + * implementations define the meaning of the data. The MSI core ignores + * this data completely. + */ +struct msi_desc_data { + union msi_domain_cookie dcookie; + union msi_instance_cookie icookie; +}; + #define MSI_MAX_INDEX ((unsigned int)USHRT_MAX) /** @@ -142,6 +174,7 @@ struct pci_msi_desc { * * @msi_index: Index of the msi descriptor * @pci: PCI specific msi descriptor data + * @data: Generic MSI descriptor data */ struct msi_desc { /* Shared device/bus type independent data */ @@ -161,7 +194,10 @@ struct msi_desc { void *write_msi_msg_data; u16 msi_index; - struct pci_msi_desc pci; + union { + struct pci_msi_desc pci; + struct msi_desc_data data; + }; }; /* --- a/include/linux/msi_api.h +++ b/include/linux/msi_api.h @@ -19,6 +19,23 @@ enum msi_domain_ids { }; /** + * union msi_instance_cookie - MSI instance cookie + * @value: u64 value store + * @ptr: Pointer to usage site specific data + * + * This cookie is handed to the IMS allocation function and stored in the + * MSI descriptor for the interrupt chip callbacks. + * + * The content of this cookie is MSI domain implementation defined. For + * PCI/IMS implementations this could be a PASID or a pointer to queue + * memory. + */ +union msi_instance_cookie { + u64 value; + void *ptr; +}; + +/** * msi_map - Mapping between MSI index and Linux interrupt number * @index: The MSI index, e.g. slot in the MSI-X table or * a software managed index if >= 0. If negative From patchwork Thu Nov 24 23:26:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25756 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3663369wrr; Thu, 24 Nov 2022 15:35:54 -0800 (PST) X-Google-Smtp-Source: AA0mqf5DY0M4TWF2hiQH8flFaTUb94nmY29ZVtEn3M/UcxfzjYuocbpgRiQ9nKQGVmjP5AYVfZmC X-Received: by 2002:a63:1151:0:b0:438:a750:99b7 with SMTP id 17-20020a631151000000b00438a75099b7mr13316712pgr.605.1669332953958; Thu, 24 Nov 2022 15:35:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669332953; cv=none; d=google.com; s=arc-20160816; b=PHaHnsJ1g+IGSHGLLiqvlnCVGBa2HuAwDllBgTg95hSFJsBHTD2sPb8E2+cWcWpEIn fPY1OgggSH0ipPG5cGWaDG+OyMy0WmzFPdIfbnC+CObkuz51FOt86d2PBAPTqkz3UGS6 CBX94oppXprAEGvm8RjzDFIE30HJhahruWjXTn+jxorg2Vr3cKg31IQN2xljjKuxbcEk Rv5lDJlrsdTtXRG+gNWBJNbs0A03qPXhnvvrThCa8K+yOS2nGDbJ7dv81DF8VU6rzq26 M2CIKwll/MO2H+G70Ul7EW842W23GzlKVAQz3uZ0kb5H7UFpgo0bhFUhcKI2jb4ws1rb NCFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=SwZQXuxaOnLugPp7faog3c8WnwPvQUn41gyOKCZSnRg=; b=nouoHiLxpe+MDFtwDwoJev2FT99ga4pRyH6Q4oMUKCpkZJPCnqr5A2MGXHD6Bto5/d 1JvDnoREBgIAWaAI1pZ4mfv8mPsJy5qyQBQ84gOWVuuuca11e+hY/y+AIVOjnX8eSbop dM9vQ6TTJ4fBUcBiF9PN+fWpOog6k4zXoA8t5t93DZ/4nUe9ds3xzLL0k6bDqi4x6e0s PyNyzASduhNUFJgnbMsqGi2qM12WzSlCJzTykBrFRqobE3AAw8JDSqIfJkQ0yBjZV6Ee yLFcvHWsy3zsfCehgpGrYwls1FEfAhjjVPn9jeQw+YwAHZBe2bL1MbdJ03uutIGI6y5j f3UQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="z/qUj+0Z"; dkim=neutral (no key) header.i=@linutronix.de header.b=VVq7nShs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p14-20020a635b0e000000b00476bfca38b0si2401824pgb.71.2022.11.24.15.35.40; Thu, 24 Nov 2022 15:35:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="z/qUj+0Z"; dkim=neutral (no key) header.i=@linutronix.de header.b=VVq7nShs; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbiKXX3X (ORCPT + 99 others); Thu, 24 Nov 2022 18:29:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229844AbiKXX2q (ORCPT ); Thu, 24 Nov 2022 18:28:46 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89117A659B; Thu, 24 Nov 2022 15:27:11 -0800 (PST) Message-ID: <20221124232326.444560717@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=SwZQXuxaOnLugPp7faog3c8WnwPvQUn41gyOKCZSnRg=; b=z/qUj+0ZX580lDfONL6nr+qyUfMMm+bqwfYDl3W0KqLSqd1NP4i6neAwJybpfqu8swNAXn VHPxTVfg/R0+HDZ5NeoOaAogMPL7LGi96EnA1kTIEn/QgHPP0ljuHKtyQXcat2Gb+VNdl7 pzBncxrUqou4blZzJ0sNFSUwbn0xGDORcf8GGwppOxsGp0Q+f51bLNQlI4mHTMOwnZNtOG tGhKSbUmH/pEn40O4efPSXEMRFJqz2HqplLt6FWKCfPTDs2ODY2qMzo0fam+ofvhpj3Ucn QJCX3Gwm+xNkwWI94KR0oDF0ym7Wr6Wwj2TqVsWqrAJmavVuj48GBBAhzUssFw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=SwZQXuxaOnLugPp7faog3c8WnwPvQUn41gyOKCZSnRg=; b=VVq7nShsZR+BBAI9DISoLr6eYJjlRI40UorrLD+fHB21R+wkHEydKh7dWrYS+rrFOYOZDr bPtALm88nUIlFtAQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 20/33] genirq/msi: Provide msi_domain_ops::prepare_desc() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:16 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422471516626939?= X-GMAIL-MSGID: =?utf-8?q?1750422471516626939?= The existing MSI domain ops msi_prepare() and set_desc() turned out to be unsuitable for implementing IMS support. msi_prepare() does not operate on the MSI descriptors. set_desc() lacks an irq_domain pointer and has a completely different purpose. Introduce a prepare_desc() op which allows IMS implementations to amend an MSI descriptor which was allocated by the core code, e.g. by adjusting the iomem base or adding some data based on the allocated index. This is way better than requiring that all IMS domain implementations preallocate the MSI descriptor and then allocate the interrupt. Signed-off-by: Thomas Gleixner --- include/linux/msi.h | 6 +++++- kernel/irq/msi.c | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -410,6 +410,8 @@ struct msi_domain_info; * @msi_init: Domain specific init function for MSI interrupts * @msi_free: Domain specific function to free a MSI interrupts * @msi_prepare: Prepare the allocation of the interrupts in the domain + * @prepare_desc: Optional function to prepare the allocated MSI descriptor + * in the domain * @set_desc: Set the msi descriptor for an interrupt * @domain_alloc_irqs: Optional function to override the default allocation * function. @@ -421,7 +423,7 @@ struct msi_domain_info; * @get_hwirq, @msi_init and @msi_free are callbacks used by the underlying * irqdomain. * - * @msi_check, @msi_prepare and @set_desc are callbacks used by the + * @msi_check, @msi_prepare, @prepare_desc and @set_desc are callbacks used by the * msi_domain_alloc/free_irqs*() variants. * * @domain_alloc_irqs, @domain_free_irqs can be used to override the @@ -444,6 +446,8 @@ struct msi_domain_ops { int (*msi_prepare)(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg); + void (*prepare_desc)(struct irq_domain *domain, msi_alloc_info_t *arg, + struct msi_desc *desc); void (*set_desc)(msi_alloc_info_t *arg, struct msi_desc *desc); int (*domain_alloc_irqs)(struct irq_domain *domain, --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -1260,6 +1260,9 @@ static int __msi_domain_alloc_irqs(struc if (WARN_ON_ONCE(allocated >= ctrl->nirqs)) return -EINVAL; + if (ops->prepare_desc) + ops->prepare_desc(domain, &arg, desc); + ops->set_desc(&arg, desc); virq = __irq_domain_alloc_irqs(domain, -1, desc->nvec_used, From patchwork Thu Nov 24 23:26:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25762 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3664918wrr; Thu, 24 Nov 2022 15:40:54 -0800 (PST) X-Google-Smtp-Source: AA0mqf7mQXuvwEAepaY8EabfDYtZ9Xyktqrq2cgIyWHxcgJDtUKCrC0/lNSGFsszNK2dZZCqiYg6 X-Received: by 2002:aa7:cc8d:0:b0:461:15f0:a574 with SMTP id p13-20020aa7cc8d000000b0046115f0a574mr32455119edt.187.1669333254127; Thu, 24 Nov 2022 15:40:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669333254; cv=none; d=google.com; s=arc-20160816; b=d1fO/Irg+B8k6rKynLgszx7SBrtuuU6/xrWd5FOY+lIdUl/JrcOy8uMMUA1iF8V5qA /Vcxkct2rczLr4pVdBuTwwO631Re28ZveSFKzv85KjrW7aQj3J3JBxRggUgKxXDq41Lt iAkQPdhcWR9Lasd9ZIPgI01OUKzRrdlBWpqSrxGQtm3JFI53lxSVBqjp3fHlPf1sDXOB kMPatlWV82qydxVTgi59xOsDf9dQFux4IA3pfEz4bQAYsmnbxsiTw8Fy6S8oGJlhEMS4 YbamG7JgjRArae4WoOT/O9095Q3IJm+fwsV2k4UMgVbQZWFSbs2vw/XAEG9F0kfAVm++ yBRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=M7KPMuDnEi8BJsEuyegZhrT26NKyJhQDz95eZFlqKwE=; b=XUdxho0eoDIie45/hDL+DNTXM1JAC/lKOe4SzYII7slS5M2OnlUcqII7CzX61WOrGC A5Mot2W8MBP8L2j2ZHqyg056XJKrQC43tjAayv9W9OH0TPnVC3nUr5kPvXBDOqpOzbeG sFD1PVEhOY4BqmzGUdlm6aURULUU2iJn/zk0vLQ8ahJpco6jIBVkvf+o7N8Mf4wtqgnr 6c/qk70mZSqFZP0PhZNQod3ivYI+SRTYUGLL5wXxqmrMV0b/KZsFlATR6axjJ3C4dhlu 0FpIXxfXMMPvKNKxPlBrUokkf2Ek8M3FSx1ZxsbB/Fd7rm5RPCpdSXiRv9UEH7Vd5f2T otvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="H4Yrw/HR"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=gs0PydzL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i13-20020a05640242cd00b004587e99bcc2si1419747edc.383.2022.11.24.15.40.30; Thu, 24 Nov 2022 15:40:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="H4Yrw/HR"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=gs0PydzL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230046AbiKXXaJ (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230031AbiKXX3p (ORCPT ); Thu, 24 Nov 2022 18:29:45 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4433F2EF65; Thu, 24 Nov 2022 15:27:48 -0800 (PST) Message-ID: <20221124232326.501359457@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=M7KPMuDnEi8BJsEuyegZhrT26NKyJhQDz95eZFlqKwE=; b=H4Yrw/HRS/LSYqDnKSaurm2WD8le2eeKsY1rI7CVowV6D6Q1tYcT1eD9kX1323Nol/a/US bEecWmm6zQzGh5e8ODQ+OvP+nGNONUW7vX3LHMnC9FH+xypyEhrTfgIecTIhJn6mpoVDo4 xqD8xZ81z3qDGHVZStkNN9Y0+dYoAcItJigzTtNsd654C3LJiCCetEmv25+xlpOMpNYVYv e4Edh3ZvIwqSRzIK49BwB1NunHt9pkVqa08PNMhTz4XJbaDPiuMt/0CFJCuDUrKUCesYGA C4FLRY8XEnH4q9c+3/l9FrIPF3GplI+PmF0NuZdBXG2cwQv69cbpEO3NoNLvBw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332379; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=M7KPMuDnEi8BJsEuyegZhrT26NKyJhQDz95eZFlqKwE=; b=gs0PydzL74C8H8Ba8sbSLOnxNIiohpwBOFp2zlATmjYvGgpUhFbeOQyATaFqX0YYoyGcCa 4kdX7AwS7++4pnBA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 21/33] genirq/msi: Provide msi_domain_alloc_irq_at() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:18 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422786015106480?= X-GMAIL-MSGID: =?utf-8?q?1750422786015106480?= For supporting post MSI-X enable allocations and for the upcoming PCI/IMS support a separate interface is required which allows not only the allocation of a specific index, but also the allocation of any, i.e. the next free index. The latter is especially required for IMS because IMS completely does away with index to functionality mappings which are often found in MSI/MSI-X implementation. But even with MSI-X there are devices where only the first few indices have a fixed functionality and the rest is freely assignable by software, e.g. to queues. msi_domain_alloc_irq_at() is also different from the range based interfaces as it always enforces that the MSI descriptor is allocated by the core code and not preallocated by the caller like the PCI/MSI[-X] enable code path does. msi_domain_alloc_irq_at() can be invoked with the index argument set to MSI_ANY_INDEX which makes the core code pick the next free index. The irq domain can provide a prepare_desc() operation callback in it's msi_domain_ops to do domain specific post allocation initialization before the actual Linux interrupt and the associated interrupt descriptor and hierarchy alloccations are conducted. The function also takes an optional @icookie argument which is of type union msi_instance_cookie. This cookie is not used by the core code and is stored in the allocated msi_desc::data::icookie. The meaning of the cookie is completely implementation defined. In case of IMS this might be a PASID or a pointer to a device queue, but for the MSI core it's opaque and not used in any way. The function returns a struct msi_map which on success contains the allocated index number and the Linux interrupt number so the caller can spare the index to Linux interrupt number lookup. On failure map::index contains the error code and map::virq is 0. Signed-off-by: Thomas Gleixner --- V2: Fix the recursive allocation issue and the index calculation (Reinette) V3: Fixup stale changelog and typos in comments (Kevin) Adopt to the reworked domain/xarray storage model --- include/linux/msi.h | 4 + include/linux/msi_api.h | 7 +++ kernel/irq/msi.c | 107 +++++++++++++++++++++++++++++++++++++++++++----- 3 files changed, 108 insertions(+), 10 deletions(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -80,6 +80,7 @@ struct pci_dev; struct platform_msi_priv_data; struct device_attribute; struct irq_domain; +struct irq_affinity_desc; void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg); #ifdef CONFIG_GENERIC_MSI_IRQ @@ -602,6 +603,9 @@ int msi_domain_alloc_irqs_range(struct d unsigned int first, unsigned int last); int msi_domain_alloc_irqs_all_locked(struct device *dev, unsigned int domid, int nirqs); +struct msi_map msi_domain_alloc_irq_at(struct device *dev, unsigned int domid, unsigned int index, + const struct irq_affinity_desc *affdesc, + union msi_instance_cookie *cookie); void msi_domain_free_irqs_range_locked(struct device *dev, unsigned int domid, unsigned int first, unsigned int last); --- a/include/linux/msi_api.h +++ b/include/linux/msi_api.h @@ -48,6 +48,13 @@ struct msi_map { int virq; }; +/* + * Constant to be used for dynamic allocations when the allocation is any + * free MSI index, which is either an entry in a hardware table or a + * software managed index. + */ +#define MSI_ANY_INDEX UINT_MAX + unsigned int msi_domain_get_virq(struct device *dev, unsigned int domid, unsigned int index); /** --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -90,17 +90,30 @@ static int msi_insert_desc(struct device int ret; hwsize = msi_domain_get_hwsize(dev, domid); - if (index >= hwsize) { - ret = -ERANGE; - goto fail; - } - desc->msi_index = index; - ret = xa_insert(xa, index, desc, GFP_KERNEL); - if (ret) - goto fail; - return 0; + if (index == MSI_ANY_INDEX) { + struct xa_limit limit = { .min = 0, .max = hwsize - 1 }; + unsigned int index; + + /* Let the xarray allocate a free index within the limit */ + ret = xa_alloc(xa, &index, desc, limit, GFP_KERNEL); + if (ret) + goto fail; + desc->msi_index = index; + return 0; + } else { + if (index >= hwsize) { + ret = -ERANGE; + goto fail; + } + + desc->msi_index = index; + ret = xa_insert(xa, index, desc, GFP_KERNEL); + if (ret) + goto fail; + return 0; + } fail: msi_free_desc(desc); return ret; @@ -294,7 +307,7 @@ int msi_setup_device_data(struct device } for (i = 0; i < MSI_MAX_DEVICE_IRQDOMAINS; i++) - xa_init(&md->__domains[i].store); + xa_init_flags(&md->__domains[i].store, XA_FLAGS_ALLOC); /* * If @dev::msi::domain is set and is a global MSI domain, copy the @@ -1407,6 +1420,80 @@ int msi_domain_alloc_irqs_all_locked(str return msi_domain_alloc_locked(dev, &ctrl); } +/** + * msi_domain_alloc_irq_at - Allocate an interrupt from a MSI interrupt domain at + * a given index - or at the next free index + * + * @dev: Pointer to device struct of the device for which the interrupts + * are allocated + * @domid: Id of the interrupt domain to operate on + * @index: Index for allocation. If @index == %MSI_ANY_INDEX the allocation + * uses the next free index. + * @affdesc: Optional pointer to an interrupt affinity descriptor structure + * @icookie: Optional pointer to a domain specific per instance cookie. If + * non-NULL the content of the cookie is stored in msi_desc::data. + * Must be NULL for MSI-X allocations + * + * This requires a MSI interrupt domain which lets the core code manage the + * MSI descriptors. + * + * Return: struct msi_map + * + * On success msi_map::index contains the allocated index number and + * msi_map::virq the corresponding Linux interrupt number + * + * On failure msi_map::index contains the error code and msi_map::virq + * is %0. + */ +struct msi_map msi_domain_alloc_irq_at(struct device *dev, unsigned int domid, unsigned int index, + const struct irq_affinity_desc *affdesc, + union msi_instance_cookie *icookie) +{ + struct msi_ctrl ctrl = { .domid = domid, .nirqs = 1, }; + struct msi_domain_info *info; + struct irq_domain *domain; + struct msi_map map = { }; + struct msi_desc *desc; + int ret; + + msi_lock_descs(dev); + domain = msi_get_device_domain(dev, domid); + if (!domain) { + map.index = -ENODEV; + goto unlock; + } + + desc = msi_alloc_desc(dev, 1, affdesc); + if (!desc) { + map.index = -ENOMEM; + goto unlock; + } + + if (icookie) + desc->data.icookie = *icookie; + + ret = msi_insert_desc(dev, desc, domid, index); + if (ret) { + map.index = ret; + goto unlock; + } + + ctrl.first = ctrl.last = desc->msi_index; + info = domain->host_data; + + ret = __msi_domain_alloc_irqs(dev, domain, &ctrl); + if (ret) { + map.index = ret; + msi_domain_free_locked(dev, &ctrl); + } else { + map.index = desc->msi_index; + map.virq = desc->irq; + } +unlock: + msi_unlock_descs(dev); + return map; +} + static void __msi_domain_free_irqs(struct device *dev, struct irq_domain *domain, struct msi_ctrl *ctrl) { From patchwork Thu Nov 24 23:26:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25759 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3663686wrr; Thu, 24 Nov 2022 15:36:51 -0800 (PST) X-Google-Smtp-Source: AA0mqf7UEWkwll0scstb65bnp82yc1eZGduBYP/ahlDfsGSuVW0Z5K5U5H3Nwmpk0BEh32dPsMXp X-Received: by 2002:a05:6402:3886:b0:463:ab08:2bc6 with SMTP id fd6-20020a056402388600b00463ab082bc6mr15292822edb.143.1669333011752; Thu, 24 Nov 2022 15:36:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669333011; cv=none; d=google.com; s=arc-20160816; b=BA5EoPyE9Wg+coOX4vJPCE8SlCiSoFhnbAmlGr1lTnXBxIKrfJWNVrTG2M0HVywc5N /Kc5nAOiC5Zc1UAeOn2RleqHnf7tJ4T9VjRhm6DxvFvs+p0g62ygZmfRFaxCJZ/cPBFT 7WKriWKMmMW6cGZxJDcF6s/32hy9YbndaTnBulm2DsWv6qmOfclWUOhcp2kj+cMWkfl3 nlJWtBUP6Ukc+tz9njMKjwxUlm6Q31soptZfds+VyXRW/KhiGpig4HJ+/wOhcfJe4miV XmxAxAxwNY5koBhcGkTqmrJbUHCxtyptyPhimP4V1rTxVoLLwhiINliel3esQXq2czVd 8PXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=U+wQx81cMa25gz7AqVU1KnfI5ICMKD3knD+g5UcDLC4=; b=kaJ7DySPcK/PBJGY9f8rq6X1o6A1J8rozQy/UGkWte+gX4PSLkYy5V2+KNKLM0DeFe hX/fuYqvaLCUypN5ctcdv16tia8MwzjEJU0tKN4DcqxriUFV1LpqA++EukhGFfyBLXn4 6znWD9N17lyEhXVgpgWE1pXnOyLrWaybXsLE4u00w5ro6IhK9sYb6xmFTP6gFUGYJQsf jL+CIAB96okGGLOkrz9rcOsh8PSKRJt0F6cTMcbrRM0MnUty9Qv5uZX+oq2PIsRqcY1R lUWrVQNUJFlwwMqksa0vlCRY8M1LSCVxgKWzQhYbJxq/c0SCtSr2Dq00zuTQggot1Rnv J3wg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=yR65BvU6; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id js6-20020a17090797c600b0078d20d71475si2157703ejc.413.2022.11.24.15.36.26; Thu, 24 Nov 2022 15:36:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=yR65BvU6; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230033AbiKXXaD (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230017AbiKXX3o (ORCPT ); Thu, 24 Nov 2022 18:29:44 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 443BE2EF68; Thu, 24 Nov 2022 15:27:49 -0800 (PST) Message-ID: <20221124232326.558843119@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=U+wQx81cMa25gz7AqVU1KnfI5ICMKD3knD+g5UcDLC4=; b=yR65BvU6uxY1TZ9/+XrTodHlBDL1Dsyt1lbD2yiEDaEpGYd4tGxoHPFQ5AB9B4vU0JrCeB eghtYMxJRPKRz3HdlcQCE0pisDoIZsa5HOKBViO6RM/r0SOxBciiByJMD+pguKDXVVs6P/ bwNHiD6sxpxOO1lIlEiOYLvMzFJw9O34V8I3H0sqgAKB0V8jBAP6O108rytoyZpK7wQMUz +6aUKlibNjJY4xxbj84MUNo1HeAf8QvJ38CbB2I8IJ5daKYz1nxgGKAxt1AjmoPBZ2rl+c yLt4M5qUhBDq3CQMpelTn/AruJhKD737EBHnouX0xF1xgTonMpQ/W8ZkyqGXvg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332380; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=U+wQx81cMa25gz7AqVU1KnfI5ICMKD3knD+g5UcDLC4=; b=KC86ODTt2mFTPKTvh3yaCHEfCGmSCUC5b2aBuEzQbscqIr8Gn4T1QqksD9ufY4ZLi+EtNr yQtJs+mN1ui5OfAg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , Jason Gunthorpe Subject: [patch V3 22/33] genirq/msi: Provide MSI_FLAG_MSIX_ALLOC_DYN References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:20 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422531904642436?= X-GMAIL-MSGID: =?utf-8?q?1750422531904642436?= Provide a new MSI feature flag in preparation for dynamic MSIX allocation after the initial MSI-X enable has been done. This needs to be an explicit MSI interrupt domain feature because quite some implementations (both interrupt domains and legacy allocation mode) have clear expectations that the allocation code is only invoked when MSI-X is about to be enabled. They either talk to hypervisors or do some other work and are not prepared to be invoked on an already MSI-X enabled device. This is also explicit MSI-X only because rewriting the size of the MSI entries is only possible when disabling MSI which in turn might cause lost interrupts on the device. Signed-off-by: Thomas Gleixner Reviewed-by: Jason Gunthorpe --- include/linux/msi.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -557,7 +557,8 @@ enum { MSI_FLAG_LEVEL_CAPABLE = (1 << 18), /* MSI-X entries must be contiguous */ MSI_FLAG_MSIX_CONTIGUOUS = (1 << 19), - + /* PCI/MSI-X vectors can be dynamically allocated/freed post MSI-X enable */ + MSI_FLAG_PCI_MSIX_ALLOC_DYN = (1 << 20), }; /** From patchwork Thu Nov 24 23:26:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25776 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3690390wrr; Thu, 24 Nov 2022 16:53:20 -0800 (PST) X-Google-Smtp-Source: AA0mqf4D7TcMr4/GS8txED9YLU3ZXqwbNOwo/SYhUgH1udhHjXeA9WZPSIpniPAotbXsDdeMJOlF X-Received: by 2002:a17:90b:46ca:b0:212:ce2d:9fd7 with SMTP id jx10-20020a17090b46ca00b00212ce2d9fd7mr38412504pjb.157.1669337600065; Thu, 24 Nov 2022 16:53:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669337600; cv=none; d=google.com; s=arc-20160816; b=YTmjYFDzyMRF60mustlOXuOh23vODO72P8XSfDq5tSInNAd4X6XQQDVP4Ed2/o4cBT fsPZ0YegX6g0+yuU51oXBXayc55us2r1gtu5VgKuleiKooTxy/O0Hu+069k8C6BTCtCc LA/KfR3QtLsgKB/IXqKIHHm+fWSmLGNRtazS6g+5f9t8fGMgedcT7uc7DO/SO0EbLh5K i7OezGWQ1e+qXcXJzskYtHDVS5Nr0gje0olDSTM8+woz9oeljAhi8QicpNSbTghQ052C r3Ua3AJAtJoNfNFPUOTi3egIen3eP/MxdLvDQc3jOk6MAP+pFRYwtn9K5DInc6wDtdCe s3Qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=aPZsUav27NBNGy9CDcKsBA6eS6rPiDVMzp1pfp5Kt2c=; b=QJX7RMO6WjZ70o30dXpq9C/pjgRQkuI7pwqgWDkJ2321TovIsGxZPafGLaHwexeXYW uWT2FYDpwKSHpJaLaGJ880uYr4HX8/LNBSY1gs/DxRlcwT8AHSHacPL8o+Cjp6qLEr48 m+q3rvB/yIEXzHjnQVMwTWRCNC1YF6TtwJ/JXZG9vuVb6BQzfF7mPfySt8lmydjLcYB8 LG6XfiqlqikM29Dc2gxuISjQO7d1gwFwXNmT9Rt8wOj1+4jYlwwJ2OFwGUT6DHtF4zF+ VnQeMJsSRLVU6SUY5hC1bX1B5kZNShnEZBLfRxlrR4sM30ee9i55FEpsQwxmMnWiousu FGUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=iBVZarSH; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 1-20020a621901000000b005734ae5e1b4si1794894pfz.362.2022.11.24.16.53.05; Thu, 24 Nov 2022 16:53:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=iBVZarSH; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229851AbiKXXaR (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230048AbiKXX3t (ORCPT ); Thu, 24 Nov 2022 18:29:49 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E00632B8C; Thu, 24 Nov 2022 15:27:50 -0800 (PST) Message-ID: <20221124232326.616292598@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=aPZsUav27NBNGy9CDcKsBA6eS6rPiDVMzp1pfp5Kt2c=; b=iBVZarSHiHR3bZpM2o1rI91WP7DQ63gQlXyn/7Oy/SU8wEufn5QJgSv8beJLwOulGg+97u FXt43U9Ltw+z3vHxwmEa/l/DPgwzg32iXRTAGVcYIXehc6NDZMacMiMz4CXsXvxD7U1SIC kdTmDscXCTBpOYJzjngcXuTwyVswbbEv61fd1mT54HDSzpj9OYiUjtQH7nOP6ConcYHyY7 ncWUWqk9dJCOa5V09yFmEeBdVIsuoYQVYpoXg0pro42hTHM0eJ5fJr0TIwAn6Hrw2ejiuM WGIgQSyw3TOKOF1v9mUsKD+MSELNugQnIBRnaY1vi13D6sdR0s4Yup5cglHtEQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332382; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=aPZsUav27NBNGy9CDcKsBA6eS6rPiDVMzp1pfp5Kt2c=; b=R/LXgCMdNvUmdN3zuonzIgD+GffVgGzDLW3zapj9TD+Rf9J+E2AqvsKFAuvlvdrVGYBS9r XvC63oc5TSEtu3Dg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 23/33] PCI/MSI: Split MSI-X descriptor setup References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:21 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750427343521193102?= X-GMAIL-MSGID: =?utf-8?q?1750427343521193102?= The upcoming mechanism to allocate MSI-X vectors after enabling MSI-X needs to share some of the MSI-X descriptor setup. The regular descriptor setup on enable has the following code flow: 1) Allocate descriptor 2) Setup descriptor with PCI specific data 3) Insert descriptor 4) Allocate interrupts which in turn scans the inserted descriptors This cannot be easily changed because the PCI/MSI code needs to handle the legacy architecture specific allocation model and the irq domain model where quite some domains have the assumption that the above flow is how it works. Ideally the code flow should look like this: 1) Invoke allocation at the MSI core 2) MSI core allocates descriptor 3) MSI core calls back into the irq domain which fills in the domain specific parts This could be done for underlying parent MSI domains which support post-enable allocation/free but that would create significantly different code pathes for MSI/MSI-X enable. Though for dynamic allocation which wants to share the allocation code with the upcoming PCI/IMS support it's the right thing to do. Split the MSI-X descriptor setup into the preallocation part which just sets the index and fills in the horrible hack of virtual IRQs and the real PCI specific MSI-X setup part which solely depends on the index in the descriptor. This allows to provide a common dynamic allocation interface at the MSI core level for both PCI/MSI-X and PCI/IMS. Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas --- drivers/pci/msi/msi.c | 72 +++++++++++++++++++++++++++++++------------------- drivers/pci/msi/msi.h | 2 + 2 files changed, 47 insertions(+), 27 deletions(-) --- a/drivers/pci/msi/msi.c +++ b/drivers/pci/msi/msi.c @@ -569,34 +569,56 @@ static void __iomem *msix_map_region(str return ioremap(phys_addr, nr_entries * PCI_MSIX_ENTRY_SIZE); } -static int msix_setup_msi_descs(struct pci_dev *dev, void __iomem *base, - struct msix_entry *entries, int nvec, - struct irq_affinity_desc *masks) +/** + * msix_prepare_msi_desc - Prepare a half initialized MSI descriptor for operation + * @dev: The PCI device for which the descriptor is prepared + * @desc: The MSI descriptor for preparation + * + * This is separate from msix_setup_msi_descs() below to handle dynamic + * allocations for MSI-X after initial enablement. + * + * Ideally the whole MSI-X setup would work that way, but there is no way to + * support this for the legacy arch_setup_msi_irqs() mechanism and for the + * fake irq domains like the x86 XEN one. Sigh... + * + * The descriptor is zeroed and only @desc::msi_index and @desc::affinity + * are set. When called from msix_setup_msi_descs() then the is_virtual + * attribute is initialized as well. + * + * Fill in the rest. + */ +void msix_prepare_msi_desc(struct pci_dev *dev, struct msi_desc *desc) +{ + desc->nvec_used = 1; + desc->pci.msi_attrib.is_msix = 1; + desc->pci.msi_attrib.is_64 = 1; + desc->pci.msi_attrib.default_irq = dev->irq; + desc->pci.mask_base = dev->msix_base; + desc->pci.msi_attrib.can_mask = !pci_msi_ignore_mask && + !desc->pci.msi_attrib.is_virtual; + + if (desc->pci.msi_attrib.can_mask) { + void __iomem *addr = pci_msix_desc_addr(desc); + + desc->pci.msix_ctrl = readl(addr + PCI_MSIX_ENTRY_VECTOR_CTRL); + } +} + +static int msix_setup_msi_descs(struct pci_dev *dev, struct msix_entry *entries, + int nvec, struct irq_affinity_desc *masks) { int ret = 0, i, vec_count = pci_msix_vec_count(dev); struct irq_affinity_desc *curmsk; struct msi_desc desc; - void __iomem *addr; memset(&desc, 0, sizeof(desc)); - desc.nvec_used = 1; - desc.pci.msi_attrib.is_msix = 1; - desc.pci.msi_attrib.is_64 = 1; - desc.pci.msi_attrib.default_irq = dev->irq; - desc.pci.mask_base = base; - for (i = 0, curmsk = masks; i < nvec; i++, curmsk++) { desc.msi_index = entries ? entries[i].entry : i; desc.affinity = masks ? curmsk : NULL; desc.pci.msi_attrib.is_virtual = desc.msi_index >= vec_count; - desc.pci.msi_attrib.can_mask = !pci_msi_ignore_mask && - !desc.pci.msi_attrib.is_virtual; - if (desc.pci.msi_attrib.can_mask) { - addr = pci_msix_desc_addr(&desc); - desc.pci.msix_ctrl = readl(addr + PCI_MSIX_ENTRY_VECTOR_CTRL); - } + msix_prepare_msi_desc(dev, &desc); ret = msi_insert_msi_desc(&dev->dev, &desc); if (ret) @@ -629,9 +651,8 @@ static void msix_mask_all(void __iomem * writel(ctrl, base + PCI_MSIX_ENTRY_VECTOR_CTRL); } -static int msix_setup_interrupts(struct pci_dev *dev, void __iomem *base, - struct msix_entry *entries, int nvec, - struct irq_affinity *affd) +static int msix_setup_interrupts(struct pci_dev *dev, struct msix_entry *entries, + int nvec, struct irq_affinity *affd) { struct irq_affinity_desc *masks = NULL; int ret; @@ -640,7 +661,7 @@ static int msix_setup_interrupts(struct masks = irq_create_affinity_masks(nvec, affd); msi_lock_descs(&dev->dev); - ret = msix_setup_msi_descs(dev, base, entries, nvec, masks); + ret = msix_setup_msi_descs(dev, entries, nvec, masks); if (ret) goto out_free; @@ -678,7 +699,6 @@ static int msix_setup_interrupts(struct static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries, int nvec, struct irq_affinity *affd) { - void __iomem *base; int ret, tsize; u16 control; @@ -696,15 +716,13 @@ static int msix_capability_init(struct p pci_read_config_word(dev, dev->msix_cap + PCI_MSIX_FLAGS, &control); /* Request & Map MSI-X table region */ tsize = msix_table_size(control); - base = msix_map_region(dev, tsize); - if (!base) { + dev->msix_base = msix_map_region(dev, tsize); + if (!dev->msix_base) { ret = -ENOMEM; goto out_disable; } - dev->msix_base = base; - - ret = msix_setup_interrupts(dev, base, entries, nvec, affd); + ret = msix_setup_interrupts(dev, entries, nvec, affd); if (ret) goto out_disable; @@ -719,7 +737,7 @@ static int msix_capability_init(struct p * which takes the MSI-X mask bits into account even * when MSI-X is disabled, which prevents MSI delivery. */ - msix_mask_all(base, tsize); + msix_mask_all(dev->msix_base, tsize); pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0); pcibios_free_irq(dev); --- a/drivers/pci/msi/msi.h +++ b/drivers/pci/msi/msi.h @@ -84,6 +84,8 @@ static inline __attribute_const__ u32 ms return (1 << (1 << desc->pci.msi_attrib.multi_cap)) - 1; } +void msix_prepare_msi_desc(struct pci_dev *dev, struct msi_desc *desc); + /* Subsystem variables */ extern int pci_msi_enable; From patchwork Thu Nov 24 23:26:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25767 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3684005wrr; Thu, 24 Nov 2022 16:32:21 -0800 (PST) X-Google-Smtp-Source: AA0mqf6/7W65zqk15orwTJRKPECmXiFZlSt24DvGm+Xb/7tCzKSaidqp1/Yocco2CuRXopfsbsJ0 X-Received: by 2002:a65:6d10:0:b0:476:fdde:e539 with SMTP id bf16-20020a656d10000000b00476fddee539mr16633675pgb.216.1669336340767; Thu, 24 Nov 2022 16:32:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669336340; cv=none; d=google.com; s=arc-20160816; b=VM3seRDhQjzwUDyhPPTAaGJrfxnqw6ERulKxtGfAbrV2Lw62VkZ0XylNT+6mZqJrzC JJ1QJXFa+zud7EUzgDOjMvSdbhB8YZS94flYYdnRBZRy4lzgJklKY/3OPjqehIJpXDdN 3ox8TrcgXjEj4/y8fE4it6nVK5tG6B9wHDoegDK+6X8wevUZLsWwxO+VGRzBkbqQQLTw u5Ss/7doIRkoQV8tRl3UcCeOcZfzrQuy9O6EpiqFfht2CVLfZm+ABSoF+O+1d71KvGfu 8lQREmyLcznTTcaVKSwO0LXFPYwY1/Su5KkHyz4r/+wUocNi4JeLq5M1IkCUgfUOVC34 pCUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=TlIkyzTTSvqR6KrJj1PQdSVRNaBSTUiGHoaI0+IdJ9c=; b=KoOTCHsaGESZaIgGJWP/0VoZKo/ZGFRI7ZFntPF7mBgZLhJqWmHOJUrAQVKaqBTCfQ m7lwEyTFKg/IQg3Aw4AYV4bF3qPkRzXCZ+xh2pHSpHAhjHxuHsq91dffT94bCrqLXX50 GN0oIuwIHVv1xkDIc7BQGbg3IGbTguDerQRi3/B/YGa88kMC8KxBmzIhJQSUbA1lm4yD +DOBbljx9quxQggi0P+DC/FHcc8LLFJK1IGxyKlU4hQTpm+cGLOld7rRGOaLGC5boxAo cWzkN6H684SAbOqR4AfIKC9z/Ytly+x7cSkP/cHY0flYY+O3f2nt+ZSvWFXIr4NzPzCf 8NiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=EUoFqgAX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o2-20020a1709026b0200b00176c891c8a0si1885317plk.6.2022.11.24.16.32.01; Thu, 24 Nov 2022 16:32:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=EUoFqgAX; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230052AbiKXXaN (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230043AbiKXX3s (ORCPT ); Thu, 24 Nov 2022 18:29:48 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E3A23C6C1; Thu, 24 Nov 2022 15:27:50 -0800 (PST) Message-ID: <20221124232326.673658806@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332383; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=TlIkyzTTSvqR6KrJj1PQdSVRNaBSTUiGHoaI0+IdJ9c=; b=EUoFqgAXkfl7YAb9sfhRXkRzdebVXqr9nvfDRvKe0xNeMGXnXUS/RjfUs0CWKgl9C/35U8 stR9uXLyUVh8zZ3Wy/97dZn806sEYliKITeL1qlE2qk/c0aIXqVfzbf8fWwuatrebatmjl g57NpwGqhmq5+zoTYAfss2YKXnzwmDlYKhDBZzZdfo0OfeH0yw5yH7zvZ49veYVFbbuWEw q79fPtz3BEOSMIk/ynGb6TvgJFGZuT5iwOFAGhQyWJS10fZgk/FgotgeYX43lBWURsCvTg EU+Q4+/pbThL08M/8oG/ayOQiWvA4jFUELPBpcQA8kLNUnWzAaI2dM9GVodocQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332383; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=TlIkyzTTSvqR6KrJj1PQdSVRNaBSTUiGHoaI0+IdJ9c=; b=vzdLTkxSP4LCa9Pxc3Pi1rv/Qg8a4qEGdY3fbmBj9MchY2As9EtRCN5fVVcBkVhl79ny3X sguIA9J1MueylECg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe , Jason Gunthorpe Subject: [patch V3 24/33] PCI/MSI: Provide prepare_desc() MSI domain op References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:23 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426022928142676?= X-GMAIL-MSGID: =?utf-8?q?1750426022928142676?= The setup of MSI descriptors for PCI/MSI-X interrupts depends partially on the MSI index for which the descriptor is initialized. Dynamic MSI-X vector allocation post MSI-X enablement allows to allocate vectors at a given index or at any free index in the available table range. The latter requires that the descriptor is initialized after the MSI core has chosen an index. Implement the prepare_desc() op in the PCI/MSI-X specific msi_domain_ops which is invoked before the core interrupt descriptor and the associated Linux interrupt number is allocated. That callback is also provided for the upcoming PCI/IMS implementations so the implementation specific interrupt domain can do their domain specific initialization of the MSI descriptors. Signed-off-by: Thomas Gleixner Reviewed-by: Jason Gunthorpe Acked-by: Bjorn Helgaas --- V2: Reworded changelog (Bjorn) --- drivers/pci/msi/irqdomain.c | 9 +++++++++ 1 file changed, 9 insertions(+) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -202,6 +202,14 @@ static void pci_irq_unmask_msix(struct i pci_msix_unmask(irq_data_get_msi_desc(data)); } +static void pci_msix_prepare_desc(struct irq_domain *domain, msi_alloc_info_t *arg, + struct msi_desc *desc) +{ + /* Don't fiddle with preallocated MSI descriptors */ + if (!desc->pci.mask_base) + msix_prepare_msi_desc(to_pci_dev(desc->dev), desc); +} + static struct msi_domain_template pci_msix_template = { .chip = { .name = "PCI-MSIX", @@ -212,6 +220,7 @@ static struct msi_domain_template pci_ms }, .ops = { + .prepare_desc = pci_msix_prepare_desc, .set_desc = pci_device_domain_set_desc, }, From patchwork Thu Nov 24 23:26:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25763 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3675473wrr; Thu, 24 Nov 2022 16:09:18 -0800 (PST) X-Google-Smtp-Source: AA0mqf6ahFlNz9Kaz3v+t2VE+5KJn7fmn3WM6z5uUmVq++ZZpKWsCqlzCDZxOs5qYglhyjiAGKSr X-Received: by 2002:aa7:cb49:0:b0:468:f307:3014 with SMTP id w9-20020aa7cb49000000b00468f3073014mr23937192edt.321.1669334958647; Thu, 24 Nov 2022 16:09:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669334958; cv=none; d=google.com; s=arc-20160816; b=0DG6ivfzS9P8kIGA66yCL2Xx69UVakakgRSadzEtywgc2vhEJsBwvpNuKw7r2lpN2J kWa1xBC1USkq5k/9sYoWA1t69MAt+2uvzX9KwaEerispYF2/AqZ9C3CchhmtihHnPcHS Ju9gXGX1IV/K+Nn1TON3pq5JUbCOICLP7YSFAHb6R5+ypIhDWU/6/iDnO0Yk02pfMALC 0Kere6A3Y5YiuJgUH7ijsybEoX27yVvbSSkcUa4tZCia6lL8VpTwlC3eAGa8LRy2q3KX Ug87PZZDxjvKTL3efMpymHByAnKD4QsTY7FfvM5Gslv5iP1iZIxBACDekdJ990BztOxL woFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=/2fKPrAwtkKgHwo/h/OzGFhuV+cQfxZ1ctTffBm7oAo=; b=xFqIqB40eP9Nk0TqupWoYASoVtuuMg6CBo2S2qSignxZiI+VcG5Yc6zRT8bo+WgUgz kXyy+xtYWLHezSf7nHqN3wRhep7E72Dnb8YGS0Wd4/ptBnc+tOk2XmEk4PRThWgLASu7 zhUUy7bIYz4h5XKGQ4FDn3oNuMh9IYRO8p6xYYfShRA2QSBChc8Pl1cO6ciuvuOAEtiL FanFBRzeFQbDtgkdnG8YF9fYWIY3fR2DNeViyPIcx2aPW3oI4bLMrD6ez7irBza6JvHn yPDrFpIAgp0BnfGurFTM2ZocSYtFyq4E9xrnsbxQO3cxasK7wNrZyLzx4pgq3XSFRXqX eSHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=fKj7xXcs; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="y4vE/JcV"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f19-20020a50a6d3000000b0046a223ca073si2142126edc.182.2022.11.24.16.08.38; Thu, 24 Nov 2022 16:09:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=fKj7xXcs; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="y4vE/JcV"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229973AbiKXXac (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229928AbiKXX3y (ORCPT ); Thu, 24 Nov 2022 18:29:54 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EB444B74A; Thu, 24 Nov 2022 15:27:55 -0800 (PST) Message-ID: <20221124232326.731233614@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/2fKPrAwtkKgHwo/h/OzGFhuV+cQfxZ1ctTffBm7oAo=; b=fKj7xXcsQJGEM21jJOr960327oTA0HXBvuCumfvcPjXgaOHso3/W53KAAYB1miy4I0h+3Y lnpsFmz6d4ORcD/wPa+Fzy3dgoVUI7Rx3AgxFmQO2ooEP0P4hxrp6+9tZ7o62OZGfNHGIa 0Xiez6a9T5mCE0lFu3lgsNmiqGNjaJZYO5IgvGyVSJdl0LI+NFQg6s1RTBt6Qg2Yx7EzQ4 x52pP2SDLp3f27d49AusfzO3p3Qqr4mge36yqsP6lSZ6aJQae+PRCInUBUhdr/QUEr6E3g B4q/vfc6z2JwgRSqMpzm+nidkmo4jAusb04aDuKRWcuhpefTluR7PCryKAsgNg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/2fKPrAwtkKgHwo/h/OzGFhuV+cQfxZ1ctTffBm7oAo=; b=y4vE/JcVoN/JVgZ77BAK4Fcc/bcFd8SNV5RbllKa+9ptcafxp9n34FgYJpV4F1+dfOJuvl yjDdBWTaEXus5rDA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 25/33] PCI/MSI: Provide post-enable dynamic allocation interfaces for MSI-X References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:24 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750424573555727434?= X-GMAIL-MSGID: =?utf-8?q?1750424573555727434?= MSI-X vectors can be allocated after the initial MSI-X enablement, but this needs explicit support of the underlying interrupt domains. Provide a function to query the ability and functions to allocate/free individual vectors post-enable. The allocation can either request a specific index in the MSI-X table or with the index argument MSI_ANY_INDEX it allocates the next free vector. The return value is a struct msi_map which on success contains both index and the Linux interrupt number. In case of failure index is negative and the Linux interrupt number is 0. The allocation function is for a single MSI-X index at a time as that's sufficient for the most urgent use case VFIO to get rid of the 'disable MSI-X, reallocate, enable-MSI-X' cycle which is prone to lost interrupts and redirections to the legacy and obviously unhandled INTx. As single index allocation is also sufficient for the use cases Jason Gunthorpe pointed out: Allocation of a MSI-X or IMS vector for a network queue. See Link below. Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas Link: https://lore.kernel.org/all/20211126232735.547996838@linutronix.de --- V2: Added Link to previous discussions (Bjorn) --- drivers/pci/msi/api.c | 67 ++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/msi/irqdomain.c | 3 + include/linux/pci.h | 6 +++ 3 files changed, 75 insertions(+), 1 deletion(-) --- a/drivers/pci/msi/api.c +++ b/drivers/pci/msi/api.c @@ -113,6 +113,73 @@ int pci_enable_msix_range(struct pci_dev EXPORT_SYMBOL(pci_enable_msix_range); /** + * pci_msix_can_alloc_dyn - Query whether dynamic allocation after enabling + * MSI-X is supported + * + * @dev: PCI device to operate on + * + * Return: True if supported, false otherwise + */ +bool pci_msix_can_alloc_dyn(struct pci_dev *dev) +{ + if (!dev->msix_cap) + return false; + + return pci_msi_domain_supports(dev, MSI_FLAG_PCI_MSIX_ALLOC_DYN, DENY_LEGACY); +} +EXPORT_SYMBOL_GPL(pci_msix_can_alloc_dyn); + +/** + * pci_msix_alloc_irq_at - Allocate an MSI-X interrupt after enabling MSI-X + * at a given MSI-X vector index or any free vector index + * + * @dev: PCI device to operate on + * @index: Index to allocate. If @index == MSI_ANY_INDEX this allocates + * the next free index in the MSI-X table + * @affdesc: Optional pointer to an affinity descriptor structure. NULL otherwise + * + * Return: A struct msi_map + * + * On success msi_map::index contains the allocated index (>= 0) and + * msi_map::virq contains the allocated Linux interrupt number (> 0). + * + * On fail msi_map::index contains the error code and msi_map::virq + * is set to 0. + */ +struct msi_map pci_msix_alloc_irq_at(struct pci_dev *dev, unsigned int index, + const struct irq_affinity_desc *affdesc) +{ + struct msi_map map = { .index = -ENOTSUPP }; + + if (!dev->msix_enabled) + return map; + + if (!pci_msix_can_alloc_dyn(dev)) + return map; + + return msi_domain_alloc_irq_at(&dev->dev, MSI_DEFAULT_DOMAIN, index, affdesc, NULL); +} +EXPORT_SYMBOL_GPL(pci_msix_alloc_irq_at); + +/** + * pci_msix_free_irq - Free an interrupt on a PCI/MSIX interrupt domain + * which was allocated via pci_msix_alloc_irq_at() + * + * @dev: The PCI device to operate on + * @map: A struct msi_map describing the interrupt to free + * as returned from the allocation function. + */ +void pci_msix_free_irq(struct pci_dev *dev, struct msi_map map) +{ + if (WARN_ON_ONCE(map.index < 0 || map.virq <= 0)) + return; + if (WARN_ON_ONCE(!pci_msix_can_alloc_dyn(dev))) + return; + msi_domain_free_irqs_range(&dev->dev, MSI_DEFAULT_DOMAIN, map.index, map.index); +} +EXPORT_SYMBOL_GPL(pci_msix_free_irq); + +/** * pci_disable_msix() - Disable MSI-X interrupt mode on device * @dev: the PCI device to operate on * --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -225,7 +225,8 @@ static struct msi_domain_template pci_ms }, .info = { - .flags = MSI_COMMON_FLAGS | MSI_FLAG_PCI_MSIX, + .flags = MSI_COMMON_FLAGS | MSI_FLAG_PCI_MSIX | + MSI_FLAG_PCI_MSIX_ALLOC_DYN, .bus_token = DOMAIN_BUS_PCI_DEVICE_MSIX, }, }; --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -38,6 +38,7 @@ #include #include #include +#include #include #include @@ -1559,6 +1560,11 @@ int pci_alloc_irq_vectors_affinity(struc unsigned int max_vecs, unsigned int flags, struct irq_affinity *affd); +bool pci_msix_can_alloc_dyn(struct pci_dev *dev); +struct msi_map pci_msix_alloc_irq_at(struct pci_dev *dev, unsigned int index, + const struct irq_affinity_desc *affdesc); +void pci_msix_free_irq(struct pci_dev *pdev, struct msi_map map); + void pci_free_irq_vectors(struct pci_dev *dev); int pci_irq_vector(struct pci_dev *dev, unsigned int nr); const struct cpumask *pci_irq_get_affinity(struct pci_dev *pdev, int vec); From patchwork Thu Nov 24 23:26:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25774 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3689587wrr; Thu, 24 Nov 2022 16:50:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf4Rk4HVFwptH1yzxfSZeLIhb1BggsjLp/pNxUy+/SqX5v6CTAw9OHZEuBfQ7ZEySqDz4JqN X-Received: by 2002:a63:5d21:0:b0:46f:188c:3312 with SMTP id r33-20020a635d21000000b0046f188c3312mr13503512pgb.562.1669337419100; Thu, 24 Nov 2022 16:50:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669337419; cv=none; d=google.com; s=arc-20160816; b=trnl4qdQhlDIjPWzccmkSVR0EdvwM+NO0brBAo9xcxGtRlXuL+8eVknqVc7yaZgNFg sYTOhSTKXqwqiIsIFOts+QJEvsBBhI4nXzqXeU9eibUT7I41Wqipq7I1dwpmhr6KV4jk /wqWEjnGA8qRn48OU3Ekvcf1Vn1cZ7Li8GPJ/PYmy/Cd/+7EcP6auBpU3Z2lnof/6OSB wn0Gj0G564Px97VdvQfrxam6GGNLUILewg8VGs4d92YZ4YFUzepRrCpM0xIWpyG6yN5N RAr1/X1wJnIq4allVoAmLcLGj/ghYUdtdg5D2i3DEI7uCPlw9JaRE8HJC1/KudVyP36a h3oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=IEK08cS7O6Uv3gBo3cg3kU1MfAmms/KMIPye21Vs0c0=; b=OS+nFDC0iH4Ci2s617EJaqdruH+17kd6pY/rNU0zk93o1Hl6TFoJVqmX8zzW+LyQW9 W8s+nU+n7X2peMEH1vn/5i5cicRujsYcoY7kM9s6mFssLPgbdVe37N9VZ8ZAhoI1PCUk 27S6vw35kNtowVBtX+KQV39KV4pjSwMjJeLl33+Fwel6h1mv+GueMNhqd7w1wFYryjT1 Cn57Ube74mIMQVVVL6m0d2DRpCJGipODUFZFL9Trfixlo+RkJP9m60wHw1O/+BTxS2Fg w/psAgLvCt0xKjmX78XAspiOWcXAz/C8cFZuKYPYweExkSuLe3ma4XGGIfOkjAKqkFDy xGLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=GWJDmfgq; dkim=neutral (no key) header.i=@linutronix.de header.b=ZRY48GSY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h13-20020a170902680d00b001869c2b3a3csi2147881plk.537.2022.11.24.16.50.06; Thu, 24 Nov 2022 16:50:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=GWJDmfgq; dkim=neutral (no key) header.i=@linutronix.de header.b=ZRY48GSY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229956AbiKXXaa (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230074AbiKXX3x (ORCPT ); Thu, 24 Nov 2022 18:29:53 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 210484B748; Thu, 24 Nov 2022 15:27:55 -0800 (PST) Message-ID: <20221124232326.787373104@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=IEK08cS7O6Uv3gBo3cg3kU1MfAmms/KMIPye21Vs0c0=; b=GWJDmfgqlPDlcLgKShBue/3ax/oc97kX1RgHr8iv9Zn2f2/PLSJ6s632PjSyOq3Rd7qieL fDTGqW0xUpCzOiO/sX6NYComheclNwtAaMgvsit/evGkb/PXnXNkGkQ+Hc3U3H2YxMSFWQ nbY+0XlXGXNu5cy0tQt3gg1NhdNXDz7ANC2fOnsyj8SpybTiB7yYE8Mg0l07dzqWrAX8Yo YP23ghg/OlZcrSmaZr2CQtAvSWKIwLLfVN5XLiLLkrxLW4Lzg6qVc0j4dsBNa9RRmUKH6K f38fM5eIdJcqAHkcYGTHo3EKvReXZRTXqnjgIbl6s+MSGyel+jvRyOaKFnHnDQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=IEK08cS7O6Uv3gBo3cg3kU1MfAmms/KMIPye21Vs0c0=; b=ZRY48GSY8X2TF4+9ZB5PNWBha3Rgy0GzxB9q2DxXum7yebn3F09iLhLaxmuk2Q4WKMRPN9 BlLD0c2ZIWQMn/AQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 26/33] x86/apic/msi: Enable MSI_FLAG_PCI_MSIX_ALLOC_DYN References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:26 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750427153318507383?= X-GMAIL-MSGID: =?utf-8?q?1750427153318507383?= x86 MSI irqdomains can handle MSI-X allocation post MSI-X enable just out of the box - on the vector domain and on the remapping domains, Add the feature flag to the supported feature list Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/msi.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/arch/x86/include/asm/msi.h +++ b/arch/x86/include/asm/msi.h @@ -63,7 +63,7 @@ struct msi_msg; u32 x86_msi_msg_get_destid(struct msi_msg *msg, bool extid); #define X86_VECTOR_MSI_FLAGS_SUPPORTED \ - (MSI_GENERIC_FLAGS_MASK | MSI_FLAG_PCI_MSIX) + (MSI_GENERIC_FLAGS_MASK | MSI_FLAG_PCI_MSIX | MSI_FLAG_PCI_MSIX_ALLOC_DYN) #define X86_VECTOR_MSI_FLAGS_REQUIRED \ (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS) From patchwork Thu Nov 24 23:26:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25769 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3685026wrr; Thu, 24 Nov 2022 16:35:14 -0800 (PST) X-Google-Smtp-Source: AA0mqf7he+ehMX+1M3xRh8LwUMPAl0xtUNKCMUedx4tLxLVXoB21m3jsZETFBiDQBEvqrHd+x/eV X-Received: by 2002:a17:902:9004:b0:17c:9a37:72fb with SMTP id a4-20020a170902900400b0017c9a3772fbmr28107648plp.82.1669336514260; Thu, 24 Nov 2022 16:35:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669336514; cv=none; d=google.com; s=arc-20160816; b=cmug7RnJjHPKqZKe/v9Fa7axANr9emqnmp3+JdakWFKojISEOR86JfLRa5O5seAsaR 2tf41hqnBIIpLA1nsO5JGlIwBX4DFn7SPfVcymwWNoQ5W9DbbvGbjyig+V/sEs5ORCZW plCzSxNpL251Yz5OGaI0IB84YS/1TV0vJj8vxxx/23vMu+HmM/KpcDnlJHmc82RRmU+g 0JU4urxxgMsg6cfD3RJI/GfEI08Ae0xdab1P/gKAIZ06+1vWqIwHeK/bF09ZgTLKRZ9p I8bYrL6HK8TNYugcGQ86R9HhHPLYBJiJLWSjkiMA0cRroq+HofTpr4w+1YIbyUL3fnhO dbQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=dO4Zi35OCCA7gHzDk5+sx7+ILHZnBqLXkJXZcofiSSc=; b=u009/o+cWNY6Ig0ICrC5FpCuYtBMNthbG5Paot3ukylSNtC9OPQToTOXUBXyhT/yq0 +/UPluhLbgIEMK8E7R+mFdPBcT59rrWkXWCiOCqLoQKHT+iYYhLcAA6Fpw2eWreUSH+7 tym4HuJDKnf1nYVoqdbWz/IPYq9iY1VcO602EevWm+cKdhzagbbs/vEXbdJvhX4YzqqU 8pN0kFQTHjhsiCaqqj4gcwB5frlNy7jr9m1SSqRNABt2YADxWxd6Kn9M7WAG0kBhc4Yv gOq83qY7VxZoLMWmXQCCnDbwzGneAHwLilYsTLUhZ8zTwL/537nqzbDUQW58dsZwrwhQ +Ghg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=vJ1ivInE; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f12-20020a65400c000000b00439ae523fd2si2566103pgp.289.2022.11.24.16.34.54; Thu, 24 Nov 2022 16:35:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=vJ1ivInE; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229984AbiKXXae (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229939AbiKXX3y (ORCPT ); Thu, 24 Nov 2022 18:29:54 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54E394EC04; Thu, 24 Nov 2022 15:27:57 -0800 (PST) Message-ID: <20221124232326.846169830@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dO4Zi35OCCA7gHzDk5+sx7+ILHZnBqLXkJXZcofiSSc=; b=vJ1ivInEGxFXD/cLi/XsGMd7gWG3RnRzT+bSNGmuipyBQatFEozxsYSg5LRSwEK/Eq2OGc OcPZYezbgEk1I2kXpuhyaEuj0tB7dluO75I6L1kZj6tUTve7qwKfhpEeKFOT+s8jrEFKJ6 QgJbaoCRSf0CKbYhEwISJNcGkuS8tucvIV6BPn16Z0Xj8/uFke4AZmOdBBeBUvcTUKencz SMhwViegYUJhy6BdiD1OzlcV7Z/OStEKHq+JW/nnJS1Skp45uU0TPFRZVOLBpZWC4bKpVa 99SQHyP2TDlfCyrY6ySUAiPLwhJZdBnCzeO2GnIPzYluWwmdxWAo8nNJWL94og== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dO4Zi35OCCA7gHzDk5+sx7+ILHZnBqLXkJXZcofiSSc=; b=bodUHha1Sn8a5HfJeWq5ctT/b+kcYXaQkmaIOJsbXjkGurYIaUgFYmHM5Y9v+EeKW61q2K teWGRf0CdndccABw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 27/33] genirq/msi: Provide constants for PCI/IMS support References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:28 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426204624888524?= X-GMAIL-MSGID: =?utf-8?q?1750426204624888524?= Provide the necessary constants for PCI/IMS support: - A new bus token for MSI irqdomain identification - A MSI feature flag for the MSI irqdomains to signal support - A secondary domain id The latter expands the device internal domain pointer storage array from 1 to 2 entries. That extra pointer is mostly unused today, but the alternative solutions would not be free either and would introduce more complexity all over the place. Trade the 8bytes for simplicity. Signed-off-by: Thomas Gleixner --- include/linux/irqdomain_defs.h | 1 + include/linux/msi.h | 2 ++ include/linux/msi_api.h | 1 + 3 files changed, 4 insertions(+) --- a/include/linux/irqdomain_defs.h +++ b/include/linux/irqdomain_defs.h @@ -25,6 +25,7 @@ enum irq_domain_bus_token { DOMAIN_BUS_PCI_DEVICE_MSIX, DOMAIN_BUS_DMAR, DOMAIN_BUS_AMDVI, + DOMAIN_BUS_PCI_DEVICE_IMS, }; #endif /* _LINUX_IRQDOMAIN_DEFS_H */ --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -559,6 +559,8 @@ enum { MSI_FLAG_MSIX_CONTIGUOUS = (1 << 19), /* PCI/MSI-X vectors can be dynamically allocated/freed post MSI-X enable */ MSI_FLAG_PCI_MSIX_ALLOC_DYN = (1 << 20), + /* Support for PCI/IMS */ + MSI_FLAG_PCI_IMS = (1 << 21), }; /** --- a/include/linux/msi_api.h +++ b/include/linux/msi_api.h @@ -15,6 +15,7 @@ struct device; */ enum msi_domain_ids { MSI_DEFAULT_DOMAIN, + MSI_SECONDARY_DOMAIN, MSI_MAX_DEVICE_IRQDOMAINS, }; From patchwork Thu Nov 24 23:26:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3664734wrr; Thu, 24 Nov 2022 15:40:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf6UpSy95GdJyjkFuwBLGWRTZHr8QzmfHW/LNYv65B4nMolvzPIAVUpkYzT/nyXAexSiFNFp X-Received: by 2002:a17:906:3993:b0:7ad:f5a9:ece3 with SMTP id h19-20020a170906399300b007adf5a9ece3mr29702873eje.635.1669333216336; Thu, 24 Nov 2022 15:40:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669333216; cv=none; d=google.com; s=arc-20160816; b=IUbMH7NSBp1yQTdeIJawZSYqSdT+n/KcvCjRPlruJkB01qxWx4M+Ptd6LWiE9cQ4O7 QhPu5FkAaBOjXu6rsVapIE1fqy8Xu8y206cC79qpeZ35cAP1dL9s9Lesf0khHeRw5yaV +iVQorwysI5qt3k7/avkbC0WoIenW4xqG+tA9GjFRumTiiPDsRvDIKyFUgLYTzsv2E28 Xr5fTxYRfbbvriwWZaBXKuQc6guEpswDZ2Ep1vHKMAkECZ6lmU8OGIyGFx8Ee7YDE3Eh uOBZjrQVwnIxz+8FtiPBZClWpAOuUQUqVtZh2OaqNrWN1FxopVt+YMx28Q6jLgUZq0Fv JGAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=pHYNj/tOODPRGvpoXQej+yHyq5YhurDakOsBIaKgkpM=; b=X7Eb6u2xZ2atISN87c0Ohlf1X8festsT2dWwyQp8X7DREFflvMfmZ3k3wp5OVWIe4k 56oPNwsTnEF1MHh9BY0ysXUjWuUTNOx49s55Jrwz+I1vI2dZXW2rTWuVSYdAbxXKp5nE pP++nTEBGeNpBfwhiFZY6ObPLUFgOjE+uZ8n01Q+YdxdG+FRxXBTX78CTGFOal4oNAkj 6UHQKkmVw75/QVgbmGC3Q1BAGLkn2KFAWa5w/yupCYiMv0SauwWqvdTMChfBE/6wFyuV oBMnHt0yP3OXbvmGM/sK43KNFZZzSdNYihF/1o0qYn0iS8ujQ8wjJJwgyL+PKiPQsbHu FLVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PyiRAtx7; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m29-20020a50999d000000b0046768efb587si929446edb.69.2022.11.24.15.39.50; Thu, 24 Nov 2022 15:40:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=PyiRAtx7; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229996AbiKXXah (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbiKXX35 (ORCPT ); Thu, 24 Nov 2022 18:29:57 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CDE114EC2C; Thu, 24 Nov 2022 15:27:57 -0800 (PST) Message-ID: <20221124232326.904316841@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=pHYNj/tOODPRGvpoXQej+yHyq5YhurDakOsBIaKgkpM=; b=PyiRAtx7MFPlpm9eN7xpUzZPic7sdE/sSBL0cZBTN7tzq7ViW7LukLk/uLyWa4zPu+qn7g hZiKoPSaEExgzeQQn6fkxk7hDJ/cBMXpzIzz2u7zQko/WUWzpTH8bdLZKpFg+LrInOw8Jn kR5XjfByqB/WTK7/NN2ecOCWoFyaIfTUMny/XfwMemWI7wCLwEYHfgD38KjEjgMi8shY+l bfJ/YfYwGK6sgyjWiX9y2AXSv11/yzr1A3A70fnhYTbgAixLeIW6tQ82jWUkJzXO6cGCu6 xzJSDAe6z7nHc55FHk9ucX/yhvaSPpfghV/KoLmr3waDQS0V3BDflxZaEVJ9fQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=pHYNj/tOODPRGvpoXQej+yHyq5YhurDakOsBIaKgkpM=; b=TvagWfbPoNQbp4EysCimvVt8l3uSwoBEmMmkZYlJWsXx67hqhyoYVMqtoiHHZW3BviDNP4 WObeRGPpMWnLkqDg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 28/33] PCI/MSI: Provide IMS (Interrupt Message Store) support References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:29 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750422746684673653?= X-GMAIL-MSGID: =?utf-8?q?1750422746684673653?= IMS (Interrupt Message Store) is a new specification which allows implementation specific storage of MSI messages contrary to the strict standard specified MSI and MSI-X message stores. This requires new device specific interrupt domains to handle the implementation defined storage which can be an array in device memory or host/guest memory which is shared with hardware queues. Add a function to create IMS domains for PCI devices. IMS domains are using the new per device domain mechanism and are configured by the device driver via a template. IMS domains are created as secondary device domains so they work side on side with MSI[-X] on the same device. The IMS domains have a few constraints: - The index space is managed by the core code. Device memory based IMS provides a storage array with a fixed size which obviously requires an index. But there is no association between index and functionality so the core can randomly allocate an index in the array. System memory based IMS does not have the concept of an index as the storage is somewhere in memory. In that case the index is purely software based to keep track of the allocations. - There is no requirement for consecutive index ranges This is currently a limitation of the MSI core and can be implemented if there is a justified use case by changing the internal storage from xarray to maple_tree. For now it's single vector allocation. - The interrupt chip must provide the following callbacks: - irq_mask() - irq_unmask() - irq_write_msi_msg() - The interrupt chip must provide the following optional callbacks when the irq_mask(), irq_unmask() and irq_write_msi_msg() callbacks cannot operate directly on hardware, e.g. in the case that the interrupt message store is in queue memory: - irq_bus_lock() - irq_bus_unlock() These callbacks are invoked from preemptible task context and are allowed to sleep. In this case the mandatory callbacks above just store the information. The irq_bus_unlock() callback is supposed to make the change effective before returning. - Interrupt affinity setting is handled by the underlying parent interrupt domain and communicated to the IMS domain via irq_write_msi_msg(). IMS domains cannot have a irq_set_affinity() callback. That's a reasonable restriction similar to the PCI/MSI device domain implementations. The domain is automatically destroyed when the PCI device is removed. Signed-off-by: Thomas Gleixner --- V3: Queue memory -> system memory (Kevin) --- drivers/pci/msi/irqdomain.c | 59 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 5 +++ 2 files changed, 64 insertions(+) --- a/drivers/pci/msi/irqdomain.c +++ b/drivers/pci/msi/irqdomain.c @@ -355,6 +355,65 @@ bool pci_msi_domain_supports(struct pci_ return (supported & feature_mask) == feature_mask; } +/** + * pci_create_ims_domain - Create a secondary IMS domain for a PCI device + * @pdev: The PCI device to operate on + * @template: The MSI info template which describes the domain + * @hwsize: The size of the hardware entry table or 0 if the domain + * is purely software managed + * @data: Optional pointer to domain specific data to be stored + * in msi_domain_info::data + * + * Return: True on success, false otherwise + * + * An IMS domain is expected to have the following constraints: + * - The index space is managed by the core code + * + * - There is no requirement for consecutive index ranges + * + * - The interrupt chip must provide the following callbacks: + * - irq_mask() + * - irq_unmask() + * - irq_write_msi_msg() + * + * - The interrupt chip must provide the following optional callbacks + * when the irq_mask(), irq_unmask() and irq_write_msi_msg() callbacks + * cannot operate directly on hardware, e.g. in the case that the + * interrupt message store is in queue memory: + * - irq_bus_lock() + * - irq_bus_unlock() + * + * These callbacks are invoked from preemptible task context and are + * allowed to sleep. In this case the mandatory callbacks above just + * store the information. The irq_bus_unlock() callback is supposed + * to make the change effective before returning. + * + * - Interrupt affinity setting is handled by the underlying parent + * interrupt domain and communicated to the IMS domain via + * irq_write_msi_msg(). + * + * The domain is automatically destroyed when the PCI device is removed. + */ +bool pci_create_ims_domain(struct pci_dev *pdev, const struct msi_domain_template *template, + unsigned int hwsize, void *data) +{ + struct irq_domain *domain = dev_get_msi_domain(&pdev->dev); + + if (!domain || !irq_domain_is_msi_parent(domain)) + return -ENOTSUPP; + + if (template->info.bus_token != DOMAIN_BUS_PCI_DEVICE_IMS || + !(template->info.flags & MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS) || + !(template->info.flags & MSI_FLAG_FREE_MSI_DESCS) || + !template->chip.irq_mask || !template->chip.irq_unmask || + !template->chip.irq_write_msi_msg || template->chip.irq_set_affinity) + return -EINVAL; + + return msi_create_device_irq_domain(&pdev->dev, MSI_SECONDARY_DOMAIN, template, + hwsize, data, NULL); +} +EXPORT_SYMBOL_GPL(pci_create_ims_domain); + /* * Users of the generic MSI infrastructure expect a device to have a single ID, * so with DMA aliases we have to pick the least-worst compromise. Devices with --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -2487,6 +2487,11 @@ static inline bool pci_is_thunderbolt_at void pci_uevent_ers(struct pci_dev *pdev, enum pci_ers_result err_type); #endif +struct msi_domain_template; + +bool pci_create_ims_domain(struct pci_dev *pdev, const struct msi_domain_template *template, + unsigned int hwsize, void *data); + #include #define pci_printk(level, pdev, fmt, arg...) \ From patchwork Thu Nov 24 23:26:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25772 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3688585wrr; Thu, 24 Nov 2022 16:47:10 -0800 (PST) X-Google-Smtp-Source: AA0mqf6NDSDh0HYaNrIlC1aObBnTDK1KrfpVs+rsKcNDTKUBsaBpZb36ViEbDDJv7W0tC1IMsKXh X-Received: by 2002:a17:903:2109:b0:186:8d7b:377e with SMTP id o9-20020a170903210900b001868d7b377emr16681871ple.19.1669337230173; Thu, 24 Nov 2022 16:47:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669337230; cv=none; d=google.com; s=arc-20160816; b=v5dJgVy+QVqo27gOrrifTxGRQ1dL4XuiJPkjB3z/MrTKh7j9epZ2Kg7nnLoKEf1fgp 8pVcXfsJX7xuk4PcS5giFIikN7P6EjnxwNy0/kAsvBl/DFJmIecxxbQ/w2xD/JpHIje/ WAbiUMMgXTpS+mX0vDO2WHM+WtBHxra8MZRyx8iv2krLZewqBq0i129SxzNALz4NxluM FftmyHUqBaNJYRQw29UogwdBXds0weU3/yd3TZpbnhIlfp1FX/45U93ZcowZIsdjNdel S9W1WdDz8+7ZqjYARuc4qiGUpfghtAdiuTY7iQOUdnL/gtVvOP2KZiI8pkq1PJRS+SiB +cdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=5RTULcrzALXI6kqcCTFqzHeG98y1iAErZ2LN9kOhvOw=; b=qL6IAu6EOP0S3wIUUIT5qDA+oNClYwzhuUCp/4FSnWx2V31tWqmbhboKT6jejkq/Rw ZOoUT2Xiq5pLhenPYOk005WT1sZ6WMMGKyoFzTcF26iI9kNSf9ig6NEztZ2cSe/B0r6r neNkKZsVq/ci6UY2fuhUy5DGGk8CfkP3EYKr9V7xYK9VYP9CtY/uWwJ4vjWLIFXELUto i/f17k5zpjqL3qiju8tpc18iZ8+kf5VrSWGhKl7ASB53usc2Gsqag6qyIi5cQ2iIQz1z mkhg3rzmMWb5p3PEXFqgZAvlPhg8MXWRJjRYrYV2al01wvjL0PVdsUO8d5zGmgdiWCh0 81XA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=z4DbH4XI; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=lu6X+IMJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f12-20020a65400c000000b00439ae523fd2si2566103pgp.289.2022.11.24.16.46.53; Thu, 24 Nov 2022 16:47:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=z4DbH4XI; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=lu6X+IMJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229999AbiKXXal (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229972AbiKXX35 (ORCPT ); Thu, 24 Nov 2022 18:29:57 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A8EA59856; Thu, 24 Nov 2022 15:27:58 -0800 (PST) Message-ID: <20221124232326.961711347@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5RTULcrzALXI6kqcCTFqzHeG98y1iAErZ2LN9kOhvOw=; b=z4DbH4XIv1kmxippL2hPCp96iWQYpvohT0sVRs5PG25Df35z6ugkXIfB7ahEPFJDmqmTG8 d+ztRHLzC6WuX9awOb1syNuT1Gwlk5hXRUPg9I8cxvNlYjP7+ARyMHQgLdf0Fj6a9LpvSh ungaxTUr3kpRsxm9k0OrPtpQErNPJ0W9CitB7HN8OksxBa+eJhm9F9CzTix/QI7/2enWmG 9biQXQElJUVHE9wwSCjT1Q9F+qMpaqdlPo/D+ykmdBRAeRs8v3kTjWcW/8NSyQtp3UjwRl jLS6X0EssAFpy97oIDN6b1cLAzXnJm9XOEoI4Jxxt414qNeKpZT7ySuguLS4pw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=5RTULcrzALXI6kqcCTFqzHeG98y1iAErZ2LN9kOhvOw=; b=lu6X+IMJmfOSIvBVNI1TXB/n/jyz7pQkN3zGH7jqRBV8dTnepVR3Op1rS8fkLO0RNNiDI5 HpAX+FhR+yvzvXCA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 29/33] PCI/MSI: Provide pci_ims_alloc/free_irq() References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:31 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426955421090963?= X-GMAIL-MSGID: =?utf-8?q?1750426955421090963?= Single vector allocation which allocates the next free index in the IMS space. The free function releases. All allocated vectors are released also via pci_free_vectors() which is also releasing MSI/MSI-X vectors. Signed-off-by: Thomas Gleixner Acked-by: Bjorn Helgaas --- V3: s/cookie/icookie/ (Kevin) --- drivers/pci/msi/api.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 3 +++ 2 files changed, 53 insertions(+) --- a/drivers/pci/msi/api.c +++ b/drivers/pci/msi/api.c @@ -361,6 +361,56 @@ const struct cpumask *pci_irq_get_affini EXPORT_SYMBOL(pci_irq_get_affinity); /** + * pci_ims_alloc_irq - Allocate an interrupt on a PCI/IMS interrupt domain + * @dev: The PCI device to operate on + * @icookie: Pointer to an IMS implementation specific cookie for this + * IMS instance (PASID, queue ID, pointer...). + * The cookie content is copied into the MSI descriptor for the + * interrupt chip callbacks or domain specific setup functions. + * @affdesc: Optional pointer to an interrupt affinity descriptor + * + * There is no index for IMS allocations as IMS is an implementation + * specific storage and does not have any direct associations between + * index, which might be a pure software construct, and device + * functionality. This association is established by the driver either via + * the index - if there is a hardware table - or in case of purely software + * managed IMS implementation the association happens via the + * irq_write_msi_msg() callback of the implementation specific interrupt + * chip, which utilizes the provided @icookie to store the MSI message in + * the appropriate place. + * + * Return: A struct msi_map + * + * On success msi_map::index contains the allocated index (>= 0) and + * msi_map::virq the allocated Linux interrupt number (> 0). + * + * On fail msi_map::index contains the error code and msi_map::virq + * is set to 0. + */ +struct msi_map pci_ims_alloc_irq(struct pci_dev *dev, union msi_instance_cookie *icookie, + const struct irq_affinity_desc *affdesc) +{ + return msi_domain_alloc_irq_at(&dev->dev, MSI_SECONDARY_DOMAIN, MSI_ANY_INDEX, + affdesc, icookie); +} +EXPORT_SYMBOL_GPL(pci_ims_alloc_irq); + +/** + * pci_ims_free_irq - Allocate an interrupt on a PCI/IMS interrupt domain + * which was allocated via pci_ims_alloc_irq() + * @dev: The PCI device to operate on + * @map: A struct msi_map describing the interrupt to free as + * returned from pci_ims_alloc_irq() + */ +void pci_ims_free_irq(struct pci_dev *dev, struct msi_map map) +{ + if (WARN_ON_ONCE(map.index < 0 || !map.virq)) + return; + msi_domain_free_irqs_range(&dev->dev, MSI_SECONDARY_DOMAIN, map.index, map.index); +} +EXPORT_SYMBOL_GPL(pci_ims_free_irq); + +/** * pci_free_irq_vectors() - Free previously allocated IRQs for a device * @dev: the PCI device to operate on * --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -2491,6 +2491,9 @@ struct msi_domain_template; bool pci_create_ims_domain(struct pci_dev *pdev, const struct msi_domain_template *template, unsigned int hwsize, void *data); +struct msi_map pci_ims_alloc_irq(struct pci_dev *pdev, union msi_instance_cookie *icookie, + const struct irq_affinity_desc *affdesc); +void pci_ims_free_irq(struct pci_dev *pdev, struct msi_map map); #include From patchwork Thu Nov 24 23:26:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25775 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3689706wrr; Thu, 24 Nov 2022 16:50:42 -0800 (PST) X-Google-Smtp-Source: AA0mqf4rGrRHeKIjKF1uW+Hj9zBojsyT5funkDA7W+DXorUBD5SbUhYC0U2tsR2M2UzHyBG9GiT7 X-Received: by 2002:a65:48cc:0:b0:434:c0ca:b376 with SMTP id o12-20020a6548cc000000b00434c0cab376mr30990480pgs.180.1669337442039; Thu, 24 Nov 2022 16:50:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669337442; cv=none; d=google.com; s=arc-20160816; b=xnf+fB5AqqoyRxVR8SvgmgyiTp5gkTupKIbok4/TS37Vx8tj0/lCP6T9f93fTuSSUr dznLIaLCRX2MKS0JCIIZ8uHxQRMtfqhUSmwNGAWvicKtVpzGJLItcPIZAftFKia9Ke8w Laq4EPDCTdds424lUa7HcvhxlJAUkxeVy8x3qfEYbgFJtSs0SMsDUhu9ZetXcovaVSqq EN8YnQ9Ib7n93ZLjao3QsAWw+e2FfsNeTUdwiG2Hnfle1Gz6L41e0t71ePdMVn7LL4os Pq+++z+PF2tpDZhBePazWJM9ZCIvNtrazsspwpsv0t4Qu0l4SVhhXCvotfvkOm3clLaP FRxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=QDMaEx7ktXTUaBjXhcwy3JcriisNhamDQjdxbqkUVcQ=; b=gGO2wszr0RT+Bxp4+8Lw90EHJILdgBsU+bhnnhWSWmnUVOapYff2fJ+rW0nKJBBEAr ls7tr4sJh3FxhGZjScuNxWfHSoWsJr71Fsbhat8Z7xmxXZJLo0aAejVzneVAmj57jB7T 8A1zuAAejAI90QgzTyblBgAOrFD/hGGYbg2M7/ffPqQz6dGj02eVUAWu3U0SQYog/vtA vBa51pA/0WkUWqvhMNyID26ht/YEEetSwuLi6vQTAgurHvbKotf3Lmm10YLZoAwKzK6F m4ZttTAuKnhEIDDxizCy7lo5+8JrrW0Qh9JnyzJSHAx6Nnmbi4o02mN/+WRnvjDW4+P5 AyJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=zUr0QOYx; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=e02d3EkP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bt21-20020a17090af01500b00213022ff9a8si5449857pjb.168.2022.11.24.16.50.29; Thu, 24 Nov 2022 16:50:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=zUr0QOYx; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=e02d3EkP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230144AbiKXXaw (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230003AbiKXXaA (ORCPT ); Thu, 24 Nov 2022 18:30:00 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EDD9A87A71; Thu, 24 Nov 2022 15:28:00 -0800 (PST) Message-ID: <20221124232327.022658817@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QDMaEx7ktXTUaBjXhcwy3JcriisNhamDQjdxbqkUVcQ=; b=zUr0QOYxgwDD1PsVVViqo27E4EsaPMbR4ALz9TfU0GC5htAX5nGPC6kA6mGwt93eEzunFY SV6YNVcQFPTwSfCmcVnJLHcVKJmea/9hHBZcbkFnnBWcMWqs62jdSUBSHL4N8+ePtaCL8B FFaR0KtJcel6lPaAZYtNjpTGRXpt7dKubo3WLJ9W9CX6n/h3XV/bLSg1oc9TgjhDZPwb3j GlFZSKS9M6+gjUjkWy8BNAyRLYckcl+6G3bOpAq+nxGJOsnvl2SLurpDuWo6Cz631V9AIs 05lrfrLGyMspEnMnsIubC+CzKiuF3CoVpv4qB6h5U1urATybLK6JYPIujJWWNQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QDMaEx7ktXTUaBjXhcwy3JcriisNhamDQjdxbqkUVcQ=; b=e02d3EkPUlNpaELh4mxVVoCi+mQ7c03Dpuxx2rMxLiCJ+d8w3kCUnmnKMKavOjEADyVpy5 OPe7q2W2AgbVSkBQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 30/33] x86/apic/msi: Enable PCI/IMS References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:32 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750427177593251696?= X-GMAIL-MSGID: =?utf-8?q?1750427177593251696?= Enable IMS in the domain init and allocation mapping code, but do not enable it on the vector domain as discussed in various threads on LKML. The interrupt remap domains can expand this setting like they do with PCI multi MSI. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/apic/msi.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/arch/x86/kernel/apic/msi.c +++ b/arch/x86/kernel/apic/msi.c @@ -184,6 +184,7 @@ static int x86_msi_prepare(struct irq_do alloc->type = X86_IRQ_ALLOC_TYPE_PCI_MSI; return 0; case DOMAIN_BUS_PCI_DEVICE_MSIX: + case DOMAIN_BUS_PCI_DEVICE_IMS: alloc->type = X86_IRQ_ALLOC_TYPE_PCI_MSIX; return 0; default: @@ -230,6 +231,10 @@ static bool x86_init_dev_msi_info(struct case DOMAIN_BUS_PCI_DEVICE_MSI: case DOMAIN_BUS_PCI_DEVICE_MSIX: break; + case DOMAIN_BUS_PCI_DEVICE_IMS: + if (!(pops->supported_flags & MSI_FLAG_PCI_IMS)) + return false; + break; default: WARN_ON_ONCE(1); return false; From patchwork Thu Nov 24 23:26:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25768 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3684859wrr; Thu, 24 Nov 2022 16:34:39 -0800 (PST) X-Google-Smtp-Source: AA0mqf65R4p0E7TW1EJYyBcBQheQG3LfSjyk+LPq9dLCOr+3Va9LTMfHszQMjjJ028miTa6jsxlF X-Received: by 2002:a63:5b0e:0:b0:476:c832:aa74 with SMTP id p14-20020a635b0e000000b00476c832aa74mr23470384pgb.456.1669336478756; Thu, 24 Nov 2022 16:34:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669336478; cv=none; d=google.com; s=arc-20160816; b=aBa3Ve85ygzmDKya3VlEUmmOJGyQtDj7j3il7dk0pCN4c7jhgp7Ve6S7QOXKRKMpCQ 9/4Vv6BzjxAuN64qySRV4bQUqWBYUUQTz+Hs2vB4DI4mHPJiGL+Gsw56l0tF9ZaEBbzJ Y4vfzvNgL+qbdLo7FPdtcuH20FchMiPOlcnAHzd1alS4/vYFMG4nTeQKvh7v7UMdzGej mAWjrDt+3HyAPwnVHP/y9v16UE+cgYcxaeatul8k+Lwu7i0IGdag6H8Plo83ZtDWcUO3 TtDxesw20GTwNunl9sv2JFy42HveDd2GqAeev0DeL9DH2A0mz290NebraQxK311SNkfU pCCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=CU97iv84FfmZ1eouToiGvE/Nlj455v2TtgsBEv4GPyg=; b=j52YMJhKOAToZCD2TfHWob/BsV+6XMDTj03J40PfsoCXBpLY8kIMntdd3Uz9+1VE07 1hMoSAO6V0wEexYJbRtXOWV1mkOZTvRa5NijxJheNgzQfilX5JohyJE4U1qQkixQsb0j edtPvOZ6de3P0AAEJUjv5k12B7A2vYvdD9lmDxPteO/7xGHYDsGVOGhXIShYIpo71LGx luhED6wauKTvWXvEUtbe1hqBEwp+kDNlu5vg+ed0wkWu6cay+iR6eowA+EJ1rjzJHYFZ XEC7Qn1iXpdBWeIHIjXg95P2RgScOJtGvVQW5TzRQ++b7n5iojC/qCUzA3Ng4p67Ro57 9glQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="mEH6I24/"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=4AG3rmFr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 1-20020a621901000000b005734ae5e1b4si1794894pfz.362.2022.11.24.16.34.25; Thu, 24 Nov 2022 16:34:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="mEH6I24/"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=4AG3rmFr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230085AbiKXXay (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230008AbiKXXaB (ORCPT ); Thu, 24 Nov 2022 18:30:01 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 717826EB76; Thu, 24 Nov 2022 15:28:02 -0800 (PST) Message-ID: <20221124232327.081482253@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=CU97iv84FfmZ1eouToiGvE/Nlj455v2TtgsBEv4GPyg=; b=mEH6I24/5JaHzfSu02ohUthSvjaDa/eetOKY9B4TW+htv7TjKS4Zxjnmte2TsmKSs+TgqB Sauyw/ycHxhL8211OkQpJT53dInJ2dJT0iy8mot93OBiaBw+AG6oXfNqK3hO8FVsLLHhPS UXnzHiHZd1r/de8cF/8mcJbNkdVdEqcUYBpAM/P5PObJbS0jnnw+CCqZDfBBPOq+BjpyI0 EKQJGWXSVfdPq7NmEFp0XFit/EQj9MS7n+G8BDZeuVrhs8jR/v3NKZZOjzebaDrDSYGjny +a0VTwOEyGmy0/D9ee3SLPe7fSWP8UqNklpA7rSgIxgeU2KeauCm7iogfLDVkw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=CU97iv84FfmZ1eouToiGvE/Nlj455v2TtgsBEv4GPyg=; b=4AG3rmFrHnCbABF7YKqqCd5OwwW3P6xrVWc0sBk8qufCuR+dAayM9fy/4QFQcJa2jK0FOi iguQdNqPP2+mJaBw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 31/33] iommu/vt-d: Enable PCI/IMS References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:34 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426167857201762?= X-GMAIL-MSGID: =?utf-8?q?1750426167857201762?= PCI/IMS works like PCI/MSI-X in the remapping. Just add the feature flag, but only when on real hardware. Virtualized IOMMUs need additional support, e.g. for PASID. Signed-off-by: Thomas Gleixner --- V3: Only enable on real hardware (Kevin) --- drivers/iommu/intel/irq_remapping.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -82,7 +82,7 @@ static const struct irq_domain_ops intel static void iommu_disable_irq_remapping(struct intel_iommu *iommu); static int __init parse_ioapics_under_ir(void); -static const struct msi_parent_ops dmar_msi_parent_ops; +static const struct msi_parent_ops dmar_msi_parent_ops, virt_dmar_msi_parent_ops; static bool ir_pre_enabled(struct intel_iommu *iommu) { @@ -577,7 +577,11 @@ static int intel_setup_irq_remapping(str irq_domain_update_bus_token(iommu->ir_domain, DOMAIN_BUS_DMAR); iommu->ir_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT; - iommu->ir_domain->msi_parent_ops = &dmar_msi_parent_ops; + + if (cap_caching_mode(iommu->cap)) + iommu->ir_domain->msi_parent_ops = &virt_dmar_msi_parent_ops; + else + iommu->ir_domain->msi_parent_ops = &dmar_msi_parent_ops; ir_table->base = page_address(pages); ir_table->bitmap = bitmap; @@ -1429,11 +1433,20 @@ static const struct irq_domain_ops intel }; static const struct msi_parent_ops dmar_msi_parent_ops = { - .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | MSI_FLAG_MULTI_PCI_MSI, + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | + MSI_FLAG_MULTI_PCI_MSI | + MSI_FLAG_PCI_IMS, .prefix = "IR-", .init_dev_msi_info = msi_parent_init_dev_msi_info, }; +static const struct msi_parent_ops virt_dmar_msi_parent_ops = { + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | + MSI_FLAG_MULTI_PCI_MSI, + .prefix = "vIR-", + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; + /* * Support of Interrupt Remapping Unit Hotplug */ From patchwork Thu Nov 24 23:26:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25773 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3689353wrr; Thu, 24 Nov 2022 16:49:29 -0800 (PST) X-Google-Smtp-Source: AA0mqf4ZEm7XqVWJ1aUuiGujGbyfaD4Qvltmv6Tu/blp0NLF1gZfk0rl1RrPm17Km/Ry8fDWtTuA X-Received: by 2002:a63:4701:0:b0:476:d784:1c44 with SMTP id u1-20020a634701000000b00476d7841c44mr15415472pga.431.1669337369466; Thu, 24 Nov 2022 16:49:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669337369; cv=none; d=google.com; s=arc-20160816; b=NwWdlire+Ht0xr4aPjfIuS3fg9JhBnU1LXHf0rD0/IFT9bUngg6Ubwmef6wIOBEdV1 J57JwbawIH0Fhdwu9qPotHUfaHpqsFdQY3O1RrGire8ZZJoEDV5FtTpGS2gsMlEvUYpc vvsfbvW/80i3z8+njD7ocqb5y1F8uiLotKwYdjim77w/W81jgmFPpcIkl90hf7B3gOm8 24QemFbKyCTXCxMkCagkyFaCmN5inCEbFkki77VZwjeaQURHmo3s8Jlics1wPx7BmkL3 Z6TNO7nVuk8h2/JjoQnxv12nhW10tcrwhXlg/MlowA2Tj6NtEGtHkQj21kj+NyNoEkkH 3UuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=/8ptqSNRM2lDaMDazh2hH5MeM7Cg1Cfp+pBZYsTAF/4=; b=z9iogdMgFmEEYki2odzdbALyPA4clAvWdBlEtJa4Ar13D03taA3DA2pi/uhGtDcs6m RXa8uLusW4o58d2b8n3MYJhu9tcjCrv0V1g5OdAO3C0s9HQ4wuu59Tge11asKlI08ziA VQcLP+1yUsNvwYl+VXbidCe2Cjqj9HY9Pb/K9m+3gnIg8FaKSEM7ZJHWQRUfjtCbSagR J6SFk3xCHpaMLF7jNN0UIoszF9f7n7PG99NMFQqOKMGvcjUw5IyMTVODVRJtbsmVgw9M fg636hx3FMPvp2NEQVm5E5MonYg/jPb8uDadEt32F5vvjHNEiACnl6Lgj21zklmefYYt mnOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=WWGH+ztk; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h13-20020a170902680d00b001869c2b3a3csi2147881plk.537.2022.11.24.16.49.16; Thu, 24 Nov 2022 16:49:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=WWGH+ztk; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230050AbiKXXa6 (ORCPT + 99 others); Thu, 24 Nov 2022 18:30:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230011AbiKXXaB (ORCPT ); Thu, 24 Nov 2022 18:30:01 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 722D68CF33; Thu, 24 Nov 2022 15:28:02 -0800 (PST) Message-ID: <20221124232327.140571546@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332396; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/8ptqSNRM2lDaMDazh2hH5MeM7Cg1Cfp+pBZYsTAF/4=; b=WWGH+ztk9ARRPXkCs//xWWHperFjxKzonkgGGDJYW5Dqx/aG3uvU8oBe1ufx0DJvtxmdrS MYEn9Vx62ip2hWr5M2DmLHvrtWHQRklP+B1uLhTthr8M8IFd7HgOKNhSQskuUDCla+JP59 WE3QBcVwTIwI07wMj7T44s2DIc+yCHwMaw1aCMH9oMD7efmATrdG9bt85OhSaYQs2P2PXd D1pb97jY8/MmPPn9qXm6cYVrJ6UEOx9e8oV+q+7bMVMl19ZUUhgxFFyPmMvqTVh+yk7zs/ OxrRF97L/y/0nULYcuMSOon+0LoLchmv8ilaPplXa2pM7b7AUSP0j85yZdRzVw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332396; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=/8ptqSNRM2lDaMDazh2hH5MeM7Cg1Cfp+pBZYsTAF/4=; b=JfV3W3IYhlVyRU9VQDQP/+KkG87oio6896IfInAUeN6yem7L5Iohw2fGrPxWEjycry7KNM DCdZd0v4WszkQpCg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 32/33] iommu/amd: Enable PCI/IMS References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:36 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750427101373234803?= X-GMAIL-MSGID: =?utf-8?q?1750427101373234803?= PCI/IMS works like PCI/MSI-X in the remapping. Just add the feature flag, but only when on real hardware. Virtualized IOMMUs need additional support. Signed-off-by: Thomas Gleixner --- V3: Only enable on real hardware (Kevin) --- drivers/iommu/amd/iommu.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -3649,11 +3649,20 @@ static struct irq_chip amd_ir_chip = { }; static const struct msi_parent_ops amdvi_msi_parent_ops = { - .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | MSI_FLAG_MULTI_PCI_MSI, + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | + MSI_FLAG_MULTI_PCI_MSI | + MSI_FLAG_PCI_IMS, .prefix = "IR-", .init_dev_msi_info = msi_parent_init_dev_msi_info, }; +static const struct msi_parent_ops virt_amdvi_msi_parent_ops = { + .supported_flags = X86_VECTOR_MSI_FLAGS_SUPPORTED | + MSI_FLAG_MULTI_PCI_MSI, + .prefix = "vIR-", + .init_dev_msi_info = msi_parent_init_dev_msi_info, +}; + int amd_iommu_create_irq_domain(struct amd_iommu *iommu) { struct fwnode_handle *fn; @@ -3670,7 +3679,11 @@ int amd_iommu_create_irq_domain(struct a irq_domain_update_bus_token(iommu->ir_domain, DOMAIN_BUS_AMDVI); iommu->ir_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT; - iommu->ir_domain->msi_parent_ops = &amdvi_msi_parent_ops; + + if (amd_iommu_np_cache) + iommu->ir_domain->msi_parent_ops = &virt_amdvi_msi_parent_ops; + else + iommu->ir_domain->msi_parent_ops = &amdvi_msi_parent_ops; return 0; } From patchwork Thu Nov 24 23:26:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 25771 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp3686322wrr; Thu, 24 Nov 2022 16:39:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf4OZQaXLjxx6V6GLQMBDP1St4zTbLs/naAejfl/pBPOLanrFMGguqW8NUuD9kczh3s5tfbf X-Received: by 2002:a17:90a:71c8:b0:218:494d:e9d2 with SMTP id m8-20020a17090a71c800b00218494de9d2mr44321644pjs.50.1669336792832; Thu, 24 Nov 2022 16:39:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669336792; cv=none; d=google.com; s=arc-20160816; b=axtC9rjfK8flEwI15dsNmiSc7SIzo826xAusAYzXxYJx/1f4mQtTmhSOd6QaImDr+S Lwh3szBX1zpgzSWbq7RL0OC8BH/QRCHqY1ClSoztP+jIHVFNtl1ttmawH1QxY9xMzKDY F7GG/fJKql0K/xOleD6bGT4Klfh4Y/l6CEquomIPgEKR6xza+ArqR+NxbgB9zl3GTSJc cMi1c5LCmP4GwFRAg8cvwMaMR4rMQ51iMkSjdK82KcqlCNrTYDJGfTp90d0olGFSJQN6 r3lLvGnmEuC68lVyhSssAx/TcFJfvdvymZZYsojUP1fGp8GKOdfDAH1LVHOCrjpKsnai AEyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=EjoFLIGfNIA85CvW/O+W8Se9j0RMzVdkCn2NwApFDRE=; b=dMhvm2iEOrCPBx0e1RH4mYooUL6/4Ibw8mzpe5EEgtvF5MAm2gDY92aHlv0IVdfjHu LpEND4rOiJHUYApmRmb4LBKNgEE3/pX2r6q7R4pSsW6WxGV/9UC7y74/SkBydhuGRWpp JO78y6II2KmEEj5vyr2thhFCunVDbJgn9Awe/0KhCKNVTf2DC9A5niQ+oYN/plh7ljBR oyit3QG9mrjvWFpuXZl/p2oBInfrPGg+rWF6Gtw3Mn57aEOekJ0dT72jjAN3+a+HLXaL 6pv7wQQqZSn3qsLdmi0NmLkqLi5J1W2VPivD+pc/Hothci0R4hkkvaqNpCndI1k0rzh2 kh/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=jgyffkKH; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id oc12-20020a17090b1c0c00b00213032c1b7esi6907727pjb.13.2022.11.24.16.39.40; Thu, 24 Nov 2022 16:39:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=jgyffkKH; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230080AbiKXXbP (ORCPT + 99 others); Thu, 24 Nov 2022 18:31:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230013AbiKXXaw (ORCPT ); Thu, 24 Nov 2022 18:30:52 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A75F8D299C; Thu, 24 Nov 2022 15:28:08 -0800 (PST) Message-ID: <20221124232327.198240498@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1669332398; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EjoFLIGfNIA85CvW/O+W8Se9j0RMzVdkCn2NwApFDRE=; b=jgyffkKHDkbJoARDjoHthE3u0MAXN/BVa0tu/GAlDc7R50l6+QsuPYEfoFT5CebVZRN0Hp UaWj1D3sIt+HWBnt3RBkhLeoXythirkm0JK6SDHGlFnXZ4PwT61PCYuvMiVVMGzRBVot31 lX88p10JiYjeKCi59GPnsHZH783gYDaFnIYfZ9bKtfMPI9cR3usJIMpDv67eqpgVJ5eCnC tOg5QSiCGIlzPIpZsgKRC8iqDMuhDF1MRfjHB3gRzmwkD02sofDld9SfRDWHtN5YxtXcHv uL9reTreRhr7gWpZDN25p5L9XwcOuURSP8mjXN5r6DF0jNPVmAZ4sRHydDeCuw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1669332398; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=EjoFLIGfNIA85CvW/O+W8Se9j0RMzVdkCn2NwApFDRE=; b=cqUYLb50xujiv8NfIza/XagWhZ6YBCcxANZRUAxMaKIVDPb0pi7DZuONkZ9/qY94cybCKY yrMPMy9oQMhG6UCQ== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Joerg Roedel , Will Deacon , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Marc Zyngier , Greg Kroah-Hartman , Jason Gunthorpe , Dave Jiang , Alex Williamson , Kevin Tian , Dan Williams , Logan Gunthorpe , Ashok Raj , Jon Mason , Allen Hubbe Subject: [patch V3 33/33] irqchip: Add IDXD Interrupt Message Store driver References: <20221124230505.073418677@linutronix.de> MIME-Version: 1.0 Date: Fri, 25 Nov 2022 00:26:37 +0100 (CET) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750426497131027382?= X-GMAIL-MSGID: =?utf-8?q?1750426497131027382?= Provide a driver for the Intel IDXD IMS implementation. The implementation uses a large message store array in device memory. The IMS domain implementation is minimal and just provides the required irq_chip callbacks and one domain callback which prepares the MSI descriptor for easy usage in the irq_chip callbacks. The necessary iobase is stored in the irqdomain and the PASID which is required for operation is handed in via msi_instance_cookie in the allocation function. Not much to see here. A few lines of code and a filled in template is all what's needed. Signed-off-by: Thomas Gleixner --- V3: Update changelog/comments (Kevin) --- drivers/irqchip/Kconfig | 7 + drivers/irqchip/Makefile | 1 drivers/irqchip/irq-pci-intel-idxd.c | 143 +++++++++++++++++++++++++++++ include/linux/irqchip/irq-pci-intel-idxd.h | 22 ++++ 4 files changed, 173 insertions(+) --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -695,4 +695,11 @@ config SUNPLUS_SP7021_INTC chained controller, routing all interrupt source in P-Chip to the primary controller on C-Chip. +config PCI_INTEL_IDXD_IMS + tristate "Intel IDXD Interrupt Message Store controller" + depends on PCI_MSI + help + Support for Intel IDXD Interrupt Message Store (IMS) controller + with IMS slot storage in a slot array in device memory + endmenu --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -121,3 +121,4 @@ obj-$(CONFIG_IRQ_IDT3243X) += irq-idt32 obj-$(CONFIG_APPLE_AIC) += irq-apple-aic.o obj-$(CONFIG_MCHP_EIC) += irq-mchp-eic.o obj-$(CONFIG_SUNPLUS_SP7021_INTC) += irq-sp7021-intc.o +obj-$(CONFIG_PCI_INTEL_IDXD_IMS) += irq-pci-intel-idxd.o --- /dev/null +++ b/drivers/irqchip/irq-pci-intel-idxd.c @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Interrupt chip and domain for Intel IDXD with hardware array based + * interrupt message store (IMS). + */ +#include +#include +#include +#include +#include + +#include + +MODULE_LICENSE("GPL"); + +/** + * struct ims_slot - The hardware layout of a slot in the memory table + * @address_lo: Lower 32bit address + * @address_hi: Upper 32bit address + * @data: Message data + * @ctrl: Control word + */ +struct ims_slot { + u32 address_lo; + u32 address_hi; + u32 data; + u32 ctrl; +} __packed; + +/* Bit to mask the interrupt in the control word */ +#define CTRL_VECTOR_MASKBIT BIT(0) +/* Bit to enable PASID in the control word */ +#define CTRL_PASID_ENABLE BIT(3) +/* Position of PASID.LSB in the control word */ +#define CTRL_PASID_SHIFT 12 + +static inline void iowrite32_and_flush(u32 value, void __iomem *addr) +{ + iowrite32(value, addr); + ioread32(addr); +} + +static void idxd_mask(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->data.dcookie.iobase; + u32 cval = (u32)desc->data.icookie.value; + + iowrite32_and_flush(cval | CTRL_VECTOR_MASKBIT, &slot->ctrl); +} + +static void idxd_unmask(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->data.dcookie.iobase; + u32 cval = (u32)desc->data.icookie.value; + + iowrite32_and_flush(cval, &slot->ctrl); +} + +static void idxd_write_msi_msg(struct irq_data *data, struct msi_msg *msg) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->data.dcookie.iobase; + + iowrite32(msg->address_lo, &slot->address_lo); + iowrite32(msg->address_hi, &slot->address_hi); + iowrite32_and_flush(msg->data, &slot->data); +} + +static void idxd_shutdown(struct irq_data *data) +{ + struct msi_desc *desc = irq_data_get_msi_desc(data); + struct ims_slot __iomem *slot = desc->data.dcookie.iobase; + + iowrite32(0, &slot->address_lo); + iowrite32(0, &slot->address_hi); + iowrite32(0, &slot->data); + iowrite32_and_flush(CTRL_VECTOR_MASKBIT, &slot->ctrl); +} + +static void idxd_prepare_desc(struct irq_domain *domain, msi_alloc_info_t *arg, + struct msi_desc *desc) +{ + struct msi_domain_info *info = domain->host_data; + struct ims_slot __iomem *slot; + + /* Set up the slot address for the irq_chip callbacks */ + slot = (__force struct ims_slot __iomem *) info->data; + slot += desc->msi_index; + desc->data.dcookie.iobase = slot; + + /* Mask the interrupt for paranoia sake */ + iowrite32_and_flush(CTRL_VECTOR_MASKBIT, &slot->ctrl); + + /* + * The caller provided PASID. Shift it to the proper position + * and set the PASID enable bit. + */ + desc->data.icookie.value <<= CTRL_PASID_SHIFT; + desc->data.icookie.value |= CTRL_PASID_ENABLE; + + arg->hwirq = desc->msi_index; +} + +static const struct msi_domain_template idxd_ims_template = { + .chip = { + .name = "PCI-IDXD", + .irq_mask = idxd_mask, + .irq_unmask = idxd_unmask, + .irq_write_msi_msg = idxd_write_msi_msg, + .irq_shutdown = idxd_shutdown, + .flags = IRQCHIP_ONESHOT_SAFE, + }, + + .ops = { + .prepare_desc = idxd_prepare_desc, + }, + + .info = { + .flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | + MSI_FLAG_FREE_MSI_DESCS | + MSI_FLAG_PCI_IMS, + .bus_token = DOMAIN_BUS_PCI_DEVICE_IMS, + }, +}; + +/** + * pci_intel_idxd_create_ims_domain - Create a IDXD IMS domain + * @pdev: IDXD PCI device to operate on + * @slots: Pointer to the mapped slot memory array + * @nr_slots: The number of slots in the array + * + * Returns: True on success, false otherwise + * + * The domain is automatically destroyed when the @pdev is destroyed + */ +bool pci_intel_idxd_create_ims_domain(struct pci_dev *pdev, void __iomem *slots, + unsigned int nr_slots) +{ + return pci_create_ims_domain(pdev, &idxd_ims_template, nr_slots, (__force void *)slots); +} +EXPORT_SYMBOL_GPL(pci_intel_idxd_create_ims_domain); --- /dev/null +++ b/include/linux/irqchip/irq-pci-intel-idxd.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* (C) Copyright 2022 Thomas Gleixner */ + +#ifndef _LINUX_IRQCHIP_IRQ_PCI_INTEL_IDXD_H +#define _LINUX_IRQCHIP_IRQ_PCI_INTEL_IDXD_H + +#include +#include +#include + +/* + * Conveniance macro to wrap the PASID for interrupt allocation + * via pci_ims_alloc_irq(pdev, INTEL_IDXD_DEV_COOKIE(pasid)) + */ +#define INTEL_IDXD_DEV_COOKIE(pasid) (union msi_instance_cookie) { .value = (pasid), } + +struct pci_dev; + +bool pci_intel_idxd_create_ims_domain(struct pci_dev *pdev, void __iomem *slots, + unsigned int nr_slots); + +#endif