From patchwork Sat May 27 05:46:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huacai Chen X-Patchwork-Id: 99762 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp175474vqr; Sat, 27 May 2023 00:04:41 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7KuJ6sHImXBUIRVivRP1JrMwZgcLmMRqWkcvfHzMsOm6iO+CaRwgiexyBBtU7dAOOQGmns X-Received: by 2002:a05:6a21:3718:b0:10c:5324:dba with SMTP id yl24-20020a056a21371800b0010c53240dbamr1705581pzb.28.1685171081142; Sat, 27 May 2023 00:04:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685171081; cv=none; d=google.com; s=arc-20160816; b=qCVjIa33ijJi+CSNJIGtBv0uthLsp7ZZqUxLd+d4jGVozXnz5mjLat/sD86IIanK7R rjWJHepTXOYiOXeiPEac2ZrEBKbk+FFC7mF3dZbvyUQlxRwB5lTlzMEE/sKoxf1mUxlg +T5177gLf2pGUr3LbHaofguf0pue1gmCdIrD0E4eOfInyhuqFym+udoe7mblS8A8GHIX q+ErWQ6Ab9l2Veu4H2mMpcDE5NpG6hvdeDo2JlBq7/3B0w9qp2nsoP9iiHiFCGHgru+s ZjJy3t3TtXhDvk6JDdRGIq2h3mwZPtcxNZ7s4egoY5liMGseNX4XbN2siah7F53vz9Gi ynCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=/KNmsBgCiB6C38hFQ9fRt0MZjL4be6P6IGIGMYOTUPI=; b=TEdk3BZz3ZmBtR7zLVo0DA5kxNvavc01yrPi0gxMRbGnp7ms2nGraydIhc+jGQOGbk yvBq3c0ZV3XycFPtzqRtbzVIjgCXFVEpn+e+i/D+4CjwuqNb9+WyJT3i2mCVC+91AveH RBHH03eaINvl5f4LO0SaAMMeh6r7mqWShZL0nUiySbb6kUE+53Qvfx2lyIRkuFF8r0fs HkxFYc8G4tuhZAMpuTtcx2ADrQ1LJU5C871WaAyhYYtx2NoSO5r2Ve+GwPpdCZ8iKLtC jrZKaqmSY2UkXQVADyNWqAASCpLZNvD4WppCFJNserWi/KqY05lE7B4OmaL8N+YdlIJT Ct+Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k62-20020a636f41000000b005349286fcc0si5717795pgc.302.2023.05.27.00.04.08; Sat, 27 May 2023 00:04:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229494AbjE0Frc (ORCPT + 99 others); Sat, 27 May 2023 01:47:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231173AbjE0Frb (ORCPT ); Sat, 27 May 2023 01:47:31 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BCAC12A for ; Fri, 26 May 2023 22:47:30 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A991660EF4 for ; Sat, 27 May 2023 05:47:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 031ECC433D2; Sat, 27 May 2023 05:47:26 +0000 (UTC) From: Huacai Chen To: Thomas Gleixner , Marc Zyngier , Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Xuefeng Li , Huacai Chen , Jiaxun Yang , Huacai Chen Subject: [PATCH 1/2] genirq/msi, platform-msi: Adjust return value of msi_domain_prepare_irqs() Date: Sat, 27 May 2023 13:46:32 +0800 Message-Id: <20230527054633.704916-2-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230527054633.704916-1-chenhuacai@loongson.cn> References: <20230527054633.704916-1-chenhuacai@loongson.cn> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767029951536936055?= X-GMAIL-MSGID: =?utf-8?q?1767029951536936055?= Adjust the return value semanteme of msi_domain_prepare_irqs(), which allows us to modify the input nvec by overriding the msi_domain_ops:: msi_prepare(). This is necessary for the later patch. Before: 0 on success, others on error. After: = 0: Success; > 0: The modified nvec; < 0: Error code. Callers are also updated. Signed-off-by: Huacai Chen --- drivers/base/platform-msi.c | 2 +- kernel/irq/msi.c | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c index f37ad34c80ec..e4a517c144e7 100644 --- a/drivers/base/platform-msi.c +++ b/drivers/base/platform-msi.c @@ -298,7 +298,7 @@ __platform_msi_create_device_domain(struct device *dev, platform_msi_set_proxy_dev(&data->arg); err = msi_domain_prepare_irqs(domain->parent, dev, nvec, &data->arg); - if (err) + if (err < 0) goto free_domain; return domain; diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 7a97bcb086bf..d151936aec05 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -1058,6 +1058,12 @@ bool msi_match_device_irq_domain(struct device *dev, unsigned int domid, return ret; } +/* + * Return Val: + * = 0: Success; + * > 0: The modified nvec; + * < 0: Error code. + */ int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) { @@ -1260,8 +1266,10 @@ static int __msi_domain_alloc_irqs(struct device *dev, struct irq_domain *domain int i, ret, virq; ret = msi_domain_prepare_irqs(domain, dev, ctrl->nirqs, &arg); - if (ret) + if (ret < 0) return ret; + if (ret > 0) + ctrl->nirqs = ret; /* * This flag is set by the PCI layer as we need to activate From patchwork Sat May 27 05:46:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huacai Chen X-Patchwork-Id: 99761 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp163168vqr; Fri, 26 May 2023 23:26:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ42h93QGDwco76uCefu4zKtEi+g6V6I0uTwv6L4wkX3cV01BOCEcbxrHZb3mDm9GRB3th8l X-Received: by 2002:a17:90a:bf04:b0:256:4b15:395b with SMTP id c4-20020a17090abf0400b002564b15395bmr1169003pjs.47.1685168813052; Fri, 26 May 2023 23:26:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685168813; cv=none; d=google.com; s=arc-20160816; b=FaLub89GDmXNh8lfkCDvD/MF+cSOU1Rzb0cUUIctDe9Bb+xyQY1/W8yoJ6eocxGCfP /oH9j7zjByXsdg8FyUBCS2Kd5Hjhg0yRIvHPz/2zTYR5dSBRd/Q/lwPAmHYnRgOuX5Xj 4LcW9g2MsNqXyUd0+mOg4gca1IVMalRSeCy3xAVKfGcqNd8DJQdrNagkzjQtyuuMCdq5 j8wRdxxuRp5HOfgAKWgl1qqiOrtkJ84Rcv3cNLg6QCaHvWDEbVJK0ub/zXIyP6YlLCz2 50vhLzgvplVrl6jWZbvSv3BGLl58fkdDpUIOoxsCPFvTw46phZlu8eWwEanna0WUKHsT 8/YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=0GfSgxGbayJ7k7drxjfxTiw0vf/ejfsUPu/0wdED5PI=; b=FFAXsLgSaMif7eocKQUB4r0qYGT9cWqYiZaxbbknEAG4BJtzvYMTSZ4A6m3mJSQTft JCoLH7fJoL9I9PUc6oqbin5JcNnc2ggAg6jmtNl5+edVFyQlq/omlS5tRzsXTkKo11zA KE31tXcuFcOAVt3jhtlY6Gk244MgEcRg9yjDxCiDe6UZO9VFX9SR7Z6bXcRZu3fSqlHJ V212ttiCYrNZx/LhrlVo9PmKNfLAEruaUSHSDb0PfnidyfeziZdbDFuPt04qGgSLKAbx XrMtPXw6F/vaIHqOFyxIvBbr6VDVzqTzbXbUoxo7ozAhUYKz82IaEZMSbd+XifA3Hdn2 bn+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t8-20020a17090a4e4800b00246a75f49desi6056814pjl.112.2023.05.26.23.26.36; Fri, 26 May 2023 23:26:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231261AbjE0Fr5 (ORCPT + 99 others); Sat, 27 May 2023 01:47:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231216AbjE0Fr4 (ORCPT ); Sat, 27 May 2023 01:47:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15AB2114 for ; Fri, 26 May 2023 22:47:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9E0D560F27 for ; Sat, 27 May 2023 05:47:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 57772C433D2; Sat, 27 May 2023 05:47:51 +0000 (UTC) From: Huacai Chen To: Thomas Gleixner , Marc Zyngier , Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Xuefeng Li , Huacai Chen , Jiaxun Yang , Huacai Chen , Juxin Gao Subject: [PATCH 2/2] irqchip/loongson-pch-msi: Add machanism to limit msi allocation Date: Sat, 27 May 2023 13:46:33 +0800 Message-Id: <20230527054633.704916-3-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230527054633.704916-1-chenhuacai@loongson.cn> References: <20230527054633.704916-1-chenhuacai@loongson.cn> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767027572852917867?= X-GMAIL-MSGID: =?utf-8?q?1767027572852917867?= Loongson machines can have as many as 256 logical cpus, but the maximum of msi vectors in one irqchip is also 256 (practically that is less than 256, because pch-pic consumes some of them). Even on a 64-core machine, 256 irqs can be easily exhausted if there are several NICs (NICs usually allocate msi irqs depending on the number of online cpus). So we want to limit the msi allocation. In this patch we add a machanism to limit msi allocation: 1, Modify input "nvec" by overriding the msi_domain_ops::msi_prepare(); 2, The default limit is 256, which is compatible with the old behavior; 3, Add a cmdline parameter "loongson_msi_limit=xxx" to control the limit. Signed-off-by: Juxin Gao Signed-off-by: Huacai Chen --- drivers/irqchip/irq-loongson-pch-msi.c | 27 ++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/drivers/irqchip/irq-loongson-pch-msi.c b/drivers/irqchip/irq-loongson-pch-msi.c index 6e1e1f011bb2..85e2e3468b8c 100644 --- a/drivers/irqchip/irq-loongson-pch-msi.c +++ b/drivers/irqchip/irq-loongson-pch-msi.c @@ -85,9 +85,36 @@ static void pch_msi_compose_msi_msg(struct irq_data *data, msg->data = data->hwirq; } +#define DEFAULT_MSI_LIMITS 256 + +static int pch_msi_limits = DEFAULT_MSI_LIMITS; + +static int __init pch_msi_limit(char *str) +{ + get_option(&str, &pch_msi_limits); + + if (pch_msi_limits <= 0) + pch_msi_limits = DEFAULT_MSI_LIMITS; + + return 0; +} + +early_param("loongson_msi_limit", pch_msi_limit); + +static int pch_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg) +{ + memset(arg, 0, sizeof(*arg)); + return clamp_val(nvec, 0, pch_msi_limits); +} + +static struct msi_domain_ops pch_msi_ops = { + .msi_prepare = pch_msi_prepare, +}; + static struct msi_domain_info pch_msi_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + .ops = &pch_msi_ops, .chip = &pch_msi_irq_chip, };