From patchwork Fri Dec 8 10:02:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 175717 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp5352697vqy; Fri, 8 Dec 2023 02:02:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IGfqJXN4GYQPc5g2SdVB9frpxahUsB4b1pz75+jXtlgULCZslFaoXRy5ttOMf3DFvC5XHIM X-Received: by 2002:a05:6a20:a217:b0:190:37a0:f343 with SMTP id u23-20020a056a20a21700b0019037a0f343mr666842pzk.34.1702029768239; Fri, 08 Dec 2023 02:02:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702029768; cv=none; d=google.com; s=arc-20160816; b=SVhA1Eu6xUL9lLsfA+OiRx61kmRgLFwf+PhTgPqaQc+GNFcAbE+U9/CfAyCZeTEC+u Wv+cXvP+85KzoI+SRy6zym7FspLHQRSZrhYxqJR89HVz0FiDo3yhKl5TnkFkUHHF9OSd OnRI3qIGoh22FgacJ5AnEjweGZ8rSgzWzWdoK3WF+c2ybvtM+3qyahG9wbFoUtWI8zbw VMtQ6+PAQzq3wShYXnvRP7KJ7/plxAPXL40eVxgY3WE0twnK9kL2PatYLEK4VT8OrAeP dVb/OgoehSoo5n4sdYiGxJVL+UaQEHfYOysHZjo2UtHujvOaAOgN1OeEheFy0jqyxnzp bJHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=bC0rZPUj3vdJkI+4wJKSoDBmkdXIOKDY9tHt0bJ5YcM=; fh=i0uHiUPKHfc1dSxTUTgp1cL8PKSGIUKO6qVCzrDTDzE=; b=hEdKYAD3slC6Kg1q/mC5ltIW1z9yUlM0AMavUseJUjEYZpM97PwGt2kbRbmoJB+GCj zJdND9gaC3EFC1rQ0EkiZJ3sPqc0H7NPRkUX5U9ryoojfNEH2h55WP9P7DpJJCl097Pr MYjSHOXpjb7CHfihYfjEvdXJHjCIt3wkojol+JFHtpycvAZg3M8QL57lk7RVtPTEa+aH Vi2Xhr8e8mmbW94ujLdAZdX8WSneDwIDGx7zrSSAg7xi4N7k0lC0NJWQ1MrPjrYDTT8S 6z6YzQ40C/NEgtnPGg88dQV0bkybgVHgYuOYQiC3sL6QpWaDoqXizhhCYlYciCToNC0C Tvlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=b76OthSm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id b2-20020a63eb42000000b005c278e32054si1253424pgk.677.2023.12.08.02.02.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Dec 2023 02:02:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=b76OthSm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 4016980F6947; Fri, 8 Dec 2023 02:02:44 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233311AbjLHKCf (ORCPT + 99 others); Fri, 8 Dec 2023 05:02:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230506AbjLHKCe (ORCPT ); Fri, 8 Dec 2023 05:02:34 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0F7E5D5B; Fri, 8 Dec 2023 02:02:40 -0800 (PST) Received: by linux.microsoft.com (Postfix, from userid 1099) id E473F20B74C0; Fri, 8 Dec 2023 02:02:38 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E473F20B74C0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1702029758; bh=bC0rZPUj3vdJkI+4wJKSoDBmkdXIOKDY9tHt0bJ5YcM=; h=From:To:Cc:Subject:Date:From; b=b76OthSm/w/9rCAyVWtY2zPWMN2U/BUue3Vv1sIOluxb3aK0za+esDZ+NuwOYvc1E 5deAKd+RE7PEQ5EH09pP4IyHiT+UhGmsiqD1BJYjjve7c2OdUTj0bsvlosi88i5MV8 55z8pbrbuCRq7XxNRNiNFoGgJpIQPwcdbimD/VWI= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com, Souradeep Chakrabarti Subject: [PATCH V5 net-next] net: mana: Assigning IRQ affinity on HT cores Date: Fri, 8 Dec 2023 02:02:34 -0800 Message-Id: <1702029754-6520-1-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 08 Dec 2023 02:02:44 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784340666827173894 X-GMAIL-MSGID: 1784707566430629103 Existing MANA design assigns IRQ to every CPU, including sibling hyper-threads. This may cause multiple IRQs to be active simultaneously in the same core and may reduce the network performance with RSS. Improve the performance by assigning IRQ to non sibling CPUs in local NUMA node. The performance improvement we are getting using ntttcp with following patch is around 15 percent with existing design and approximately 11 percent, when trying to assign one IRQ in each core across NUMA nodes, if enough cores are present. Suggested-by: Yury Norov Signed-off-by: Souradeep Chakrabarti --- V4 -> V5: * Fixed use of for_each_numa_hop_mask in irq_setup, by using visited_cpus to mask previous node mask. * Changed the way IRQ0 is gettng assigned, to assign it a separate CPU if possible rather than assigning it to a already assigned CPU. * Added Suggested-by tag. V3 -> V4: * Used for_each_numa_hop_mask() macro and simplified the code. Thanks to Yury Norov for the suggestion. * Added code to assign hwc irq separately in mana_gd_setup_irqs. V2 -> V3: * Created a helper function to get the next NUMA with CPU. * Added some error checks for unsuccessful memory allocation. * Fixed some comments on the code. V1 -> V2: * Simplified the code by removing filter_mask_list and using avail_cpus. * Addressed infinite loop issue when there are numa nodes with no CPUs. * Addressed uses of local numa node instead of 0 to start. * Removed uses of BUG_ON. * Placed cpus_read_lock in parent function to avoid num_online_cpus to get changed before function finishes the affinity assignment. --- .../net/ethernet/microsoft/mana/gdma_main.c | 92 +++++++++++++++++-- 1 file changed, 83 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 6367de0c2c2e..18e8908c5d29 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1243,15 +1243,56 @@ void mana_gd_free_res_map(struct gdma_resource *r) r->size = 0; } +static int irq_setup(int *irqs, int nvec, int start_numa_node) +{ + int w, cnt, cpu, err = 0, i = 0; + int next_node = start_numa_node; + const struct cpumask *next, *prev = cpu_none_mask; + cpumask_var_t curr, cpus; + + if (!zalloc_cpumask_var(&curr, GFP_KERNEL)) { + err = -ENOMEM; + return err; + } + if (!zalloc_cpumask_var(&cpus, GFP_KERNEL)) { + err = -ENOMEM; + return err; + } + + rcu_read_lock(); + for_each_numa_hop_mask(next, next_node) { + cpumask_andnot(curr, next, prev); + for (w = cpumask_weight(curr), cnt = 0; cnt < w; ) { + cpumask_copy(cpus, curr); + for_each_cpu(cpu, cpus) { + irq_set_affinity_and_hint(irqs[i], topology_sibling_cpumask(cpu)); + if (++i == nvec) + goto done; + cpumask_andnot(cpus, cpus, topology_sibling_cpumask(cpu)); + ++cnt; + } + } + prev = next; + } +done: + rcu_read_unlock(); + free_cpumask_var(curr); + free_cpumask_var(cpus); + return err; +} + static int mana_gd_setup_irqs(struct pci_dev *pdev) { - unsigned int max_queues_per_port = num_online_cpus(); struct gdma_context *gc = pci_get_drvdata(pdev); + unsigned int max_queues_per_port; struct gdma_irq_context *gic; unsigned int max_irqs, cpu; - int nvec, irq; + int start_irq_index = 1; + int nvec, *irqs, irq; int err, i = 0, j; + cpus_read_lock(); + max_queues_per_port = num_online_cpus(); if (max_queues_per_port > MANA_MAX_NUM_QUEUES) max_queues_per_port = MANA_MAX_NUM_QUEUES; @@ -1261,6 +1302,14 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) nvec = pci_alloc_irq_vectors(pdev, 2, max_irqs, PCI_IRQ_MSIX); if (nvec < 0) return nvec; + if (nvec <= num_online_cpus()) + start_irq_index = 0; + + irqs = kmalloc_array((nvec - start_irq_index), sizeof(int), GFP_KERNEL); + if (!irqs) { + err = -ENOMEM; + goto free_irq_vector; + } gc->irq_contexts = kcalloc(nvec, sizeof(struct gdma_irq_context), GFP_KERNEL); @@ -1287,21 +1336,44 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) goto free_irq; } - err = request_irq(irq, mana_gd_intr, 0, gic->name, gic); - if (err) - goto free_irq; - - cpu = cpumask_local_spread(i, gc->numa_node); - irq_set_affinity_and_hint(irq, cpumask_of(cpu)); + if (!i) { + err = request_irq(irq, mana_gd_intr, 0, gic->name, gic); + if (err) + goto free_irq; + + /* If number of IRQ is one extra than number of online CPUs, + * then we need to assign IRQ0 (hwc irq) and IRQ1 to + * same CPU. + * Else we will use different CPUs for IRQ0 and IRQ1. + * Also we are using cpumask_local_spread instead of + * cpumask_first for the node, because the node can be + * mem only. + */ + if (start_irq_index) { + cpu = cpumask_local_spread(i, gc->numa_node); + irq_set_affinity_and_hint(irq, cpumask_of(cpu)); + } else { + irqs[start_irq_index] = irq; + } + } else { + irqs[i - start_irq_index] = irq; + err = request_irq(irqs[i - start_irq_index], mana_gd_intr, 0, + gic->name, gic); + if (err) + goto free_irq; + } } + err = irq_setup(irqs, (nvec - start_irq_index), gc->numa_node); + if (err) + goto free_irq; err = mana_gd_alloc_res_map(nvec, &gc->msix_resource); if (err) goto free_irq; gc->max_num_msix = nvec; gc->num_msix_usable = nvec; - + cpus_read_unlock(); return 0; free_irq: @@ -1314,8 +1386,10 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) } kfree(gc->irq_contexts); + kfree(irqs); gc->irq_contexts = NULL; free_irq_vector: + cpus_read_unlock(); pci_free_irq_vectors(pdev); return err; }