From patchwork Tue Jan 9 10:51:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 186315 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp27634dyi; Tue, 9 Jan 2024 02:53:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IFVNnRcOEQrsnp/vO9m643NUBU6mtpx8BKYph/ZL7xoQW1kHsCs1nI+Yo3Na7LuK2lPULWQ X-Received: by 2002:a05:6902:2313:b0:db7:dacf:5a0f with SMTP id do19-20020a056902231300b00db7dacf5a0fmr2743899ybb.131.1704797618080; Tue, 09 Jan 2024 02:53:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704797618; cv=none; d=google.com; s=arc-20160816; b=tqkCYXpb/imbZlYyCpptW/a2a6Sls0eHjPxzad0byQZ178M8/uKxZSuxslx//JaCWF B4SFrgJHi8WAq8xJnySy197dSLzeX+Io6BfoCmMTlouWSjID9ht6A8DU8+l+OAMoSVn8 U/OWDAhik34MwhvejcyNP6vyAusfz3rbdX/KVLJwBcFIiDae5iAQMwrN34QuLk3UzAJN Mhanv0YsJ91cO8kU2c3NG7W1Jm4mYfib8qpO9tXxkDpmBlSpKY7kIw+75ty0cHYI8f/8 wVHavxKMQA5iJzeDWuAD6HnipBNObee6bV/hs3e+XPaTAdOXWzdk35oUzmyA6k0aI7m7 wDSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=Pqlx90j8yIcRTng9j5EAodGwIHIWZSW2GV/qQf/cfoM=; fh=tZ+d+kaZjTJMarg6XpXtL9/z8hMSaArJbuj7ZVP9x6U=; b=EEFwVi7KBP+Lhcz6sfzCEuyMJdiJwQZAn1bHxBQOZgEWpFrDl03kIfgy6dag2gHoVk Cp0vJfMcWxZ5deXqHogcf+iDKSiRP4na8/Fx7G3e7SUpvIkUY5nBWA1Z5jmI2pT4OMyG xjSqRdFf2gKJOe91bcOkTgoo9zTay8S+t7k9ERNADq9Gym/NRyhkQybyPA+GVLvZCHk6 vzhLeL0jNM9WvMgIiVMTbiy0bSBmR6vYy916/V2Dt/l5T2zuU/Wd4s+mJbng+otHQRvf 1WfG9efik5RfguH7Ads2NnOCngCqOR66ZOJrqRbhx+JMvRdO5bkiB+q1UAMijbzGS9Hn URkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linux.microsoft.com header.s=default header.b=XiZAh96x; spf=pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id bb3-20020a05622a1b0300b004254d515f8esi1791501qtb.330.2024.01.09.02.53.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 02:53:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linux.microsoft.com header.s=default header.b=XiZAh96x; spf=pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D7F561C23D3C for ; Tue, 9 Jan 2024 10:53:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC15F38FB9; Tue, 9 Jan 2024 10:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="XiZAh96x" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D63B5374FE; Tue, 9 Jan 2024 10:51:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1099) id 5BF6820B3CC4; Tue, 9 Jan 2024 02:51:34 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5BF6820B3CC4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1704797494; bh=8KNJQqGWQfH/h7F1Q5rQO5g9we6YWyykhE+RQJsMV2o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XiZAh96x0GrkVeksmPP/EDDkT/8ig5CE2RNqsmwIkJv+CUAn9TwCYK0Fk0oZ9yoIV /YfUIVAK7/pDKml/bNYgCXIGxgVdYzScP1ZZS4SZgWg1IldxLz2iZWHXG4+4XtDUbL dvCq1n0okooemTO+NW3v2Sq44ARD4sqXbKCno0ek= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com Subject: [PATCH 3/4 net-next] net: mana: add a function to spread IRQs per CPUs Date: Tue, 9 Jan 2024 02:51:17 -0800 Message-Id: <1704797478-32377-4-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> References: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787609867109450797 X-GMAIL-MSGID: 1787609867109450797 From: Yury Norov Souradeep investigated that the driver performs faster if IRQs are spread on CPUs with the following heuristics: 1. No more than one IRQ per CPU, if possible; 2. NUMA locality is the second priority; 3. Sibling dislocality is the last priority. Let's consider this topology: Node 0 1 Core 0 1 2 3 CPU 0 1 2 3 4 5 6 7 The most performant IRQ distribution based on the above topology and heuristics may look like this: IRQ Nodes Cores CPUs 0 1 0 0-1 1 1 1 2-3 2 1 0 0-1 3 1 1 2-3 4 2 2 4-5 5 2 3 6-7 6 2 2 4-5 7 2 3 6-7 The irq_setup() routine introduced in this patch leverages the for_each_numa_hop_mask() iterator and assigns IRQs to sibling groups as described above. According to [1], for NUMA-aware but sibling-ignorant IRQ distribution based on cpumask_local_spread() performance test results look like this: /ntttcp -r -m 16 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 08:05:20 INFO: 17 threads created 08:05:28 INFO: Network activity progressing... 08:06:28 INFO: Test run completed. 08:06:28 INFO: Test cycle finished. 08:06:28 INFO: ##### Totals: ##### 08:06:28 INFO: test duration :60.00 seconds 08:06:28 INFO: total bytes :630292053310 08:06:28 INFO: throughput :84.04Gbps 08:06:28 INFO: retrans segs :4 08:06:28 INFO: cpu cores :192 08:06:28 INFO: cpu speed :3799.725MHz 08:06:28 INFO: user :0.05% 08:06:28 INFO: system :1.60% 08:06:28 INFO: idle :96.41% 08:06:28 INFO: iowait :0.00% 08:06:28 INFO: softirq :1.94% 08:06:28 INFO: cycles/byte :2.50 08:06:28 INFO: cpu busy (all) :534.41% For NUMA- and sibling-aware IRQ distribution, the same test works 15% faster: /ntttcp -r -m 16 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 08:08:51 INFO: 17 threads created 08:08:56 INFO: Network activity progressing... 08:09:56 INFO: Test run completed. 08:09:56 INFO: Test cycle finished. 08:09:56 INFO: ##### Totals: ##### 08:09:56 INFO: test duration :60.00 seconds 08:09:56 INFO: total bytes :741966608384 08:09:56 INFO: throughput :98.93Gbps 08:09:56 INFO: retrans segs :6 08:09:56 INFO: cpu cores :192 08:09:56 INFO: cpu speed :3799.791MHz 08:09:56 INFO: user :0.06% 08:09:56 INFO: system :1.81% 08:09:56 INFO: idle :96.18% 08:09:56 INFO: iowait :0.00% 08:09:56 INFO: softirq :1.95% 08:09:56 INFO: cycles/byte :2.25 08:09:56 INFO: cpu busy (all) :569.22% [1] https://lore.kernel.org/all/20231211063726.GA4977@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/ Signed-off-by: Yury Norov Co-developed-by: Souradeep Chakrabarti --- .../net/ethernet/microsoft/mana/gdma_main.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 6367de0c2c2e..6a967d6be01e 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1243,6 +1243,35 @@ void mana_gd_free_res_map(struct gdma_resource *r) r->size = 0; } +static __maybe_unused int irq_setup(unsigned int *irqs, unsigned int len, int node) +{ + const struct cpumask *next, *prev = cpu_none_mask; + cpumask_var_t cpus __free(free_cpumask_var); + int cpu, weight; + + if (!alloc_cpumask_var(&cpus, GFP_KERNEL)) + return -ENOMEM; + + rcu_read_lock(); + for_each_numa_hop_mask(next, node) { + weight = cpumask_weight_andnot(next, prev); + while (weight > 0) { + cpumask_andnot(cpus, next, prev); + for_each_cpu(cpu, cpus) { + if (len-- == 0) + goto done; + irq_set_affinity_and_hint(*irqs++, topology_sibling_cpumask(cpu)); + cpumask_andnot(cpus, cpus, topology_sibling_cpumask(cpu)); + --weight; + } + } + prev = next; + } +done: + rcu_read_unlock(); + return 0; +} + static int mana_gd_setup_irqs(struct pci_dev *pdev) { unsigned int max_queues_per_port = num_online_cpus();