From patchwork Tue Jan 9 10:51:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 186312 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp27344dyi; Tue, 9 Jan 2024 02:52:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IGcPD/U62chTSGORJgSjNJS1NhnkSbdbKDAklAOfe2kC+0WJ/SPjFAGtD/sJiQh10n5ke75 X-Received: by 2002:a05:6902:120d:b0:dbd:5cd5:4d53 with SMTP id s13-20020a056902120d00b00dbd5cd54d53mr2682124ybu.96.1704797561710; Tue, 09 Jan 2024 02:52:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704797561; cv=none; d=google.com; s=arc-20160816; b=d+/cYsXXboRlJaicmh42JExpVrIYW8EHuy0S9qNMdvIvX/ApMeM92QwVyvn/exj6Ry rxTkB9R+zsidUzhIwcbNULtRewBQ34lR/JH/sK+P1xIwowJ+HyJCOHVPdbCZWrf3wFHc j1F0h7iomRaLCqT463IKdsZxY0T2B7MViWqXa4q28mNMbrCzmqTxPCS23zLV4cHBnyp7 CyPdCgMzRYo4h5TJeQCox3pSCmXMz7kos3unqXJwrnxvrgg6m7fxjZ8YHTjDMwBpMTma RL0OMoSlmiH+fMidEI9sXFGiyZxmj/6qnvH2ZSirYP3X6O7bB9lr+UUGXnkjmVzn5HHY nsdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=Qh4vCcMbs6V3VMTJ7hXH2GEKRppPqxUeLFTB/Ig1nok=; fh=tZ+d+kaZjTJMarg6XpXtL9/z8hMSaArJbuj7ZVP9x6U=; b=mhK8FJ5sZ+wMYQP/w4VR1ZDcPhgdxMAM8D3ceRCC+p6N7+S5psWnLnarjG7g3bwEJQ 9x9wBYtDwgpwB6g7gT4+uxXWcGyFr67QxHcblbHp84L+xPXSnHt1cFLdiElKEVr43+Ze Hh0xQKcWOM7ErSRfd166t/fuxJaK7VOMbbtuG/MVLoMkbAtXk5W7d54SxvA/0Of9lV1l 03Yb4oSjNFJCi0WxDn0I6FQ5uloEmbzZUGQOBpgl2ZRxQ9WzTJrhzCluW5vq+R6hBj4W /Ee3dYZ2a1vUoifVZX0GLMBxp0ARG+WJf6Av+vM5VeD0o7QMugASC/ZtW8xIl7v1Sfgv QMiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=FwE6pRuy; spf=pass (google.com: domain of linux-kernel+bounces-20727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20727-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id e16-20020ac84e50000000b00429aa3a8958si497652qtw.365.2024.01.09.02.52.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 02:52:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=FwE6pRuy; spf=pass (google.com: domain of linux-kernel+bounces-20727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20727-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7DB9D1C23E40 for ; Tue, 9 Jan 2024 10:52:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 86696381C0; Tue, 9 Jan 2024 10:51:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="FwE6pRuy" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8F0ED3715E; Tue, 9 Jan 2024 10:51:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1099) id 6373F20AEC8A; Tue, 9 Jan 2024 02:51:31 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 6373F20AEC8A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1704797491; bh=Qh4vCcMbs6V3VMTJ7hXH2GEKRppPqxUeLFTB/Ig1nok=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FwE6pRuyaI+HJkDB3Afk1M4hhFkZwSOQlWfPwZXinmuveNFMzrjnBbyvhUeNLDXdE N7NHLkaiU/olpqoZ2mqgwZMV5emDY4NEb3+rDHQ4ihr8HG7ZnsASjlOMz/CNEOLQU4 qRgCuvSHZq6rV52WdrNTpoXf4wp8WyW1BQ5LrM1s= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com Subject: [PATCH 1/4 net-next] cpumask: add cpumask_weight_andnot() Date: Tue, 9 Jan 2024 02:51:15 -0800 Message-Id: <1704797478-32377-2-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> References: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787609808105878005 X-GMAIL-MSGID: 1787609808105878005 From: Yury Norov Similarly to cpumask_weight_and(), cpumask_weight_andnot() is a handy helper that may help to avoid creating an intermediate mask just to calculate number of bits that set in a 1st given mask, and clear in 2nd one. Signed-off-by: Yury Norov Reviewed-by: Jacob Keller --- include/linux/bitmap.h | 12 ++++++++++++ include/linux/cpumask.h | 13 +++++++++++++ lib/bitmap.c | 7 +++++++ 3 files changed, 32 insertions(+) diff --git a/include/linux/bitmap.h b/include/linux/bitmap.h index 99451431e4d6..5814e9ee40ba 100644 --- a/include/linux/bitmap.h +++ b/include/linux/bitmap.h @@ -54,6 +54,7 @@ struct device; * bitmap_full(src, nbits) Are all bits set in *src? * bitmap_weight(src, nbits) Hamming Weight: number set bits * bitmap_weight_and(src1, src2, nbits) Hamming Weight of and'ed bitmap + * bitmap_weight_andnot(src1, src2, nbits) Hamming Weight of andnot'ed bitmap * bitmap_set(dst, pos, nbits) Set specified bit area * bitmap_clear(dst, pos, nbits) Clear specified bit area * bitmap_find_next_zero_area(buf, len, pos, n, mask) Find bit free area @@ -169,6 +170,8 @@ bool __bitmap_subset(const unsigned long *bitmap1, unsigned int __bitmap_weight(const unsigned long *bitmap, unsigned int nbits); unsigned int __bitmap_weight_and(const unsigned long *bitmap1, const unsigned long *bitmap2, unsigned int nbits); +unsigned int __bitmap_weight_andnot(const unsigned long *bitmap1, + const unsigned long *bitmap2, unsigned int nbits); void __bitmap_set(unsigned long *map, unsigned int start, int len); void __bitmap_clear(unsigned long *map, unsigned int start, int len); @@ -425,6 +428,15 @@ unsigned long bitmap_weight_and(const unsigned long *src1, return __bitmap_weight_and(src1, src2, nbits); } +static __always_inline +unsigned long bitmap_weight_andnot(const unsigned long *src1, + const unsigned long *src2, unsigned int nbits) +{ + if (small_const_nbits(nbits)) + return hweight_long(*src1 & ~(*src2) & BITMAP_LAST_WORD_MASK(nbits)); + return __bitmap_weight_andnot(src1, src2, nbits); +} + static __always_inline void bitmap_set(unsigned long *map, unsigned int start, unsigned int nbits) { diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index cfb545841a2c..228c23eb36d2 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -719,6 +719,19 @@ static inline unsigned int cpumask_weight_and(const struct cpumask *srcp1, return bitmap_weight_and(cpumask_bits(srcp1), cpumask_bits(srcp2), small_cpumask_bits); } +/** + * cpumask_weight_andnot - Count of bits in (*srcp1 & ~*srcp2) + * @srcp1: the cpumask to count bits (< nr_cpu_ids) in. + * @srcp2: the cpumask to count bits (< nr_cpu_ids) in. + * + * Return: count of bits set in both *srcp1 and *srcp2 + */ +static inline unsigned int cpumask_weight_andnot(const struct cpumask *srcp1, + const struct cpumask *srcp2) +{ + return bitmap_weight_andnot(cpumask_bits(srcp1), cpumask_bits(srcp2), small_cpumask_bits); +} + /** * cpumask_shift_right - *dstp = *srcp >> n * @dstp: the cpumask result diff --git a/lib/bitmap.c b/lib/bitmap.c index 09522af227f1..b97692854966 100644 --- a/lib/bitmap.c +++ b/lib/bitmap.c @@ -348,6 +348,13 @@ unsigned int __bitmap_weight_and(const unsigned long *bitmap1, } EXPORT_SYMBOL(__bitmap_weight_and); +unsigned int __bitmap_weight_andnot(const unsigned long *bitmap1, + const unsigned long *bitmap2, unsigned int bits) +{ + return BITMAP_WEIGHT(bitmap1[idx] & ~bitmap2[idx], bits); +} +EXPORT_SYMBOL(__bitmap_weight_andnot); + void __bitmap_set(unsigned long *map, unsigned int start, int len) { unsigned long *p = map + BIT_WORD(start); From patchwork Tue Jan 9 10:51:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 186314 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp27518dyi; Tue, 9 Jan 2024 02:53:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IExdYX5W7OsfsFpFZaKiymulIrRS6U8OkFygXeGyCkwhB4XwpCv8hh+bisjH9BnVJerCIH2 X-Received: by 2002:a17:90a:db95:b0:28d:62bd:b53c with SMTP id h21-20020a17090adb9500b0028d62bdb53cmr1535000pjv.95.1704797590393; Tue, 09 Jan 2024 02:53:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704797590; cv=none; d=google.com; s=arc-20160816; b=wGvOW27xJB053USph+b41kqLZdmR1FC6f8bzYVBlINtMe8BN/GzDqB7s6sHBEZ5sQP SnXe3Laf/Kr5WX/WihY0hppILJugnbkAh/c1sahI3gjLsMWGgK7XYV8O1pgCgXlNmRDj eDk7M74PX0s4kUFnZhxg/Tlnv5hBRZhjWD/OdEIRGcptVWst63HVAUoXfJ0+Utkh9owz 90Rx0HkAxdSzDqvXk8Ogd+FSvgvWh6dVTze7CBSK4tZD5IxGCEJGYPZloWof+4WgDeOB kcXRI43oncF3+sRaMPH3HmDILbTVs2AmvEjCsl1VvfVCsU6BlHtIDRAvMdk7Cdt4VM7q ZDuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=ZPRu3SXlIUr+8W26d0LPvybxMOuIzrvNRUu535P0XZM=; fh=tZ+d+kaZjTJMarg6XpXtL9/z8hMSaArJbuj7ZVP9x6U=; b=YIev025539TfcUuIom3MHtjbUJkcbWa3yqTDecbwQzJqGQsdKZJh1vYzk1Z+Nu03IM l80S3oDAmn9N3DNd6+IAZ22Uk8n1LqIdi2owZz9a0OhoQVE0vFDXbmiY0VP6Z6Ttt8tB fw0/bmSjLDXkFM4SgXnuNGykypx74C6+MFL5Tzb12WF0e55uy6IDIRG9pcsZMlzfPLcg DIKKnjqrkpzQNn9rOOJufv6SDIcQVCeW+InkHvk8NZzo3DjWY/ssYFo56ETgX6IMraXb RMglbipuq2iV1+tvS1ncpztS8Dtn/GNZ9uFId7151G6hV+Bv64TA0/UR2xpP5HBhNvmQ MzpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=ru1tJY2x; spf=pass (google.com: domain of linux-kernel+bounces-20729-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20729-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id z10-20020a17090ad78a00b0028cfa2f740asi6983838pju.12.2024.01.09.02.53.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 02:53:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20729-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=ru1tJY2x; spf=pass (google.com: domain of linux-kernel+bounces-20729-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20729-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3339B287C7D for ; Tue, 9 Jan 2024 10:53:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3AEB338DEC; Tue, 9 Jan 2024 10:51:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="ru1tJY2x" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 77B003717B; Tue, 9 Jan 2024 10:51:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1099) id 323D120B3CC1; Tue, 9 Jan 2024 02:51:33 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 323D120B3CC1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1704797493; bh=ZPRu3SXlIUr+8W26d0LPvybxMOuIzrvNRUu535P0XZM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ru1tJY2xhw1R3FF8i2Jn9+MXzu9/ixQze9DSxOEGohcTqRzL4KF1ln5codcIrQK99 de7jWi7E4cKKZHtXz5nfwUvUyCqvAcVI2xzgazGFzzUfDrlQsI4Ww4Z/HsSudPQaFx EbDQOymGQxIkglZjhfqVCW5hx667UmQ5ZCIZ/ns8= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com Subject: [PATCH 2/4 net-next] cpumask: define cleanup function for cpumasks Date: Tue, 9 Jan 2024 02:51:16 -0800 Message-Id: <1704797478-32377-3-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> References: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787609837807377706 X-GMAIL-MSGID: 1787609837807377706 From: Yury Norov Now we can simplify code that allocates cpumasks for local needs. Signed-off-by: Yury Norov --- include/linux/cpumask.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 228c23eb36d2..1c29947db848 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -7,6 +7,7 @@ * set of CPUs in a system, one bit position per CPU number. In general, * only nr_cpu_ids (<= NR_CPUS) bits are valid. */ +#include #include #include #include @@ -990,6 +991,8 @@ static inline bool cpumask_available(cpumask_var_t mask) } #endif /* CONFIG_CPUMASK_OFFSTACK */ +DEFINE_FREE(free_cpumask_var, struct cpumask *, if (_T) free_cpumask_var(_T)); + /* It's common to want to use cpu_all_mask in struct member initializers, * so it has to refer to an address rather than a pointer. */ extern const DECLARE_BITMAP(cpu_all_bits, NR_CPUS); From patchwork Tue Jan 9 10:51:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 186315 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp27634dyi; Tue, 9 Jan 2024 02:53:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IFVNnRcOEQrsnp/vO9m643NUBU6mtpx8BKYph/ZL7xoQW1kHsCs1nI+Yo3Na7LuK2lPULWQ X-Received: by 2002:a05:6902:2313:b0:db7:dacf:5a0f with SMTP id do19-20020a056902231300b00db7dacf5a0fmr2743899ybb.131.1704797618080; Tue, 09 Jan 2024 02:53:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704797618; cv=none; d=google.com; s=arc-20160816; b=tqkCYXpb/imbZlYyCpptW/a2a6Sls0eHjPxzad0byQZ178M8/uKxZSuxslx//JaCWF B4SFrgJHi8WAq8xJnySy197dSLzeX+Io6BfoCmMTlouWSjID9ht6A8DU8+l+OAMoSVn8 U/OWDAhik34MwhvejcyNP6vyAusfz3rbdX/KVLJwBcFIiDae5iAQMwrN34QuLk3UzAJN Mhanv0YsJ91cO8kU2c3NG7W1Jm4mYfib8qpO9tXxkDpmBlSpKY7kIw+75ty0cHYI8f/8 wVHavxKMQA5iJzeDWuAD6HnipBNObee6bV/hs3e+XPaTAdOXWzdk35oUzmyA6k0aI7m7 wDSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=Pqlx90j8yIcRTng9j5EAodGwIHIWZSW2GV/qQf/cfoM=; fh=tZ+d+kaZjTJMarg6XpXtL9/z8hMSaArJbuj7ZVP9x6U=; b=EEFwVi7KBP+Lhcz6sfzCEuyMJdiJwQZAn1bHxBQOZgEWpFrDl03kIfgy6dag2gHoVk Cp0vJfMcWxZ5deXqHogcf+iDKSiRP4na8/Fx7G3e7SUpvIkUY5nBWA1Z5jmI2pT4OMyG xjSqRdFf2gKJOe91bcOkTgoo9zTay8S+t7k9ERNADq9Gym/NRyhkQybyPA+GVLvZCHk6 vzhLeL0jNM9WvMgIiVMTbiy0bSBmR6vYy916/V2Dt/l5T2zuU/Wd4s+mJbng+otHQRvf 1WfG9efik5RfguH7Ads2NnOCngCqOR66ZOJrqRbhx+JMvRdO5bkiB+q1UAMijbzGS9Hn URkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linux.microsoft.com header.s=default header.b=XiZAh96x; spf=pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id bb3-20020a05622a1b0300b004254d515f8esi1791501qtb.330.2024.01.09.02.53.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 02:53:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linux.microsoft.com header.s=default header.b=XiZAh96x; spf=pass (google.com: domain of linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D7F561C23D3C for ; Tue, 9 Jan 2024 10:53:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC15F38FB9; Tue, 9 Jan 2024 10:51:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="XiZAh96x" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D63B5374FE; Tue, 9 Jan 2024 10:51:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1099) id 5BF6820B3CC4; Tue, 9 Jan 2024 02:51:34 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5BF6820B3CC4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1704797494; bh=8KNJQqGWQfH/h7F1Q5rQO5g9we6YWyykhE+RQJsMV2o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XiZAh96x0GrkVeksmPP/EDDkT/8ig5CE2RNqsmwIkJv+CUAn9TwCYK0Fk0oZ9yoIV /YfUIVAK7/pDKml/bNYgCXIGxgVdYzScP1ZZS4SZgWg1IldxLz2iZWHXG4+4XtDUbL dvCq1n0okooemTO+NW3v2Sq44ARD4sqXbKCno0ek= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com Subject: [PATCH 3/4 net-next] net: mana: add a function to spread IRQs per CPUs Date: Tue, 9 Jan 2024 02:51:17 -0800 Message-Id: <1704797478-32377-4-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> References: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787609867109450797 X-GMAIL-MSGID: 1787609867109450797 From: Yury Norov Souradeep investigated that the driver performs faster if IRQs are spread on CPUs with the following heuristics: 1. No more than one IRQ per CPU, if possible; 2. NUMA locality is the second priority; 3. Sibling dislocality is the last priority. Let's consider this topology: Node 0 1 Core 0 1 2 3 CPU 0 1 2 3 4 5 6 7 The most performant IRQ distribution based on the above topology and heuristics may look like this: IRQ Nodes Cores CPUs 0 1 0 0-1 1 1 1 2-3 2 1 0 0-1 3 1 1 2-3 4 2 2 4-5 5 2 3 6-7 6 2 2 4-5 7 2 3 6-7 The irq_setup() routine introduced in this patch leverages the for_each_numa_hop_mask() iterator and assigns IRQs to sibling groups as described above. According to [1], for NUMA-aware but sibling-ignorant IRQ distribution based on cpumask_local_spread() performance test results look like this: /ntttcp -r -m 16 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 08:05:20 INFO: 17 threads created 08:05:28 INFO: Network activity progressing... 08:06:28 INFO: Test run completed. 08:06:28 INFO: Test cycle finished. 08:06:28 INFO: ##### Totals: ##### 08:06:28 INFO: test duration :60.00 seconds 08:06:28 INFO: total bytes :630292053310 08:06:28 INFO: throughput :84.04Gbps 08:06:28 INFO: retrans segs :4 08:06:28 INFO: cpu cores :192 08:06:28 INFO: cpu speed :3799.725MHz 08:06:28 INFO: user :0.05% 08:06:28 INFO: system :1.60% 08:06:28 INFO: idle :96.41% 08:06:28 INFO: iowait :0.00% 08:06:28 INFO: softirq :1.94% 08:06:28 INFO: cycles/byte :2.50 08:06:28 INFO: cpu busy (all) :534.41% For NUMA- and sibling-aware IRQ distribution, the same test works 15% faster: /ntttcp -r -m 16 NTTTCP for Linux 1.4.0 --------------------------------------------------------- 08:08:51 INFO: 17 threads created 08:08:56 INFO: Network activity progressing... 08:09:56 INFO: Test run completed. 08:09:56 INFO: Test cycle finished. 08:09:56 INFO: ##### Totals: ##### 08:09:56 INFO: test duration :60.00 seconds 08:09:56 INFO: total bytes :741966608384 08:09:56 INFO: throughput :98.93Gbps 08:09:56 INFO: retrans segs :6 08:09:56 INFO: cpu cores :192 08:09:56 INFO: cpu speed :3799.791MHz 08:09:56 INFO: user :0.06% 08:09:56 INFO: system :1.81% 08:09:56 INFO: idle :96.18% 08:09:56 INFO: iowait :0.00% 08:09:56 INFO: softirq :1.95% 08:09:56 INFO: cycles/byte :2.25 08:09:56 INFO: cpu busy (all) :569.22% [1] https://lore.kernel.org/all/20231211063726.GA4977@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/ Signed-off-by: Yury Norov Co-developed-by: Souradeep Chakrabarti --- .../net/ethernet/microsoft/mana/gdma_main.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 6367de0c2c2e..6a967d6be01e 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1243,6 +1243,35 @@ void mana_gd_free_res_map(struct gdma_resource *r) r->size = 0; } +static __maybe_unused int irq_setup(unsigned int *irqs, unsigned int len, int node) +{ + const struct cpumask *next, *prev = cpu_none_mask; + cpumask_var_t cpus __free(free_cpumask_var); + int cpu, weight; + + if (!alloc_cpumask_var(&cpus, GFP_KERNEL)) + return -ENOMEM; + + rcu_read_lock(); + for_each_numa_hop_mask(next, node) { + weight = cpumask_weight_andnot(next, prev); + while (weight > 0) { + cpumask_andnot(cpus, next, prev); + for_each_cpu(cpu, cpus) { + if (len-- == 0) + goto done; + irq_set_affinity_and_hint(*irqs++, topology_sibling_cpumask(cpu)); + cpumask_andnot(cpus, cpus, topology_sibling_cpumask(cpu)); + --weight; + } + } + prev = next; + } +done: + rcu_read_unlock(); + return 0; +} + static int mana_gd_setup_irqs(struct pci_dev *pdev) { unsigned int max_queues_per_port = num_online_cpus(); From patchwork Tue Jan 9 10:51:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 186316 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp27694dyi; Tue, 9 Jan 2024 02:53:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IE1XfJF2ukU0WrtKoH5L7UVG57pW1X2rM9zEFXx4AT056vUcb/jtJCMzIi+nVXE5AcbjeCW X-Received: by 2002:a7b:c5c5:0:b0:40d:3be5:c8b7 with SMTP id n5-20020a7bc5c5000000b0040d3be5c8b7mr1437158wmk.117.1704797626674; Tue, 09 Jan 2024 02:53:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704797626; cv=none; d=google.com; s=arc-20160816; b=bAaujR0IV0pQv833Bob5IjSi3HzfUpkCdIIsZY0kE64FcQepoF5iK2Ie21aaVftZjI hRdUDhrLD5qjUpJfe0o2xK8wSXpjCrrgqN/RO8HpFFKIbu3UNg4s0Z17yOco3shose8t DZbhu1fZghkqrVzFy3sMYYDzsHZ6CNr0xJF4Ekg5gx8BsM+nRtzKSQFDKQnzgnTuOyhi bozwHfETb4pjelY9q0BkU/Mb7mbuVpsvK2qNh30Xorlxcd0ZwDRcJhT8x+QXKZ1HeKv5 B0hjZkzlkY/ETyWMY5APIle6O3Cb1M+E34wgZtQ3KwIAB+2EVyQwiXmKYHbSilnKgzDu IYXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=BjA1llB0tV0DNKtNQwDzCat1URmop7G8PpQu5Aagoj4=; fh=i0uHiUPKHfc1dSxTUTgp1cL8PKSGIUKO6qVCzrDTDzE=; b=nGitTCa260KNE/32Myv+5nR6MD0ZMf2rNwvtBGmAAOMWBMf8JKFqNCgQIQznxYBbPF 2BUsSJ7Dwud5YMvw2HeKH9f7fzXlcimUOUDhz8zydmxTjqy+xU0ENisPFsJZx9prv216 0ReqIIC9SLCM1rqxzOcAHwkRhF4AfbVA53AGrZXldElaG3I81v8V5E2HIBzVwVjbriOv 4GjaPCr+1A/wC12hsXjrnp9xSYrBhUCjJ3nZao57vFlMDIKaSI1EnQtKTZipa92MbVVl vaggkKjI5BkQMjkW0saKAgwEahqO7HlBlsbo/deClzSRbSO1UbVaR3ZYxLxoAmOwIVKV affg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=YNAZu+v3; spf=pass (google.com: domain of linux-kernel+bounces-20731-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20731-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u22-20020a170906069600b00a2a2ad85e13si649554ejb.926.2024.01.09.02.53.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 02:53:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20731-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=YNAZu+v3; spf=pass (google.com: domain of linux-kernel+bounces-20731-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20731-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 470911F250BF for ; Tue, 9 Jan 2024 10:53:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0AF0539852; Tue, 9 Jan 2024 10:51:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="YNAZu+v3" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 25AE1381D6; Tue, 9 Jan 2024 10:51:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1099) id F252120B3CC2; Tue, 9 Jan 2024 02:51:35 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com F252120B3CC2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1704797496; bh=BjA1llB0tV0DNKtNQwDzCat1URmop7G8PpQu5Aagoj4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YNAZu+v3p4jhxsCl+dNOLV5ZCD7oKcv0KmwZforvUJgtpnAVrwmY5zXCykrDI7SM0 5qPsJwpcp8fMmt2KGvayIzMFDV2Je6kTsr9GFsBfsvnoLfUusVmBcZFAMofUQCNBcX JnjYnZxDIfX/8zx/JP/KF/Zn1HLvqXAK+xcEjDPQ= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, yury.norov@gmail.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, paulros@microsoft.com, Souradeep Chakrabarti Subject: [PATCH 4/4 net-next] net: mana: Assigning IRQ affinity on HT cores Date: Tue, 9 Jan 2024 02:51:18 -0800 Message-Id: <1704797478-32377-5-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> References: <1704797478-32377-1-git-send-email-schakrabarti@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787609764438613194 X-GMAIL-MSGID: 1787609876402517336 Existing MANA design assigns IRQ to every CPU, including sibling hyper-threads. This may cause multiple IRQs to be active simultaneously in the same core and may reduce the network performance. Improve the performance by assigning IRQ to non sibling CPUs in local NUMA node. The performance improvement we are getting using ntttcp with following patch is around 15 percent against existing design and approximately 11 percent, when trying to assign one IRQ in each core across NUMA nodes, if enough cores are present. Signed-off-by: Souradeep Chakrabarti --- .../net/ethernet/microsoft/mana/gdma_main.c | 58 +++++++++++++++---- 1 file changed, 48 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c index 6a967d6be01e..6715d6939bc7 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1274,13 +1274,16 @@ static __maybe_unused int irq_setup(unsigned int *irqs, unsigned int len, int no static int mana_gd_setup_irqs(struct pci_dev *pdev) { - unsigned int max_queues_per_port = num_online_cpus(); struct gdma_context *gc = pci_get_drvdata(pdev); + unsigned int max_queues_per_port; struct gdma_irq_context *gic; unsigned int max_irqs, cpu; - int nvec, irq; + int start_irq_index = 1; + int nvec, *irqs, irq; int err, i = 0, j; + cpus_read_lock(); + max_queues_per_port = num_online_cpus(); if (max_queues_per_port > MANA_MAX_NUM_QUEUES) max_queues_per_port = MANA_MAX_NUM_QUEUES; @@ -1288,8 +1291,18 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) max_irqs = max_queues_per_port + 1; nvec = pci_alloc_irq_vectors(pdev, 2, max_irqs, PCI_IRQ_MSIX); - if (nvec < 0) + if (nvec < 0) { + cpus_read_unlock(); return nvec; + } + if (nvec <= num_online_cpus()) + start_irq_index = 0; + + irqs = kmalloc_array((nvec - start_irq_index), sizeof(int), GFP_KERNEL); + if (!irqs) { + err = -ENOMEM; + goto free_irq_vector; + } gc->irq_contexts = kcalloc(nvec, sizeof(struct gdma_irq_context), GFP_KERNEL); @@ -1316,21 +1329,44 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) goto free_irq; } - err = request_irq(irq, mana_gd_intr, 0, gic->name, gic); - if (err) - goto free_irq; - - cpu = cpumask_local_spread(i, gc->numa_node); - irq_set_affinity_and_hint(irq, cpumask_of(cpu)); + if (!i) { + err = request_irq(irq, mana_gd_intr, 0, gic->name, gic); + if (err) + goto free_irq; + + /* If number of IRQ is one extra than number of online CPUs, + * then we need to assign IRQ0 (hwc irq) and IRQ1 to + * same CPU. + * Else we will use different CPUs for IRQ0 and IRQ1. + * Also we are using cpumask_local_spread instead of + * cpumask_first for the node, because the node can be + * mem only. + */ + if (start_irq_index) { + cpu = cpumask_local_spread(i, gc->numa_node); + irq_set_affinity_and_hint(irq, cpumask_of(cpu)); + } else { + irqs[start_irq_index] = irq; + } + } else { + irqs[i - start_irq_index] = irq; + err = request_irq(irqs[i - start_irq_index], mana_gd_intr, 0, + gic->name, gic); + if (err) + goto free_irq; + } } + err = irq_setup(irqs, (nvec - start_irq_index), gc->numa_node); + if (err) + goto free_irq; err = mana_gd_alloc_res_map(nvec, &gc->msix_resource); if (err) goto free_irq; gc->max_num_msix = nvec; gc->num_msix_usable = nvec; - + cpus_read_unlock(); return 0; free_irq: @@ -1343,8 +1379,10 @@ static int mana_gd_setup_irqs(struct pci_dev *pdev) } kfree(gc->irq_contexts); + kfree(irqs); gc->irq_contexts = NULL; free_irq_vector: + cpus_read_unlock(); pci_free_irq_vectors(pdev); return err; }