From patchwork Fri Dec 8 02:52:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gang Li X-Patchwork-Id: 175537 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp5207746vqy; Thu, 7 Dec 2023 18:53:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IE5+MgUcrlypI0LKBMC5trRioFawUB+oH0cUbXSwyZrp28cvkemDsMXUmHJeY/7lSyGep8i X-Received: by 2002:a17:90a:9406:b0:286:6cc0:62a3 with SMTP id r6-20020a17090a940600b002866cc062a3mr296767pjo.34.1702004012528; Thu, 07 Dec 2023 18:53:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702004012; cv=none; d=google.com; s=arc-20160816; b=nkna1QQtg57QDz1iLQRoy2i9mv8a/8nN09kUoWxHE0RWwoj7vx30uFUDlkZBNtOI7V ObBZYhcD8LwQc400EEGkVYCczZO2LPJ3vexd21ps2ted87fLpstGiydUGkATtgKEdfi1 Nb/f6lISUzxHniNzdA8tvG2kD8EyJXGx05AjxJvfK/JOzoqWQoXZldwZdDza1hd2L+/i /UitidSVSnCMVUTBMsI6H3T/bGZHCEg2iqtLNh205faUaPCNpy7gcg+ByATHTLt6CUzY 9eHXIX/64AMSHURsyhiOwexiktFJ4DIKNYEejQYZFjmmPqDiQs3hdn5zKxEwApp8Vn8g WQ4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xBnw5RqwYO2IqY80uM+vPTSc8/TzM8Sh0FOmLemG2G4=; fh=ALVB2amNZc7wpwPN7EU8i4C3eL0OH515L0QoCIRQrQE=; b=UmYHsqnX/UFa0eQLzqOAv4HdsFkVGn9fQZwS+zWv60NCSyQSZjBUW/LfEzejHaPLTQ iiajGhok8AZqQmLAQ4feh2ewV0YEb53+33z+NzcuYu5tojAI6xR/BWopa3EQJdF+uYs/ y/bQ0UaCyasROpekbiuCj9ysUn67cWN/lbWN7UVzrMsTRGX6RXf1lA0JE7yql89srkru +h1ycCKNHl5eihHLS6igH5jDPWesK6evxecD7cV1wPuA06EwMNw7yjyg9FBwwiL0Iqn/ IzqJmKK3Cz50/bKaQ8o4nNTva+LbYbOg8S8G50TLBDqW6O1i7+tSVD2QVDjipoJh6S9e rEgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=esddMIrN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id br12-20020a17090b0f0c00b00286b69fb2d3si841685pjb.87.2023.12.07.18.53.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 18:53:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=esddMIrN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id E0CC98179079; Thu, 7 Dec 2023 18:53:28 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1573039AbjLHCxO (ORCPT + 99 others); Thu, 7 Dec 2023 21:53:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1573014AbjLHCxJ (ORCPT ); Thu, 7 Dec 2023 21:53:09 -0500 Received: from out-187.mta0.migadu.com (out-187.mta0.migadu.com [IPv6:2001:41d0:1004:224b::bb]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 298C010CA for ; Thu, 7 Dec 2023 18:53:15 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1702003993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xBnw5RqwYO2IqY80uM+vPTSc8/TzM8Sh0FOmLemG2G4=; b=esddMIrNubz11Tjk5hcSDYsiI76jCdFHcT7RuRX5vkHNO/tt71jh8T95AJcRJ0EkjT5FPP nyp6uRRHoc0E4ynoEJCj3cDTTNvgLfqjFHV2Cv3uc8gZ6cNzKUTvW7ji1pnYkE7HVckU18 dtXx/KCzGEPds0Qw2r7tjuJ+Dt6mSrc= From: Gang Li To: David Hildenbrand , David Rientjes , Mike Kravetz , Muchun Song , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, ligang.bdlg@bytedance.com, Gang Li Subject: [RFC PATCH v2 3/5] padata: dispatch works on different nodes Date: Fri, 8 Dec 2023 10:52:38 +0800 Message-Id: <20231208025240.4744-4-gang.li@linux.dev> In-Reply-To: <20231208025240.4744-1-gang.li@linux.dev> References: <20231208025240.4744-1-gang.li@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 07 Dec 2023 18:53:29 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784680559326301080 X-GMAIL-MSGID: 1784680559326301080 When a group of tasks that access different nodes are scheduled on the same node, they may encounter bandwidth bottlenecks and access latency. Thus, numa_aware flag is introduced here, allowing tasks to be distributed across different nodes to fully utilize the advantage of multi-node systems. Signed-off-by: Gang Li --- include/linux/padata.h | 2 ++ kernel/padata.c | 8 ++++++-- mm/mm_init.c | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 495b16b6b4d72..f6c58c30ed96a 100644 --- a/include/linux/padata.h +++ b/include/linux/padata.h @@ -137,6 +137,7 @@ struct padata_shell { * appropriate for one worker thread to do at once. * @max_threads: Max threads to use for the job, actual number may be less * depending on task size and minimum chunk size. + * @numa_aware: Dispatch jobs to different nodes. */ struct padata_mt_job { void (*thread_fn)(unsigned long start, unsigned long end, void *arg); @@ -146,6 +147,7 @@ struct padata_mt_job { unsigned long align; unsigned long min_chunk; int max_threads; + bool numa_aware; }; /** diff --git a/kernel/padata.c b/kernel/padata.c index 179fb1518070c..80f82c563e46a 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -485,7 +485,7 @@ void __init padata_do_multithreaded(struct padata_mt_job *job) struct padata_work my_work, *pw; struct padata_mt_job_state ps; LIST_HEAD(works); - int nworks; + int nworks, nid; if (job->size == 0) return; @@ -517,7 +517,11 @@ void __init padata_do_multithreaded(struct padata_mt_job *job) ps.chunk_size = roundup(ps.chunk_size, job->align); list_for_each_entry(pw, &works, pw_list) - queue_work(system_unbound_wq, &pw->pw_work); + if (job->numa_aware) + queue_work_node((++nid % num_node_state(N_MEMORY)), + system_unbound_wq, &pw->pw_work); + else + queue_work(system_unbound_wq, &pw->pw_work); /* Use the current thread, which saves starting a workqueue worker. */ padata_work_init(&my_work, padata_mt_helper, &ps, PADATA_WORK_ONSTACK); diff --git a/mm/mm_init.c b/mm/mm_init.c index 077bfe393b5e2..1226f0c81fcb3 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2234,6 +2234,7 @@ static int __init deferred_init_memmap(void *data) .align = PAGES_PER_SECTION, .min_chunk = PAGES_PER_SECTION, .max_threads = max_threads, + .numa_aware = false, }; padata_do_multithreaded(&job);