From patchwork Thu Nov 2 03:23:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 160771 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp101764vqu; Wed, 1 Nov 2023 20:26:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH3lio75Z8ToVOqOMDT9Oc/WhXT0K0F+eQ9rmIWiR8PgfxjzZm8kywOWMX04agdL+I+aWNy X-Received: by 2002:a05:690c:dc6:b0:5ad:4975:c860 with SMTP id db6-20020a05690c0dc600b005ad4975c860mr22328656ywb.39.1698895575542; Wed, 01 Nov 2023 20:26:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698895575; cv=none; d=google.com; s=arc-20160816; b=IhNee8tBlfKRz28SS8Z4juvlY3X/14RyWxUM8LLd4j2wVqH/XeLTa5+oeEFxjGEKGr pdQD0qgyx464l5m9EY1lZmdyi2XdCML164vK+E7JbQjx6wLjrt8xxvSmG3rvDEoZihaC d5pZnj7VX8gXxkWU5lLbD4d5eNy3oFhUAPgRH+AphaePjTTa4ZFLgW1cKkGTkJUNQnzA ZwfTrZW8gA2TfZAgcAuwG3o/WItYRyygZIA2D2LYU8tDVL4eechWcKZi95hBtaPDnerw RzN7Gy8cfM6ZgU0BIhawQLyDRJ/OyKtFfaGspsws0h8l27hrYCuAFeeUkdqE9uF1mMIe jWoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=f6Me2hDPNAg0QwflvoQbBjNc8QPP2obW8dtDV2OvsdM=; fh=Dd5ryTbz4gXtDU9jzXqg2hD87XusQgPqunF08GDmhnA=; b=zSxdyBQvKXNXMpy3JnSI/qMfWhqTqt/3FDEhUFuKOuAOQRAGBrv6PucFd9yO8bk5pv qp8iYkv3WOP/UPUYNlZI3xijuVd/B5YAF55JcY4+AAKM8WkZCQFURwiGe7mcm1fZL886 WCfe29PbsCRUjUkBnMkCpkhD3Mb6zUQmnweGcNS6AzXeb1Ppb2exv58unPBCYYDJEgWj 5Sg3Y0CYBat2HaMSyQS0u1gIRA1YVxjYoA/A+5+70jU6pLyWOppIMQTWh6PDI0NWebvu UgLxNTXfNQMK9bvWUUYuFqPGvQOIK8zvvS0Pr0LuUSia90WV18+LtjqsP31D3ihWEW/T Nfxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=BPwts8cC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id h1-20020a636c01000000b005b9a1494637si1096662pgc.317.2023.11.01.20.26.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 20:26:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=BPwts8cC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 5E34B8062DDB; Wed, 1 Nov 2023 20:25:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348396AbjKBDZI (ORCPT + 35 others); Wed, 1 Nov 2023 23:25:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348366AbjKBDZA (ORCPT ); Wed, 1 Nov 2023 23:25:00 -0400 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [IPv6:2001:41d0:203:375::ad]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EF8A128 for ; Wed, 1 Nov 2023 20:24:54 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1698895493; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f6Me2hDPNAg0QwflvoQbBjNc8QPP2obW8dtDV2OvsdM=; b=BPwts8cCYtipUTdyFWricBp8OGyxnWzOEVZlgGagJ9J46S70yP6mdX0Zikc3F0F2DBJvXT /yMeu2ntPNaXcbiqN3TobVLGvOh9MDaYMJyNpEP8dSexDMJlp27RoUaeceT+iumgN10Z/7 Sn3Ry4EaMJKSfkAaTC3ksH7jqeLcYEc= From: chengming.zhou@linux.dev To: vbabka@suse.cz, cl@linux.com, penberg@kernel.org Cc: rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v5 4/9] slub: Prepare __slab_free() for unfrozen partial slab out of node partial list Date: Thu, 2 Nov 2023 03:23:25 +0000 Message-Id: <20231102032330.1036151-5-chengming.zhou@linux.dev> In-Reply-To: <20231102032330.1036151-1-chengming.zhou@linux.dev> References: <20231102032330.1036151-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Wed, 01 Nov 2023 20:25:40 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781421126918864295 X-GMAIL-MSGID: 1781421126918864295 From: Chengming Zhou Now the partially empty slub will be frozen when taken out of node partial list, so the __slab_free() will know from "was_frozen" that the partially empty slab is not on node partial list and is a cpu or cpu partial slab of some cpu. But we will change this, make partial slabs leave the node partial list with unfrozen state, so we need to change __slab_free() to use the new slab_test_node_partial() we just introduced. Signed-off-by: Chengming Zhou Reviewed-by: Vlastimil Babka Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/slub.c b/mm/slub.c index eed8ae0dbaf9..1880b483350e 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3631,6 +3631,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, unsigned long counters; struct kmem_cache_node *n = NULL; unsigned long flags; + bool on_node_partial; stat(s, FREE_SLOWPATH); @@ -3678,6 +3679,7 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, */ spin_lock_irqsave(&n->list_lock, flags); + on_node_partial = slab_test_node_partial(slab); } } @@ -3706,6 +3708,15 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, return; } + /* + * This slab was partially empty but not on the per-node partial list, + * in which case we shouldn't manipulate its list, just return. + */ + if (prior && !on_node_partial) { + spin_unlock_irqrestore(&n->list_lock, flags); + return; + } + if (unlikely(!new.inuse && n->nr_partial >= s->min_partial)) goto slab_empty;