Message ID | 20221024081435.204970-9-bhe@redhat.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp318600wru; Mon, 24 Oct 2022 01:23:01 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7KlBhTN0TMKj1ByQmom3EehIDeru+fFwBJcXOltn7n1zxqUEyy8l0sXqoAsmeba+CksYsE X-Received: by 2002:a17:907:6e18:b0:78d:b374:8989 with SMTP id sd24-20020a1709076e1800b0078db3748989mr26429020ejc.552.1666599781492; Mon, 24 Oct 2022 01:23:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666599781; cv=none; d=google.com; s=arc-20160816; b=eIJNNgt8Azv8VcjX7BP2MCTar01cX7Oyb8SI66VABqj3KXz1hYMe2fsIwv/OJZNy6R r4oYPBE1uIjr0Zn5Hw6vlMyu2lvtm9Kt1+wea++2ABGNLoL/lWtpEWdMUAWFGwv9xIQF of+00NBrE1fUEmYpB2AAx7Kml+3/MTzW3DCfdBtt3ACOZPxR1x7Q/ZV+w2FhaYOZVub8 UyILM66svXpuP0eAlbRJE8e4HEt8aE9MK5HUJyn99jmg6gy4AXs8LruxK9o3yaEuMThw HDBWkwhRxz0hlI0gXLxzN/NmBYyPz3roycBWOys9xBlofTv7MMKITwOaIcrNo7T6xzoi K87A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oB5FHCqdQzRcpJOgYsd1zcN5d+GwusZuRHj9Ye7P85Q=; b=h2K91iQtVbNP3goiRryf2RtbnEsfR5cXvIS538vha2Jmj6t2wCRjPLba8VX3g4jW/k aLxfp2e/SahYDTm/Fsg2ZBxw5UxD3OPOucpYMYlksF4Jj82Y850lj6YXNtS6CHoK+Knq gDeV5eS6v4hrK2nuuyijp0D9BJuKW7aPQa9z8NxJZz8HR27nDtQie7Njkp2XTKJFIG2o wtvAWa4gs0PfcXUimgMjKKNB2oJIeUrsU15OR+zIPrPJoIvONMndTRgn9INgJVp/Sx9W vD8Xu91ZeT10ef+sCub6tx5f0r1GePkjeUZrSBNGyklKfg28UU88q9Cd1zRtq5BRFetu mrKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="MVVPlPV/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c19-20020aa7d613000000b00461c8f80e72si1675772edr.629.2022.10.24.01.22.37; Mon, 24 Oct 2022 01:23:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="MVVPlPV/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230016AbiJXISr (ORCPT <rfc822;pwkd43@gmail.com> + 99 others); Mon, 24 Oct 2022 04:18:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229891AbiJXISM (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 24 Oct 2022 04:18:12 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A516E65008 for <linux-kernel@vger.kernel.org>; Mon, 24 Oct 2022 01:16:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666599330; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oB5FHCqdQzRcpJOgYsd1zcN5d+GwusZuRHj9Ye7P85Q=; b=MVVPlPV/zqchGBRP0L9jJ4IXX1nNCVDaaVg/AR6d6GrM6k8ReNJrJzgu59nRLD4umPcNJg Sdvk4qEZNx2AdpLs5tmLwSiaIbzCScmh/rz+B9XVM5E3adN44QTZfqoZlDkcNm6ZKgug86 L5kBwgxIK7jeeB1ZdKOPMYcpWaEpSVY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-md_8dzoMOzCwdktsUExyUg-1; Mon, 24 Oct 2022 04:15:20 -0400 X-MC-Unique: md_8dzoMOzCwdktsUExyUg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 70B7E887400; Mon, 24 Oct 2022 08:15:19 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (ovpn-12-35.pek2.redhat.com [10.72.12.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D63C40C6F9F; Mon, 24 Oct 2022 08:15:12 +0000 (UTC) From: Baoquan He <bhe@redhat.com> To: linux-kernel@vger.kernel.org Cc: Dennis Zhou <dennis@kernel.org>, Tejun Heo <tj@kernel.org>, Christoph Lameter <cl@linux.com>, Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org, Baoquan He <bhe@redhat.com>, Pekka Enberg <penberg@kernel.org>, David Rientjes <rientjes@google.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>, Vlastimil Babka <vbabka@suse.cz>, Roman Gushchin <roman.gushchin@linux.dev>, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH 8/8] mm/slub, percpu: correct the calculation of early percpu allocation size Date: Mon, 24 Oct 2022 16:14:35 +0800 Message-Id: <20221024081435.204970-9-bhe@redhat.com> In-Reply-To: <20221024081435.204970-1-bhe@redhat.com> References: <20221024081435.204970-1-bhe@redhat.com> MIME-Version: 1.0 Content-type: text/plain Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747556532866231562?= X-GMAIL-MSGID: =?utf-8?q?1747556532866231562?= |
Series |
Cleanup and optimization patches for percpu
|
|
Commit Message
Baoquan He
Oct. 24, 2022, 8:14 a.m. UTC
SLUB allocator relies on percpu allocator to initialize its ->cpu_slab
during early boot. For that, the dynamic chunk of percpu which serves
the early allocation need be large enough to satisfy the kmalloc
creation.
However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't
consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that
with correct calculation.
Signed-off-by: Baoquan He <bhe@redhat.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
---
mm/slub.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Comments
On Mon, Oct 24, 2022 at 04:14:35PM +0800, Baoquan He wrote: > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > during early boot. For that, the dynamic chunk of percpu which serves > the early allocation need be large enough to satisfy the kmalloc > creation. > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > with correct calculation. > > Signed-off-by: Baoquan He <bhe@redhat.com> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > --- > mm/slub.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/mm/slub.c b/mm/slub.c > index 157527d7101b..8ac3bb9a122a 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -4017,7 +4017,8 @@ init_kmem_cache_node(struct kmem_cache_node *n) > static inline int alloc_kmem_cache_cpus(struct kmem_cache *s) > { > BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE < > - KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)); > + NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * > + sizeof(struct kmem_cache_cpu)); > > /* > * Must align to double word boundary for the double cmpxchg Looks good to me. Acked-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Thanks! > -- > 2.34.1 >
On Mon, Oct 24, 2022 at 04:14:35PM +0800, Baoquan He wrote: > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > during early boot. For that, the dynamic chunk of percpu which serves > the early allocation need be large enough to satisfy the kmalloc > creation. > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > with correct calculation. > > Signed-off-by: Baoquan He <bhe@redhat.com> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > --- > mm/slub.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/mm/slub.c b/mm/slub.c > index 157527d7101b..8ac3bb9a122a 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -4017,7 +4017,8 @@ init_kmem_cache_node(struct kmem_cache_node *n) > static inline int alloc_kmem_cache_cpus(struct kmem_cache *s) > { > BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE < > - KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)); > + NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * > + sizeof(struct kmem_cache_cpu)); > > /* > * Must align to double word boundary for the double cmpxchg > -- > 2.34.1 > Acked-by: Dennis Zhou <dennis@kernel.org> Thanks, Dennis
On 10/24/22 10:14, Baoquan He wrote: > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > during early boot. For that, the dynamic chunk of percpu which serves > the early allocation need be large enough to satisfy the kmalloc > creation. > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > with correct calculation. > > Signed-off-by: Baoquan He <bhe@redhat.com> > Cc: Christoph Lameter <cl@linux.com> > Cc: Pekka Enberg <penberg@kernel.org> > Cc: David Rientjes <rientjes@google.com> > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Vlastimil Babka <vbabka@suse.cz> > Cc: Roman Gushchin <roman.gushchin@linux.dev> > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > --- > mm/slub.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) As only slub is touched and there's no prerequsities in the previous patches, I took this to the slab tree, branch slab/for-6.2/cleanups Thanks! > > diff --git a/mm/slub.c b/mm/slub.c > index 157527d7101b..8ac3bb9a122a 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -4017,7 +4017,8 @@ init_kmem_cache_node(struct kmem_cache_node *n) > static inline int alloc_kmem_cache_cpus(struct kmem_cache *s) > { > BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE < > - KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)); > + NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * > + sizeof(struct kmem_cache_cpu)); > > /* > * Must align to double word boundary for the double cmpxchg
On 11/06/22 at 09:56pm, Vlastimil Babka wrote: > On 10/24/22 10:14, Baoquan He wrote: > > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > > during early boot. For that, the dynamic chunk of percpu which serves > > the early allocation need be large enough to satisfy the kmalloc > > creation. > > > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > > with correct calculation. > > > > Signed-off-by: Baoquan He <bhe@redhat.com> > > Cc: Christoph Lameter <cl@linux.com> > > Cc: Pekka Enberg <penberg@kernel.org> > > Cc: David Rientjes <rientjes@google.com> > > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > Cc: Andrew Morton <akpm@linux-foundation.org> > > Cc: Vlastimil Babka <vbabka@suse.cz> > > Cc: Roman Gushchin <roman.gushchin@linux.dev> > > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > > --- > > mm/slub.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > As only slub is touched and there's no prerequsities in the previous > patches, I took this to the slab tree, branch > slab/for-6.2/cleanups Yes, it only changes slub code. Thanks for taking it. I will resend v2 with the left 7 percpu only patches with update.
Hi Baoquan, On Mon, Nov 07, 2022 at 12:35:56PM +0800, Baoquan He wrote: > On 11/06/22 at 09:56pm, Vlastimil Babka wrote: > > On 10/24/22 10:14, Baoquan He wrote: > > > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > > > during early boot. For that, the dynamic chunk of percpu which serves > > > the early allocation need be large enough to satisfy the kmalloc > > > creation. > > > > > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > > > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > > > with correct calculation. > > > > > > Signed-off-by: Baoquan He <bhe@redhat.com> > > > Cc: Christoph Lameter <cl@linux.com> > > > Cc: Pekka Enberg <penberg@kernel.org> > > > Cc: David Rientjes <rientjes@google.com> > > > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > Cc: Vlastimil Babka <vbabka@suse.cz> > > > Cc: Roman Gushchin <roman.gushchin@linux.dev> > > > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > > > --- > > > mm/slub.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > As only slub is touched and there's no prerequsities in the previous > > patches, I took this to the slab tree, branch > > slab/for-6.2/cleanups > > Yes, it only changes slub code. Thanks for taking it. > > I will resend v2 with the left 7 percpu only patches with update. > > Don't worry about resending them, I'll pick them up tomorrow morning. Thanks, Dennis
On 11/06/22 at 11:20pm, Dennis Zhou wrote: > Hi Baoquan, > > On Mon, Nov 07, 2022 at 12:35:56PM +0800, Baoquan He wrote: > > On 11/06/22 at 09:56pm, Vlastimil Babka wrote: > > > On 10/24/22 10:14, Baoquan He wrote: > > > > SLUB allocator relies on percpu allocator to initialize its ->cpu_slab > > > > during early boot. For that, the dynamic chunk of percpu which serves > > > > the early allocation need be large enough to satisfy the kmalloc > > > > creation. > > > > > > > > However, the current BUILD_BUG_ON() in alloc_kmem_cache_cpus() doesn't > > > > consider the kmalloc array with NR_KMALLOC_TYPES length. Fix that > > > > with correct calculation. > > > > > > > > Signed-off-by: Baoquan He <bhe@redhat.com> > > > > Cc: Christoph Lameter <cl@linux.com> > > > > Cc: Pekka Enberg <penberg@kernel.org> > > > > Cc: David Rientjes <rientjes@google.com> > > > > Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > > > Cc: Andrew Morton <akpm@linux-foundation.org> > > > > Cc: Vlastimil Babka <vbabka@suse.cz> > > > > Cc: Roman Gushchin <roman.gushchin@linux.dev> > > > > Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> > > > > --- > > > > mm/slub.c | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > As only slub is touched and there's no prerequsities in the previous > > > patches, I took this to the slab tree, branch > > > slab/for-6.2/cleanups > > > > Yes, it only changes slub code. Thanks for taking it. > > > > I will resend v2 with the left 7 percpu only patches with update. > > > > > > Don't worry about resending them, I'll pick them up tomorrow morning. That's great. Thanks a lot, Dennis.
diff --git a/mm/slub.c b/mm/slub.c index 157527d7101b..8ac3bb9a122a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4017,7 +4017,8 @@ init_kmem_cache_node(struct kmem_cache_node *n) static inline int alloc_kmem_cache_cpus(struct kmem_cache *s) { BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE < - KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)); + NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * + sizeof(struct kmem_cache_cpu)); /* * Must align to double word boundary for the double cmpxchg