From patchwork Fri Sep 15 10:59:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matteo Rizzo X-Patchwork-Id: 140435 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1044851vqi; Fri, 15 Sep 2023 06:26:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFihRanbPHCWu7SGh+gX9ZFrBnkYXe6HxSqB9QHZvrHus+P0ADfqYs2TBpWgnNxhyq4BN24 X-Received: by 2002:a05:6a00:124b:b0:68f:e810:e894 with SMTP id u11-20020a056a00124b00b0068fe810e894mr1690257pfi.33.1694784369176; Fri, 15 Sep 2023 06:26:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694784369; cv=none; d=google.com; s=arc-20160816; b=FGQdGNfkFH5ZPb66gv9hRo9XEiclWDxEaEAnsgxDK92aC8zqN5+gKMhWDcIsMpGm8s hQmTR2Fsm1YDcy50gURNgb9/QVWGa2N5RUZoy4Xm+oRvfgovXGPGTk/U7/566uuhWdZw WnUcAdTshGjaFyQeORqXBvu5JxdZL9kt4BtsQOjSrWkiq9onAXSduPOBXyRRh3c+x5fJ FMOH7w2ACoh1rYfHOOcahLHFnR7sMznZXNaxi/6LC8YWZba5AolnPvyqBKfhxg/rbbbW tOxGk50GT2X+/wT8T/s/kuiNXR6pPPH0tlGdYwoSJb5t0K1OOwwssIpzLwwFE5IVWLoT EasA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; fh=hxrcP/evkFXgdiqiA6zEdvE31LUTUVye/z/fSCqBw68=; b=USZd7fxROef/HMmwMOC3p3FL2WaOWt8Z/ae15jURYxfhbBqHSyybyE4CaDofrzcKWo iLucUTNfrTyJAwT6BS2C3mN2xEgu2EJCV1W4sVuEeCtc+tDnPxM/ERxonQco1zJKIrj3 /gKTshOyh1CXJwPvpgTokwNyf1TwMA8uD5gEPsLG3b/j60jBvRR6SmjNOuL2y8QQyZhC vOEHy3UBfTDA3AWPoSd9VLOernYkC+e5WAWVHvtA3+RZedsKMV0iZ6EVDFTfRSvn4n4R DM/pqVjC7TMCBJkJnOaHtlNZsl+20YZhWnewQuWpz/iPjs7WVvTkP8pySs0uRTNcaeoz PzHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=y7rwNgx5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id j6-20020a63fc06000000b00565df122f43si3237758pgi.202.2023.09.15.06.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 06:26:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=y7rwNgx5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id C71D8823626C; Fri, 15 Sep 2023 04:00:27 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234330AbjIOLAL (ORCPT + 32 others); Fri, 15 Sep 2023 07:00:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234335AbjIOLAF (ORCPT ); Fri, 15 Sep 2023 07:00:05 -0400 Received: from mail-ej1-x64a.google.com (mail-ej1-x64a.google.com [IPv6:2a00:1450:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44335189 for ; Fri, 15 Sep 2023 03:59:52 -0700 (PDT) Received: by mail-ej1-x64a.google.com with SMTP id a640c23a62f3a-94a35b0d4ceso145489966b.3 for ; Fri, 15 Sep 2023 03:59:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694775590; x=1695380390; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; b=y7rwNgx5k+DXr11V2BHWhy4llRFlcX1PV5xnO8aUypv9+5X3Q1RLH6DBPt5Ei6aH/q MxB3tr91+IBbtWdSO/Iz60Ku74D9gg4vdywXgL7Kdwzd0JxS7ke/fhSDQEP3g57rh4dy lwKdhFHtZoGTAVhH9eQRNbWgKxcSO2VGa4oI9/yXHSCmM1u6xHDDSg7lZyjwRmqcmDZt mXjdu88rV0iq2Ilc4zAVdsNx9jp/dk3Ytqc+GcVimQFc1xJnw/GSGP6MxLx3iyQFyO47 yd+xZQuBp0PT/s0HZufJpypXiSURxaYzoSiDjTvy0d6q++ttBaNr9Dcm5fvlCM6Zhdqq bgYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694775590; x=1695380390; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; b=lUMKXh+d1waWUecK8zb2EWDX78PGi/gbIjPRMiZR7zdJlxq5RujmSlevWIAmjQ27V2 sRybpVyS8OyFaC5FgdcLuizczwDtgdyaxWfaOH9wgAQQT1eVzO041SAzS0dZdCW3lLCu I7krtZNS3l8W1eCUDqe7nPvsbVW2rDAFJuLRu6BblvVl486kErbYJZrQfr6t9UEp5IUZ 2lXQTGX7ca1i4xZD31H4o8Ik9QLgdjFA1ivn1GUSbyMH0mXrdurKXqEs81KuKXRLB/mT sZtBOZitsZqKagH9/hHBBaA+tLgS+BSREivigqyPNA0naYdRowlGip0yS8OrEAbRYZVZ FVCA== X-Gm-Message-State: AOJu0YyDz6JiF+WS6alWMsrZDU88DUF4SxNaGB7A10J0u+tUtqNojTMA CeUuDeQOfoaAfDAxltjQn3L1kKf7Cwmk4nfSGg== X-Received: from mr-cloudtop2.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:2a6]) (user=matteorizzo job=sendgmr) by 2002:a17:907:890:b0:9ad:a751:2ea3 with SMTP id zt16-20020a170907089000b009ada7512ea3mr6741ejb.6.1694775590539; Fri, 15 Sep 2023 03:59:50 -0700 (PDT) Date: Fri, 15 Sep 2023 10:59:24 +0000 In-Reply-To: <20230915105933.495735-1-matteorizzo@google.com> Mime-Version: 1.0 References: <20230915105933.495735-1-matteorizzo@google.com> X-Mailer: git-send-email 2.42.0.459.ge4e396fd5e-goog Message-ID: <20230915105933.495735-6-matteorizzo@google.com> Subject: [RFC PATCH 05/14] mm/slub: create folio_set/clear_slab helpers From: Matteo Rizzo To: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org Cc: jannh@google.com, matteorizzo@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com X-Spam-Status: No, score=-8.4 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 15 Sep 2023 04:00:27 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777110214272653265 X-GMAIL-MSGID: 1777110214272653265 From: Jann Horn This is refactoring in preparation for SLAB_VIRTUAL. Extract this code to separate functions so that it's not duplicated in the code that allocates and frees page with SLAB_VIRTUAL enabled. Signed-off-by: Jann Horn Co-developed-by: Matteo Rizzo Signed-off-by: Matteo Rizzo Reviewed-by: Kees Cook --- mm/slub.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index ad33d9e1601d..9b87afade125 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1849,6 +1849,26 @@ static void *setup_object(struct kmem_cache *s, void *object) /* * Slab allocation and freeing */ + +static void folio_set_slab(struct folio *folio, struct slab *slab) +{ + __folio_set_slab(folio); + /* Make the flag visible before any changes to folio->mapping */ + smp_wmb(); + + if (folio_is_pfmemalloc(folio)) + slab_set_pfmemalloc(slab); +} + +static void folio_clear_slab(struct folio *folio, struct slab *slab) +{ + __slab_clear_pfmemalloc(slab); + folio->mapping = NULL; + /* Make the mapping reset visible before clearing the flag */ + smp_wmb(); + __folio_clear_slab(folio); +} + static inline struct slab *alloc_slab_page(gfp_t flags, int node, struct kmem_cache_order_objects oo) { @@ -1865,11 +1885,7 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node, return NULL; slab = folio_slab(folio); - __folio_set_slab(folio); - /* Make the flag visible before any changes to folio->mapping */ - smp_wmb(); - if (folio_is_pfmemalloc(folio)) - slab_set_pfmemalloc(slab); + folio_set_slab(folio, slab); return slab; } @@ -2067,11 +2083,7 @@ static void __free_slab(struct kmem_cache *s, struct slab *slab) int order = folio_order(folio); int pages = 1 << order; - __slab_clear_pfmemalloc(slab); - folio->mapping = NULL; - /* Make the mapping reset visible before clearing the flag */ - smp_wmb(); - __folio_clear_slab(folio); + folio_clear_slab(folio, slab); mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); __free_pages(&folio->page, order);