Message ID | 20231109073133.792-2-justinjiang@vivo.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b129:0:b0:403:3b70:6f57 with SMTP id q9csp271893vqs; Wed, 8 Nov 2023 23:32:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IEPJvC5Yn68TZGkm/lwyMwMqxaEKYcsruTmCQZL72Z/FJbvbLawP4N7VFhbmiFyfWDqMcQF X-Received: by 2002:a05:6358:3417:b0:168:d31f:9fe3 with SMTP id h23-20020a056358341700b00168d31f9fe3mr4597458rwd.23.1699515128414; Wed, 08 Nov 2023 23:32:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699515128; cv=pass; d=google.com; s=arc-20160816; b=HuGtIptMY2nJBLSRQn0hL5ROsxtm8H/ZJJIIj8/lDxvjqLpaHJyIHfNavZeSPAjoSH HdeH2VhVScsgU6TUfKI6QxSyfE9g6MuNIccxwHZsO54y+FVpSNKj0/MM+F5BajDo1MIN uqQuQmw0opyvQ6I12ezM4wd9T+5eERYL1Kh/KyHmPj11v8T677x2M7B/06zSyGvKbEaT wMrRWNVKA+CttBxkJYOFyN+g1+y+JzdKGPEar8IcYdQM+ZkxZq6nZOspa5QFKlh4vhtZ 3C6ENjDb/CkjDt86pz4lRG0FEKXnICeMKrmIu9qSKOS4nWZaK1zw1jpE2fBJQE0k3JBA Ba2A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hLJUZqO0wkBfRI5tWq/EbvA+6ceM75bugAlYQzeQXzg=; fh=TWbg/BkjXAHwJZ7W8qs2AL6SRSosoByTcBt4OMKwj4Y=; b=GRYYhnSIlWQ1SHhZaLUcdGaprOHDzZqvAfNXTikM+UPEroKodHCo0yXr7NzhY4/WAK zaY3usvoTlyj4b1XqtxyaCyyOYdcgJuwQ/qrW2C5EKpzNxSqxy2Tv0Dwppvcam6JzEYw LkapnWp54qqHPE70KBO7931OMouY1fJ+eDmy6kVjufNTzxxEmdMj5JKfwhD/gjd935C3 8RNqH7sPiXBaknhvPhv+7spfFjJzgDXPnhU9TLOIcvmr5Gei3nH4tlFBRjPAK0+uHncd LaqyO5JhdJJTz5ZaHVEQM7AjoDvVG/8zCu+M9TKuNCAK5plkgxkXWrKNTm0/k48GElba S6pw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@vivo.com header.s=selector2 header.b=gRRPwP4Q; arc=pass (i=1 spf=pass spfdomain=vivo.com dkim=pass dkdomain=vivo.com dmarc=pass fromdomain=vivo.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=vivo.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id t127-20020a632d85000000b005b93908de64si6240828pgt.704.2023.11.08.23.32.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 23:32:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@vivo.com header.s=selector2 header.b=gRRPwP4Q; arc=pass (i=1 spf=pass spfdomain=vivo.com dkim=pass dkdomain=vivo.com dmarc=pass fromdomain=vivo.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=vivo.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 0C270831EDAB; Wed, 8 Nov 2023 23:32:06 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232670AbjKIHby (ORCPT <rfc822;jaysivo@gmail.com> + 32 others); Thu, 9 Nov 2023 02:31:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232592AbjKIHbv (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 9 Nov 2023 02:31:51 -0500 Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2116.outbound.protection.outlook.com [40.107.255.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 809592582 for <linux-kernel@vger.kernel.org>; Wed, 8 Nov 2023 23:31:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iOjZRhEQijlMAlpUZPUvQidG0fPXdH+wpiLqgRvQRUGstf1AY4e+XQWcz5ucAmRtn0JWCMyBueLH+TKPjFN/odOD/yDGjwoy0fSc0TTjiD4IWZksJPaB1iJgOv5CeP7g/agDBxrViBe/ICX6muiTQ9EmFgpOZelsTBpzHJLfLs4ieDrxtLwD8VabP2f433TmT04S/JsW6x5SDuivYYLZCWjcwBwSa0aTA0CM5StAtC/R8YOqQIdqAb+rdadHxNIcpx0mEo5nPVLve3DvexYs2Up6+9uAmIMHDIKjKS1DW/Psx/JVFFxL75Bmqg8bcPSTP8fOZRwQcGD8fD4OkLv6ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hLJUZqO0wkBfRI5tWq/EbvA+6ceM75bugAlYQzeQXzg=; b=OOJGJM103DpWtj1ko5/kZO+VWQ5nSH9e7CCEb4FKZUZRlKqxkO+uRVT0IYSWQ23omTA1GoV6/KQnHBtdqlROifTGyQ6cMBXVNHxnmnU6ZO4DpCnVW2/D3dHoh9ypqRQYJmriGJCJSayBuk1AjzWgVzAnsb/R7P2g/a+hdc9BZmnd5pdzN49wwB1TsxzKXsypZQEx0nY3PD9VKFJRhIhY44LTknfopx5ZS9cu6X4ar029iSd07bctxgDdg/6r+y8EfTTr6+55g0RSW447uPHRvXca43HluOgodb6ivwsBlHcx/pSFnXzsb6Pt+RLXlwLPrqw4xqNJ/Mbh4kpWSpL08Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hLJUZqO0wkBfRI5tWq/EbvA+6ceM75bugAlYQzeQXzg=; b=gRRPwP4QgNI3Ri+O3tDS5apOfqBe+S18W7GX8TEKs/DmJkwTMX9kIzK22ydSX8Dx1N43L8Yl6ZYIpvzJCSDwQ0tFI1+aWVo3TCddSzc4zukeGpwYVMaEOCy6RTNNb2Bh/NzFjy9upe/PcaIJS1Slvq8z9d3a2yyZcJhV14IsXANPnsUWYbCaozTU4kAM7z4bJpuAaEIctiLZn5Vbv/ximf9s3cgf77w7LF2JS2S1pg+4aVWhziq3uinN7ipaxIIwIY7AIAXl9pDBxAZnQdjjHBqgZ8gLQNNfBhhzKY1n1Qb/36gicDHKZkYPQ4us4uejZQ4FpBmphbrmsKcuImGcLg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from JH0PR06MB6849.apcprd06.prod.outlook.com (2603:1096:990:47::12) by PUZPR06MB5933.apcprd06.prod.outlook.com (2603:1096:301:11a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.29; Thu, 9 Nov 2023 07:31:43 +0000 Received: from JH0PR06MB6849.apcprd06.prod.outlook.com ([fe80::bdde:31ee:f13c:79fb]) by JH0PR06MB6849.apcprd06.prod.outlook.com ([fe80::bdde:31ee:f13c:79fb%4]) with mapi id 15.20.6954.027; Thu, 9 Nov 2023 07:31:43 +0000 From: Zhiguo Jiang <justinjiang@vivo.com> To: Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Johannes Weiner <hannes@cmpxchg.org>, opensource.kernel@vivo.com, Zhiguo Jiang <justinjiang@vivo.com> Subject: [PATCH v2 1/1] mm: ALLOC_HIGHATOMIC flag allocation issue Date: Thu, 9 Nov 2023 15:31:33 +0800 Message-ID: <20231109073133.792-2-justinjiang@vivo.com> X-Mailer: git-send-email 2.41.0.windows.3 In-Reply-To: <20231109073133.792-1-justinjiang@vivo.com> References: <20231109073133.792-1-justinjiang@vivo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SG2PR06CA0219.apcprd06.prod.outlook.com (2603:1096:4:68::27) To JH0PR06MB6849.apcprd06.prod.outlook.com (2603:1096:990:47::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: JH0PR06MB6849:EE_|PUZPR06MB5933:EE_ X-MS-Office365-Filtering-Correlation-Id: eba322d7-08c8-4bf7-0690-08dbe0f5eb9a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Iw2M49gfK1SDxOKSiGKJxyCCU/iUQL7JKnS7az65DHnSTNJzr4BEwCtkG6m/CfdLAp4L7IMZuRoHL7NsrIyEN0ZnZMsqxrBxZwDDNBNxMmoNFuwfRezp574ZsDaEbhtTdoE/MZygKN5HjDFjU5brSu/qeJ+GP7lpGpMp9yzMh7TZIIjGY55VsT3hAS5824kHZdIt9/ufFvCBTU2fZlfVdF3CYcf8ZqDRkZQvEBONLCKOOY7ov8MVsTp4rGBq1MBJcBTrv1T8R7cBv8dSwC1V6ytpaC+4D/fC+i6R9DQqPkmy4i2dLyUjAR+2ZRzq12D0+ced96QGNYQg4M0M776M8d35EHJR6YhPdQBbwMsDdufUDgDGtkzV6ZI8SpWprxDhGDdANoaM4L3P2ohhH6Z2jN3QYUZmA74hnqK/SJBTAvegl+JnIDIP3K/IuvSnXvjsuO8vVHCWH+3zmWc7n8yoQ3hH73xY+OcDMGsxmN2eQ0s7UgZwtnfUbhq9MdCDtpkGw78/DeA0wOQEoS3KREeM4U1RcSx6dwvdDhq0xLu0IHbZPegjatpwaPfP0KlmfdOCAde7ksqqLyQRegrjuPhffHJyBgN6GpzN4iReoh+3RWzFG8TTABFSdv7f30yMidrs X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:JH0PR06MB6849.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(39860400002)(366004)(376002)(136003)(346002)(396003)(230922051799003)(1800799009)(64100799003)(451199024)(186009)(478600001)(107886003)(41300700001)(6486002)(6506007)(52116002)(6512007)(2616005)(316002)(8676002)(4326008)(83380400001)(8936002)(26005)(5660300002)(6666004)(54906003)(66556008)(66476007)(66946007)(38100700002)(2906002)(86362001)(1076003)(36756003)(38350700005);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: uaThiQEL6Z7hEzoSRHgs9w+rTCFyK6Z5Q7hsMZ9ayWF6GEG7DajqDxGfv6yqKDk3tQRFjYRJ71TE+taWkdcO4o4wTSqnvsi5QyXJ7KCOLR2QdtxyBXxeufLiM9YB6DBNEW2fxLTgmiosOz7L603quktjClvMk+SZRm4sTIGDUFQd7fJ86C5giFjgC32u34iVTPa2CIyIgnqw/Vdvh4XRN2JPdzFb3BqMIUyRxP9AcPHVuX5/qfvhrZE1Y7kp5iN0oRA+xvhT8HKqw8sx92JRQ5iuOaVMamOKabiwFStOr61oRSb/0GZRYP+F+hH32+Rb+9QspRjZdK5RjnrcEcdTbgpecCAyWmwVOzNYSl8zIZbgQp4Nc5omU0Na5MbzSIbUly90b5LA0wwLs51fR4dsOqw6CuGLMXPGl5OkYjre/m/q7OhsQddmrabNHzj87ub2m9rGQyN0D8LLfrTGc2sSKb5foB2izVWS9UUaPs1RFBBmdXjvlPZzhrHHDEDAr945sS33P+h/qQPSDID/1oKy1BcbZrm/isuZ5RtFlp4lhsxsCvSauKLlBtGHar7CwBF1dmzoLwF8gG0ZU30TGcUj8Kvjmdp9HBvn06YxWJazvDUnKiYdzHP65tSBzPXDmPS/n9eCbl65VxN/p9yvL4JgvTRiHu5pvWGvq+cjT87xkSxUdUnwtZ5OahKYZkxigr/RcBNDmbFvU8eD4R1oYYsiljRNk7R7IYSbdF3CGafbXVwub6vnCU0AN/p+LO+i6QgdaZxzJk5lqYv7T6mJdtujAYOb5ufaLNVtkwxyytXqpFMrgL6ReELBLzctGVG0y1NVR/KJ90xLkw3Zb7G6d2m3zpzs+VEquW8AtJVV+U25dCrzqJroqGcrVZkVeLvUHqlAyRdp0VZp4x6Ijtjb0Etm6T+2kMn6UJt4QHQDT6sWxmHCtgr876b8DvbZcrIPAva3NmyXjpdersQ6xWgrkPdlJqsCb1EQ83tU+YrDPjCp/fOiV1YKH/yX4y/+TbdLRdxPjFkXi6SQR8RLiBP2iec8oynm6N3ewrhXeou/29HnH0LbnT75Dntut5S9h1S9CAps1q4fFftOD5GYwz74eZk+PDh09e4DPzw0Ti39t/gZ5f8cMMm4MuMM+urk9marhit9NOsVgzAJgVWBrWJHd2VW63YVw5PqNGErqZIgpCFM5ZX38gHaF1W6gGigjGqdGgdtMo2rAtr2etjFq3qEsdUNlVGOSO/qunCUK/KXjzztTinAYYCujynBHkZ82eZnafhbLBIsuyUX4LmgncbidRdfeRIu1cHSf13mickhRymL7M9TH1DqemF5HYP6Prepk20aps6uegE4bGp27abOYtkGZUTrT9h7E5Rfm9JRm8CBBEujrgHo/jeEfi27sak2p+6fZCuF5ITZti0Ei/0Z6grtIpAA/MpVTurG77IfKxmt96e5h0VsS7cn+t0u8Z7n4jVypqMkj3c0pMzvOVKGD0iK42/pVrGAbkQnuCieGVfWnEdp2GhW0+SrVn4PXUSOvzabHEDYjZEIVl4iLG84R+Nffw36NloPuE2wxYAwKB19ywEnDlPZstKMWICQYlVC1d45 X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: eba322d7-08c8-4bf7-0690-08dbe0f5eb9a X-MS-Exchange-CrossTenant-AuthSource: JH0PR06MB6849.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2023 07:31:42.8710 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zcI/yR5Y/KTeHVH1nCD2ksRhMgvNwQn2BJjDT8LbnKgV++x7tH5VZkDrw0RSa8mxvx+U0c94BSRl+3PxgWJuYA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PUZPR06MB5933 Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 08 Nov 2023 23:32:06 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782070773308870199 X-GMAIL-MSGID: 1782070775211898962 |
Series |
mm: ALLOC_HIGHATOMIC flag allocation issue
|
|
Commit Message
Zhiguo Jiang
Nov. 9, 2023, 7:31 a.m. UTC
Add a new bool* argument to pass return flag instead of *alloc_flags
and add the related comments.
Signed-off-by: Zhiguo Jiang <justinjiang@vivo.com>
---
Changelog:
v1->v2:
1. Delete ALLOC_PCPLIST macro.
2. In rmqueue() add a new bool* argument to pass return flag
instead of alloc_flags.
3. Add comments for reader understanding.
mm/internal.h | 1 -
mm/page_alloc.c | 41 ++++++++++++++++++++++++++++++-----------
2 files changed, 30 insertions(+), 12 deletions(-)
Comments
On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: > Add a new bool* argument to pass return flag instead of *alloc_flags > and add the related comments. Please retain (and update) the changelog with each version of a patch. For reviewers (please), here's the v1 changelog: : In case of alloc_flags contain ALLOC_HIGHATOMIC and alloc order : is order1/2/3/10 in rmqueue(), if pages are alloced successfully : from pcplist cache, and move a free pageblock from the alloced : migratetype freelist to MIGRATE_HIGHATOMIC freelist, rather than : alloc from MIGRATE_HIGHATOMIC freelist firstly, so this will result : in an increasing number of pages on the MIGRATE_HIGHATOMIC freelist, : pages in other migratetype freelist are reduced and more likely to : allocation failure. : : Currently the sequence of ALLOC_HIGHATOMIC allocation is: : pcplist cache --> buddy (batch >> order) allocation migratetype : freelist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation : migratetype freelist. : : Due to the fact that requesting pages from the pcplist cache is faster : than buddy, the sequence of modifying the ALLOC_HIGHATOMIC allocation is: : pcplist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation : migrate freelist. : : This patch can solve the allocation failure of the Non-ALLOC_HIGHATOMIC : alloc_flag due to excessive pages reservations in MIGRATE_HIGHATOMIC : freelists. > --- a/mm/internal.h > +++ b/mm/internal.h > @@ -905,7 +905,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, > #endif > #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ > #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */ > -#define ALLOC_PCPLIST 0x1000 /* Allocations from pcplist */ > > /* Flags that allow allocations below the min watermark. */ > #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index 67cec88164b1..3c84c3e3eeb0 100755 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -2854,6 +2854,11 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, > int batch = nr_pcp_alloc(pcp, zone, order); > int alloced; > > + /* > + * If pcplist is empty and alloc_flags is with ALLOC_HIGHATOMIC, > + * it should alloc from buddy highatomic migrate freelist firstly > + * to ensure quick and successful allocation. > + */ > if (alloc_flags & ALLOC_HIGHATOMIC) > goto out; > > @@ -2925,8 +2930,8 @@ __no_sanitize_memory > static inline > struct page *rmqueue(struct zone *preferred_zone, > struct zone *zone, unsigned int order, > - gfp_t gfp_flags, unsigned int *alloc_flags, > - int migratetype) > + gfp_t gfp_flags, unsigned int alloc_flags, > + int migratetype, bool *highatomc_allocation) > { > struct page *page; > > @@ -2938,19 +2943,33 @@ struct page *rmqueue(struct zone *preferred_zone, > > if (likely(pcp_allowed_order(order))) { > page = rmqueue_pcplist(preferred_zone, zone, order, > - migratetype, *alloc_flags); > - if (likely(page)) { > - *alloc_flags |= ALLOC_PCPLIST; > + migratetype, alloc_flags); > + if (likely(page)) > goto out; > - } > } > > - page = rmqueue_buddy(preferred_zone, zone, order, *alloc_flags, > + page = rmqueue_buddy(preferred_zone, zone, order, alloc_flags, > migratetype); > > + /* > + * The high-order atomic allocation pageblock reserved conditions: > + * > + * If the high-order atomic allocation page is alloced from pcplist, > + * the highatomic pageblock does not need to be reserved, which can > + * void to migrate an increasing number of pages into buddy > + * MIGRATE_HIGHATOMIC freelist and lead to an increasing risk of > + * allocation failure on other buddy migrate freelists. > + * > + * If the high-order atomic allocation page is alloced from buddy > + * highatomic migrate freelist, regardless of whether the allocation > + * is successful or not, the highatomic pageblock can try to be > + * reserved. > + */ > + if (unlikely(alloc_flags & ALLOC_HIGHATOMIC)) > + *highatomc_allocation = true; > out: > /* Separate test+clear to avoid unnecessary atomics */ > - if ((*alloc_flags & ALLOC_KSWAPD) && > + if ((alloc_flags & ALLOC_KSWAPD) && > unlikely(test_bit(ZONE_BOOSTED_WATERMARK, &zone->flags))) { > clear_bit(ZONE_BOOSTED_WATERMARK, &zone->flags); > wakeup_kswapd(zone, 0, 0, zone_idx(zone)); > @@ -3218,6 +3237,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, > struct pglist_data *last_pgdat = NULL; > bool last_pgdat_dirty_ok = false; > bool no_fallback; > + bool highatomc_allocation = false; > > retry: > /* > @@ -3349,7 +3369,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, > > try_this_zone: > page = rmqueue(ac->preferred_zoneref->zone, zone, order, > - gfp_mask, &alloc_flags, ac->migratetype); > + gfp_mask, alloc_flags, ac->migratetype, &highatomc_allocation); > if (page) { > prep_new_page(page, order, gfp_mask, alloc_flags); > > @@ -3357,8 +3377,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, > * If this is a high-order atomic allocation then check > * if the pageblock should be reserved for the future > */ > - if (unlikely(alloc_flags & ALLOC_HIGHATOMIC) && > - unlikely(!(alloc_flags & ALLOC_PCPLIST))) > + if (unlikely(highatomc_allocation)) > reserve_highatomic_pageblock(page, zone); > > return page; > -- > 2.39.0
在 2023/11/10 1:49, Andrew Morton 写道: > On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: > >> Add a new bool* argument to pass return flag instead of *alloc_flags >> and add the related comments. > Please retain (and update) the changelog with each version of a patch. ok,I will supplement it if there is a later version. > > For reviewers (please), here's the v1 changelog: > > : In case of alloc_flags contain ALLOC_HIGHATOMIC and alloc order > : is order1/2/3/10 in rmqueue(), if pages are alloced successfully > : from pcplist cache, and move a free pageblock from the alloced > : migratetype freelist to MIGRATE_HIGHATOMIC freelist, rather than > : alloc from MIGRATE_HIGHATOMIC freelist firstly, so this will result > : in an increasing number of pages on the MIGRATE_HIGHATOMIC freelist, > : pages in other migratetype freelist are reduced and more likely to > : allocation failure. > : > : Currently the sequence of ALLOC_HIGHATOMIC allocation is: > : pcplist cache --> buddy (batch >> order) allocation migratetype > : freelist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation > : migratetype freelist. > : > : Due to the fact that requesting pages from the pcplist cache is faster > : than buddy, the sequence of modifying the ALLOC_HIGHATOMIC allocation is: > : pcplist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation > : migrate freelist. > : > : This patch can solve the allocation failure of the Non-ALLOC_HIGHATOMIC > : alloc_flag due to excessive pages reservations in MIGRATE_HIGHATOMIC > : freelists. > >> --- a/mm/internal.h >> +++ b/mm/internal.h >> @@ -905,7 +905,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, >> #endif >> #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ >> #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */ >> -#define ALLOC_PCPLIST 0x1000 /* Allocations from pcplist */ >> >> /* Flags that allow allocations below the min watermark. */ >> #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >> index 67cec88164b1..3c84c3e3eeb0 100755 >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -2854,6 +2854,11 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, >> int batch = nr_pcp_alloc(pcp, zone, order); >> int alloced; >> >> + /* >> + * If pcplist is empty and alloc_flags is with ALLOC_HIGHATOMIC, >> + * it should alloc from buddy highatomic migrate freelist firstly >> + * to ensure quick and successful allocation. >> + */ >> if (alloc_flags & ALLOC_HIGHATOMIC) >> goto out; >> >> @@ -2925,8 +2930,8 @@ __no_sanitize_memory >> static inline >> struct page *rmqueue(struct zone *preferred_zone, >> struct zone *zone, unsigned int order, >> - gfp_t gfp_flags, unsigned int *alloc_flags, >> - int migratetype) >> + gfp_t gfp_flags, unsigned int alloc_flags, >> + int migratetype, bool *highatomc_allocation) >> { >> struct page *page; >> >> @@ -2938,19 +2943,33 @@ struct page *rmqueue(struct zone *preferred_zone, >> >> if (likely(pcp_allowed_order(order))) { >> page = rmqueue_pcplist(preferred_zone, zone, order, >> - migratetype, *alloc_flags); >> - if (likely(page)) { >> - *alloc_flags |= ALLOC_PCPLIST; >> + migratetype, alloc_flags); >> + if (likely(page)) >> goto out; >> - } >> } >> >> - page = rmqueue_buddy(preferred_zone, zone, order, *alloc_flags, >> + page = rmqueue_buddy(preferred_zone, zone, order, alloc_flags, >> migratetype); >> >> + /* >> + * The high-order atomic allocation pageblock reserved conditions: >> + * >> + * If the high-order atomic allocation page is alloced from pcplist, >> + * the highatomic pageblock does not need to be reserved, which can >> + * void to migrate an increasing number of pages into buddy >> + * MIGRATE_HIGHATOMIC freelist and lead to an increasing risk of >> + * allocation failure on other buddy migrate freelists. >> + * >> + * If the high-order atomic allocation page is alloced from buddy >> + * highatomic migrate freelist, regardless of whether the allocation >> + * is successful or not, the highatomic pageblock can try to be >> + * reserved. >> + */ >> + if (unlikely(alloc_flags & ALLOC_HIGHATOMIC)) >> + *highatomc_allocation = true; >> out: >> /* Separate test+clear to avoid unnecessary atomics */ >> - if ((*alloc_flags & ALLOC_KSWAPD) && >> + if ((alloc_flags & ALLOC_KSWAPD) && >> unlikely(test_bit(ZONE_BOOSTED_WATERMARK, &zone->flags))) { >> clear_bit(ZONE_BOOSTED_WATERMARK, &zone->flags); >> wakeup_kswapd(zone, 0, 0, zone_idx(zone)); >> @@ -3218,6 +3237,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> struct pglist_data *last_pgdat = NULL; >> bool last_pgdat_dirty_ok = false; >> bool no_fallback; >> + bool highatomc_allocation = false; >> >> retry: >> /* >> @@ -3349,7 +3369,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> >> try_this_zone: >> page = rmqueue(ac->preferred_zoneref->zone, zone, order, >> - gfp_mask, &alloc_flags, ac->migratetype); >> + gfp_mask, alloc_flags, ac->migratetype, &highatomc_allocation); >> if (page) { >> prep_new_page(page, order, gfp_mask, alloc_flags); >> >> @@ -3357,8 +3377,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> * If this is a high-order atomic allocation then check >> * if the pageblock should be reserved for the future >> */ >> - if (unlikely(alloc_flags & ALLOC_HIGHATOMIC) && >> - unlikely(!(alloc_flags & ALLOC_PCPLIST))) >> + if (unlikely(highatomc_allocation)) >> reserve_highatomic_pageblock(page, zone); >> >> return page; >> -- >> 2.39.0
在 2023/11/10 1:49, Andrew Morton 写道: > On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: > >> Add a new bool* argument to pass return flag instead of *alloc_flags >> and add the related comments. > Please retain (and update) the changelog with each version of a patch. ok, supplement in v3 patch: https://lore.kernel.org/all/20231110020840.1031-1-justinjiang@vivo.com/ > > For reviewers (please), here's the v1 changelog: > > : In case of alloc_flags contain ALLOC_HIGHATOMIC and alloc order > : is order1/2/3/10 in rmqueue(), if pages are alloced successfully > : from pcplist cache, and move a free pageblock from the alloced > : migratetype freelist to MIGRATE_HIGHATOMIC freelist, rather than > : alloc from MIGRATE_HIGHATOMIC freelist firstly, so this will result > : in an increasing number of pages on the MIGRATE_HIGHATOMIC freelist, > : pages in other migratetype freelist are reduced and more likely to > : allocation failure. > : > : Currently the sequence of ALLOC_HIGHATOMIC allocation is: > : pcplist cache --> buddy (batch >> order) allocation migratetype > : freelist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation > : migratetype freelist. > : > : Due to the fact that requesting pages from the pcplist cache is faster > : than buddy, the sequence of modifying the ALLOC_HIGHATOMIC allocation is: > : pcplist --> buddy MIGRATE_HIGHATOMIC freelist --> buddy allocation > : migrate freelist. > : > : This patch can solve the allocation failure of the Non-ALLOC_HIGHATOMIC > : alloc_flag due to excessive pages reservations in MIGRATE_HIGHATOMIC > : freelists. > >> --- a/mm/internal.h >> +++ b/mm/internal.h >> @@ -905,7 +905,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, >> #endif >> #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ >> #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */ >> -#define ALLOC_PCPLIST 0x1000 /* Allocations from pcplist */ >> >> /* Flags that allow allocations below the min watermark. */ >> #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >> index 67cec88164b1..3c84c3e3eeb0 100755 >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -2854,6 +2854,11 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, >> int batch = nr_pcp_alloc(pcp, zone, order); >> int alloced; >> >> + /* >> + * If pcplist is empty and alloc_flags is with ALLOC_HIGHATOMIC, >> + * it should alloc from buddy highatomic migrate freelist firstly >> + * to ensure quick and successful allocation. >> + */ >> if (alloc_flags & ALLOC_HIGHATOMIC) >> goto out; >> >> @@ -2925,8 +2930,8 @@ __no_sanitize_memory >> static inline >> struct page *rmqueue(struct zone *preferred_zone, >> struct zone *zone, unsigned int order, >> - gfp_t gfp_flags, unsigned int *alloc_flags, >> - int migratetype) >> + gfp_t gfp_flags, unsigned int alloc_flags, >> + int migratetype, bool *highatomc_allocation) >> { >> struct page *page; >> >> @@ -2938,19 +2943,33 @@ struct page *rmqueue(struct zone *preferred_zone, >> >> if (likely(pcp_allowed_order(order))) { >> page = rmqueue_pcplist(preferred_zone, zone, order, >> - migratetype, *alloc_flags); >> - if (likely(page)) { >> - *alloc_flags |= ALLOC_PCPLIST; >> + migratetype, alloc_flags); >> + if (likely(page)) >> goto out; >> - } >> } >> >> - page = rmqueue_buddy(preferred_zone, zone, order, *alloc_flags, >> + page = rmqueue_buddy(preferred_zone, zone, order, alloc_flags, >> migratetype); >> >> + /* >> + * The high-order atomic allocation pageblock reserved conditions: >> + * >> + * If the high-order atomic allocation page is alloced from pcplist, >> + * the highatomic pageblock does not need to be reserved, which can >> + * void to migrate an increasing number of pages into buddy >> + * MIGRATE_HIGHATOMIC freelist and lead to an increasing risk of >> + * allocation failure on other buddy migrate freelists. >> + * >> + * If the high-order atomic allocation page is alloced from buddy >> + * highatomic migrate freelist, regardless of whether the allocation >> + * is successful or not, the highatomic pageblock can try to be >> + * reserved. >> + */ >> + if (unlikely(alloc_flags & ALLOC_HIGHATOMIC)) >> + *highatomc_allocation = true; >> out: >> /* Separate test+clear to avoid unnecessary atomics */ >> - if ((*alloc_flags & ALLOC_KSWAPD) && >> + if ((alloc_flags & ALLOC_KSWAPD) && >> unlikely(test_bit(ZONE_BOOSTED_WATERMARK, &zone->flags))) { >> clear_bit(ZONE_BOOSTED_WATERMARK, &zone->flags); >> wakeup_kswapd(zone, 0, 0, zone_idx(zone)); >> @@ -3218,6 +3237,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> struct pglist_data *last_pgdat = NULL; >> bool last_pgdat_dirty_ok = false; >> bool no_fallback; >> + bool highatomc_allocation = false; >> >> retry: >> /* >> @@ -3349,7 +3369,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> >> try_this_zone: >> page = rmqueue(ac->preferred_zoneref->zone, zone, order, >> - gfp_mask, &alloc_flags, ac->migratetype); >> + gfp_mask, alloc_flags, ac->migratetype, &highatomc_allocation); >> if (page) { >> prep_new_page(page, order, gfp_mask, alloc_flags); >> >> @@ -3357,8 +3377,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, >> * If this is a high-order atomic allocation then check >> * if the pageblock should be reserved for the future >> */ >> - if (unlikely(alloc_flags & ALLOC_HIGHATOMIC) && >> - unlikely(!(alloc_flags & ALLOC_PCPLIST))) >> + if (unlikely(highatomc_allocation)) >> reserve_highatomic_pageblock(page, zone); >> >> return page; >> -- >> 2.39.0
On Thu, Nov 09, 2023 at 09:49:54AM -0800, Andrew Morton wrote: > On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: > > > Add a new bool* argument to pass return flag instead of *alloc_flags > > and add the related comments. > > Please retain (and update) the changelog with each version of a patch. > > For reviewers (please), here's the v1 changelog: This patch isn't diffed against the current tree. It can't be reviewed.
在 2023/11/11 14:25, Matthew Wilcox 写道: > On Thu, Nov 09, 2023 at 09:49:54AM -0800, Andrew Morton wrote: >> On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: >> >>> Add a new bool* argument to pass return flag instead of *alloc_flags >>> and add the related comments. >> Please retain (and update) the changelog with each version of a patch. >> >> For reviewers (please), here's the v1 changelog: > This patch isn't diffed against the current tree. It can't be reviewed. patch v1: https://lore.kernel.org/all/20231108065408.1861-1-justinjiang@vivo.com/ patch v2: https://lore.kernel.org/all/20231109073133.792-2-justinjiang@vivo.com/ patch v3: https://lore.kernel.org/all/20231110020840.1031-1-justinjiang@vivo.com/ >
On Mon, Nov 13, 2023 at 09:04:49AM +0800, zhiguojiang wrote: > > > 在 2023/11/11 14:25, Matthew Wilcox 写道: > > On Thu, Nov 09, 2023 at 09:49:54AM -0800, Andrew Morton wrote: > > > On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: > > > > > > > Add a new bool* argument to pass return flag instead of *alloc_flags > > > > and add the related comments. > > > Please retain (and update) the changelog with each version of a patch. > > > > > > For reviewers (please), here's the v1 changelog: > > This patch isn't diffed against the current tree. It can't be reviewed. > patch v1: > https://lore.kernel.org/all/20231108065408.1861-1-justinjiang@vivo.com/ > patch v2: > https://lore.kernel.org/all/20231109073133.792-2-justinjiang@vivo.com/ > patch v3: > https://lore.kernel.org/all/20231110020840.1031-1-justinjiang@vivo.com/ None of those are diffed against the current tree. You need to send a patch that applies cleanly to either linux-next or current Linus head. Not a patch on top of an earlier version of your patch.
在 2023/11/13 9:36, Matthew Wilcox 写道: > On Mon, Nov 13, 2023 at 09:04:49AM +0800, zhiguojiang wrote: >> >> 在 2023/11/11 14:25, Matthew Wilcox 写道: >>> On Thu, Nov 09, 2023 at 09:49:54AM -0800, Andrew Morton wrote: >>>> On Thu, 9 Nov 2023 15:31:33 +0800 Zhiguo Jiang <justinjiang@vivo.com> wrote: >>>> >>>>> Add a new bool* argument to pass return flag instead of *alloc_flags >>>>> and add the related comments. >>>> Please retain (and update) the changelog with each version of a patch. >>>> >>>> For reviewers (please), here's the v1 changelog: >>> This patch isn't diffed against the current tree. It can't be reviewed. >> patch v1: >> https://lore.kernel.org/all/20231108065408.1861-1-justinjiang@vivo.com/ >> patch v2: >> https://lore.kernel.org/all/20231109073133.792-2-justinjiang@vivo.com/ >> patch v3: >> https://lore.kernel.org/all/20231110020840.1031-1-justinjiang@vivo.com/ > None of those are diffed against the current tree. You need to send a > patch that applies cleanly to either linux-next or current Linus head. > Not a patch on top of an earlier version of your patch. Modification has been made based on today's latest patch from linux-next, thanks. Patch: https://lore.kernel.org/all/20231113030343.1984-1-justinjiang@vivo.com/
diff --git a/mm/internal.h b/mm/internal.h index 98c14b16ce81..1d67c141902d 100755 --- a/mm/internal.h +++ b/mm/internal.h @@ -905,7 +905,6 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, #endif #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */ -#define ALLOC_PCPLIST 0x1000 /* Allocations from pcplist */ /* Flags that allow allocations below the min watermark. */ #define ALLOC_RESERVES (ALLOC_NON_BLOCK|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 67cec88164b1..3c84c3e3eeb0 100755 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2854,6 +2854,11 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, int batch = nr_pcp_alloc(pcp, zone, order); int alloced; + /* + * If pcplist is empty and alloc_flags is with ALLOC_HIGHATOMIC, + * it should alloc from buddy highatomic migrate freelist firstly + * to ensure quick and successful allocation. + */ if (alloc_flags & ALLOC_HIGHATOMIC) goto out; @@ -2925,8 +2930,8 @@ __no_sanitize_memory static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, - gfp_t gfp_flags, unsigned int *alloc_flags, - int migratetype) + gfp_t gfp_flags, unsigned int alloc_flags, + int migratetype, bool *highatomc_allocation) { struct page *page; @@ -2938,19 +2943,33 @@ struct page *rmqueue(struct zone *preferred_zone, if (likely(pcp_allowed_order(order))) { page = rmqueue_pcplist(preferred_zone, zone, order, - migratetype, *alloc_flags); - if (likely(page)) { - *alloc_flags |= ALLOC_PCPLIST; + migratetype, alloc_flags); + if (likely(page)) goto out; - } } - page = rmqueue_buddy(preferred_zone, zone, order, *alloc_flags, + page = rmqueue_buddy(preferred_zone, zone, order, alloc_flags, migratetype); + /* + * The high-order atomic allocation pageblock reserved conditions: + * + * If the high-order atomic allocation page is alloced from pcplist, + * the highatomic pageblock does not need to be reserved, which can + * void to migrate an increasing number of pages into buddy + * MIGRATE_HIGHATOMIC freelist and lead to an increasing risk of + * allocation failure on other buddy migrate freelists. + * + * If the high-order atomic allocation page is alloced from buddy + * highatomic migrate freelist, regardless of whether the allocation + * is successful or not, the highatomic pageblock can try to be + * reserved. + */ + if (unlikely(alloc_flags & ALLOC_HIGHATOMIC)) + *highatomc_allocation = true; out: /* Separate test+clear to avoid unnecessary atomics */ - if ((*alloc_flags & ALLOC_KSWAPD) && + if ((alloc_flags & ALLOC_KSWAPD) && unlikely(test_bit(ZONE_BOOSTED_WATERMARK, &zone->flags))) { clear_bit(ZONE_BOOSTED_WATERMARK, &zone->flags); wakeup_kswapd(zone, 0, 0, zone_idx(zone)); @@ -3218,6 +3237,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, struct pglist_data *last_pgdat = NULL; bool last_pgdat_dirty_ok = false; bool no_fallback; + bool highatomc_allocation = false; retry: /* @@ -3349,7 +3369,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, try_this_zone: page = rmqueue(ac->preferred_zoneref->zone, zone, order, - gfp_mask, &alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, &highatomc_allocation); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags); @@ -3357,8 +3377,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, * If this is a high-order atomic allocation then check * if the pageblock should be reserved for the future */ - if (unlikely(alloc_flags & ALLOC_HIGHATOMIC) && - unlikely(!(alloc_flags & ALLOC_PCPLIST))) + if (unlikely(highatomc_allocation)) reserve_highatomic_pageblock(page, zone); return page;