From patchwork Wed Sep 27 01:37:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yajun Deng X-Patchwork-Id: 145105 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp2327181vqu; Tue, 26 Sep 2023 19:16:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHzFNBgGPkl9bovlj3ngtMsgEHp08aHCl6KF7kqvys3j/20/Tw4kAs7yxfGUn8PHfoJHvAG X-Received: by 2002:a05:6a00:1988:b0:68e:2eed:5ab0 with SMTP id d8-20020a056a00198800b0068e2eed5ab0mr866294pfl.7.1695780980510; Tue, 26 Sep 2023 19:16:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695780980; cv=none; d=google.com; s=arc-20160816; b=FUdOfniM857YO0u6tncPXWoW4HAeOH5UoLNwKhsNmhdVztJtYQCvE1/p4lNmDkNPOy V9gmB5/qPwpPu7EHjACu0JMLsed7FMdgbQ5he/MtBXWKEvmhrwSdvWgbyUal2jiCEfN0 uMcnLtM/Jr36U3uVXqjtr3/AGj3KJg8JG72SqFN1LMM7XcOq7SGq+pM5Su6EVvTzdwil qfQX4FQddjWUnyyQIOhji5Ur6b5jbyItpdR0q2pyHSoZ9oXfW9C126GZXob66ycLRqky NKgmmFnlUfpaacXXQSJOvuBVIl3yWVM+WocgVHC5QSBFyRu6GxJnwCgcaE+jNqlnwbjy i9/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Nz6caoQZVPJenPUxtn5KQteC75zflIiT4pdokHAsttw=; fh=QZU3guN9hBCNbG3XWCsoVTJE3Km/GRJW/frPYUuNAVA=; b=a+Hfx1pXkqT8WfqWvY+39h8xORt5Z/1Nery75AAh1YRzJsNkG47/34gUNZ17o++PlU bwg+eioNT2HaIKfMhegL+TaaqhkmwbpuNPxSECBc0N1j8jXAjcw4SJ7L1K7W61Y1t/Xl HbGPWBN17o0ogtGNG9L8CU3tvl+3nsG8TcuQtDkctrQ57m1KLc/NEhv4Zbzdj6XjawUm vbiQNdInMBlzS7AeipZMkWFD5YI9na1EVKUsSgg0N18YN6joK/kC5k/O2liMAiq/iRLv /Jj8IKCwidcRrHVU5Vq7QPv7rd2/XW3tKhCXYsrn5ldKDMLV5zfAJn2aDw5OH1LUEd5x fovQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=F+C2wGLK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id ei16-20020a056a0080d000b00690c1e51cdfsi13327828pfb.188.2023.09.26.19.16.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Sep 2023 19:16:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=F+C2wGLK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 5941E8271412; Tue, 26 Sep 2023 19:14:51 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234563AbjI0COq (ORCPT + 28 others); Tue, 26 Sep 2023 22:14:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235074AbjI0CMp (ORCPT ); Tue, 26 Sep 2023 22:12:45 -0400 Received: from out-208.mta1.migadu.com (out-208.mta1.migadu.com [95.215.58.208]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C9049C for ; Tue, 26 Sep 2023 18:38:08 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1695778686; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Nz6caoQZVPJenPUxtn5KQteC75zflIiT4pdokHAsttw=; b=F+C2wGLKKtN0ueSSw//MQ/ZSjyS6aCRiZyJ6YlOb1fuekh72Nu5ff5Z6H39On8vrOt6mxj rhBzAklP1gljRJdrJg3BZS4d6PV4hLD8FHKRZo0RTFybBUTBwUnNJ3lcX5+4e4TwLKRzZ2 t5Kg4LL9wEzupxrNacEixn6KSw4KXgo= From: Yajun Deng To: rppt@kernel.org, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yajun Deng Subject: [PATCH] memblock: don't run loop in memblock_add_range() twice Date: Wed, 27 Sep 2023 09:37:52 +0800 Message-Id: <20230927013752.2515238-1-yajun.deng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 26 Sep 2023 19:14:51 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778155237270420071 X-GMAIL-MSGID: 1778155237270420071 There is round twice in memblock_add_range(). The first counts the number of regions needed to accommodate the new area. The second actually inserts them. But the first round isn't really needed, we just need to check the counts before inserting them. Check the count before calling memblock_insert_region(). If the count is equal to the maximum value, it needs to resize the array. Otherwise, insert it directly. To avoid nested calls to memblock_add_range(), we need to call memblock_reserve() out of memblock_double_array(). Signed-off-by: Yajun Deng --- mm/memblock.c | 117 ++++++++++++++++++++++++-------------------------- 1 file changed, 57 insertions(+), 60 deletions(-) diff --git a/mm/memblock.c b/mm/memblock.c index 5a88d6d24d79..3f44c84f5d0b 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -400,6 +400,8 @@ void __init memblock_discard(void) * @type: memblock type of the regions array being doubled * @new_area_start: starting address of memory range to avoid overlap with * @new_area_size: size of memory range to avoid overlap with + * @new_reserve_base: starting address of new array + * @new_reserve_size: size of new array * * Double the size of the @type regions array. If memblock is being used to * allocate memory for a new reserved regions array and there is a previously @@ -412,7 +414,9 @@ void __init memblock_discard(void) */ static int __init_memblock memblock_double_array(struct memblock_type *type, phys_addr_t new_area_start, - phys_addr_t new_area_size) + phys_addr_t new_area_size, + phys_addr_t *new_reserve_base, + phys_addr_t *new_reserve_size) { struct memblock_region *new_array, *old_array; phys_addr_t old_alloc_size, new_alloc_size; @@ -490,11 +494,13 @@ static int __init_memblock memblock_double_array(struct memblock_type *type, memblock_free(old_array, old_alloc_size); /* - * Reserve the new array if that comes from the memblock. Otherwise, we - * needn't do it + * Keep the address and size if that comes from the memblock. Otherwise, + * we needn't do it. */ - if (!use_slab) - BUG_ON(memblock_reserve(addr, new_alloc_size)); + if (!use_slab) { + *new_reserve_base = addr; + *new_reserve_size = new_alloc_size; + } /* Update slab flag */ *in_slab = use_slab; @@ -588,11 +594,12 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, phys_addr_t base, phys_addr_t size, int nid, enum memblock_flags flags) { - bool insert = false; phys_addr_t obase = base; phys_addr_t end = base + memblock_cap_size(base, &size); - int idx, nr_new, start_rgn = -1, end_rgn; + phys_addr_t new_base = 0, new_size; + int idx, start_rgn = -1, end_rgn; struct memblock_region *rgn; + unsigned long ocnt = type->cnt; if (!size) return 0; @@ -608,25 +615,6 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, return 0; } - /* - * The worst case is when new range overlaps all existing regions, - * then we'll need type->cnt + 1 empty regions in @type. So if - * type->cnt * 2 + 1 is less than or equal to type->max, we know - * that there is enough empty regions in @type, and we can insert - * regions directly. - */ - if (type->cnt * 2 + 1 <= type->max) - insert = true; - -repeat: - /* - * The following is executed twice. Once with %false @insert and - * then with %true. The first counts the number of regions needed - * to accommodate the new area. The second actually inserts them. - */ - base = obase; - nr_new = 0; - for_each_memblock_type(idx, type, rgn) { phys_addr_t rbase = rgn->base; phys_addr_t rend = rbase + rgn->size; @@ -644,15 +632,23 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, WARN_ON(nid != memblock_get_region_node(rgn)); #endif WARN_ON(flags != rgn->flags); - nr_new++; - if (insert) { - if (start_rgn == -1) - start_rgn = idx; - end_rgn = idx + 1; - memblock_insert_region(type, idx++, base, - rbase - base, nid, - flags); - } + + /* + * If type->cnt is equal to type->max, it means there's + * not enough empty region and the array needs to be + * resized. Otherwise, insert it directly. + */ + if ((type->cnt == type->max) && + memblock_double_array(type, obase, size, + &new_base, &new_size)) + return -ENOMEM; + + if (start_rgn == -1) + start_rgn = idx; + end_rgn = idx + 1; + memblock_insert_region(type, idx++, base, + rbase - base, nid, + flags); } /* area below @rend is dealt with, forget about it */ base = min(rend, end); @@ -660,33 +656,28 @@ static int __init_memblock memblock_add_range(struct memblock_type *type, /* insert the remaining portion */ if (base < end) { - nr_new++; - if (insert) { - if (start_rgn == -1) - start_rgn = idx; - end_rgn = idx + 1; - memblock_insert_region(type, idx, base, end - base, - nid, flags); - } + if ((type->cnt == type->max) && + memblock_double_array(type, obase, size, + &new_base, &new_size)) + return -ENOMEM; + + if (start_rgn == -1) + start_rgn = idx; + end_rgn = idx + 1; + memblock_insert_region(type, idx, base, end - base, + nid, flags); } - if (!nr_new) + if (ocnt == type->cnt) return 0; - /* - * If this was the first round, resize array and repeat for actual - * insertions; otherwise, merge and return. - */ - if (!insert) { - while (type->cnt + nr_new > type->max) - if (memblock_double_array(type, obase, size) < 0) - return -ENOMEM; - insert = true; - goto repeat; - } else { - memblock_merge_regions(type, start_rgn, end_rgn); - return 0; - } + memblock_merge_regions(type, start_rgn, end_rgn); + + /* Reserve the new array */ + if (new_base) + memblock_reserve(new_base, new_size); + + return 0; } /** @@ -755,6 +746,7 @@ static int __init_memblock memblock_isolate_range(struct memblock_type *type, int *start_rgn, int *end_rgn) { phys_addr_t end = base + memblock_cap_size(base, &size); + phys_addr_t new_base = 0, new_size; int idx; struct memblock_region *rgn; @@ -764,10 +756,15 @@ static int __init_memblock memblock_isolate_range(struct memblock_type *type, return 0; /* we'll create at most two more regions */ - while (type->cnt + 2 > type->max) - if (memblock_double_array(type, base, size) < 0) + if (type->cnt + 2 > type->max) { + if (memblock_double_array(type, base, size, + &new_base, &new_size)) return -ENOMEM; + if (new_base) + memblock_reserve(new_base, new_size); + } + for_each_memblock_type(idx, type, rgn) { phys_addr_t rbase = rgn->base; phys_addr_t rend = rbase + rgn->size;