From patchwork Mon Nov 21 17:12:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Gondois X-Patchwork-Id: 23961 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1719510wrr; Mon, 21 Nov 2022 09:16:38 -0800 (PST) X-Google-Smtp-Source: AA0mqf5BYxVYb9bIvmD1Onegyi8Ji1j6lcyVf3QyKllRrkxLaFKa45rDtg9112FXj4gMt62LcCJJ X-Received: by 2002:aa7:8513:0:b0:563:3d81:7faf with SMTP id v19-20020aa78513000000b005633d817fafmr21237898pfn.45.1669050997881; Mon, 21 Nov 2022 09:16:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669050997; cv=none; d=google.com; s=arc-20160816; b=Vt3hAUlpaRJPI9IP17+UPxOFcV1xibxH5VoMoo2ehYUdFo4NabNUUWJZEicnA004a9 UvH8lUBAPMk31xb0tf6A82Q0KGCH86SLc59o+NqAtsmBsnFgPTNbKLh4GkV2xHYbe/Gw l8IHFHRxJA4zS5RThbsX36crfg9v6pzFQeVKq1qfZT3dInmIQWlc1nTroESrKxly95cY eFrcub+v+8CXL+HhFji0lkXBuQ0KQ65dMQiHRIyQQzvOqX3Z8q8GrFvSWT0ikDDo/y/b TOva7WzVeuNrBfIaFuFiTwHNDdZvVr5ZSO6RELJs+ccUCOO+pigPoAYmZ3pg2BD9vFd+ Ah0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=TEOOSEraparsXFshAYUORzRTqPtb9qfHtug5jgmtXF4=; b=Cpd0DDgnMxuCDKoI/Hfe8JrBf9RrXGhT40n3e5SWyjkkB+svG1pHPPj4MsgC8iSA+l fkhLUp5jM8thlWaFhpg6Q0qusmGQIAAwkMzg/7l8kkWxsEmOV3oBrzGo4IIINXAQ1QDw Z0vflN1MboIiaqO9b/F8lHtfcPN5vXHx9SB4Yu3Z1/PfX6xK+UJwRVlIiWI2A2OS8qNN uZl6sFucApXpDZDFEW6nnNqHZ4qZhyT9cqsxgaqyeObVk7Q8NRaUE4WJWqj6TANfUF1B UYswOtTDY+q2Qx5WYoU8gXbZVVgA2eWO1p8obnnk/S19T+pfk3z6YtbwsWBIgztTt044 XQog== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f19-20020a63f113000000b00476e845a78bsi11795367pgi.126.2022.11.21.09.16.23; Mon, 21 Nov 2022 09:16:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231522AbiKURPt (ORCPT + 99 others); Mon, 21 Nov 2022 12:15:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231330AbiKURPW (ORCPT ); Mon, 21 Nov 2022 12:15:22 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5125AD9070; Mon, 21 Nov 2022 09:13:30 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 413701FB; Mon, 21 Nov 2022 09:13:36 -0800 (PST) Received: from pierre123.arm.com (unknown [10.57.7.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1DCA73F587; Mon, 21 Nov 2022 09:13:24 -0800 (PST) From: Pierre Gondois To: linux-kernel@vger.kernel.org Cc: Ionela.Voinescu@arm.com, Rob.Herring@arm.com, Pierre Gondois , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Rafael J. Wysocki" , Len Brown , Sudeep Holla , Greg Kroah-Hartman , Jeremy Linton , Conor Dooley , Gavin Shan , Alexei Starovoitov , SeongJae Park , Jakub Kicinski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-acpi@vger.kernel.org Subject: [PATCH v2 5/5] arch_topology: Build cacheinfo from primary CPU Date: Mon, 21 Nov 2022 18:12:13 +0100 Message-Id: <20221121171217.3581004-6-pierre.gondois@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221121171217.3581004-1-pierre.gondois@arm.com> References: <20221121171217.3581004-1-pierre.gondois@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750126744837357513?= X-GMAIL-MSGID: =?utf-8?q?1750126819219797533?= commit 3fcbf1c77d08 ("arch_topology: Fix cache attributes detection in the CPU hotplug path") adds a call to detect_cache_attributes() to populate the cacheinfo before updating the siblings mask. detect_cache_attributes() allocates memory and can take the PPTT mutex (on ACPI platforms). On PREEMPT_RT kernels, on secondary CPUs, this triggers a: 'BUG: sleeping function called from invalid context' [1] as the code is executed with preemption and interrupts disabled. The primary CPU was previously storing the cache information using the now removed (struct cpu_topology).llc_id: commit 5b8dc787ce4a ("arch_topology: Drop LLC identifier stash from the CPU topology") allocate_cache_info() tries to build the cacheinfo from the primary CPU prior secondary CPUs boot, if the DT/ACPI description contains cache information. If allocate_cache_info() fails, then fallback to the current state for the cacheinfo allocation. [1] will be triggered in such case. When unplugging a CPU, the cacheinfo memory cannot be freed. If it was, then the memory would be allocated early by the re-plugged CPU and would trigger [1]. Note that populate_cache_leaves() might be called multiple times due to populate_leaves being moved up. This is required since detect_cache_attributes() might be called with per_cpu_cacheinfo(cpu) being allocated but not populated. [1]: [ 7.560791] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 [ 7.560794] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/111 [ 7.560796] preempt_count: 1, expected: 0 [ 7.560797] RCU nest depth: 1, expected: 1 [ 7.560799] 3 locks held by swapper/111/0: [ 7.560800] #0: ffff403e406cae98 (&pcp->lock){+.+.}-{3:3}, at: get_page_from_freelist+0x218/0x12c8 [ 7.560811] #1: ffffc5f8ed09f8e8 (rcu_read_lock){....}-{1:3}, at: rt_spin_trylock+0x48/0xf0 [ 7.560820] #2: ffff403f400b4fd8 (&zone->lock){+.+.}-{3:3}, at: rmqueue_bulk+0x64/0xa80 [ 7.560824] irq event stamp: 0 [ 7.560825] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 7.560827] hardirqs last disabled at (0): [] copy_process+0x5dc/0x1ab8 [ 7.560830] softirqs last enabled at (0): [] copy_process+0x5dc/0x1ab8 [ 7.560833] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 7.560834] Preemption disabled at: [ 7.560835] [] migrate_enable+0x30/0x130 [ 7.560838] CPU: 111 PID: 0 Comm: swapper/111 Tainted: G W 6.0.0-rc4-rt6-[...] [ 7.560841] Call trace: [...] [ 7.560870] __kmalloc+0xbc/0x1e8 [ 7.560873] detect_cache_attributes+0x2d4/0x5f0 [ 7.560876] update_siblings_masks+0x30/0x368 [ 7.560880] store_cpu_topology+0x78/0xb8 [ 7.560883] secondary_start_kernel+0xd0/0x198 [ 7.560885] __secondary_switched+0xb0/0xb4 Signed-off-by: Pierre Gondois Reviewed-by: Sudeep Holla --- drivers/base/arch_topology.c | 10 ++++++- drivers/base/cacheinfo.c | 56 +++++++++++++++++++++++++++++------- include/linux/cacheinfo.h | 1 + 3 files changed, 55 insertions(+), 12 deletions(-) diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index e7d6e6657ffa..54be88f658a0 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -825,7 +825,7 @@ __weak int __init parse_acpi_topology(void) #if defined(CONFIG_ARM64) || defined(CONFIG_RISCV) void __init init_cpu_topology(void) { - int ret; + int cpu, ret; reset_cpu_topology(); ret = parse_acpi_topology(); @@ -840,6 +840,14 @@ void __init init_cpu_topology(void) reset_cpu_topology(); return; } + + for_each_possible_cpu(cpu) { + ret = allocate_cache_info(cpu); + if (ret) { + pr_err("Early cacheinfo failed, ret = %d\n", ret); + break; + } + } } void store_cpu_topology(unsigned int cpuid) diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c index 6f6cd120c4f1..429a55e7c5e2 100644 --- a/drivers/base/cacheinfo.c +++ b/drivers/base/cacheinfo.c @@ -371,10 +371,6 @@ static void free_cache_attributes(unsigned int cpu) return; cache_shared_cpu_map_remove(cpu); - - kfree(per_cpu_cacheinfo(cpu)); - per_cpu_cacheinfo(cpu) = NULL; - cache_leaves(cpu) = 0; } int __weak init_cache_level(unsigned int cpu) @@ -387,18 +383,56 @@ int __weak populate_cache_leaves(unsigned int cpu) return -ENOENT; } +int allocate_cache_info(unsigned int cpu) +{ + struct cpu_cacheinfo *this_cpu_ci; + unsigned int levels, split_levels; + int ret; + + if (acpi_disabled) { + ret = init_of_cache_level(cpu); + if (ret < 0) + return ret; + } else { + ret = acpi_get_cache_info(cpu, &levels, &split_levels); + if (ret < 0) + return ret; + + this_cpu_ci = get_cpu_cacheinfo(cpu); + this_cpu_ci->num_levels = levels; + /* + * This assumes that: + * - there cannot be any split caches (data/instruction) + * above a unified cache + * - data/instruction caches come by pair + */ + this_cpu_ci->num_leaves = levels + split_levels; + } + if (!cache_leaves(cpu)) + return -ENOENT; + + per_cpu_cacheinfo(cpu) = kcalloc(cache_leaves(cpu), + sizeof(struct cacheinfo), GFP_ATOMIC); + if (per_cpu_cacheinfo(cpu) == NULL) { + cache_leaves(cpu) = 0; + return -ENOMEM; + } + + return ret; +} + int detect_cache_attributes(unsigned int cpu) { int ret; - /* Since early detection of the cacheinfo is allowed via this - * function and this also gets called as CPU hotplug callbacks via - * cacheinfo_cpu_online, the initialisation can be skipped and only - * CPU maps can be updated as the CPU online status would be update - * if called via cacheinfo_cpu_online path. + /* Since early initialization/allocation of the cacheinfo is allowed + * via allocate_cache_info() and this also gets called as CPU hotplug + * callbacks via cacheinfo_cpu_online, the init/alloc can be skipped + * as it will happen only once (the cacheinfo memory is never freed). + * Just populate the cacheinfo. */ if (per_cpu_cacheinfo(cpu)) - goto update_cpu_map; + goto populate_leaves; if (init_cache_level(cpu) || !cache_leaves(cpu)) return -ENOENT; @@ -410,6 +444,7 @@ int detect_cache_attributes(unsigned int cpu) return -ENOMEM; } +populate_leaves: /* * populate_cache_leaves() may completely setup the cache leaves and * shared_cpu_map or it may leave it partially setup. @@ -418,7 +453,6 @@ int detect_cache_attributes(unsigned int cpu) if (ret) goto free_ci; -update_cpu_map: /* * For systems using DT for cache hierarchy, fw_token * and shared_cpu_map will be set up here only if they are diff --git a/include/linux/cacheinfo.h b/include/linux/cacheinfo.h index f992d81d211f..7d390806b788 100644 --- a/include/linux/cacheinfo.h +++ b/include/linux/cacheinfo.h @@ -85,6 +85,7 @@ int populate_cache_leaves(unsigned int cpu); int cache_setup_acpi(unsigned int cpu); bool last_level_cache_is_valid(unsigned int cpu); bool last_level_cache_is_shared(unsigned int cpu_x, unsigned int cpu_y); +int allocate_cache_info(unsigned int cpu); int detect_cache_attributes(unsigned int cpu); #ifndef CONFIG_ACPI_PPTT /*