Message ID | 20221108110424.166896-1-pierre.gondois@arm.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2632802wru; Tue, 8 Nov 2022 03:06:48 -0800 (PST) X-Google-Smtp-Source: AMsMyM6kIf1Ig3nWKam3CUlPwdxa1qO7P4utZUv86hOXyimTbLCzChHZmMItUSl/zLPaxrcF/t9y X-Received: by 2002:a17:907:3f24:b0:7ad:dcf7:c48 with SMTP id hq36-20020a1709073f2400b007addcf70c48mr43325269ejc.547.1667905608770; Tue, 08 Nov 2022 03:06:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1667905608; cv=none; d=google.com; s=arc-20160816; b=mZO6ns8/q6zU5TKyl+K9TtnFGVY7JQCSAP0tKV4QakszdOqaXiFqRLDLylUelsLEMN J29MnTEOlPLhAXQRtZdf5tIeRvo5ST+wXfWz/IgzDS3srINZD9S0D9k1cu3jrKQRoiiO 3LJKBSul8DT4IVFxRb1Rx3I+E39+/oIQQcAeTI3eW3ELywgxF7eEqZLNc2Wp1bZoBQJ8 oxCyZyuJ9k8ZHMkD2DGqhkzN80nDYiCoNUrYvGex/G79F/u0VyFfBw8Es4syhm0hQiwF LalvXz0UeGvLdy9f9iHlXTU0u5Rci3t1nIeHVWIImVkBJPuNRnqW8L8hd/2XDSfSZuvU gojQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=Y5JNdHCNxwWuShMPJ0KEzIyG7bbZFk3y1QTGZiO+lcQ=; b=RUKsLQD43FuZWlbOBYjEPg5zKhyQzH1pZMOxjee+d3+tTGdxmCWzvHngyTyc3r33zU 5ePzJ5IYL/1urEV1YrzFfU5FuIj3nzz5/6nLTDaxjl1fdkgTaq4knCrRMCoUonBQoFj6 5a/cgUXnrzlhYHIawYBSN3wmz9llBDNk47GFYp76anQV6oy5vCILOrK6+15XBes7q6pE +KUSuX+wAn8qHMfucttJLcACvkb9AyzCvg5JBEiecndjUBVEEl8VvFD/hYqdQieFXD46 eVDbfK3OaR4ElgVoVBAO8I271NzjvDuXrToFFttN9ZPucoe7sTBx+CMGJ88ZxxHfmfDT XFaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s20-20020aa7d794000000b004570e4887desi10440592edq.437.2022.11.08.03.06.24; Tue, 08 Nov 2022 03:06:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233844AbiKHLGC (ORCPT <rfc822;hjfbswb@gmail.com> + 99 others); Tue, 8 Nov 2022 06:06:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233882AbiKHLGA (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 8 Nov 2022 06:06:00 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 20BAF47323; Tue, 8 Nov 2022 03:05:58 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 007941FB; Tue, 8 Nov 2022 03:06:04 -0800 (PST) Received: from pierre123.arm.com (unknown [10.57.5.33]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 26E713F534; Tue, 8 Nov 2022 03:05:55 -0800 (PST) From: Pierre Gondois <pierre.gondois@arm.com> To: linux-kernel@vger.kernel.org Cc: Pierre Gondois <pierre.gondois@arm.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, Albert Ou <aou@eecs.berkeley.edu>, "Rafael J. Wysocki" <rafael@kernel.org>, Len Brown <lenb@kernel.org>, Sudeep Holla <sudeep.holla@arm.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Gavin Shan <gshan@redhat.com>, SeongJae Park <sj@kernel.org>, Jakub Kicinski <kuba@kernel.org>, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-acpi@vger.kernel.org Subject: [PATCH 0/5] arch_topology: Build cacheinfo from primary CPU Date: Tue, 8 Nov 2022 12:04:16 +0100 Message-Id: <20221108110424.166896-1-pierre.gondois@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748925791984863350?= X-GMAIL-MSGID: =?utf-8?q?1748925791984863350?= |
Series |
arch_topology: Build cacheinfo from primary CPU
|
|
Message
Pierre Gondois
Nov. 8, 2022, 11:04 a.m. UTC
[1] and [2] build the CPU topology from the cacheinfo information for both DT/ACPI based systems and remove (struct cpu_topology).llc_id which was used by ACPI only. Creating the cacheinfo for secondary CPUs is done during early boot. Preemption and interrupts are disabled at this stage. On PREEMPT_RT kernels, allocating memory (and parsing the PPTT table for ACPI based systems) triggers a: 'BUG: sleeping function called from invalid context' [4] To prevent this bug, allocate the cacheinfo from the primary CPU when preemption and interrupts are enabled and before booting secondary CPUs. The cache levels/leaves are computed from DT/ACPI PPTT information only, without relying on the arm64 CLIDR_EL1 register. If no cache information is found in the DT/ACPI PPTT, then fallback to the current state, triggering [4] on PREEMPT_RT kernels. Patches to update the arm64 device trees that have incomplete cacheinfo (mostly for missing the 'cache-level' or 'cache-unified' property) have been sent at [3]. Tested platforms: - ACPI + PPTT: Ampere Altra, Ampere eMAG, Cavium ThunderX2, Kunpeng 920, Juno-r2 - DT: rb5, db845c, Juno-r2 [1] https://lore.kernel.org/all/20220704101605.1318280-1-sudeep.holla@arm.com/ [2] https://lore.kernel.org/all/20220720-arch_topo_fixes-v3-0-43d696288e84@arm.com/ [3] https://lore.kernel.org/all/20221107155825.1644604-1-pierre.gondois@arm.com/ [4] On an Ampere Altra, with PREEMPT_RT kernel based on v6.0.0-rc4: [ 7.560791] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 [ 7.560794] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/111 [ 7.560796] preempt_count: 1, expected: 0 [ 7.560797] RCU nest depth: 1, expected: 1 [ 7.560799] 3 locks held by swapper/111/0: [ 7.560800] #0: ffff403e406cae98 (&pcp->lock){+.+.}-{3:3}, at: get_page_from_freelist+0x218/0x12c8 [ 7.560811] #1: ffffc5f8ed09f8e8 (rcu_read_lock){....}-{1:3}, at: rt_spin_trylock+0x48/0xf0 [ 7.560820] #2: ffff403f400b4fd8 (&zone->lock){+.+.}-{3:3}, at: rmqueue_bulk+0x64/0xa80 [ 7.560824] irq event stamp: 0 [ 7.560825] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 7.560827] hardirqs last disabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 [ 7.560830] softirqs last enabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 [ 7.560833] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 7.560834] Preemption disabled at: [ 7.560835] [<ffffc5f8e9fd3c28>] migrate_enable+0x30/0x130 [ 7.560838] CPU: 111 PID: 0 Comm: swapper/111 Tainted: G W 6.0.0-rc4-[...] [ 7.560841] Call trace: [...] [ 7.560870] __kmalloc+0xbc/0x1e8 [ 7.560873] detect_cache_attributes+0x2d4/0x5f0 [ 7.560876] update_siblings_masks+0x30/0x368 [ 7.560880] store_cpu_topology+0x78/0xb8 [ 7.560883] secondary_start_kernel+0xd0/0x198 [ 7.560885] __secondary_switched+0xb0/0xb4 Pierre Gondois (5): cacheinfo: Use riscv's init_cache_level() as generic OF implem cacheinfo: Return error code in init_of_cache_level() ACPI: PPTT: Remove acpi_find_cache_levels() ACPI: PPTT: Update acpi_find_last_cache_level() to acpi_get_cache_info() arch_topology: Build cacheinfo from primary CPU arch/arm64/kernel/cacheinfo.c | 9 ++- arch/riscv/kernel/cacheinfo.c | 39 +------------ drivers/acpi/pptt.c | 86 +++++++++++++++++------------ drivers/base/arch_topology.c | 10 +++- drivers/base/cacheinfo.c | 101 ++++++++++++++++++++++++++++++---- include/linux/cacheinfo.h | 10 +++- 6 files changed, 164 insertions(+), 91 deletions(-)
Comments
+ Rob Herring + Jeremy Linton On 11/8/22 12:04, Pierre Gondois wrote: > [1] and [2] build the CPU topology from the cacheinfo information for > both DT/ACPI based systems and remove (struct cpu_topology).llc_id > which was used by ACPI only. > > Creating the cacheinfo for secondary CPUs is done during early boot. > Preemption and interrupts are disabled at this stage. On PREEMPT_RT > kernels, allocating memory (and parsing the PPTT table for ACPI based > systems) triggers a: > 'BUG: sleeping function called from invalid context' [4] > > To prevent this bug, allocate the cacheinfo from the primary CPU when > preemption and interrupts are enabled and before booting secondary > CPUs. The cache levels/leaves are computed from DT/ACPI PPTT information > only, without relying on the arm64 CLIDR_EL1 register. > If no cache information is found in the DT/ACPI PPTT, then fallback > to the current state, triggering [4] on PREEMPT_RT kernels. > > Patches to update the arm64 device trees that have incomplete cacheinfo > (mostly for missing the 'cache-level' or 'cache-unified' property) > have been sent at [3]. > > Tested platforms: > - ACPI + PPTT: Ampere Altra, Ampere eMAG, Cavium ThunderX2, > Kunpeng 920, Juno-r2 > - DT: rb5, db845c, Juno-r2 > > [1] https://lore.kernel.org/all/20220704101605.1318280-1-sudeep.holla@arm.com/ > [2] https://lore.kernel.org/all/20220720-arch_topo_fixes-v3-0-43d696288e84@arm.com/ > [3] https://lore.kernel.org/all/20221107155825.1644604-1-pierre.gondois@arm.com/ > [4] On an Ampere Altra, with PREEMPT_RT kernel based on v6.0.0-rc4: > > > [ 7.560791] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 > [ 7.560794] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/111 > [ 7.560796] preempt_count: 1, expected: 0 > [ 7.560797] RCU nest depth: 1, expected: 1 > [ 7.560799] 3 locks held by swapper/111/0: > [ 7.560800] #0: ffff403e406cae98 (&pcp->lock){+.+.}-{3:3}, at: get_page_from_freelist+0x218/0x12c8 > [ 7.560811] #1: ffffc5f8ed09f8e8 (rcu_read_lock){....}-{1:3}, at: rt_spin_trylock+0x48/0xf0 > [ 7.560820] #2: ffff403f400b4fd8 (&zone->lock){+.+.}-{3:3}, at: rmqueue_bulk+0x64/0xa80 > [ 7.560824] irq event stamp: 0 > [ 7.560825] hardirqs last enabled at (0): [<0000000000000000>] 0x0 > [ 7.560827] hardirqs last disabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 > [ 7.560830] softirqs last enabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 > [ 7.560833] softirqs last disabled at (0): [<0000000000000000>] 0x0 > [ 7.560834] Preemption disabled at: > [ 7.560835] [<ffffc5f8e9fd3c28>] migrate_enable+0x30/0x130 > [ 7.560838] CPU: 111 PID: 0 Comm: swapper/111 Tainted: G W 6.0.0-rc4-[...] > [ 7.560841] Call trace: > [...] > [ 7.560870] __kmalloc+0xbc/0x1e8 > [ 7.560873] detect_cache_attributes+0x2d4/0x5f0 > [ 7.560876] update_siblings_masks+0x30/0x368 > [ 7.560880] store_cpu_topology+0x78/0xb8 > [ 7.560883] secondary_start_kernel+0xd0/0x198 > [ 7.560885] __secondary_switched+0xb0/0xb4 > > Pierre Gondois (5): > cacheinfo: Use riscv's init_cache_level() as generic OF implem > cacheinfo: Return error code in init_of_cache_level() > ACPI: PPTT: Remove acpi_find_cache_levels() > ACPI: PPTT: Update acpi_find_last_cache_level() to > acpi_get_cache_info() > arch_topology: Build cacheinfo from primary CPU > > arch/arm64/kernel/cacheinfo.c | 9 ++- > arch/riscv/kernel/cacheinfo.c | 39 +------------ > drivers/acpi/pptt.c | 86 +++++++++++++++++------------ > drivers/base/arch_topology.c | 10 +++- > drivers/base/cacheinfo.c | 101 ++++++++++++++++++++++++++++++---- > include/linux/cacheinfo.h | 10 +++- > 6 files changed, 164 insertions(+), 91 deletions(-) >
On Tue, Nov 08, 2022 at 12:04:16PM +0100, Pierre Gondois wrote: > [1] and [2] build the CPU topology from the cacheinfo information for > both DT/ACPI based systems and remove (struct cpu_topology).llc_id > which was used by ACPI only. > > Creating the cacheinfo for secondary CPUs is done during early boot. > Preemption and interrupts are disabled at this stage. On PREEMPT_RT > kernels, allocating memory (and parsing the PPTT table for ACPI based > systems) triggers a: > 'BUG: sleeping function called from invalid context' [4] > > To prevent this bug, allocate the cacheinfo from the primary CPU when > preemption and interrupts are enabled and before booting secondary > CPUs. The cache levels/leaves are computed from DT/ACPI PPTT information > only, without relying on the arm64 CLIDR_EL1 register. > If no cache information is found in the DT/ACPI PPTT, then fallback > to the current state, triggering [4] on PREEMPT_RT kernels. > > Patches to update the arm64 device trees that have incomplete cacheinfo > (mostly for missing the 'cache-level' or 'cache-unified' property) > have been sent at [3]. FWIW checkpatch whinges a bit at your final two patches [a]. The RISC-V code movement looks okay though, no new warnings and the like. [a] - https://patchwork.kernel.org/project/linux-riscv/list/?series=693175 > > Tested platforms: > - ACPI + PPTT: Ampere Altra, Ampere eMAG, Cavium ThunderX2, > Kunpeng 920, Juno-r2 > - DT: rb5, db845c, Juno-r2 > > [1] https://lore.kernel.org/all/20220704101605.1318280-1-sudeep.holla@arm.com/ > [2] https://lore.kernel.org/all/20220720-arch_topo_fixes-v3-0-43d696288e84@arm.com/ > [3] https://lore.kernel.org/all/20221107155825.1644604-1-pierre.gondois@arm.com/ > [4] On an Ampere Altra, with PREEMPT_RT kernel based on v6.0.0-rc4: > > > [ 7.560791] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 > [ 7.560794] in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/111 > [ 7.560796] preempt_count: 1, expected: 0 > [ 7.560797] RCU nest depth: 1, expected: 1 > [ 7.560799] 3 locks held by swapper/111/0: > [ 7.560800] #0: ffff403e406cae98 (&pcp->lock){+.+.}-{3:3}, at: get_page_from_freelist+0x218/0x12c8 > [ 7.560811] #1: ffffc5f8ed09f8e8 (rcu_read_lock){....}-{1:3}, at: rt_spin_trylock+0x48/0xf0 > [ 7.560820] #2: ffff403f400b4fd8 (&zone->lock){+.+.}-{3:3}, at: rmqueue_bulk+0x64/0xa80 > [ 7.560824] irq event stamp: 0 > [ 7.560825] hardirqs last enabled at (0): [<0000000000000000>] 0x0 > [ 7.560827] hardirqs last disabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 > [ 7.560830] softirqs last enabled at (0): [<ffffc5f8e9f7d594>] copy_process+0x5dc/0x1ab8 > [ 7.560833] softirqs last disabled at (0): [<0000000000000000>] 0x0 > [ 7.560834] Preemption disabled at: > [ 7.560835] [<ffffc5f8e9fd3c28>] migrate_enable+0x30/0x130 > [ 7.560838] CPU: 111 PID: 0 Comm: swapper/111 Tainted: G W 6.0.0-rc4-[...] > [ 7.560841] Call trace: > [...] > [ 7.560870] __kmalloc+0xbc/0x1e8 > [ 7.560873] detect_cache_attributes+0x2d4/0x5f0 > [ 7.560876] update_siblings_masks+0x30/0x368 > [ 7.560880] store_cpu_topology+0x78/0xb8 > [ 7.560883] secondary_start_kernel+0xd0/0x198 > [ 7.560885] __secondary_switched+0xb0/0xb4 > > Pierre Gondois (5): > cacheinfo: Use riscv's init_cache_level() as generic OF implem > cacheinfo: Return error code in init_of_cache_level() > ACPI: PPTT: Remove acpi_find_cache_levels() > ACPI: PPTT: Update acpi_find_last_cache_level() to > acpi_get_cache_info() > arch_topology: Build cacheinfo from primary CPU > > arch/arm64/kernel/cacheinfo.c | 9 ++- > arch/riscv/kernel/cacheinfo.c | 39 +------------ > drivers/acpi/pptt.c | 86 +++++++++++++++++------------ > drivers/base/arch_topology.c | 10 +++- > drivers/base/cacheinfo.c | 101 ++++++++++++++++++++++++++++++---- > include/linux/cacheinfo.h | 10 +++- > 6 files changed, 164 insertions(+), 91 deletions(-) > > -- > 2.25.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel