From patchwork Tue Feb 7 23:04:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 54110 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3128688wrn; Tue, 7 Feb 2023 15:09:08 -0800 (PST) X-Google-Smtp-Source: AK7set+cByI3ZRWD7C/c5hzDzGqN7BY49pfq7IPyfEWIc0qd+FEzbTmBTm5MTHhHcwvldvhKk78X X-Received: by 2002:a17:906:2ed4:b0:888:a32d:b50 with SMTP id s20-20020a1709062ed400b00888a32d0b50mr5090203eji.40.1675811348273; Tue, 07 Feb 2023 15:09:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675811348; cv=none; d=google.com; s=arc-20160816; b=Tdknk5VqYMemzjIlm19IDOTeqxsgmOaSgEamXd2qxvYheJevVg+PpzN3qRApF0VRw2 LCQdPgDZJcI3gXbAeBFo7lZf5sIfcID982j3jevVBkqIonkETYqGL7wih7HkF6BGeCFT 7vlLz7l51SbGRsAe17nbHVsJ9NbJ+2RWay7CtPNtuorTjoHGClOTvx5yhQROUr0owMw3 fvnWJwr3XRLYmSWS/vygf7p4l7jTjJ0pMb7gl3dSyRqqTkjq+2lwcSXvdj6ZBoKBD5Kb RXtOs9pOvbQw2I13ThsgnMZYiMJTljJD5ysEnfKBZH62uZANarfj+4L59NGwfcLQHrvG SPrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KVqUJYOx2vTuosUua8hJoGt0geRYeRPq5H8k23kPkBg=; b=WDY33luL/x44xpf2GwVV21BGmz5fsK9129TGEkzX9mNiLJvnvh2BPNj/BCdmyO1bmS TppSxra16/vq7/ApjbObz0eiE2tngDjQD2LEW9tQuSPOZimAkSfASC4gCQEagOZRLeRC +vXykXKTXOrxr0jfJdmhfATYWydwRR4AxfLx1/jaVY7OS8R+9sZk5Tqd62+e1uYcsEmW nnFqK4kAKrdDOlzdb6NZ6m7cS56BBJJ8Q7aAzG6iNs6+2pXVzWKbv0vNIqVgYd3kzt7l w4v26vVgs8RaB/WqoPwttFVMl8oH9tvhSaPK0EZeX1qham1LrO3FvfTZAIjNJrwLqKAs MHpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="K/XKPjBx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g16-20020a1709065d1000b008778924dd6esi1273765ejt.958.2023.02.07.15.08.44; Tue, 07 Feb 2023 15:09:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="K/XKPjBx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229709AbjBGXGp (ORCPT + 99 others); Tue, 7 Feb 2023 18:06:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230224AbjBGXF2 (ORCPT ); Tue, 7 Feb 2023 18:05:28 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7948D40BC5 for ; Tue, 7 Feb 2023 15:04:49 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id m16-20020a05600c3b1000b003dc4050c94aso196937wms.4 for ; Tue, 07 Feb 2023 15:04:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KVqUJYOx2vTuosUua8hJoGt0geRYeRPq5H8k23kPkBg=; b=K/XKPjBxiVvpGnCBsaHZtP2MmfmWYWpHG1S9ROqPdhk3ZU25g/JOlC+1MkQQErcuGq aONBfjQdDeEo8jMzDH3KrgKg2XEZOAhYuB17+A2g8pVL3/ADw6KQNGnoisVnwPqJ5SFi UXa0Xf/0IgntZAtCY3uyoJA/fMaeKNtZEhGCWxIUnlczw8Gp7HPlvDIeMFGDNVO7buQ8 lhMtml6oj25JiR4LpJ2OdEqpI8kxrAi0e6fDTRKQpWZi6OdDen2rlSYuhPvBunudbzXZ Vg0c59FaCOlEtUptoe12uB9Os165CWXPoSJhwUQ+94sebZ/KKvWoeAGea+d7zhHC1TTl Pdbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KVqUJYOx2vTuosUua8hJoGt0geRYeRPq5H8k23kPkBg=; b=azjYey3qfTNP7vPltG2TqkOUxOOK4FIx1esDizggc0/P4fnh/jpj8EfuHsaBwF/Me9 MKeZ/Ep/hAukdFC6g2BvYzwTdBsfsY7e9YX0XW9p1wR6QSJe1QI/UpZg/pnsR5IwPoDM l9gJCmWaJgV2+YGGthVO+w4KhAIeZ/FFFu5xo64N5eVDO4TJL1ORCAPeziw7ERXRPm3B uQvakn41O2Dvm/EsoRnkMfx4KL9jx2meDqnQNmBwgF2sYi19RgDnln41e6zrVQoDKfJt J+y/M/JEvgN0ETsAuFA3mwu5035uYXD2nGufMKUUF9S/DHmM9VyxWFYif6SA/CVg62BK 0qjQ== X-Gm-Message-State: AO0yUKWX+y+GNdr5Eo1q1PNcOMakAip6TSxMh8lgVtKuKD+g3zNgIiwV mvhesGpfV6GrQo6VHVlP7Eb51w== X-Received: by 2002:a05:600c:807:b0:3dc:eaef:c1bb with SMTP id k7-20020a05600c080700b003dceaefc1bbmr4449372wmp.35.1675811088984; Tue, 07 Feb 2023 15:04:48 -0800 (PST) Received: from usaari01.cust.communityfibre.co.uk ([2a02:6b6a:b566:0:c04f:2463:c151:8b87]) by smtp.gmail.com with ESMTPSA id j14-20020a05600c190e00b003dcc82ce53fsm146485wmq.38.2023.02.07.15.04.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Feb 2023 15:04:48 -0800 (PST) From: Usama Arif To: dwmw2@infradead.org, tglx@linutronix.de, kim.phillips@amd.com Cc: arjan@linux.intel.com, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org, pbonzini@redhat.com, paulmck@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com, thomas.lendacky@amd.com, seanjc@google.com, pmenzel@molgen.mpg.de, fam.zheng@bytedance.com, punit.agrawal@bytedance.com, simon.evans@bytedance.com, liangma@liangbit.com, David Woodhouse , Usama Arif Subject: [PATCH v7 9/9] x86/smpboot: Serialize topology updates for secondary bringup Date: Tue, 7 Feb 2023 23:04:36 +0000 Message-Id: <20230207230436.2690891-10-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230207230436.2690891-1-usama.arif@bytedance.com> References: <20230207230436.2690891-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757215560080552802?= X-GMAIL-MSGID: =?utf-8?q?1757215560080552802?= From: David Woodhouse The toplogy update is performed by the AP via smp_callin() after the BSP has called do_wait_cpu_initialized(), setting the AP's bit in cpu_callout_mask to allow it to proceed. In preparation to enable further parallelism of AP bringup, add locking to serialize the update even if multiple APs are (in future) permitted to proceed through the next stages of bringup in parallel. Without such ordering (and with that future extra parallelism), confusion ensues: [ 1.360149] x86: Booting SMP configuration: [ 1.360221] .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 [ 1.366225] .... node #1, CPUs: #24 #25 #26 #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 [ 1.370219] .... node #0, CPUs: #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 #65 #66 #67 #68 #69 #70 #71 [ 1.378226] .... node #1, CPUs: #72 #73 #74 #75 #76 #77 #78 #79 #80 #81 #82 #83 #84 #85 #86 #87 #88 #89 #90 #91 #92 #93 #94 #95 [ 1.382037] Brought 96 CPUs to x86/cpu:kick in 72232606 cycles [ 0.104104] smpboot: CPU 26 Converting physical 0 to logical die 1 [ 0.104104] smpboot: CPU 27 Converting physical 1 to logical package 2 [ 0.104104] smpboot: CPU 24 Converting physical 1 to logical package 3 [ 0.104104] smpboot: CPU 27 Converting physical 0 to logical die 2 [ 0.104104] smpboot: CPU 25 Converting physical 1 to logical package 4 [ 1.385609] Brought 96 CPUs to x86/cpu:wait-init in 9269218 cycles [ 1.395285] Brought CPUs online in 28930764 cycles [ 1.395469] smp: Brought up 2 nodes, 96 CPUs [ 1.395689] smpboot: Max logical packages: 2 [ 1.396222] smpboot: Total of 96 processors activated (576000.00 BogoMIPS) [Usama Arif: fixed rebase conflict] Signed-off-by: David Woodhouse Signed-off-by: Usama Arif --- arch/x86/include/asm/smp.h | 4 +- arch/x86/include/asm/topology.h | 2 - arch/x86/kernel/cpu/common.c | 6 +-- arch/x86/kernel/smpboot.c | 73 ++++++++++++++++++++------------- arch/x86/xen/smp_pv.c | 4 +- 5 files changed, 48 insertions(+), 41 deletions(-) diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index 33c0d5fd8af6..b4b29e052b6e 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -48,8 +48,6 @@ struct smp_ops { }; /* Globals due to paravirt */ -extern void set_cpu_sibling_map(int cpu); - #ifdef CONFIG_SMP extern struct smp_ops smp_ops; @@ -137,7 +135,7 @@ void native_send_call_func_single_ipi(int cpu); void x86_idle_thread_init(unsigned int cpu, struct task_struct *idle); void smp_store_boot_cpu_info(void); -void smp_store_cpu_info(int id); +void smp_store_cpu_info(int id, bool force_single_core); asmlinkage __visible void smp_reboot_interrupt(void); __visible void smp_reschedule_interrupt(struct pt_regs *regs); diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index 458c891a8273..4bccbd949a99 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -136,8 +136,6 @@ static inline int topology_max_smt_threads(void) return __max_smt_threads; } -int topology_update_package_map(unsigned int apicid, unsigned int cpu); -int topology_update_die_map(unsigned int dieid, unsigned int cpu); int topology_phys_to_logical_pkg(unsigned int pkg); int topology_phys_to_logical_die(unsigned int die, unsigned int cpu); bool topology_is_primary_thread(unsigned int cpu); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9cfca3d7d0e2..cf1a4eff7a76 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1766,7 +1766,7 @@ static void generic_identify(struct cpuinfo_x86 *c) * Validate that ACPI/mptables have the same information about the * effective APIC id and update the package map. */ -static void validate_apic_and_package_id(struct cpuinfo_x86 *c) +static void validate_apic_id(struct cpuinfo_x86 *c) { #ifdef CONFIG_SMP unsigned int apicid, cpu = smp_processor_id(); @@ -1777,8 +1777,6 @@ static void validate_apic_and_package_id(struct cpuinfo_x86 *c) pr_err(FW_BUG "CPU%u: APIC id mismatch. Firmware: %x APIC: %x\n", cpu, apicid, c->initial_apicid); } - BUG_ON(topology_update_package_map(c->phys_proc_id, cpu)); - BUG_ON(topology_update_die_map(c->cpu_die_id, cpu)); #else c->logical_proc_id = 0; #endif @@ -1969,7 +1967,7 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c) #ifdef CONFIG_X86_32 enable_sep_cpu(); #endif - validate_apic_and_package_id(c); + validate_apic_id(c); x86_spec_ctrl_setup_ap(); update_srbds_msr(); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 7e7bcab6676e..8ffec5de2e2e 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -180,16 +180,12 @@ static void smp_callin(void) apic_ap_setup(); /* - * Save our processor parameters. Note: this information - * is needed for clock calibration. - */ - smp_store_cpu_info(cpuid); - - /* + * Save our processor parameters and update topology. + * Note: this information is needed for clock calibration. * The topology information must be up to date before * calibrate_delay() and notify_cpu_starting(). */ - set_cpu_sibling_map(raw_smp_processor_id()); + smp_store_cpu_info(cpuid, false); ap_init_aperfmperf(); @@ -244,6 +240,12 @@ static void notrace start_secondary(void *unused) * smp_callout_mask to release them. */ cpu_init_secondary(); + + /* + * Even though notify_cpu_starting() will do this, it does so too late + * as the AP may already have triggered lockdep splats by then. See + * commit 29368e093 ("x86/smpboot: Move rcu_cpu_starting() earlier"). + */ rcu_cpu_starting(raw_smp_processor_id()); x86_cpuinit.early_percpu_clock_init(); @@ -352,7 +354,7 @@ EXPORT_SYMBOL(topology_phys_to_logical_die); * @pkg: The physical package id as retrieved via CPUID * @cpu: The cpu for which this is updated */ -int topology_update_package_map(unsigned int pkg, unsigned int cpu) +static int topology_update_package_map(unsigned int pkg, unsigned int cpu) { int new; @@ -375,7 +377,7 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu) * @die: The die id as retrieved via CPUID * @cpu: The cpu for which this is updated */ -int topology_update_die_map(unsigned int die, unsigned int cpu) +static int topology_update_die_map(unsigned int die, unsigned int cpu) { int new; @@ -406,25 +408,7 @@ void __init smp_store_boot_cpu_info(void) c->initialized = true; } -/* - * The bootstrap kernel entry code has set these up. Save them for - * a given CPU - */ -void smp_store_cpu_info(int id) -{ - struct cpuinfo_x86 *c = &cpu_data(id); - - /* Copy boot_cpu_data only on the first bringup */ - if (!c->initialized) - *c = boot_cpu_data; - c->cpu_index = id; - /* - * During boot time, CPU0 has this setup already. Save the info when - * bringing up AP or offlined CPU0. - */ - identify_secondary_cpu(c); - c->initialized = true; -} +static arch_spinlock_t topology_lock = __ARCH_SPIN_LOCK_UNLOCKED; static bool topology_same_node(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) @@ -630,7 +614,7 @@ static struct sched_domain_topology_level x86_topology[] = { */ static bool x86_has_numa_in_package; -void set_cpu_sibling_map(int cpu) +static void set_cpu_sibling_map(int cpu) { bool has_smt = smp_num_siblings > 1; bool has_mp = has_smt || boot_cpu_data.x86_max_cores > 1; @@ -709,6 +693,37 @@ void set_cpu_sibling_map(int cpu) } } +/* + * The bootstrap kernel entry code has set these up. Save them for + * a given CPU + */ +void smp_store_cpu_info(int id, bool force_single_core) +{ + struct cpuinfo_x86 *c = &cpu_data(id); + + /* Copy boot_cpu_data only on the first bringup */ + if (!c->initialized) + *c = boot_cpu_data; + c->cpu_index = id; + /* + * During boot time, CPU0 has this setup already. Save the info when + * bringing up AP or offlined CPU0. + */ + identify_secondary_cpu(c); + + arch_spin_lock(&topology_lock); + BUG_ON(topology_update_package_map(c->phys_proc_id, id)); + BUG_ON(topology_update_die_map(c->cpu_die_id, id)); + c->initialized = true; + + /* For Xen PV */ + if (force_single_core) + c->x86_max_cores = 1; + + set_cpu_sibling_map(id); + arch_spin_unlock(&topology_lock); +} + /* maps the cpu to the sched domain representing multi-core */ const struct cpumask *cpu_coregroup_mask(int cpu) { diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c index 6175f2c5c822..09f94f940689 100644 --- a/arch/x86/xen/smp_pv.c +++ b/arch/x86/xen/smp_pv.c @@ -71,9 +71,7 @@ static void cpu_bringup(void) xen_enable_syscall(); } cpu = smp_processor_id(); - smp_store_cpu_info(cpu); - cpu_data(cpu).x86_max_cores = 1; - set_cpu_sibling_map(cpu); + smp_store_cpu_info(cpu, true); speculative_store_bypass_ht_init();