From patchwork Thu Mar 2 11:12:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 6027 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp4174805wrd; Thu, 2 Mar 2023 03:17:05 -0800 (PST) X-Google-Smtp-Source: AK7set9YrlJKvqBx7827IFvE0W8oLjg7Gp9WnQfjYLV6gCR79AQAzF7PkypNwCmrWQq37os+aXEe X-Received: by 2002:aa7:d546:0:b0:4ac:d3bc:cb0d with SMTP id u6-20020aa7d546000000b004acd3bccb0dmr9474325edr.3.1677755824945; Thu, 02 Mar 2023 03:17:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677755824; cv=none; d=google.com; s=arc-20160816; b=yizV64JyKW54r6K3Oy91G8LR+ld8gFbZlaSuOUrmlJ2YpxzLjRobb+rzuMjhYYonXt rd+2hs/+l9CFRSkZNaB/2IMHLoa/NUk1nHBNY81Y3lscamtni54F+pJy0ZmAxoY7DeBv mChH0tGAN/0ks1uS5fhrVy36TM/rIJp/N1DxbS/SMBPCMGMPO35++GqRQxaJDDVA9XsY 2figo1zzWUtm7GLYu39z5UOQ4qAlZDa21vx4GnEAKlOGNDQqPMfzX9Wwn7W6KbAQscMo ZlSuTc3KJU5NzI2oKO+EoAAcEKfvz6DkQLORWn5MKIHqjpdDAmT0OoV8VxOyiSSeLLrG L1Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=SmbgYBfsiP69zAShGBqEQbH1PuV83xuFSvcJJxGMMf4=; b=N18L8mxz8Qnvhe9lVTxsUuzrnPuJw3ILS8/5EHI2iI0CINYVTrdHOjpEQSgi6zBpMd CzcjD3/TCa/ff/Jh0htMT2az4XE20N722bb63Q+MCyLO73TsQqiWWBXKM5cyglw4JwpG ujW1RejPaZz77aSkI/d2JM0JAtZCl3SViIajPZi5iFLZCx/Oba5uiQsLm690AD+MfC0i 0HvkR93CY/lmIdD8X9yeE3fLhreQ+NSRg6Z6U2PowfQwEAanX8lIKa3aoQOP4HaBeSLZ cgAP2fEQNN/jWgApkyiPC50M4jRGBQ+fpUhKxiLtR5QOg6v/yTu5PZ3Cpk+rtJ98DAL+ ZBnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=DrUlMSLx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w3-20020a056402128300b004abb522bf9bsi7118971edv.142.2023.03.02.03.16.42; Thu, 02 Mar 2023 03:17:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=DrUlMSLx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230227AbjCBLNE (ORCPT + 99 others); Thu, 2 Mar 2023 06:13:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbjCBLM6 (ORCPT ); Thu, 2 Mar 2023 06:12:58 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7245C3E082 for ; Thu, 2 Mar 2023 03:12:32 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id j2so16119920wrh.9 for ; Thu, 02 Mar 2023 03:12:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=SmbgYBfsiP69zAShGBqEQbH1PuV83xuFSvcJJxGMMf4=; b=DrUlMSLxCfk+wenY5dGy8zdp8YdI8rwkdVkiM9b4xwhFqbBcP8gKQGiVekkShAMDGb siTZabe8yR4OsN5yFqEf8tWw23l9QEZd1B8mQN/xwWdY2GwRCgDaxdwXnDBXjWcOhFep zmavEFKBv6RbLcB3Y4wZJ7XWc2Fw37ZaUdqhU2vo9HMB8jdBrrHp5RvaQSaEkhP0n5dw ozot/+J4p8hyqVnp0u0amlEz5wYTixfb5YB7sMyyJkGiJlNuYRzrFRAk57eMqqeeEF3P UCxjArfwFZ+h97ZOj0DuNvdL4gSBPKxwVkKyrRIUuILFpN+yMY6Wce6+3NqU2ghB2TcL RRJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SmbgYBfsiP69zAShGBqEQbH1PuV83xuFSvcJJxGMMf4=; b=kjAZDS8M++0LPY0gBhanrztmRDXCzaRIi/H6aWQmyUtTdvjoxHBtMbbpWGBmCYOEeV Zn8FQF+Xg2B6jNVdOSrIDRtW+Iqh1avCClxKanZhcPRCIzJchnLNwctVs59/xla0BeFN mTqfkUXeFtyeNoh3PFqQiNVixYhhOKoRfOyzQMVQ2xyBIExvIJlv2FSr9w5ZPvvbOVhJ n7xPK1p4dqWb7EUCLjZayCKEsROkYIQUblOxCi+1Nz9FRVZaw1ScNjCWzXsJjzBylYzx wtyfj97yNDrZEWgJALJQNKUKjL1RZxFhNiYF/0L84DlMUdsUC7VCJPuYXi0BqbjOpFD9 +3lg== X-Gm-Message-State: AO0yUKUt6dPbGGBiJVkhbPlaVX2hLoSjO1yr43o/Ticxxk3gDuOO5tp8 khDGzm1SaQ0VHjdPFe3bFj2HmQ== X-Received: by 2002:adf:fecd:0:b0:2c7:13e4:2094 with SMTP id q13-20020adffecd000000b002c713e42094mr7740020wrs.42.1677755550902; Thu, 02 Mar 2023 03:12:30 -0800 (PST) Received: from usaari01.cust.communityfibre.co.uk ([2a02:6b6a:b566:0:11aa:3c13:d3e:eb29]) by smtp.gmail.com with ESMTPSA id a5-20020a5d4565000000b002c3f81c51b6sm14724830wrc.90.2023.03.02.03.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 03:12:30 -0800 (PST) From: Usama Arif To: dwmw2@infradead.org, tglx@linutronix.de, kim.phillips@amd.com, brgerst@gmail.com Cc: piotrgorski@cachyos.org, oleksandr@natalenko.name, arjan@linux.intel.com, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org, pbonzini@redhat.com, paulmck@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com, thomas.lendacky@amd.com, seanjc@google.com, pmenzel@molgen.mpg.de, fam.zheng@bytedance.com, punit.agrawal@bytedance.com, simon.evans@bytedance.com, liangma@liangbit.com, Usama Arif Subject: [PATCH v13 00/11] Parallel CPU bringup for x86_64 Date: Thu, 2 Mar 2023 11:12:16 +0000 Message-Id: <20230302111227.2102545-1-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759254491929699838?= X-GMAIL-MSGID: =?utf-8?q?1759254491929699838?= The main code change over v12 is to fix the build error when CONFIG_FORCE_NR_CPUS is present. The commit message for removing initial stack has also been improved, typos have been fixed and extra comments have been added to make code clearer. Thanks, Usama Changes across versions: v2: Cut it back to just INIT/SIPI/SIPI in parallel for now, nothing more v3: Clean up x2apic patch, add MTRR optimisation, lock topology update in preparation for more parallelisation. v4: Fixes to the real mode parallelisation patch spotted by SeanC, to avoid scribbling on initial_gs in common_cpu_up(), and to allow all 24 bits of the physical X2APIC ID to be used. That patch still needs a Signed-off-by from its original author, who once claimed not to remember writing it at all. But now we've fixed it, hopefully he'll admit it now :) v5: rebase to v6.1 and remeasure performance, disable parallel bringup for AMD CPUs. v6: rebase to v6.2-rc6, disabled parallel boot on amd as a cpu bug and reused timer calibration for secondary CPUs. v7: [David Woodhouse] iterate over all possible CPUs to find any existing cluster mask in alloc_clustermask. (patch 1/9) Keep parallel AMD support enabled in AMD, using APIC ID in CPUID leaf 0x0B (for x2APIC mode) or CPUID leaf 0x01 where 8 bits are sufficient. Included sanity checks for APIC id from 0x0B. (patch 6/9) Removed patch for reusing timer calibration for secondary CPUs. commit message and code improvements. v8: Fix CPU0 hotplug by setting up the initial_gs, initial_stack and early_gdt_descr. Drop trampoline lock and bail if APIC ID not found in find_cpunr. Code comments improved and debug prints added. v9: Drop patch to avoid repeated saves of MTRR at boot time. rebased and retested at v6.2-rc8. added kernel doc for no_parallel_bringup and made do_parallel_bringup __ro_after_init. v10: Fixed suspend/resume not working with parallel smpboot. rebased and retested to 6.2. fixed checkpatch errors. v11: Added patches from Brian Gerst to remove the global variables initial_gs, initial_stack, and early_gdt_descr from the 64-bit boot code (https://lore.kernel.org/all/20230222221301.245890-1-brgerst@gmail.com/). v12: Fixed compilation errors, acquire tr_lock for every stack setup in trampoline_64.S. Rearranged commits for a cleaner git history. v13: Fix build error with CONFIG_FORCE_NR_CPUS. Commit message improved, typos fixed and extra comments added. Brian Gerst (3): x86/smpboot: Remove initial_stack on 64-bit x86/smpboot: Remove early_gdt_descr on 64-bit x86/smpboot: Remove initial_gs David Woodhouse (8): x86/apic/x2apic: Allow CPU cluster_mask to be populated in parallel cpu/hotplug: Move idle_thread_get() to cpu/hotplug: Add dynamic parallel bringup states before CPUHP_BRINGUP_CPU x86/smpboot: Reference count on smpboot_setup_warm_reset_vector() x86/smpboot: Split up native_cpu_up into separate phases and document them x86/smpboot: Support parallel startup of secondary CPUs x86/smpboot: Send INIT/SIPI/SIPI to secondary CPUs in parallel x86/smpboot: Serialize topology updates for secondary bringup .../admin-guide/kernel-parameters.txt | 3 + arch/x86/include/asm/processor.h | 6 +- arch/x86/include/asm/realmode.h | 4 +- arch/x86/include/asm/smp.h | 15 +- arch/x86/include/asm/topology.h | 2 - arch/x86/kernel/acpi/sleep.c | 30 +- arch/x86/kernel/apic/apic.c | 2 +- arch/x86/kernel/apic/x2apic_cluster.c | 126 ++++--- arch/x86/kernel/asm-offsets.c | 1 + arch/x86/kernel/cpu/common.c | 6 +- arch/x86/kernel/head_64.S | 132 +++++-- arch/x86/kernel/smpboot.c | 350 +++++++++++++----- arch/x86/realmode/init.c | 3 + arch/x86/realmode/rm/trampoline_64.S | 27 +- arch/x86/xen/smp_pv.c | 4 +- arch/x86/xen/xen-head.S | 2 +- include/linux/cpuhotplug.h | 2 + include/linux/smpboot.h | 7 + kernel/cpu.c | 31 +- kernel/smpboot.h | 2 - 20 files changed, 556 insertions(+), 199 deletions(-) Signed-off-by: David Woodhouse