From patchwork Thu Feb 23 19:11:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 61066 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp485574wrd; Thu, 23 Feb 2023 11:15:53 -0800 (PST) X-Google-Smtp-Source: AK7set9fupYXmBFxogMoENhVDhe8wRSxNICC0Kv3wyaJVGtWTkeSEVRt/AdftQZSx3YN20g5ybsq X-Received: by 2002:a17:906:5d:b0:88c:f97c:7c87 with SMTP id 29-20020a170906005d00b0088cf97c7c87mr19505332ejg.2.1677179752916; Thu, 23 Feb 2023 11:15:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677179752; cv=none; d=google.com; s=arc-20160816; b=VvbyjyG2XC+wNkBJUYaGgLS2iu3wrgnv5phg1wDVkkdZiM9wCdrd1U7PYITYN+tyc/ 7vJJX/ujrI46uVW8qtZthHF6LRKsiej43dF6cI4TBpg2otaKFFKAAUXuuIVkfHQfdhJG G/nc9b3MWefYAJRbd6/lcyNjxyxlGc9mH0hV9jrz2YoBwVWelXnKV3FK2YvJrcB9u3b2 i/VYg440qTKjBRcIJ03IIoRDUAlHmkOUGto6yMEUUm9W0foibULV2BxTx0F4fD91BaQa QT+0OdK6qq7/y/hs3BkwvLxJx3aLnEQePeo5eJEDCr8I0VX/S2CMCisCabIh8UldUqdq pQWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wL4H6dkwiqQh7dcmIrLqt3HZrti5VJpIfMc44frmzHc=; b=cfgI5/p1o4ZPPIoOdkm+UtpK8aGNw/GNT1itiAc94XIGhF2aSuHAuG5K9NIM6rT9cL 0zPzcoRtnjmgrMbdBajl77Fbj0zukryF/PCszdHDiNK/nYaTugNa7Q0kzXeb49fPNoGy IQnVMdJWnFAVnPGZu3UzvH6lIZzzQwi1rGEhOWgSELVjD3+UGuN3DnBvKCJodHZYz15X qpOXD3HJvl/1NZILyVwCHGMF7VILE+Ji/AlNUdcYJcHIk4PtloQ7Zxo8w5GgCFrOKpD7 dKb5XE3F8dhFryx6Pf4zB47NeLecE8VA/gafIZ7y2mr6XoGxLm32jcKDfVdwTYbdB+HQ rO4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Z+5BzDDr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ep5-20020a1709069b4500b008c9851d114asi15785342ejc.607.2023.02.23.11.15.29; Thu, 23 Feb 2023 11:15:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=Z+5BzDDr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232670AbjBWTNr (ORCPT + 99 others); Thu, 23 Feb 2023 14:13:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231937AbjBWTNN (ORCPT ); Thu, 23 Feb 2023 14:13:13 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B6155BBBB for ; Thu, 23 Feb 2023 11:12:23 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id l7-20020a05600c4f0700b003e79fa98ce1so194802wmq.2 for ; Thu, 23 Feb 2023 11:12:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wL4H6dkwiqQh7dcmIrLqt3HZrti5VJpIfMc44frmzHc=; b=Z+5BzDDrPRuwK1aoLnVaWnL5dhdfm1qA4XC8szYEgLwxcNNkKR7N1jrdZUW8KEWp0r ooYGJUtsbba00Y/hcT4cdNn2uc+6w59DOT51XHq8wkNlj1qgwWVTvjDjUiFC3ZFQToAM Ms/4rjIXTj2kdOLZklcn7MemtWoO5VOYBGMjSb5QcoQKZg2bNnOZRgeY3otdukJX0KNl BffMhFJReoM18BJOkeYVYcZVKuaGFxn5NJNeSccfuIgvSMAAOL6Gq4tvOWjQeorYSe8d JHZI4Woj5seaPOAbsH+RKe4gXlSOJ5xsX4e0EjXjAxBad9J0hkle5NxDW9QiNVwK7Dd4 3Jng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wL4H6dkwiqQh7dcmIrLqt3HZrti5VJpIfMc44frmzHc=; b=WXY848qiE8f7JCNeCRy2fzHG5k29GatnNIrWmU2HMBeMMWz/6z+mR3Q8ergrhYIOGa D91NihJx5/9vzqfPwKFZEcZViAH/RYVI0NtNGHCDc8ayDBOvXVJN7Zop/TmTIIT5AaZL YVq9em0MR5o9J3C9V+CI3iaPAL8TYXd/62XqySimmdZBhKP7HW2IW1i+ZaQk0O88WnEq afsIpqtKC1XiUEA7E47flV+PHfJYUfvKvVfmAJFyEC7F9SAhe9tB+CWdTCxOVTav4qWF lZjNu8EE7ryKTRdUMirvvQIvGzcc8G3ik9LsHO2RmIPilmPKuXPKqpTbCywd9h8Q5KId iriQ== X-Gm-Message-State: AO0yUKV7+vyb5Km3iLoz7MiY0QPVjNTGOxQjbqgBB84CfK+so+ZBfqo+ zibZ2maOFLIYHnMIsAfBSjnMrw== X-Received: by 2002:a05:600c:3b9f:b0:3e9:9fc5:2c6d with SMTP id n31-20020a05600c3b9f00b003e99fc52c6dmr4257178wms.14.1677179511475; Thu, 23 Feb 2023 11:11:51 -0800 (PST) Received: from usaari01.cust.communityfibre.co.uk ([2a02:6b6a:b566:0:5ee0:5af0:64bd:6198]) by smtp.gmail.com with ESMTPSA id b15-20020a5d4b8f000000b002c561805a4csm12957286wrt.45.2023.02.23.11.11.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Feb 2023 11:11:51 -0800 (PST) From: Usama Arif To: dwmw2@infradead.org, tglx@linutronix.de, kim.phillips@amd.com, brgerst@gmail.com Cc: piotrgorski@cachyos.org, oleksandr@natalenko.name, arjan@linux.intel.com, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org, pbonzini@redhat.com, paulmck@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com, thomas.lendacky@amd.com, seanjc@google.com, pmenzel@molgen.mpg.de, fam.zheng@bytedance.com, punit.agrawal@bytedance.com, simon.evans@bytedance.com, liangma@liangbit.com, David Woodhouse , Usama Arif Subject: [PATCH v11 09/12] x86/smpboot: Remove initial_stack on 64-bit Date: Thu, 23 Feb 2023 19:11:37 +0000 Message-Id: <20230223191140.4155012-10-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230223191140.4155012-1-usama.arif@bytedance.com> References: <20230223191140.4155012-1-usama.arif@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758650436140892984?= X-GMAIL-MSGID: =?utf-8?q?1758650436140892984?= From: Brian Gerst Load RSP from current_task->thread.sp instead. Signed-off-by: Brian Gerst Reviewed-by: David Woodhouse Signed-off-by: David Woodhouse Tested-by: Usama Arif Signed-off-by: Usama Arif --- arch/x86/include/asm/processor.h | 6 +++++- arch/x86/kernel/acpi/sleep.c | 2 +- arch/x86/kernel/head_64.S | 35 ++++++++++++++++++-------------- arch/x86/xen/xen-head.S | 2 +- 4 files changed, 27 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 4e35c66edeb7..9c4a5c4d46c1 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -648,7 +648,11 @@ static inline void spin_lock_prefetch(const void *x) #define KSTK_ESP(task) (task_pt_regs(task)->sp) #else -#define INIT_THREAD { } +extern unsigned long __end_init_task[]; + +#define INIT_THREAD { \ + .sp = (unsigned long)&__end_init_task - PTREGS_SIZE, \ +} extern unsigned long KSTK_ESP(struct task_struct *task); diff --git a/arch/x86/kernel/acpi/sleep.c b/arch/x86/kernel/acpi/sleep.c index 47e75c056cb5..008fda8b1982 100644 --- a/arch/x86/kernel/acpi/sleep.c +++ b/arch/x86/kernel/acpi/sleep.c @@ -113,7 +113,7 @@ int x86_acpi_suspend_lowlevel(void) saved_magic = 0x12345678; #else /* CONFIG_64BIT */ #ifdef CONFIG_SMP - initial_stack = (unsigned long)temp_stack + sizeof(temp_stack); + current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack); early_gdt_descr.address = (unsigned long)get_cpu_gdt_rw(smp_processor_id()); initial_gs = per_cpu_offset(smp_processor_id()); diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index c32e5b06a9ce..f7905ba4b992 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -62,8 +62,8 @@ SYM_CODE_START_NOALIGN(startup_64) * tables and then reload them. */ - /* Set up the stack for verify_cpu(), similar to initial_stack below */ - leaq (__end_init_task - FRAME_SIZE)(%rip), %rsp + /* Set up the stack for verify_cpu() */ + leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp leaq _text(%rip), %rdi @@ -245,11 +245,11 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) #ifdef CONFIG_SMP /* * Is this the boot CPU coming up? If so everything is available - * in initial_gs, initial_stack and early_gdt_descr. + * in initial_gs and early_gdt_descr. */ movl smpboot_control(%rip), %edx testl $STARTUP_SECONDARY, %edx - jz .Lsetup_cpu + jz .Linit_cpu0_data /* * For parallel boot, the APIC ID is retrieved from CPUID, and then @@ -302,6 +302,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) hlt jmp 1b +.Linit_cpu0_data: + movq __per_cpu_offset(%rip), %rdx + jmp .Lsetup_cpu + .Linit_cpu_data: /* Get the per cpu offset for the given CPU# which is in ECX */ leaq __per_cpu_offset(%rip), %rbx @@ -314,13 +318,21 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) addq %rbx, %rcx movq %rcx, early_gdt_descr_base(%rip) - /* Find the idle task stack */ - movq pcpu_hot + X86_current_task(%rbx), %rcx - movq TASK_threadsp(%rcx), %rcx - movq %rcx, initial_stack(%rip) + movq %rbx, %rdx +#else + xorl %edx, %edx #endif /* CONFIG_SMP */ .Lsetup_cpu: + /* + * Setup a boot time stack - Any secondary CPU will have lost its stack + * by now because the cr3-switch above unmaps the real-mode stack + * + * RDX contains the per-cpu offset + */ + movq pcpu_hot + X86_current_task(%rdx), %rax + movq TASK_threadsp(%rax), %rsp + /* * We must switch to a new descriptor in kernel space for the GDT * because soon the kernel won't have access anymore to the userspace @@ -355,12 +367,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) movl initial_gs+4(%rip),%edx wrmsr - /* - * Setup a boot time stack - Any secondary CPU will have lost its stack - * by now because the cr3-switch above unmaps the real-mode stack - */ - movq initial_stack(%rip), %rsp - /* Drop the realmode protection. For the boot CPU the pointer is NULL! */ movq trampoline_lock(%rip), %rax testq %rax, %rax @@ -517,7 +523,6 @@ SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb) * The FRAME_SIZE gap is a convention which helps the in-kernel unwinder * reliably detect the end of the stack. */ -SYM_DATA(initial_stack, .quad init_thread_union + THREAD_SIZE - FRAME_SIZE) SYM_DATA(trampoline_lock, .quad 0); __FINITDATA diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index ffaa62167f6e..6bd391476656 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -49,7 +49,7 @@ SYM_CODE_START(startup_xen) ANNOTATE_NOENDBR cld - mov initial_stack(%rip), %rsp + leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp /* Set up %gs. *