From patchwork Fri Feb 24 07:01:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 61177 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp765374wrd; Thu, 23 Feb 2023 23:57:57 -0800 (PST) X-Google-Smtp-Source: AK7set/VpOFZxdP3YClCRXXxhV90VvzPZZRylR6DBSwsYaAWcT84rbIK5FdoNcSaa/grnMjNkyAj X-Received: by 2002:a17:90a:5c:b0:237:9cc7:28a6 with SMTP id 28-20020a17090a005c00b002379cc728a6mr539469pjb.26.1677225477396; Thu, 23 Feb 2023 23:57:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677225477; cv=none; d=google.com; s=arc-20160816; b=w9qrxRSvSk5EkFE9YMaHBytrLvyEITF4I5en1lod8H2bAN+Yo4x2s00Cyah6SR4uom LVlmvSvpN7EHNltAVjCHlF5i9bwLkuooHGLwvMCk22bMHkbn4Dq2E0uXdOoMCVk/H0io mc44UTEmGcZ7oyWASO5A7svvusvkGg76UpOek+W5Rux1XVGgqEFlZSrVh4HkLOXr68zE EfK2Pk4IY5JHpzMfUYU2mY6UaE7XhRmK6NoDaLH9GvlTUxhNaYyD1gxnwRySh8C3zDPw 7KqSfr0R4ByHFIOYNHrJACcW8yIgEVNMNlWFL5Jlr2pf+yOQbvgj1v/KupsmtgXmDGVk Y3Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=J1n5yfAjEPgOcVYrOlZlvMqkva+uHhUnvCfKuH7R9TY=; b=Xc0+MRvC96FTgOgKPbvq8M3mzqC7ZI0dZKnEhHC2eJkwGxrDUeCCcdvq4sEA0pL4aL 9e1RB4SCrVKjxvE5q4k1JKvZf7txo4xA+x+6ZjghgKXo8kfmliDNwLjdw+VjP5IUWfp/ eEv59T3w/2q7cmRTY2IvWSutoltGq3xhSPg8gvQDa81azx5WKExK4WJgG7c66dDiyTyB TMfDt0mz3MgYxXhKhCasnTarns2YBXp+UxAHhG0DV1LQCI/zcFRPLWYMg7DA5iEKDC+N 9vnKKyMi7T4IB/S/1PBVkzzARD1lGFTqQv8NWnhpxsFxpiXKxV4cYWKNa0ofEszUJaEI qiRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Wq2WDW4F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nk5-20020a17090b194500b0023747b030e7si1810091pjb.105.2023.02.23.23.57.43; Thu, 23 Feb 2023 23:57:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Wq2WDW4F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229940AbjBXHaJ (ORCPT + 99 others); Fri, 24 Feb 2023 02:30:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229506AbjBXH3i (ORCPT ); Fri, 24 Feb 2023 02:29:38 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F00D83B640; Thu, 23 Feb 2023 23:28:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677223722; x=1708759722; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QTItTBCG8ChkQChY9t9Ee1Y9V+C1tr1gSMal5Lb+Ivk=; b=Wq2WDW4FP9cs2rZhE3J8Ku9G3XmHnpAL2SxFSvOZJ9+Qx/A2A9obmbiq p8rHgQBBhw1c5DWT47IiEr5laNP8RN5n51VgsuniYSYCumFnUPYEK45Xt JvbqU7rmRFa4hrQkXy9redbkD0DTVm4fWwsKSX1fBMHyrTAIRuor0opWY dhfwnWc8gqO5GYFSAk7ecbdzeRXdnOfJggPiFREbtxXjZpRNBdziYGcKa eVkHzSN0VmBktTj596xL2c36gq3IkBcXPgW703sRDSqVyqaGz4j6nQcr2 240nu2aRUCQ98EbCXmQ2nx7iMdV79xrALjzp/IDekXEQVAvdGoltF97IH g==; X-IronPort-AV: E=McAfee;i="6500,9779,10630"; a="334836184" X-IronPort-AV: E=Sophos;i="5.97,324,1669104000"; d="scan'208";a="334836184" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2023 23:27:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10630"; a="815639287" X-IronPort-AV: E=Sophos;i="5.97,324,1669104000"; d="scan'208";a="815639287" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga001.fm.intel.com with ESMTP; 23 Feb 2023 23:27:26 -0800 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org, andrew.cooper3@citrix.com, seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com Subject: [RFC PATCH v3 31/32] x86/fred: allow dynamic stack frame size Date: Thu, 23 Feb 2023 23:01:44 -0800 Message-Id: <20230224070145.3572-32-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230224070145.3572-1-xin3.li@intel.com> References: <20230224070145.3572-1-xin3.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758698382000750360?= X-GMAIL-MSGID: =?utf-8?q?1758698382000750360?= A FRED stack frame could contain different amount of information for different event types, or perhaps even for different instances of the same event type. Thus we need to eliminate the need of any advance information of the stack frame size to allow dynamic stack frame size. Implement it through: 1) add a new field user_pt_regs to thread_info, and initialize it with a pointer to a virtual pt_regs structure at the top of a thread stack. 2) save a pointer to the user-space pt_regs structure created by fred_entrypoint_user() to user_pt_regs in fred_entry_from_user(). 3) initialize the init_thread_info's user_pt_regs with a pointer to a virtual pt_regs structure at the top of init stack. This approach also works for IDT, thus we unify the code. Suggested-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/entry/entry_32.S | 2 +- arch/x86/entry/entry_fred.c | 2 ++ arch/x86/include/asm/entry-common.h | 3 +++ arch/x86/include/asm/processor.h | 12 +++------ arch/x86/include/asm/switch_to.h | 3 +-- arch/x86/include/asm/thread_info.h | 41 ++++------------------------- arch/x86/kernel/head_32.S | 3 +-- arch/x86/kernel/process.c | 5 ++++ kernel/fork.c | 6 +++++ 9 files changed, 27 insertions(+), 50 deletions(-) diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S index 91397f58ac30..5adc4cf33d92 100644 --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -1244,7 +1244,7 @@ SYM_CODE_START(rewind_stack_and_make_dead) xorl %ebp, %ebp movl PER_CPU_VAR(pcpu_hot + X86_top_of_stack), %esi - leal -TOP_OF_KERNEL_STACK_PADDING-PTREGS_SIZE(%esi), %esp + leal -PTREGS_SIZE(%esi), %esp call make_task_dead 1: jmp 1b diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c index 8d3e144670d6..a72167c83923 100644 --- a/arch/x86/entry/entry_fred.c +++ b/arch/x86/entry/entry_fred.c @@ -178,6 +178,8 @@ __visible noinstr void fred_entry_from_user(struct pt_regs *regs) [EVENT_TYPE_OTHER] = fred_syscall_slow }; + current->thread_info.user_pt_regs = regs; + /* * FRED employs a two-level event dispatch mechanism, with * the first-level on the type of an event and the second-level diff --git a/arch/x86/include/asm/entry-common.h b/arch/x86/include/asm/entry-common.h index 117903881fe4..5b7d0f47f188 100644 --- a/arch/x86/include/asm/entry-common.h +++ b/arch/x86/include/asm/entry-common.h @@ -12,6 +12,9 @@ /* Check that the stack and regs on entry from user mode are sane. */ static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) { + if (!cpu_feature_enabled(X86_FEATURE_FRED)) + current->thread_info.user_pt_regs = regs; + if (IS_ENABLED(CONFIG_DEBUG_ENTRY)) { /* * Make sure that the entry code gave us a sensible EFLAGS diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 8d73004e4cac..4a50d2a2c14b 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -626,17 +626,11 @@ static inline void spin_lock_prefetch(const void *x) prefetchw(x); } -#define TOP_OF_INIT_STACK ((unsigned long)&init_stack + sizeof(init_stack) - \ - TOP_OF_KERNEL_STACK_PADDING) +#define TOP_OF_INIT_STACK ((unsigned long)&init_stack + sizeof(init_stack)) -#define task_top_of_stack(task) ((unsigned long)(task_pt_regs(task) + 1)) +#define task_top_of_stack(task) ((unsigned long)task_stack_page(task) + THREAD_SIZE) -#define task_pt_regs(task) \ -({ \ - unsigned long __ptr = (unsigned long)task_stack_page(task); \ - __ptr += THREAD_SIZE - TOP_OF_KERNEL_STACK_PADDING; \ - ((struct pt_regs *)__ptr) - 1; \ -}) +#define task_pt_regs(task) ((task)->thread_info.user_pt_regs) #ifdef CONFIG_X86_32 #define INIT_THREAD { \ diff --git a/arch/x86/include/asm/switch_to.h b/arch/x86/include/asm/switch_to.h index 00fd85abc1d2..0a31da150808 100644 --- a/arch/x86/include/asm/switch_to.h +++ b/arch/x86/include/asm/switch_to.h @@ -72,8 +72,7 @@ static inline void update_task_stack(struct task_struct *task) /* * Will use WRMSRNS/WRMSRLIST for performance once it's upstreamed. */ - wrmsrl(MSR_IA32_FRED_RSP0, - task_top_of_stack(task) + TOP_OF_KERNEL_STACK_PADDING); + wrmsrl(MSR_IA32_FRED_RSP0, task_top_of_stack(task)); } else if (cpu_feature_enabled(X86_FEATURE_XENPV)) { /* Xen PV enters the kernel on the thread stack. */ load_sp0(task_top_of_stack(task)); diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index 998483078d5f..ced0a01e0a3e 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -13,42 +13,6 @@ #include #include -/* - * TOP_OF_KERNEL_STACK_PADDING is a number of unused bytes that we - * reserve at the top of the kernel stack. We do it because of a nasty - * 32-bit corner case. On x86_32, the hardware stack frame is - * variable-length. Except for vm86 mode, struct pt_regs assumes a - * maximum-length frame. If we enter from CPL 0, the top 8 bytes of - * pt_regs don't actually exist. Ordinarily this doesn't matter, but it - * does in at least one case: - * - * If we take an NMI early enough in SYSENTER, then we can end up with - * pt_regs that extends above sp0. On the way out, in the espfix code, - * we can read the saved SS value, but that value will be above sp0. - * Without this offset, that can result in a page fault. (We are - * careful that, in this case, the value we read doesn't matter.) - * - * In vm86 mode, the hardware frame is much longer still, so add 16 - * bytes to make room for the real-mode segments. - * - * x86-64 has a fixed-length stack frame, but it depends on whether - * or not FRED is enabled. Future versions of FRED might make this - * dynamic, but for now it is always 2 words longer. - */ -#ifdef CONFIG_X86_32 -# ifdef CONFIG_VM86 -# define TOP_OF_KERNEL_STACK_PADDING 16 -# else -# define TOP_OF_KERNEL_STACK_PADDING 8 -# endif -#else /* x86-64 */ -# ifdef CONFIG_X86_FRED -# define TOP_OF_KERNEL_STACK_PADDING (2*8) -# else -# define TOP_OF_KERNEL_STACK_PADDING 0 -# endif -#endif - /* * low level task data that entry.S needs immediate access to * - this struct should fit entirely inside of one cache line @@ -56,6 +20,7 @@ */ #ifndef __ASSEMBLY__ struct task_struct; +struct pt_regs; #include #include @@ -66,11 +31,14 @@ struct thread_info { #ifdef CONFIG_SMP u32 cpu; /* current CPU */ #endif + struct pt_regs *user_pt_regs; }; +#define INIT_TASK_PT_REGS ((struct pt_regs *)TOP_OF_INIT_STACK - 1) #define INIT_THREAD_INFO(tsk) \ { \ .flags = 0, \ + .user_pt_regs = INIT_TASK_PT_REGS, \ } #else /* !__ASSEMBLY__ */ @@ -240,6 +208,7 @@ static inline int arch_within_stack_frames(const void * const stack, extern void arch_task_cache_init(void); extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src); +extern void arch_init_user_pt_regs(struct task_struct *tsk); extern void arch_release_task_struct(struct task_struct *tsk); extern void arch_setup_new_exec(void); #define arch_setup_new_exec arch_setup_new_exec diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S index 67c8ed99144b..0201ddcd7576 100644 --- a/arch/x86/kernel/head_32.S +++ b/arch/x86/kernel/head_32.S @@ -517,8 +517,7 @@ SYM_DATA_END(initial_page_table) * reliably detect the end of the stack. */ SYM_DATA(initial_stack, - .long init_thread_union + THREAD_SIZE - - SIZEOF_PTREGS - TOP_OF_KERNEL_STACK_PADDING) + .long init_thread_union + THREAD_SIZE - SIZEOF_PTREGS) __INITRODATA int_msg: diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index e57cd31bfec4..e176a376bde6 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -98,6 +98,11 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) return 0; } +void arch_init_user_pt_regs(struct task_struct *tsk) +{ + tsk->thread_info.user_pt_regs = (struct pt_regs *)task_top_of_stack(tsk)- 1; +} + #ifdef CONFIG_X86_64 void arch_release_task_struct(struct task_struct *tsk) { diff --git a/kernel/fork.c b/kernel/fork.c index 82b2b5846aae..90d5dbd6b522 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -961,6 +961,10 @@ int __weak arch_dup_task_struct(struct task_struct *dst, return 0; } +void __weak arch_init_user_pt_regs(struct task_struct *tsk) +{ +} + void set_task_stack_end_magic(struct task_struct *tsk) { unsigned long *stackend; @@ -988,6 +992,8 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) if (err) goto free_tsk; + arch_init_user_pt_regs(tsk); + #ifdef CONFIG_THREAD_INFO_IN_TASK refcount_set(&tsk->stack_refcount, 1); #endif