From patchwork Mon Jul 31 06:41:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Xin3" X-Patchwork-Id: 128430 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp1867061vqg; Mon, 31 Jul 2023 01:13:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlHObJ54I+AA6zg7Y0QV2q/+Dw+FxkXVt9bNNbH1CHFP7f+Vi06Omh2ekZUxmcdyx6ktZBQB X-Received: by 2002:a05:6a00:158b:b0:687:2d68:f30d with SMTP id u11-20020a056a00158b00b006872d68f30dmr4616296pfk.23.1690791228837; Mon, 31 Jul 2023 01:13:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690791228; cv=none; d=google.com; s=arc-20160816; b=Iz7MJ0eqVwo0XsPZ/l4zf1BRed7MPPjApcysQTBxii5QlryvS0lV7ldemqHv8rRYWz rnDRmRFNEcRTJ3yRUnJ+Qt/oQxwhro+0oJ7w9PLMYpPlnow0qDP5AAaybPAHAvWc7JOK Pupez+xnLrScksX0ZoNy2LkYsYVh58BtdDJuiApMeed2g/kgJpXBY2rR/FAkfdV+4c58 8uMqx3oMg3sq0CRjCT8SLHMbK0fkktpAPqnZ7KBrWtmp+x9utLiGTg7d2/ccsIsHAoYw H6V/CtfD8CQKwfESVuQY/mhSFRwR7eXhvXKuW6lTGeiEEuNFF3wHnU3z49miH8yl70ZM 9sfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=4ywlqJNwXqhs6zqTSjvdVma1hmO/yFjs69iEIb1OLow=; fh=runz8kYc7mvoALl4f6YJmLswtUHBUyF5+hFgjyimmLU=; b=G0fOjnO75PTFeZXYnkakDc2r2z9Psw2tBuAmDhTcFWxmxNSHNoBGEYfKJ8Qw1V4HLM GgBphhJrtepp9HX4VNaKhMhQaX1VYMrFwCpds4C5uHnGKTcA1seNVsye5dVcdH2dchIY N/RvkY0zqDl3aRWVIATtwQhZsp7lsejustSoYdwN09UEWTSQSiPLRRZEwoXllB1Mxwkp MKpz2+hHo8xrCuxjgoBP3P/7mIt46BAKBxbg2KpYNy0lu2jMjIf+hfNoDnXzVgBt+0tT yj/wkzhZ6cipSspkeex0OO+yYqPLRn9pDoarolJTisgNzf4fHV2OzgL0OIop+y28ij6r 6vpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=hqS7oJxN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cp20-20020a056a00349400b00666ad95e625si2355236pfb.337.2023.07.31.01.13.36; Mon, 31 Jul 2023 01:13:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=hqS7oJxN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229729AbjGaHMw (ORCPT + 99 others); Mon, 31 Jul 2023 03:12:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229666AbjGaHMP (ORCPT ); Mon, 31 Jul 2023 03:12:15 -0400 Received: from mgamail.intel.com (unknown [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 652B02D64; Mon, 31 Jul 2023 00:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1690787415; x=1722323415; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=U03M3kZ8BDHSBlTSXJIlVDmlKpE5VAGPdolj0bAVLvI=; b=hqS7oJxNq6ZZZzeV+NUfK51rzbHMkZOfqLmedvkViRR3x5xA7qmLmkps 17CMtf08yfUiWvthJ/CcBw4vgvHfI4S9w9caabFoP2bOxaLQTUSLH+YB0 xJCyqU8tKrStMFdlZcrXWJoqdfHz3LitJ5dGL82sZOV4fJN/5FH8BRAvu uki/5SwBUOEHnhx9wzHBnNJZvMqX6TuCo8YezwGO2gNw+XaNq9rp6NwiC qnkxYGzCPtTY3WCcN9L5lBAysR3cgv0A8VwcfG+LPnfBfM9TCqnsiVLzo HBMQPq16W338ry5SnkBnIQ4L5PZLGcQAx8TjT6zwMFXZsRexf8JR/+HAO w==; X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="432750071" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="432750071" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2023 00:10:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10787"; a="798157778" X-IronPort-AV: E=Sophos;i="6.01,244,1684825200"; d="scan'208";a="798157778" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga004.fm.intel.com with ESMTP; 31 Jul 2023 00:10:05 -0700 From: Xin Li To: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, linux-hyperv@vger.kernel.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org Cc: Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Oleg Nesterov , Tony Luck , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Sean Christopherson , Peter Zijlstra , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Josh Poimboeuf , "Paul E . McKenney" , Catalin Marinas , Randy Dunlap , Steven Rostedt , Kim Phillips , Xin Li , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Liam R . Howlett" , Sebastian Reichel , "Kirill A . Shutemov" , Suren Baghdasaryan , Pawan Gupta , Jiaxi Chen , Babu Moger , Jim Mattson , Sandipan Das , Lai Jiangshan , Hans de Goede , Reinette Chatre , Daniel Sneddon , Breno Leitao , Nikunj A Dadhania , Brian Gerst , Sami Tolvanen , Alexander Potapenko , Andrew Morton , Arnd Bergmann , "Eric W . Biederman" , Kees Cook , Masami Hiramatsu , Masahiro Yamada , Ze Gao , Fei Li , Conghui , Ashok Raj , "Jason A . Donenfeld" , Mark Rutland , Jacob Pan , Jiapeng Chong , Jane Malalane , David Woodhouse , Boris Ostrovsky , Arnaldo Carvalho de Melo , Yantengsi , Christophe Leroy , Sathvika Vasireddy Subject: [PATCH v9 30/36] x86/fred: Fixup fault on ERETU by jumping to fred_entrypoint_user Date: Sun, 30 Jul 2023 23:41:27 -0700 Message-Id: <20230731064133.3881-1-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772923103602829056 X-GMAIL-MSGID: 1772923103602829056 If the stack frame contains an invalid user context (e.g. due to invalid SS, a non-canonical RIP, etc.) the ERETU instruction will trap (#SS or #GP). From a Linux point of view, this really should be considered a user space failure, so use the standard fault fixup mechanism to intercept the fault, fix up the exception frame, and redirect execution to fred_entrypoint_user. The end result is that it appears just as if the hardware had taken the exception immediately after completing the transition to user space. Suggested-by: H. Peter Anvin (Intel) Tested-by: Shan Kang Signed-off-by: Xin Li --- Changes since v8: * Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack. Changes since v6: * Add a comment to explain why it is safe to write to the previous FRED stack frame. (Lai Jiangshan). Changes since v5: * Move the NMI bit from an invalid stack frame, which caused ERETU to fault, to the fault handler's stack frame, thus to unblock NMI ASAP if NMI is blocked (Lai Jiangshan). --- arch/x86/entry/entry_64_fred.S | 5 +- arch/x86/include/asm/extable_fixup_types.h | 4 +- arch/x86/mm/extable.c | 79 ++++++++++++++++++++++ 3 files changed, 86 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64_fred.S b/arch/x86/entry/entry_64_fred.S index 4ae12d557db3..d24bf7f10ac8 100644 --- a/arch/x86/entry/entry_64_fred.S +++ b/arch/x86/entry/entry_64_fred.S @@ -3,6 +3,7 @@ * The actual FRED entry points. */ +#include #include #include "calling.h" @@ -34,7 +35,9 @@ SYM_CODE_START_NOALIGN(fred_entrypoint_user) call fred_entry_from_user SYM_INNER_LABEL(fred_exit_user, SYM_L_GLOBAL) FRED_EXIT - ERETU +1: ERETU + + _ASM_EXTABLE_TYPE(1b, fred_entrypoint_user, EX_TYPE_ERETU) SYM_CODE_END(fred_entrypoint_user) .fill fred_entrypoint_kernel - ., 1, 0xcc diff --git a/arch/x86/include/asm/extable_fixup_types.h b/arch/x86/include/asm/extable_fixup_types.h index 991e31cfde94..1585c798a02f 100644 --- a/arch/x86/include/asm/extable_fixup_types.h +++ b/arch/x86/include/asm/extable_fixup_types.h @@ -64,6 +64,8 @@ #define EX_TYPE_UCOPY_LEN4 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(4)) #define EX_TYPE_UCOPY_LEN8 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(8)) -#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ +#define EX_TYPE_ZEROPAD 20 /* longword load with zeropad on fault */ + +#define EX_TYPE_ERETU 21 #endif diff --git a/arch/x86/mm/extable.c b/arch/x86/mm/extable.c index 271dcb2deabc..0874f29e85ef 100644 --- a/arch/x86/mm/extable.c +++ b/arch/x86/mm/extable.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -223,6 +224,80 @@ static bool ex_handler_ucopy_len(const struct exception_table_entry *fixup, return ex_handler_uaccess(fixup, regs, trapnr, fault_address); } +#ifdef CONFIG_X86_FRED +static bool ex_handler_eretu(const struct exception_table_entry *fixup, + struct pt_regs *regs, unsigned long error_code) +{ + struct pt_regs *uregs = (struct pt_regs *) + (regs->sp - offsetof(struct pt_regs, orig_ax)); + unsigned short ss = uregs->ss; + unsigned short cs = uregs->cs; + + /* + * Move the NMI bit from the invalid stack frame, which caused ERETU + * to fault, to the fault handler's stack frame, thus to unblock NMI + * with the fault handler's ERETS instruction ASAP if NMI is blocked. + */ + regs->nmi = uregs->nmi; + + /* + * Sync event information to uregs, i.e., the ERETU return frame, but + * is it safe to write to the ERETU return frame which is just above + * current event stack frame? + * + * The RSP used by FRED to push a stack frame is not the value in %rsp, + * it is calculated from %rsp with the following 2 steps: + * 1) RSP = %rsp - (IA32_FRED_CONFIG & 0x1c0) // Reserve N*64 bytes + * 2) RSP = RSP & ~0x3f // Align to a 64-byte cache line + * when an event delivery doesn't trigger a stack level change. + * + * Here is an example with N*64 (N=1) bytes reserved: + * + * 64-byte cache line ==> ______________ + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| + * 64-byte cache line ==> |__Error_code__| <== ERETU return frame + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * |______________| + * 64-byte cache line ==> |______________| <== RSP after step 1) and 2) + * |___Reserved___| + * |__Event_data__| + * |_____SS_______| + * |_____RSP______| + * |_____FLAGS____| + * |_____CS_______| + * |_____IP_______| + * 64-byte cache line ==> |__Error_code__| <== ERETS return frame + * + * Thus a new FRED stack frame will always be pushed below a previous + * FRED stack frame ((N*64) bytes may be reserved between), and it is + * safe to write to a previous FRED stack frame as they never overlap. + */ + fred_info(uregs)->edata = fred_event_data(regs); + uregs->ssx = regs->ssx; + uregs->ss = ss; + /* The NMI bit was moved away above */ + uregs->nmi = 0; + uregs->csx = regs->csx; + uregs->sl = 0; + uregs->wfe = 0; + uregs->cs = cs; + uregs->orig_ax = error_code; + + return ex_handler_default(fixup, regs); +} +#endif + int ex_get_fixup_type(unsigned long ip) { const struct exception_table_entry *e = search_exception_tables(ip); @@ -300,6 +375,10 @@ int fixup_exception(struct pt_regs *regs, int trapnr, unsigned long error_code, return ex_handler_ucopy_len(e, regs, trapnr, fault_addr, reg, imm); case EX_TYPE_ZEROPAD: return ex_handler_zeropad(e, regs, fault_addr); +#ifdef CONFIG_X86_FRED + case EX_TYPE_ERETU: + return ex_handler_eretu(e, regs, error_code); +#endif } BUG(); }