From patchwork Sat Feb 18 21:14:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 59016 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp555966wrn; Sat, 18 Feb 2023 13:22:51 -0800 (PST) X-Google-Smtp-Source: AK7set82Ye/e80vFEtyoB9I6E8FreK0VvHp6BsPP7Q6d3vQjKZ0Dw8Z6C4dvj1WWbsSbB1Gss3mu X-Received: by 2002:a17:903:18e:b0:19a:5933:936e with SMTP id z14-20020a170903018e00b0019a5933936emr1426450plg.38.1676755371528; Sat, 18 Feb 2023 13:22:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676755371; cv=none; d=google.com; s=arc-20160816; b=WWsLlM1dDudqml+Fg6lm5lODRktiqvzPq1PJ5NHEYPoQ0e1BYPDymF0kJ0qKNx0llj go2LgxZ/ACl1UZz3EPRzVf+S8wFSOUFzK0hI4SoPbvctjEX19tw9IbZaz/mQqiUb36TP qh4Hwr6H4EP2SbW/4R8TS0NJwLBL9KNDhDhA5Ozy2oHUohj71gPZwQd7/0s7spgoVJmf G7r+sXAE3U+JKyT9fU8oJseBcAOGmVjw51VYdj+k9+q5Sn7AsQp6UiETrOb7KS1sJJ7n 1i0brzK3E9Cddm18ojwawv62W1T6tNsnH5p/qGu/pmwFbK+cLfoBKI82cDFN3DBiv0ZJ 62jA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=PzXH/uJcd68ZZoNWbbCBuIigR/v2lSDbJ+EW86dLo/w=; b=LDszjcuYEtAUlpM7YELq1vz/upjdM0UK41rkO4TZbVHU/TWjVynnha/ortNuvsE/B2 0EAUDsKuCivSkT8vqa/GpJf1e5/dnxYWS8G/shd3cUL3zt+sWtkIYRTPh4GvV3oA0jU5 XZsMfIkEfqTZ/YuyO0PFsKK0b2FYvjtnhK9j8s6g9Nf+JLJoMCK3QwcV+Hu7NRiKTDrQ PFo5oNAjx47dPMnRuPQpgPtYAEKbXsWGWY28puVpXqE2jzj2tInvdFKdhsyIE8XnJkSd C886NMgM1nCgzAJTheGZVOw6ILAGp/kp+ifuwJ65GkFYHgUYZB61q6pdO3zgNIrTSTya 7GEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="ZqAd/wk5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q9-20020a170902bd8900b0019bf9b4b5f1si3458849pls.629.2023.02.18.13.22.38; Sat, 18 Feb 2023 13:22:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="ZqAd/wk5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230315AbjBRVWR (ORCPT + 99 others); Sat, 18 Feb 2023 16:22:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230179AbjBRVVf (ORCPT ); Sat, 18 Feb 2023 16:21:35 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A71211968B; Sat, 18 Feb 2023 13:18:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676755133; x=1708291133; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=qEjm0xHlFad5ywdqKhdVQEfywUJWji/1KowdSbu4Qj8=; b=ZqAd/wk5GufB34BMVVOQPCcyg1OH9jKpBbEmbkD7+S7czx+8bnDXqWDS qcJYkYKwZCvE+cPnQFLhmOplClo5/QPvXc+Tata1hMkbaDL0mHoymGlke cIstKF/yU5GahpdCox4lbo+uDrmT58XciQ5dnQEWKd3z5QcVM5opoBMn0 j++FLBnO2IEo2K/9Q/U3ztBq64YiiWA+vmv0ERzUNZscWh1GfVogXXTBV irUqgIzS49JGedT5XlPei+WPj3r3rvg0QIN2C79Ymm/x+ziXmUDKrbqbD frsQhT5jEbkAldX3TXsRpdp6+yQHL0UvYf9pxTqf9u6CIGu83Im+LKENk A==; X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="418427762" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="418427762" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:16:22 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="664241723" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="664241723" Received: from adityava-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.80.223]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:16:21 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com, Yu-cheng Yu Subject: [PATCH v6 31/41] x86/shstk: Introduce routines modifying shstk Date: Sat, 18 Feb 2023 13:14:23 -0800 Message-Id: <20230218211433.26859-32-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230218211433.26859-1-rick.p.edgecombe@intel.com> References: <20230218211433.26859-1-rick.p.edgecombe@intel.com> X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758205440190432653?= X-GMAIL-MSGID: =?utf-8?q?1758205440190432653?= From: Yu-cheng Yu Shadow stacks are normally written to via CALL/RET or specific CET instructions like RSTORSSP/SAVEPREVSSP. However during some Linux operations the kernel will need to write to directly using the ring-0 only WRUSS instruction. A shadow stack restore token marks a restore point of the shadow stack, and the address in a token must point directly above the token, which is within the same shadow stack. This is distinctively different from other pointers on the shadow stack, since those pointers point to executable code area. Introduce token setup and verify routines. Also introduce WRUSS, which is a kernel-mode instruction but writes directly to user shadow stack. In future patches that enable shadow stack to work with signals, the kernel will need something to denote the point in the stack where sigreturn may be called. This will prevent attackers calling sigreturn at arbitrary places in the stack, in order to help prevent SROP attacks. To do this, something that can only be written by the kernel needs to be placed on the shadow stack. This can be accomplished by setting bit 63 in the frame written to the shadow stack. Userspace return addresses can't have this bit set as it is in the kernel range. It is also can't be a valid restore token. Tested-by: Pengfei Xu Tested-by: John Allen Reviewed-by: Kees Cook Signed-off-by: Yu-cheng Yu Co-developed-by: Rick Edgecombe Signed-off-by: Rick Edgecombe Cc: Kees Cook --- v5: - Fix typo in commit log v3: - Drop shstk_check_rstor_token() - Fail put_shstk_data() if bit 63 is set in the data (Kees) - Add comment in create_rstor_token() (Kees) - Pull in create_rstor_token() changes from future patch (Kees) v2: - Add data helpers for writing to shadow stack. v1: - Use xsave helpers. --- arch/x86/include/asm/special_insns.h | 13 +++++ arch/x86/kernel/shstk.c | 73 ++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index de48d1389936..d6cd9344f6c7 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -202,6 +202,19 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } +#ifdef CONFIG_X86_USER_SHADOW_STACK +static inline int write_user_shstk_64(u64 __user *addr, u64 val) +{ + asm_volatile_goto("1: wrussq %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} +#endif /* CONFIG_X86_USER_SHADOW_STACK */ + #define nop() asm volatile ("nop") static inline void serialize(void) diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 1d30295e0066..13c02747386f 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -25,6 +25,8 @@ #include #include +#define SS_FRAME_SIZE 8 + static bool features_enabled(unsigned long features) { return current->thread.features & features; @@ -40,6 +42,35 @@ static void features_clr(unsigned long features) current->thread.features &= ~features; } +/* + * Create a restore token on the shadow stack. A token is always 8-byte + * and aligned to 8. + */ +static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) +{ + unsigned long addr; + + /* Token must be aligned */ + if (!IS_ALIGNED(ssp, 8)) + return -EINVAL; + + addr = ssp - SS_FRAME_SIZE; + + /* + * SSP is aligned, so reserved bits and mode bit are a zero, just mark + * the token 64-bit. + */ + ssp |= BIT(0); + + if (write_user_shstk_64((u64 __user *)addr, (u64)ssp)) + return -EFAULT; + + if (token_addr) + *token_addr = addr; + + return 0; +} + static unsigned long alloc_shstk(unsigned long size) { int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; @@ -159,6 +190,48 @@ int shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, return 0; } +static unsigned long get_user_shstk_addr(void) +{ + unsigned long long ssp; + + fpregs_lock_and_load(); + + rdmsrl(MSR_IA32_PL3_SSP, ssp); + + fpregs_unlock(); + + return ssp; +} + +static int put_shstk_data(u64 __user *addr, u64 data) +{ + if (WARN_ON_ONCE(data & BIT(63))) + return -EINVAL; + + /* + * Mark the high bit so that the sigframe can't be processed as a + * return address. + */ + if (write_user_shstk_64(addr, data | BIT(63))) + return -EFAULT; + return 0; +} + +static int get_shstk_data(unsigned long *data, unsigned long __user *addr) +{ + unsigned long ldata; + + if (unlikely(get_user(ldata, addr))) + return -EFAULT; + + if (!(ldata & BIT(63))) + return -EINVAL; + + *data = ldata & ~BIT(63); + + return 0; +} + void shstk_free(struct task_struct *tsk) { struct thread_shstk *shstk = &tsk->thread.shstk;