From patchwork Wed Jul 19 22:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 122896 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp2769950vqt; Wed, 19 Jul 2023 16:23:55 -0700 (PDT) X-Google-Smtp-Source: APBJJlHFJaeuipBaZ8tk+N3+IvgoYwYGzJZ+hxPAYkriT4h+MMaAdxiuyDMaZV5ICXiUTkexIWQ6 X-Received: by 2002:a05:6a00:986:b0:682:9162:720f with SMTP id u6-20020a056a00098600b006829162720fmr7242802pfg.6.1689809034726; Wed, 19 Jul 2023 16:23:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689809034; cv=none; d=google.com; s=arc-20160816; b=BdW+3tIJGummz9Ba2zk5Rp6FM/QYpjG+/yy14GkCiqtSjsdPQM8Ls7bJ3vYmx8nz9N 8krtzbR02DEhW79ir/QPQ9h1xaZFE3gcjW54VxeN3rhRkBpx/KfB+5MbKpvGzymXG4wV XNXaC5q0rO7GQJwnrW24bKz2CHPHs+BqggbHlkx8qM8xMCNq5mQqYOePixO5Hk3tcpZv FBOcG8Tn7nh/LSvUMgmT0T7y3XeLrpirqExLmX389J700vcC3O0ff9NFTQo8jrLdCpZm +FP9Rkyx8+AwsECI4miALOM7neAzHzcp/LIkd4msyQMavWpcN/YnjiwY4E94Nc1KzYjy 5CwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:cc:subject:to:reply-to:sender:from :dkim-signature:dkim-signature:date; bh=Obgqh2InRwJaOGFGWbKwYAzB6x4FmDvW8UkrpzRGVG8=; fh=cYKsLYKlfL5GLDpOt1a9DaF93PQ8azQhO0iBCt+veh0=; b=GVvbrxHEdHrsnuTzEXCu15oji/faidm4lQAUWROzqHkEzQ26GVAK0azX6EtIH2ICC3 xor7hbCD4ixlvRqRaG86wooMv5fDXgqt5o+7ZGQr5bPiZgx0JTKvxNnabC64lLOCrUn2 lOUHPHSDo2s8yv5sK77MJIvnp+WmuCZUMLLbglFbibLyXAXawbUl99aEem+U70ezrpH7 7QZ3AUgHK4WcddVUGry3P4isw2TBUWiM5ICdKZs1hPhtBQNILQstc+bEvvfXlBaVfTt6 oxNHFAHdj1riRgDONSiZLDc+bm20QrSRBNnfPvIEp4yCsnf8dBcoFToOoyfKV36BPTJJ M7GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=oEKRiOy2; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ei17-20020a056a0080d100b00678ba339933si33141pfb.160.2023.07.19.16.23.39; Wed, 19 Jul 2023 16:23:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=oEKRiOy2; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231414AbjGSWtt (ORCPT + 99 others); Wed, 19 Jul 2023 18:49:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230147AbjGSWro (ORCPT ); Wed, 19 Jul 2023 18:47:44 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75C302108; Wed, 19 Jul 2023 15:47:34 -0700 (PDT) Date: Wed, 19 Jul 2023 22:47:31 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1689806852; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Obgqh2InRwJaOGFGWbKwYAzB6x4FmDvW8UkrpzRGVG8=; b=oEKRiOy2rNRu/7VinLYY9BVGRGNMk7Ub7SarjZlG2YFMMdsw+hWzr6B8zsYe0u/fhzG67h CET20RQrZOAe728cSoudg2Tu5VEcBB4Gkjefv5gYtxgEEoSczP6b7L6b4uf1hZyeta/IXs /CwBaecdy/6OhZ58yY/fS4LyFY5/pN8/uashwKV3AVAtBqTP7GzcZmT/0aMmE8h4XnM9q+ 94eNiQ8KQS4JKdgCx+i9E8bN+WFklEs/gHF1Brbmr2ojbJRoV3rpxsS+/p9oBZj+EdYtG5 inz0L4vVoL3enYqFa92kYwQQLQrC8N70iCrbH+SU9JTs78iaAzm/AaXoXq1fuw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1689806852; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Obgqh2InRwJaOGFGWbKwYAzB6x4FmDvW8UkrpzRGVG8=; b=AIibaOC3BYgPH1eakQ9HVW337bbGL1ygfZ0HIl4zPQHZinH+m1wkfBO3lb3ITq/4L8U+8h prqmgCoETmIY9aCw== From: "tip-bot2 for Rick Edgecombe" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/shstk] x86/shstk: Add user-mode shadow stack support Cc: "Yu-cheng Yu" , Rick Edgecombe , Dave Hansen , "Borislav Petkov (AMD)" , Kees Cook , "Mike Rapoport (IBM)" , Pengfei Xu , John Allen , x86@kernel.org, linux-kernel@vger.kernel.org MIME-Version: 1.0 Message-ID: <168980685187.28540.17478294224587896576.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771893197915154763 X-GMAIL-MSGID: 1771893197915154763 The following commit has been merged into the x86/shstk branch of tip: Commit-ID: 10e0e11755d88afefedd4e29fb529d03ff454c24 Gitweb: https://git.kernel.org/tip/10e0e11755d88afefedd4e29fb529d03ff454c24 Author: Rick Edgecombe AuthorDate: Mon, 12 Jun 2023 17:10:54 -07:00 Committer: Rick Edgecombe CommitterDate: Tue, 11 Jul 2023 14:12:50 -07:00 x86/shstk: Add user-mode shadow stack support Introduce basic shadow stack enabling/disabling/allocation routines. A task's shadow stack is allocated from memory with VM_SHADOW_STACK flag and has a fixed size of min(RLIMIT_STACK, 4GB). Keep the task's shadow stack address and size in thread_struct. This will be copied when cloning new threads, but needs to be cleared during exec, so add a function to do this. 32 bit shadow stack is not expected to have many users and it will complicate the signal implementation. So do not support IA32 emulation or x32. Co-developed-by: Yu-cheng Yu Signed-off-by: Yu-cheng Yu Signed-off-by: Rick Edgecombe Signed-off-by: Dave Hansen Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook Link: https://lore.kernel.org/all/20230613001108.3040476-29-rick.p.edgecombe%40intel.com --- arch/x86/include/asm/processor.h | 2 +- arch/x86/include/asm/shstk.h | 7 +- arch/x86/include/uapi/asm/prctl.h | 3 +- arch/x86/kernel/shstk.c | 145 +++++++++++++++++++++++++++++- 4 files changed, 157 insertions(+) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 4e35f40..b216ac8 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -479,6 +479,8 @@ struct thread_struct { #ifdef CONFIG_X86_USER_SHADOW_STACK unsigned long features; unsigned long features_locked; + + struct thread_shstk shstk; #endif /* Floating point and extended processor state */ diff --git a/arch/x86/include/asm/shstk.h b/arch/x86/include/asm/shstk.h index ec75380..2b1f7c9 100644 --- a/arch/x86/include/asm/shstk.h +++ b/arch/x86/include/asm/shstk.h @@ -8,12 +8,19 @@ struct task_struct; #ifdef CONFIG_X86_USER_SHADOW_STACK +struct thread_shstk { + u64 base; + u64 size; +}; + long shstk_prctl(struct task_struct *task, int option, unsigned long features); void reset_thread_features(void); +void shstk_free(struct task_struct *p); #else static inline long shstk_prctl(struct task_struct *task, int option, unsigned long arg2) { return -EINVAL; } static inline void reset_thread_features(void) {} +static inline void shstk_free(struct task_struct *p) {} #endif /* CONFIG_X86_USER_SHADOW_STACK */ #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/uapi/asm/prctl.h b/arch/x86/include/uapi/asm/prctl.h index 1cd44ec..6a8e0e1 100644 --- a/arch/x86/include/uapi/asm/prctl.h +++ b/arch/x86/include/uapi/asm/prctl.h @@ -34,4 +34,7 @@ #define ARCH_SHSTK_DISABLE 0x5002 #define ARCH_SHSTK_LOCK 0x5003 +/* ARCH_SHSTK_ features bits */ +#define ARCH_SHSTK_SHSTK (1ULL << 0) + #endif /* _ASM_X86_PRCTL_H */ diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 41ed655..3cb8522 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -8,14 +8,159 @@ #include #include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include #include +static bool features_enabled(unsigned long features) +{ + return current->thread.features & features; +} + +static void features_set(unsigned long features) +{ + current->thread.features |= features; +} + +static void features_clr(unsigned long features) +{ + current->thread.features &= ~features; +} + +static unsigned long alloc_shstk(unsigned long size) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; + struct mm_struct *mm = current->mm; + unsigned long addr, unused; + + mmap_write_lock(mm); + addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + + mmap_write_unlock(mm); + + return addr; +} + +static unsigned long adjust_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + +static void unmap_shadow_stack(u64 base, u64 size) +{ + while (1) { + int r; + + r = vm_munmap(base, size); + + /* + * vm_munmap() returns -EINTR when mmap_lock is held by + * something else, and that lock should not be held for a + * long time. Retry it for the case. + */ + if (r == -EINTR) { + cond_resched(); + continue; + } + + /* + * For all other types of vm_munmap() failure, either the + * system is out of memory or there is bug. + */ + WARN_ON_ONCE(r); + break; + } +} + +static int shstk_setup(void) +{ + struct thread_shstk *shstk = ¤t->thread.shstk; + unsigned long addr, size; + + /* Already enabled */ + if (features_enabled(ARCH_SHSTK_SHSTK)) + return 0; + + /* Also not supported for 32 bit and x32 */ + if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || in_32bit_syscall()) + return -EOPNOTSUPP; + + size = adjust_shstk_size(0); + addr = alloc_shstk(size); + if (IS_ERR_VALUE(addr)) + return PTR_ERR((void *)addr); + + fpregs_lock_and_load(); + wrmsrl(MSR_IA32_PL3_SSP, addr + size); + wrmsrl(MSR_IA32_U_CET, CET_SHSTK_EN); + fpregs_unlock(); + + shstk->base = addr; + shstk->size = size; + features_set(ARCH_SHSTK_SHSTK); + + return 0; +} + void reset_thread_features(void) { + memset(¤t->thread.shstk, 0, sizeof(struct thread_shstk)); current->thread.features = 0; current->thread.features_locked = 0; } +void shstk_free(struct task_struct *tsk) +{ + struct thread_shstk *shstk = &tsk->thread.shstk; + + if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || + !features_enabled(ARCH_SHSTK_SHSTK)) + return; + + if (!tsk->mm) + return; + + unmap_shadow_stack(shstk->base, shstk->size); +} + +static int shstk_disable(void) +{ + if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK)) + return -EOPNOTSUPP; + + /* Already disabled? */ + if (!features_enabled(ARCH_SHSTK_SHSTK)) + return 0; + + fpregs_lock_and_load(); + /* Disable WRSS too when disabling shadow stack */ + wrmsrl(MSR_IA32_U_CET, 0); + wrmsrl(MSR_IA32_PL3_SSP, 0); + fpregs_unlock(); + + shstk_free(current); + features_clr(ARCH_SHSTK_SHSTK); + + return 0; +} + long shstk_prctl(struct task_struct *task, int option, unsigned long features) { if (option == ARCH_SHSTK_LOCK) {