From patchwork Mon Feb 27 22:29:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 5913 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp2682301wrd; Mon, 27 Feb 2023 14:33:31 -0800 (PST) X-Google-Smtp-Source: AK7set9C0L753VU15t9/giBjxnvyH8X990Y/L7xbpjC65BlvsznTZRrBqRAjYz5h1PDyMg6igoJN X-Received: by 2002:a17:907:c71a:b0:8e0:2887:8263 with SMTP id ty26-20020a170907c71a00b008e028878263mr330003ejc.39.1677537211817; Mon, 27 Feb 2023 14:33:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677537211; cv=none; d=google.com; s=arc-20160816; b=dyPOvnXevV9jUQ5QK5zg1w/XVq24+sdqxl4MQldLpPeuRlUPx+NtgmdB8Mre8Ym3NV S8xs+TdibNUedp6JrEFw30GEztr5Y4eOeWHsfO3JzSJ362DMidtERBzcjxROqCSRUw3W jYBEzqrTSRUf93/rQAM3AamhF/0PDIfy8zTTYfSFWwTu5SVPkXkzZnUymvS9DLqoKtmP DirvMD5NRWmHyuni6NoQm+S0groNKoB4sy08xrC6hCC4Ele8IMyiUsQQKskouOlFt2ez B+Y/819zklXqgYykmFnjvQr04YATaNDJWj8QJm4n115bhS+w7kOCVdK0Dd/UUp/UxPev V8GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=5HdbklBkPekiXe8PnrG3ZCQoVaTjjLytNWI9dALaCUQ=; b=0WCsqQqvoRqfc3LsX0PoX8RjZUSkwrR43AoXC+8f643m1S3aZloIQ5h8hk1vj20kiR GVBMSZUbSIy026AQMzGEWYQURz2Whh4xhnwr9K1punYCeZtwAZOmsEySR5+CV3Fpf+MJ dE2ecPdXRqJchq8yDUkLIkhiBNPl/+9S9uughSj4IrkbytVKe2OeRtMhWa5atVGa6Kqz ovmzTfv0WYx7kSBd5manLbS9i0lxIB4WQOr8svkbZsRrSCZLHPNQ2TptZu3Vv3rfVKFP 4MYzO1hXGiuFP+EdPc02sCWv5Ns5dpoXHmi5166G7q1dMuDGkhjCpfFPAW03IuCXMapH o1DQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CY3GFHSK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b2-20020a056402138200b004aba73a1814si9926454edv.6.2023.02.27.14.33.08; Mon, 27 Feb 2023 14:33:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CY3GFHSK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229549AbjB0Wb3 (ORCPT + 99 others); Mon, 27 Feb 2023 17:31:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229882AbjB0Wb1 (ORCPT ); Mon, 27 Feb 2023 17:31:27 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C84025BBA; Mon, 27 Feb 2023 14:31:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677537086; x=1709073086; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=wgxjh9QgD3JL/mbNIdl6volkZkIB/jWnSXcuFGCrqwg=; b=CY3GFHSKn2onDY6JzsM1AC1wf7tGcSGSgBaetbbYbg2cBsyyBwrFv7EA 1uobanq6HIisq/xL6ZVhi0p9UcE1W2B7Iu6ZsnuIC7RE+FbtVhsAGpl61 CegWNlZL7X+Z1tQ5kAsi9Jcu8BE52n3H1Vbtfn6vADGeb7B+M5x2Yd92s M3oHUkgakpl5Y+rk4nQyUQAvsMoQTG2/9xTFNBdOqnfnvlWt0gFETwCn2 FrAWHQxt4iVlyv2SxkrSWLan7C+Ovoa5sCgSfiJ/t8AUQoT39dMYaREhU 79r75Rc6eq36SvL4ad/2JXyrgHjRSICyHq5oz1xyHYkbA3uWX0xoI5L0f w==; X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="313656974" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="313656974" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:07 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10634"; a="848024329" X-IronPort-AV: E=Sophos;i="5.98,220,1673942400"; d="scan'208";a="848024329" Received: from leonqu-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.72.19]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Feb 2023 14:31:06 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v7 00/41] Shadow stacks for userspace Date: Mon, 27 Feb 2023 14:29:16 -0800 Message-Id: <20230227222957.24501-1-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759025259504178977?= X-GMAIL-MSGID: =?utf-8?q?1759025259504178977?= Hi, This series implements Shadow Stacks for userspace using x86's Control-flow Enforcement Technology (CET). CET consists of two related security features: shadow stacks and indirect branch tracking. This series implements just the shadow stack part of this feature, and just for userspace. The main use case for shadow stack is providing protection against return oriented programming attacks. It works by maintaining a secondary (shadow) stack using a special memory type that has protections against modification. When executing a CALL instruction, the processor pushes the return address to both the normal stack and to the special permission shadow stack. Upon RET, the processor pops the shadow stack copy and compares it to the normal stack copy. For more details, see the coverletter from v1 [0]. The changes for this version are some more cleanup of comment and commit log verbiage, and small refactor in the memory accounting patch. There was also some feedback from David Hildenbrand about adding GUP tests for the !FOLL_FORCE case. This is currently planned for a fast follow on patch. Previous version [1]. Thanks, Rick [0] https://lore.kernel.org/lkml/20220130211838.8382-1-rick.p.edgecombe@intel.com/ [1] https://lore.kernel.org/lkml/20230218211433.26859-1-rick.p.edgecombe@intel.com/ Kirill A. Shutemov (1): x86: Introduce userspace API for shadow stack Mike Rapoport (1): x86/shstk: Add ARCH_SHSTK_UNLOCK Rick Edgecombe (19): x86/fpu: Add helper for modifying xstate x86: Move control protection handler to separate file mm: Introduce pte_mkwrite_kernel() s390/mm: Introduce pmd_mkwrite_kernel() mm: Make pte_mkwrite() take a VMA x86/mm: Introduce _PAGE_SAVED_DIRTY x86/mm: Start actually marking _PAGE_SAVED_DIRTY x86/mm: Teach pte_mkwrite() about stack memory mm: Don't allow write GUPs to shadow stack memory x86/mm: Introduce MAP_ABOVE4G mm: Warn on shadow stack memory in wrong vma x86/mm: Warn if create Write=0,Dirty=1 with raw prot x86/shstk: Introduce map_shadow_stack syscall x86/shstk: Support WRSS for userspace x86: Expose thread features in /proc/$PID/status x86/shstk: Wire in shadow stack interface selftests/x86: Add shadow stack test x86/fpu: Add helper for initing features x86/shstk: Add ARCH_SHSTK_STATUS Yu-cheng Yu (20): Documentation/x86: Add CET shadow stack description x86/shstk: Add Kconfig option for shadow stack x86/cpufeatures: Add CPU feature flags for shadow stacks x86/cpufeatures: Enable CET CR4 bit for shadow stack x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states x86/shstk: Add user control-protection fault handler x86/mm: Remove _PAGE_DIRTY from kernel RO pages x86/mm: Move pmd_write(), pud_write() up in the file x86/mm: Update ptep/pmdp_set_wrprotect() for _PAGE_SAVED_DIRTY mm: Move VM_UFFD_MINOR_BIT from 37 to 38 mm: Introduce VM_SHADOW_STACK for shadow stack memory x86/mm: Check shadow stack page fault errors mm: Add guard pages around a shadow stack. mm/mmap: Add shadow stack pages to memory accounting mm: Re-introduce vm_flags to do_mmap() x86/shstk: Add user-mode shadow stack support x86/shstk: Handle thread shadow stack x86/shstk: Introduce routines modifying shstk x86/shstk: Handle signals for shadow stack x86: Add PTRACE interface for shadow stack Documentation/filesystems/proc.rst | 1 + Documentation/mm/arch_pgtable_helpers.rst | 9 +- Documentation/x86/index.rst | 1 + Documentation/x86/shstk.rst | 176 +++++ arch/alpha/include/asm/pgtable.h | 6 +- arch/arc/include/asm/hugepage.h | 2 +- arch/arc/include/asm/pgtable-bits-arcv2.h | 7 +- arch/arm/include/asm/pgtable-3level.h | 7 +- arch/arm/include/asm/pgtable.h | 2 +- arch/arm/kernel/signal.c | 2 +- arch/arm64/include/asm/pgtable.h | 9 +- arch/arm64/kernel/signal.c | 2 +- arch/arm64/kernel/signal32.c | 2 +- arch/arm64/mm/trans_pgd.c | 4 +- arch/csky/include/asm/pgtable.h | 2 +- arch/hexagon/include/asm/pgtable.h | 2 +- arch/ia64/include/asm/pgtable.h | 2 +- arch/loongarch/include/asm/pgtable.h | 4 +- arch/m68k/include/asm/mcf_pgtable.h | 2 +- arch/m68k/include/asm/motorola_pgtable.h | 6 +- arch/m68k/include/asm/sun3_pgtable.h | 6 +- arch/microblaze/include/asm/pgtable.h | 2 +- arch/mips/include/asm/pgtable.h | 6 +- arch/nios2/include/asm/pgtable.h | 2 +- arch/openrisc/include/asm/pgtable.h | 2 +- arch/parisc/include/asm/pgtable.h | 6 +- arch/powerpc/include/asm/book3s/32/pgtable.h | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 4 +- arch/powerpc/include/asm/nohash/32/pgtable.h | 2 +- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 2 +- arch/powerpc/include/asm/nohash/64/pgtable.h | 2 +- arch/riscv/include/asm/pgtable.h | 6 +- arch/s390/include/asm/hugetlb.h | 4 +- arch/s390/include/asm/pgtable.h | 14 +- arch/s390/mm/pageattr.c | 4 +- arch/sh/include/asm/pgtable_32.h | 10 +- arch/sparc/include/asm/pgtable_32.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 6 +- arch/sparc/kernel/signal32.c | 2 +- arch/sparc/kernel/signal_64.c | 2 +- arch/um/include/asm/pgtable.h | 2 +- arch/x86/Kconfig | 24 + arch/x86/Kconfig.assembler | 5 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/disabled-features.h | 16 +- arch/x86/include/asm/fpu/api.h | 9 + arch/x86/include/asm/fpu/regset.h | 7 +- arch/x86/include/asm/fpu/sched.h | 3 +- arch/x86/include/asm/fpu/types.h | 16 +- arch/x86/include/asm/fpu/xstate.h | 6 +- arch/x86/include/asm/idtentry.h | 2 +- arch/x86/include/asm/mmu_context.h | 2 + arch/x86/include/asm/msr.h | 11 + arch/x86/include/asm/pgtable.h | 322 +++++++- arch/x86/include/asm/pgtable_types.h | 56 +- arch/x86/include/asm/processor.h | 8 + arch/x86/include/asm/shstk.h | 40 + arch/x86/include/asm/special_insns.h | 13 + arch/x86/include/asm/tlbflush.h | 3 +- arch/x86/include/asm/trap_pf.h | 2 + arch/x86/include/asm/traps.h | 12 + arch/x86/include/uapi/asm/mman.h | 4 + arch/x86/include/uapi/asm/prctl.h | 12 + arch/x86/kernel/Makefile | 4 + arch/x86/kernel/cet.c | 152 ++++ arch/x86/kernel/cpu/common.c | 35 +- arch/x86/kernel/cpu/cpuid-deps.c | 1 + arch/x86/kernel/cpu/proc.c | 23 + arch/x86/kernel/fpu/core.c | 59 +- arch/x86/kernel/fpu/regset.c | 86 +++ arch/x86/kernel/fpu/xstate.c | 148 ++-- arch/x86/kernel/fpu/xstate.h | 6 + arch/x86/kernel/idt.c | 2 +- arch/x86/kernel/process.c | 18 +- arch/x86/kernel/process_64.c | 9 +- arch/x86/kernel/ptrace.c | 12 + arch/x86/kernel/shstk.c | 491 +++++++++++++ arch/x86/kernel/signal.c | 1 + arch/x86/kernel/signal_32.c | 2 +- arch/x86/kernel/signal_64.c | 8 +- arch/x86/kernel/sys_x86_64.c | 6 +- arch/x86/kernel/traps.c | 87 --- arch/x86/mm/fault.c | 31 + arch/x86/mm/pat/set_memory.c | 4 +- arch/x86/mm/pgtable.c | 38 + arch/x86/xen/enlighten_pv.c | 2 +- arch/x86/xen/mmu_pv.c | 2 +- arch/x86/xen/xen-asm.S | 2 +- arch/xtensa/include/asm/pgtable.h | 2 +- fs/aio.c | 2 +- fs/proc/array.c | 6 + fs/proc/task_mmu.c | 3 + include/asm-generic/hugetlb.h | 4 +- include/linux/mm.h | 46 +- include/linux/mman.h | 4 + include/linux/pgtable.h | 14 + include/linux/proc_fs.h | 2 + include/linux/syscalls.h | 1 + include/uapi/asm-generic/siginfo.h | 3 +- include/uapi/asm-generic/unistd.h | 2 +- include/uapi/linux/elf.h | 2 + ipc/shm.c | 2 +- kernel/sys_ni.c | 1 + mm/debug_vm_pgtable.c | 16 +- mm/gup.c | 2 +- mm/huge_memory.c | 7 +- mm/hugetlb.c | 4 +- mm/internal.h | 8 +- mm/memory.c | 5 +- mm/migrate_device.c | 2 +- mm/mmap.c | 10 +- mm/mprotect.c | 2 +- mm/nommu.c | 4 +- mm/userfaultfd.c | 2 +- mm/util.c | 2 +- tools/testing/selftests/x86/Makefile | 2 +- .../testing/selftests/x86/test_shadow_stack.c | 695 ++++++++++++++++++ 118 files changed, 2669 insertions(+), 327 deletions(-) create mode 100644 Documentation/x86/shstk.rst create mode 100644 arch/x86/include/asm/shstk.h create mode 100644 arch/x86/kernel/cet.c create mode 100644 arch/x86/kernel/shstk.c create mode 100644 tools/testing/selftests/x86/test_shadow_stack.c