[v5,00/39] Shadow stacks for userspace

  Hi,

This series implements Shadow Stacks for userspace using x86's Control-flow 
Enforcement Technology (CET). CET consists of two related security features: 
shadow stacks and indirect branch tracking. This series implements just the 
shadow stack part of this feature, and just for userspace.

The main use case for shadow stack is providing protection against return 
oriented programming attacks. It works by maintaining a secondary (shadow) 
stack using a special memory type that has protections against modification. 
When executing a CALL instruction, the processor pushes the return address to 
both the normal stack and to the special permission shadow stack. Upon RET, 
the processor pops the shadow stack copy and compares it to the normal stack 
copy. For more details, see the coverletter from v1 [0].

The main change in this version is the removal of the attempt to prevent 32 bit 
signals from being registered with shadow stack enabled. Peterz originally 
raised the issue that shadow stack support in 32 bit signals was in a half 
working state. The reason for that was 32 bit signals are not easy to support 
for shadow stack, and also there is not a huge demand for shadow stack support 
in 32 bit apps using 32 bit emulation on 64 bit kernels. At that point the 
solution was to prevent shadow stack from being enabled on 32 bit processes. 
But Peterz pointed that 64 bit apps can transition to 32 bit outside of kernel
interaction by making a far call to a 32 bit segment.

So the next solution was to prevent 32 bit signals from being registered when
shadow stack was enabled. This turned out to be hard to do, due to signals
being per-process and shadow stack being per task.

But it turns out this far call scenario was already mostly not possible due to 
the HW not supporting shadow stacks located outside of the 32 bit address space 
when in 32 bit mode. During the transition to 32 bit mode with an SSP pointing 
outside of the 32 bit address space, HW generates a #GP which in turn triggers 
a segfault. So basically there is already a barrier in place for this far call 
scenario for the most part. Creation of shadow stack memory is tightly 
controlled, so the solution in this version is just to *ensure* that shadow 
stacks can never be allocated in the 32 bit address space. For more information 
see the new patch: "x86/mm: Introduce MAP_ABOVE4G", and the documentation in 
patch 1.

Additionally:
 - A smattering of small changes from Boris and Kees
 - Fixed my spellcheck setup and then fixed a bunch of spelling issues in the
   commit logs.
 - An update to the pte_modify() PAGE_COW solution
 
I left tested-by tags in place per discussion with testers. Testers, please
retest.

Previous version [1].

[0] https://lore.kernel.org/lkml/20220130211838.8382-1-rick.p.edgecombe@intel.com/
[1] https://lore.kernel.org/lkml/20221203003606.6838-1-rick.p.edgecombe@intel.com/

Kirill A. Shutemov (1):
  x86: Introduce userspace API for shadow stack

Mike Rapoport (1):
  x86/shstk: Add ARCH_SHSTK_UNLOCK

Rick Edgecombe (14):
  x86/fpu: Add helper for modifying xstate
  x86/mm: Introduce _PAGE_COW
  x86/mm: Start actually marking _PAGE_COW
  mm: Handle faultless write upgrades for shstk
  mm: Don't allow write GUPs to shadow stack memory
  x86/mm: Introduce MAP_ABOVE4G
  mm: Warn on shadow stack memory in wrong vma
  x86/shstk: Introduce map_shadow_stack syscall
  x86/shstk: Support WRSS for userspace
  x86: Expose thread features in /proc/$PID/status
  x86/shstk: Wire in shadow stack interface
  selftests/x86: Add shadow stack test
  x86/fpu: Add helper for initing features
  x86/shstk: Add ARCH_SHSTK_STATUS

Yu-cheng Yu (23):
  Documentation/x86: Add CET shadow stack description
  x86/shstk: Add Kconfig option for shadow stack
  x86/cpufeatures: Add CPU feature flags for shadow stacks
  x86/cpufeatures: Enable CET CR4 bit for shadow stack
  x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states
  x86: Add user control-protection fault handler
  x86/mm: Remove _PAGE_DIRTY from kernel RO pages
  x86/mm: Move pmd_write(), pud_write() up in the file
  x86/mm: Update pte_modify for _PAGE_COW
  x86/mm: Update ptep_set_wrprotect() and pmdp_set_wrprotect() for
    transition from _PAGE_DIRTY to _PAGE_COW
  mm: Move VM_UFFD_MINOR_BIT from 37 to 38
  mm: Introduce VM_SHADOW_STACK for shadow stack memory
  x86/mm: Check shadow stack page fault errors
  x86/mm: Update maybe_mkwrite() for shadow stack
  mm: Fixup places that call pte_mkwrite() directly
  mm: Add guard pages around a shadow stack.
  mm/mmap: Add shadow stack pages to memory accounting
  mm: Re-introduce vm_flags to do_mmap()
  x86/shstk: Add user-mode shadow stack support
  x86/shstk: Handle thread shadow stack
  x86/shstk: Introduce routines modifying shstk
  x86/shstk: Handle signals for shadow stack
  x86: Add PTRACE interface for shadow stack

 Documentation/filesystems/proc.rst            |   1 +
 Documentation/x86/index.rst                   |   1 +
 Documentation/x86/shstk.rst                   | 176 +++++
 arch/arm/kernel/signal.c                      |   2 +-
 arch/arm64/kernel/signal.c                    |   2 +-
 arch/arm64/kernel/signal32.c                  |   2 +-
 arch/sparc/kernel/signal32.c                  |   2 +-
 arch/sparc/kernel/signal_64.c                 |   2 +-
 arch/x86/Kconfig                              |  24 +
 arch/x86/Kconfig.assembler                    |   5 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 arch/x86/include/asm/cpufeatures.h            |   2 +
 arch/x86/include/asm/disabled-features.h      |  16 +-
 arch/x86/include/asm/fpu/api.h                |   9 +
 arch/x86/include/asm/fpu/regset.h             |   7 +-
 arch/x86/include/asm/fpu/sched.h              |   3 +-
 arch/x86/include/asm/fpu/types.h              |  16 +-
 arch/x86/include/asm/fpu/xstate.h             |   6 +-
 arch/x86/include/asm/idtentry.h               |   2 +-
 arch/x86/include/asm/mmu_context.h            |   2 +
 arch/x86/include/asm/msr.h                    |  11 +
 arch/x86/include/asm/pgtable.h                | 338 ++++++++-
 arch/x86/include/asm/pgtable_types.h          |  65 +-
 arch/x86/include/asm/processor.h              |   8 +
 arch/x86/include/asm/shstk.h                  |  40 ++
 arch/x86/include/asm/special_insns.h          |  13 +
 arch/x86/include/asm/tlbflush.h               |   3 +-
 arch/x86/include/asm/trap_pf.h                |   2 +
 arch/x86/include/asm/traps.h                  |  12 +
 arch/x86/include/uapi/asm/mman.h              |   4 +
 arch/x86/include/uapi/asm/prctl.h             |  12 +
 arch/x86/kernel/Makefile                      |   4 +
 arch/x86/kernel/cet.c                         | 152 ++++
 arch/x86/kernel/cpu/common.c                  |  35 +-
 arch/x86/kernel/cpu/cpuid-deps.c              |   1 +
 arch/x86/kernel/cpu/proc.c                    |  23 +
 arch/x86/kernel/fpu/core.c                    |  59 +-
 arch/x86/kernel/fpu/regset.c                  |  87 +++
 arch/x86/kernel/fpu/xstate.c                  | 148 ++--
 arch/x86/kernel/fpu/xstate.h                  |   6 +
 arch/x86/kernel/idt.c                         |   2 +-
 arch/x86/kernel/process.c                     |  18 +-
 arch/x86/kernel/process_64.c                  |   9 +-
 arch/x86/kernel/ptrace.c                      |  12 +
 arch/x86/kernel/shstk.c                       | 492 +++++++++++++
 arch/x86/kernel/signal.c                      |   1 +
 arch/x86/kernel/signal_32.c                   |   2 +-
 arch/x86/kernel/signal_64.c                   |   8 +-
 arch/x86/kernel/sys_x86_64.c                  |   6 +-
 arch/x86/kernel/traps.c                       |  87 ---
 arch/x86/mm/fault.c                           |  38 +
 arch/x86/mm/pat/set_memory.c                  |   2 +-
 arch/x86/mm/pgtable.c                         |   6 +
 arch/x86/xen/enlighten_pv.c                   |   2 +-
 arch/x86/xen/xen-asm.S                        |   2 +-
 fs/aio.c                                      |   2 +-
 fs/proc/array.c                               |   6 +
 fs/proc/task_mmu.c                            |   3 +
 include/linux/mm.h                            |  59 +-
 include/linux/mman.h                          |   4 +
 include/linux/pgtable.h                       |  35 +
 include/linux/proc_fs.h                       |   2 +
 include/linux/syscalls.h                      |   1 +
 include/uapi/asm-generic/siginfo.h            |   3 +-
 include/uapi/asm-generic/unistd.h             |   2 +-
 include/uapi/linux/elf.h                      |   2 +
 ipc/shm.c                                     |   2 +-
 kernel/sys_ni.c                               |   1 +
 mm/gup.c                                      |   2 +-
 mm/huge_memory.c                              |  12 +-
 mm/memory.c                                   |   7 +-
 mm/migrate_device.c                           |   4 +-
 mm/mmap.c                                     |  12 +-
 mm/nommu.c                                    |   4 +-
 mm/userfaultfd.c                              |  10 +-
 mm/util.c                                     |   2 +-
 tools/testing/selftests/x86/Makefile          |   4 +-
 .../testing/selftests/x86/test_shadow_stack.c | 667 ++++++++++++++++++
 78 files changed, 2578 insertions(+), 259 deletions(-)
 create mode 100644 Documentation/x86/shstk.rst
 create mode 100644 arch/x86/include/asm/shstk.h
 create mode 100644 arch/x86/kernel/cet.c
 create mode 100644 arch/x86/kernel/shstk.c
 create mode 100644 tools/testing/selftests/x86/test_shadow_stack.c
  

On Thu, 19 Jan 2023 13:22:38 -0800 Rick Edgecombe <rick.p.edgecombe@intel.com> wrote:

> SHSTK

Sounds like me trying to swear in Russian while drunk.

Is there any chance of s/shstk/shadow_stack/g?
  

On Thu, 2023-01-19 at 14:26 -0800, Andrew Morton wrote:
> On Thu, 19 Jan 2023 13:22:38 -0800 Rick Edgecombe <
> rick.p.edgecombe@intel.com> wrote:
> 
> > SHSTK
> 
> Sounds like me trying to swear in Russian while drunk.
> 
> Is there any chance of s/shstk/shadow_stack/g?

I'm fine with the name change. I think shstk got debated and picked
early in the history of the series before I got involved. "shstk" is
nice and short, but it's not completely clear what it is unless you
already know about shadow stack. So there is a tradeoff of clarity and
line length/wrapping. Does anyone else have any strong opinions?
  

On 1/19/23 3:22 PM, Rick Edgecombe wrote:
> I left tested-by tags in place per discussion with testers. Testers, please
> retest.

Re-tested on my AMD system (Dell PowerEdge R6515 w/ EPYC 7713) and it looks
like everything is still working properly.

The selftests seem to run cleanly:

[INFO]	new_ssp = 7ff19be0dff8, *new_ssp = 7ff19be0e001
[INFO]	changing ssp from 7ff19c7f1ff0 to 7ff19be0dff8
[INFO]	ssp is now 7ff19be0e000
[OK]	Shadow stack pivot
[OK]	Shadow stack faults
[INFO]	Corrupting shadow stack
[INFO]	Generated shadow stack violation successfully
[OK]	Shadow stack violation test
[INFO]	Gup read -> shstk access success
[INFO]	Gup write -> shstk access success
[INFO]	Violation from normal write
[INFO]	Gup read -> write access success
[INFO]	Violation from normal write
[INFO]	Gup write -> write access success
[INFO]	Cow gup write -> write access success
[OK]	Shadow gup test
[INFO]	Violation from shstk access
[OK]	mprotect() test
[OK]	Userfaultfd test
[OK]	32 bit test

Additionally, I could see the control protection messages in dmesg when
running the shstk violation test from here:
https://gitlab.com/cet-software/cet-smoke-test

ld-linux-x86-64[99764] control protection ip:401139 sp:7fff025507d8 ssp:7f186e017fd8 error:1(near ret) in shstk1[401000+1000]

Tested-by: John Allen <john.allen@amd.com>
  

On Fri, Jan 20, 2023 at 05:27:30PM +0000, Edgecombe, Rick P wrote:
> On Thu, 2023-01-19 at 14:26 -0800, Andrew Morton wrote:
> > On Thu, 19 Jan 2023 13:22:38 -0800 Rick Edgecombe <
> > rick.p.edgecombe@intel.com> wrote:
> > 
> > > SHSTK
> > 
> > Sounds like me trying to swear in Russian while drunk.
> > 
> > Is there any chance of s/shstk/shadow_stack/g?
> 
> I'm fine with the name change. I think shstk got debated and picked
> early in the history of the series before I got involved. "shstk" is
> nice and short, but it's not completely clear what it is unless you
> already know about shadow stack. So there is a tradeoff of clarity and
> line length/wrapping. Does anyone else have any strong opinions?

I prefer SHSTK because it specifically means x86's hardware shadow
stack from CET. Lots of things can (and have) implemented things called
"shadow stack".
  

On Thu, Jan 19, 2023 at 01:22:38PM -0800, Rick Edgecombe wrote:
> Hi,
> 
> This series implements Shadow Stacks for userspace using x86's Control-flow 
> Enforcement Technology (CET). CET consists of two related security features: 
> shadow stacks and indirect branch tracking. This series implements just the 
> shadow stack part of this feature, and just for userspace.

Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>

> Previous version [1].
> 
> [0] https://lore.kernel.org/lkml/20220130211838.8382-1-rick.p.edgecombe@intel.com/
> [1] https://lore.kernel.org/lkml/20221203003606.6838-1-rick.p.edgecombe@intel.com/
> 
> Kirill A. Shutemov (1):
>   x86: Introduce userspace API for shadow stack
> 
> Mike Rapoport (1):
>   x86/shstk: Add ARCH_SHSTK_UNLOCK
> 
> Rick Edgecombe (14):
>   x86/fpu: Add helper for modifying xstate
>   x86/mm: Introduce _PAGE_COW
>   x86/mm: Start actually marking _PAGE_COW
>   mm: Handle faultless write upgrades for shstk
>   mm: Don't allow write GUPs to shadow stack memory
>   x86/mm: Introduce MAP_ABOVE4G
>   mm: Warn on shadow stack memory in wrong vma
>   x86/shstk: Introduce map_shadow_stack syscall
>   x86/shstk: Support WRSS for userspace
>   x86: Expose thread features in /proc/$PID/status
>   x86/shstk: Wire in shadow stack interface
>   selftests/x86: Add shadow stack test
>   x86/fpu: Add helper for initing features
>   x86/shstk: Add ARCH_SHSTK_STATUS
> 
> Yu-cheng Yu (23):
>   Documentation/x86: Add CET shadow stack description
>   x86/shstk: Add Kconfig option for shadow stack
>   x86/cpufeatures: Add CPU feature flags for shadow stacks
>   x86/cpufeatures: Enable CET CR4 bit for shadow stack
>   x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states
>   x86: Add user control-protection fault handler
>   x86/mm: Remove _PAGE_DIRTY from kernel RO pages
>   x86/mm: Move pmd_write(), pud_write() up in the file
>   x86/mm: Update pte_modify for _PAGE_COW
>   x86/mm: Update ptep_set_wrprotect() and pmdp_set_wrprotect() for
>     transition from _PAGE_DIRTY to _PAGE_COW
>   mm: Move VM_UFFD_MINOR_BIT from 37 to 38
>   mm: Introduce VM_SHADOW_STACK for shadow stack memory
>   x86/mm: Check shadow stack page fault errors
>   x86/mm: Update maybe_mkwrite() for shadow stack
>   mm: Fixup places that call pte_mkwrite() directly
>   mm: Add guard pages around a shadow stack.
>   mm/mmap: Add shadow stack pages to memory accounting
>   mm: Re-introduce vm_flags to do_mmap()
>   x86/shstk: Add user-mode shadow stack support
>   x86/shstk: Handle thread shadow stack
>   x86/shstk: Introduce routines modifying shstk
>   x86/shstk: Handle signals for shadow stack
>   x86: Add PTRACE interface for shadow stack
> 
>  Documentation/filesystems/proc.rst            |   1 +
>  Documentation/x86/index.rst                   |   1 +
>  Documentation/x86/shstk.rst                   | 176 +++++
>  arch/arm/kernel/signal.c                      |   2 +-
>  arch/arm64/kernel/signal.c                    |   2 +-
>  arch/arm64/kernel/signal32.c                  |   2 +-
>  arch/sparc/kernel/signal32.c                  |   2 +-
>  arch/sparc/kernel/signal_64.c                 |   2 +-
>  arch/x86/Kconfig                              |  24 +
>  arch/x86/Kconfig.assembler                    |   5 +
>  arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
>  arch/x86/include/asm/cpufeatures.h            |   2 +
>  arch/x86/include/asm/disabled-features.h      |  16 +-
>  arch/x86/include/asm/fpu/api.h                |   9 +
>  arch/x86/include/asm/fpu/regset.h             |   7 +-
>  arch/x86/include/asm/fpu/sched.h              |   3 +-
>  arch/x86/include/asm/fpu/types.h              |  16 +-
>  arch/x86/include/asm/fpu/xstate.h             |   6 +-
>  arch/x86/include/asm/idtentry.h               |   2 +-
>  arch/x86/include/asm/mmu_context.h            |   2 +
>  arch/x86/include/asm/msr.h                    |  11 +
>  arch/x86/include/asm/pgtable.h                | 338 ++++++++-
>  arch/x86/include/asm/pgtable_types.h          |  65 +-
>  arch/x86/include/asm/processor.h              |   8 +
>  arch/x86/include/asm/shstk.h                  |  40 ++
>  arch/x86/include/asm/special_insns.h          |  13 +
>  arch/x86/include/asm/tlbflush.h               |   3 +-
>  arch/x86/include/asm/trap_pf.h                |   2 +
>  arch/x86/include/asm/traps.h                  |  12 +
>  arch/x86/include/uapi/asm/mman.h              |   4 +
>  arch/x86/include/uapi/asm/prctl.h             |  12 +
>  arch/x86/kernel/Makefile                      |   4 +
>  arch/x86/kernel/cet.c                         | 152 ++++
>  arch/x86/kernel/cpu/common.c                  |  35 +-
>  arch/x86/kernel/cpu/cpuid-deps.c              |   1 +
>  arch/x86/kernel/cpu/proc.c                    |  23 +
>  arch/x86/kernel/fpu/core.c                    |  59 +-
>  arch/x86/kernel/fpu/regset.c                  |  87 +++
>  arch/x86/kernel/fpu/xstate.c                  | 148 ++--
>  arch/x86/kernel/fpu/xstate.h                  |   6 +
>  arch/x86/kernel/idt.c                         |   2 +-
>  arch/x86/kernel/process.c                     |  18 +-
>  arch/x86/kernel/process_64.c                  |   9 +-
>  arch/x86/kernel/ptrace.c                      |  12 +
>  arch/x86/kernel/shstk.c                       | 492 +++++++++++++
>  arch/x86/kernel/signal.c                      |   1 +
>  arch/x86/kernel/signal_32.c                   |   2 +-
>  arch/x86/kernel/signal_64.c                   |   8 +-
>  arch/x86/kernel/sys_x86_64.c                  |   6 +-
>  arch/x86/kernel/traps.c                       |  87 ---
>  arch/x86/mm/fault.c                           |  38 +
>  arch/x86/mm/pat/set_memory.c                  |   2 +-
>  arch/x86/mm/pgtable.c                         |   6 +
>  arch/x86/xen/enlighten_pv.c                   |   2 +-
>  arch/x86/xen/xen-asm.S                        |   2 +-
>  fs/aio.c                                      |   2 +-
>  fs/proc/array.c                               |   6 +
>  fs/proc/task_mmu.c                            |   3 +
>  include/linux/mm.h                            |  59 +-
>  include/linux/mman.h                          |   4 +
>  include/linux/pgtable.h                       |  35 +
>  include/linux/proc_fs.h                       |   2 +
>  include/linux/syscalls.h                      |   1 +
>  include/uapi/asm-generic/siginfo.h            |   3 +-
>  include/uapi/asm-generic/unistd.h             |   2 +-
>  include/uapi/linux/elf.h                      |   2 +
>  ipc/shm.c                                     |   2 +-
>  kernel/sys_ni.c                               |   1 +
>  mm/gup.c                                      |   2 +-
>  mm/huge_memory.c                              |  12 +-
>  mm/memory.c                                   |   7 +-
>  mm/migrate_device.c                           |   4 +-
>  mm/mmap.c                                     |  12 +-
>  mm/nommu.c                                    |   4 +-
>  mm/userfaultfd.c                              |  10 +-
>  mm/util.c                                     |   2 +-
>  tools/testing/selftests/x86/Makefile          |   4 +-
>  .../testing/selftests/x86/test_shadow_stack.c | 667 ++++++++++++++++++
>  78 files changed, 2578 insertions(+), 259 deletions(-)
>  create mode 100644 Documentation/x86/shstk.rst
>  create mode 100644 arch/x86/include/asm/shstk.h
>  create mode 100644 arch/x86/kernel/cet.c
>  create mode 100644 arch/x86/kernel/shstk.c
>  create mode 100644 tools/testing/selftests/x86/test_shadow_stack.c
> 
> -- 
> 2.17.1
>
  

On Fri, 2023-01-20 at 11:19 -0800, Kees Cook wrote:
> On Fri, Jan 20, 2023 at 05:27:30PM +0000, Edgecombe, Rick P wrote:
> > On Thu, 2023-01-19 at 14:26 -0800, Andrew Morton wrote:
> > > On Thu, 19 Jan 2023 13:22:38 -0800 Rick Edgecombe <
> > > rick.p.edgecombe@intel.com> wrote:
> > > 
> > > > SHSTK
> > > 
> > > Sounds like me trying to swear in Russian while drunk.
> > > 
> > > Is there any chance of s/shstk/shadow_stack/g?
> > 
> > I'm fine with the name change. I think shstk got debated and picked
> > early in the history of the series before I got involved. "shstk"
> > is
> > nice and short, but it's not completely clear what it is unless you
> > already know about shadow stack. So there is a tradeoff of clarity
> > and
> > line length/wrapping. Does anyone else have any strong opinions?
> 
> I prefer SHSTK because it specifically means x86's hardware shadow
> stack from CET. Lots of things can (and have) implemented things
> called
> "shadow stack".

This makes sense to, especially if we can hide it more from the core-mm 
code per David Hildebrand's suggestion. I guess I'll leave it for now
unless anyone else has a stronger opinion.