[v6,00/41] Shadow stacks for userspace

  Hi,

This series implements Shadow Stacks for userspace using x86's Control-flow 
Enforcement Technology (CET). CET consists of two related security features: 
shadow stacks and indirect branch tracking. This series implements just the 
shadow stack part of this feature, and just for userspace.

The main use case for shadow stack is providing protection against return 
oriented programming attacks. It works by maintaining a secondary (shadow) 
stack using a special memory type that has protections against modification. 
When executing a CALL instruction, the processor pushes the return address to 
both the normal stack and to the special permission shadow stack. Upon RET, 
the processor pops the shadow stack copy and compares it to the normal stack 
copy. For more details, see the coverletter from v1 [0].

The main changes in this version are the MM suggestions by David Hildenbrand
to have pte_mkwrite() take a vma, and rename _PAGE_COW.
The former is split over three patches:
mm: Introduce pte_mkwrite_kernel()
s390/mm: Introduce pmd_mkwrite_kernel()
mm: Make pte_mkwrite() take a VMA

With these changes, and an adjustment to "mm: Warn on shadow stack memory in
wrong vma", references to "shstk" are now only in x86 arch code, hopefully
addressing Andrew Morton's concerns. There are still a couple VM_SHADOW_STACK 
references, which seems to be in keeping with the treatment of other
VM_HIGH_ARCH flags. If other shadow stack implementations end up with identical
logic, it can easily be refactored at that point.

There was also some more feedback from Boris which was incorporated.

I left tested-by tags in place per discussion with testers. Testers, please
retest.

Previous version [1].

Thanks,
Rick


[0] https://lore.kernel.org/lkml/20220130211838.8382-1-rick.p.edgecombe@intel.com/
[1] https://lore.kernel.org/lkml/20230119212317.8324-1-rick.p.edgecombe@intel.com/


Kirill A. Shutemov (1):
  x86: Introduce userspace API for shadow stack

Mike Rapoport (1):
  x86/shstk: Add ARCH_SHSTK_UNLOCK

Rick Edgecombe (19):
  x86/fpu: Add helper for modifying xstate
  x86: Move control protection handler to separate file
  mm: Introduce pte_mkwrite_kernel()
  s390/mm: Introduce pmd_mkwrite_kernel()
  mm: Make pte_mkwrite() take a VMA
  x86/mm: Introduce _PAGE_SAVED_DIRTY
  x86/mm: Start actually marking _PAGE_SAVED_DIRTY
  x86/mm: Teach pte_mkwrite() about stack memory
  mm: Don't allow write GUPs to shadow stack memory
  x86/mm: Introduce MAP_ABOVE4G
  mm: Warn on shadow stack memory in wrong vma
  x86/mm: Warn if create Write=0,Dirty=1 with raw prot
  x86/shstk: Introduce map_shadow_stack syscall
  x86/shstk: Support WRSS for userspace
  x86: Expose thread features in /proc/$PID/status
  x86/shstk: Wire in shadow stack interface
  selftests/x86: Add shadow stack test
  x86/fpu: Add helper for initing features
  x86/shstk: Add ARCH_SHSTK_STATUS

Yu-cheng Yu (20):
  Documentation/x86: Add CET shadow stack description
  x86/shstk: Add Kconfig option for shadow stack
  x86/cpufeatures: Add CPU feature flags for shadow stacks
  x86/cpufeatures: Enable CET CR4 bit for shadow stack
  x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states
  x86/shstk: Add user control-protection fault handler
  x86/mm: Remove _PAGE_DIRTY from kernel RO pages
  x86/mm: Move pmd_write(), pud_write() up in the file
  x86/mm: Update ptep/pmdp_set_wrprotect() for _PAGE_SAVED_DIRTY
  mm: Move VM_UFFD_MINOR_BIT from 37 to 38
  mm: Introduce VM_SHADOW_STACK for shadow stack memory
  x86/mm: Check shadow stack page fault errors
  mm: Add guard pages around a shadow stack.
  mm/mmap: Add shadow stack pages to memory accounting
  mm: Re-introduce vm_flags to do_mmap()
  x86/shstk: Add user-mode shadow stack support
  x86/shstk: Handle thread shadow stack
  x86/shstk: Introduce routines modifying shstk
  x86/shstk: Handle signals for shadow stack
  x86: Add PTRACE interface for shadow stack

 Documentation/filesystems/proc.rst            |   1 +
 Documentation/mm/arch_pgtable_helpers.rst     |   9 +-
 Documentation/x86/index.rst                   |   1 +
 Documentation/x86/shstk.rst                   | 176 +++++
 arch/alpha/include/asm/pgtable.h              |   6 +-
 arch/arc/include/asm/hugepage.h               |   2 +-
 arch/arc/include/asm/pgtable-bits-arcv2.h     |   7 +-
 arch/arm/include/asm/pgtable-3level.h         |   7 +-
 arch/arm/include/asm/pgtable.h                |   2 +-
 arch/arm/kernel/signal.c                      |   2 +-
 arch/arm64/include/asm/pgtable.h              |   9 +-
 arch/arm64/kernel/signal.c                    |   2 +-
 arch/arm64/kernel/signal32.c                  |   2 +-
 arch/arm64/mm/trans_pgd.c                     |   4 +-
 arch/csky/include/asm/pgtable.h               |   2 +-
 arch/hexagon/include/asm/pgtable.h            |   2 +-
 arch/ia64/include/asm/pgtable.h               |   2 +-
 arch/loongarch/include/asm/pgtable.h          |   4 +-
 arch/m68k/include/asm/mcf_pgtable.h           |   2 +-
 arch/m68k/include/asm/motorola_pgtable.h      |   6 +-
 arch/m68k/include/asm/sun3_pgtable.h          |   6 +-
 arch/microblaze/include/asm/pgtable.h         |   2 +-
 arch/mips/include/asm/pgtable.h               |   6 +-
 arch/nios2/include/asm/pgtable.h              |   2 +-
 arch/openrisc/include/asm/pgtable.h           |   2 +-
 arch/parisc/include/asm/pgtable.h             |   6 +-
 arch/powerpc/include/asm/book3s/32/pgtable.h  |   2 +-
 arch/powerpc/include/asm/book3s/64/pgtable.h  |   4 +-
 arch/powerpc/include/asm/nohash/32/pgtable.h  |   2 +-
 arch/powerpc/include/asm/nohash/32/pte-8xx.h  |   2 +-
 arch/powerpc/include/asm/nohash/64/pgtable.h  |   2 +-
 arch/riscv/include/asm/pgtable.h              |   6 +-
 arch/s390/include/asm/hugetlb.h               |   4 +-
 arch/s390/include/asm/pgtable.h               |  14 +-
 arch/s390/mm/pageattr.c                       |   4 +-
 arch/sh/include/asm/pgtable_32.h              |  10 +-
 arch/sparc/include/asm/pgtable_32.h           |   2 +-
 arch/sparc/include/asm/pgtable_64.h           |   6 +-
 arch/sparc/kernel/signal32.c                  |   2 +-
 arch/sparc/kernel/signal_64.c                 |   2 +-
 arch/um/include/asm/pgtable.h                 |   2 +-
 arch/x86/Kconfig                              |  24 +
 arch/x86/Kconfig.assembler                    |   5 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 arch/x86/include/asm/cpufeatures.h            |   2 +
 arch/x86/include/asm/disabled-features.h      |  16 +-
 arch/x86/include/asm/fpu/api.h                |   9 +
 arch/x86/include/asm/fpu/regset.h             |   7 +-
 arch/x86/include/asm/fpu/sched.h              |   3 +-
 arch/x86/include/asm/fpu/types.h              |  16 +-
 arch/x86/include/asm/fpu/xstate.h             |   6 +-
 arch/x86/include/asm/idtentry.h               |   2 +-
 arch/x86/include/asm/mmu_context.h            |   2 +
 arch/x86/include/asm/msr.h                    |  11 +
 arch/x86/include/asm/pgtable.h                | 322 ++++++++-
 arch/x86/include/asm/pgtable_types.h          |  71 +-
 arch/x86/include/asm/processor.h              |   8 +
 arch/x86/include/asm/shstk.h                  |  40 ++
 arch/x86/include/asm/special_insns.h          |  13 +
 arch/x86/include/asm/tlbflush.h               |   3 +-
 arch/x86/include/asm/trap_pf.h                |   2 +
 arch/x86/include/asm/traps.h                  |  12 +
 arch/x86/include/uapi/asm/mman.h              |   4 +
 arch/x86/include/uapi/asm/prctl.h             |  12 +
 arch/x86/kernel/Makefile                      |   4 +
 arch/x86/kernel/cet.c                         | 152 ++++
 arch/x86/kernel/cpu/common.c                  |  35 +-
 arch/x86/kernel/cpu/cpuid-deps.c              |   1 +
 arch/x86/kernel/cpu/proc.c                    |  23 +
 arch/x86/kernel/fpu/core.c                    |  59 +-
 arch/x86/kernel/fpu/regset.c                  |  86 +++
 arch/x86/kernel/fpu/xstate.c                  | 148 ++--
 arch/x86/kernel/fpu/xstate.h                  |   6 +
 arch/x86/kernel/idt.c                         |   2 +-
 arch/x86/kernel/process.c                     |  18 +-
 arch/x86/kernel/process_64.c                  |   9 +-
 arch/x86/kernel/ptrace.c                      |  12 +
 arch/x86/kernel/shstk.c                       | 491 +++++++++++++
 arch/x86/kernel/signal.c                      |   1 +
 arch/x86/kernel/signal_32.c                   |   2 +-
 arch/x86/kernel/signal_64.c                   |   8 +-
 arch/x86/kernel/sys_x86_64.c                  |   6 +-
 arch/x86/kernel/traps.c                       |  87 ---
 arch/x86/mm/fault.c                           |  38 +
 arch/x86/mm/pat/set_memory.c                  |   4 +-
 arch/x86/mm/pgtable.c                         |  38 +
 arch/x86/xen/enlighten_pv.c                   |   2 +-
 arch/x86/xen/mmu_pv.c                         |   2 +-
 arch/x86/xen/xen-asm.S                        |   2 +-
 arch/xtensa/include/asm/pgtable.h             |   2 +-
 fs/aio.c                                      |   2 +-
 fs/proc/array.c                               |   6 +
 fs/proc/task_mmu.c                            |   3 +
 include/asm-generic/hugetlb.h                 |   4 +-
 include/linux/mm.h                            |  46 +-
 include/linux/mman.h                          |   4 +
 include/linux/pgtable.h                       |  14 +
 include/linux/proc_fs.h                       |   2 +
 include/linux/syscalls.h                      |   1 +
 include/uapi/asm-generic/siginfo.h            |   3 +-
 include/uapi/asm-generic/unistd.h             |   2 +-
 include/uapi/linux/elf.h                      |   2 +
 ipc/shm.c                                     |   2 +-
 kernel/sys_ni.c                               |   1 +
 mm/debug_vm_pgtable.c                         |  16 +-
 mm/gup.c                                      |   2 +-
 mm/huge_memory.c                              |   7 +-
 mm/hugetlb.c                                  |   4 +-
 mm/memory.c                                   |   5 +-
 mm/migrate_device.c                           |   2 +-
 mm/mmap.c                                     |  12 +-
 mm/mprotect.c                                 |   2 +-
 mm/nommu.c                                    |   4 +-
 mm/userfaultfd.c                              |   2 +-
 mm/util.c                                     |   2 +-
 tools/testing/selftests/x86/Makefile          |   4 +-
 .../testing/selftests/x86/test_shadow_stack.c | 676 ++++++++++++++++++
 117 files changed, 2671 insertions(+), 324 deletions(-)
 create mode 100644 Documentation/x86/shstk.rst
 create mode 100644 arch/x86/include/asm/shstk.h
 create mode 100644 arch/x86/kernel/cet.c
 create mode 100644 arch/x86/kernel/shstk.c
 create mode 100644 tools/testing/selftests/x86/test_shadow_stack.c
  

On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> This series implements Shadow Stacks for userspace using x86's Control-flow 
> Enforcement Technology (CET). CET consists of two related security features: 
> shadow stacks and indirect branch tracking. This series implements just the 
> shadow stack part of this feature, and just for userspace.

Okay, I've done some bare metal testing, and it all looks happy. The
selftest passes, and I can can see the stack address mismatch get
detected if I explicitly rewrite the saved function pointer on the stack:

[INFO] Want normal flow
[INFO] Found 0x401890 @ 0x7fff47cf2ef8
[INFO] Normal execution flow
[INFO] Want to redirect
[INFO] Found 0x401890 @ 0x7fff47cf2ef8
[INFO] Hijacked execution flow
[INFO] Enabling shadow stack
[INFO] Want to redirect
[INFO] Found 0x401890 @ 0x7fff47cf2ef8
Segmentation fault (core dumped)

Tested-by: Kees Cook <keescook@chromium.org>
  

On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> Hi,
> 
> This series implements Shadow Stacks for userspace using x86's Control-flow 
> Enforcement Technology (CET). CET consists of two related security features: 
> shadow stacks and indirect branch tracking. This series implements just the 
> shadow stack part of this feature, and just for userspace.

For the series

Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
  

On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> I left tested-by tags in place per discussion with testers. Testers, please
> retest.

v6 is still working well on my AMD system (Dell PowerEdge
R6515 w/ EPYC 7713).

The selftests run cleanly:

[INFO]  new_ssp = 7f53069ffff8, *new_ssp = 7f5306a00001
[INFO]  changing ssp from 7f53071ffff0 to 7f53069ffff8
[INFO]  ssp is now 7f5306a00000
[OK]    Shadow stack pivot
[OK]    Shadow stack faults
[INFO]  Corrupting shadow stack
[INFO]  Generated shadow stack violation successfully
[OK]    Shadow stack violation test
[INFO]  Gup read -> shstk access success
[INFO]  Gup write -> shstk access success
[INFO]  Violation from normal write
[INFO]  Gup read -> write access success
[INFO]  Violation from normal write
[INFO]  Gup write -> write access success
[INFO]  Cow gup write -> write access success
[OK]    Shadow gup test
[INFO]  Violation from shstk access
[OK]    mprotect() test
[OK]    Userfaultfd test
[OK]    32 bit test

And I can see the control protection messages in dmesg when
running the shstk violation test from here:
https://gitlab.com/cet-software/cet-smoke-test

ld-linux-x86-64[51598] control protection ip:401139 sp:7ffd68b1b7c8 ssp:7fb433578fd8 error:1(near ret) in shstk1[401000+1000]

Tested-by: John Allen <john.allen@amd.com>
  

On Mon, 2023-02-20 at 08:50 +0200, Mike Rapoport wrote:
> On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> > Hi,
> > 
> > This series implements Shadow Stacks for userspace using x86's
> > Control-flow 
> > Enforcement Technology (CET). CET consists of two related security
> > features: 
> > shadow stacks and indirect branch tracking. This series implements
> > just the 
> > shadow stack part of this feature, and just for userspace.
> 
> For the series
> 
> Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>

Thanks Mike! Sorry forgot to add it since last time.
  

On Sun, 2023-02-19 at 19:42 -0800, Kees Cook wrote:
> On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> > This series implements Shadow Stacks for userspace using x86's
> > Control-flow 
> > Enforcement Technology (CET). CET consists of two related security
> > features: 
> > shadow stacks and indirect branch tracking. This series implements
> > just the 
> > shadow stack part of this feature, and just for userspace.
> 
> Okay, I've done some bare metal testing, and it all looks happy. The
> selftest passes, and I can can see the stack address mismatch get
> detected if I explicitly rewrite the saved function pointer on the
> stack:
> 
> [INFO] Want normal flow
> [INFO] Found 0x401890 @ 0x7fff47cf2ef8
> [INFO] Normal execution flow
> [INFO] Want to redirect
> [INFO] Found 0x401890 @ 0x7fff47cf2ef8
> [INFO] Hijacked execution flow
> [INFO] Enabling shadow stack
> [INFO] Want to redirect
> [INFO] Found 0x401890 @ 0x7fff47cf2ef8
> Segmentation fault (core dumped)
> 
> Tested-by: Kees Cook <keescook@chromium.org>

Thanks and for the other tags!
  

Hi Rick,

On 2023-02-18 at 13:13:52 -0800, Rick Edgecombe wrote:
> Hi,
> 
...
> 
> I left tested-by tags in place per discussion with testers. Testers, please
> retest.
> 

1. Tested kself-test from user space shstk on ADL-S, TGL-U without Glibc shstk
support in CentOS 8 stream OS:

// From the test_shadow_stack code in this patch series:
# ./test_shadow_stack
[INFO]  new_ssp = 7f014ac2dff8, *new_ssp = 7f014ac2e001
[INFO]  changing ssp from 7f014a1ffff0 to 7f014ac2dff8
[INFO]  ssp is now 7f014ac2e000
[OK]    Shadow stack pivot
[OK]    Shadow stack faults
[INFO]  Corrupting shadow stack
[INFO]  Generated shadow stack violation successfully
[OK]    Shadow stack violation test
[INFO]  Gup read -> shstk access success
[INFO]  Gup write -> shstk access success
[INFO]  Violation from normal write
[INFO]  Gup read -> write access success
[INFO]  Violation from normal write
[INFO]  Gup write -> write access success
[INFO]  Cow gup write -> write access success
[OK]    Shadow gup test
[INFO]  Violation from shstk access
[OK]    mprotect() test
[OK]    Userfaultfd test
[OK]    32 bit test

// shstk violation without SHSTK glibc support
// Code link: https://github.com/intel/lkvs/blob/main/cet/shstk_cp.c
# ./shstk_cp
[PASS]  Enable SHSTK successfully
[PASS]  Disabling shadow stack successfully
[PASS]  Re-enable shadow stack successfully
[PASS]  SHSTK enabled, ssp:7fa3bfe00000
[INFO]  do_hack() change address for return:
[INFO]  Before,ssp:7fa3bfdffff8,*ssp:40133f,rbp:0x7ffc23b5b440,*rbp:7ffc23b5b480,*(rbp+1):40133f
[INFO]  After, ssp:7fa3bfdffff8,*ssp:40133f,rbp:0x7ffc23b5b440,*rbp:7ffc23b5b480,*(rbp+1):401146
Segmentation fault (core dumped)

Dmesg:
[1117184.518588] shstk_cp[1523882] control protection ip:40122c sp:7ffc23b5b448 ssp:7fa3bfdffff8 error:1(near ret) in shstk_cp[401000+1000]

// shstk ARCH_SHSTK_STATUS read/set test without SHSTK Glibc support
// Code link: https://github.com/intel/lkvs/blob/main/cet/shstk_unlock_test.c
# ./shstk_unlock_test
[PASS]  Parent process enable SHSTK.
[PASS]  Parent pid:1522040, ssp:0x7f57fc400000
[INFO]  pid:1522040, ssp:0x7f57fc3ffff8, *ssp:401799
[PASS]  Unlock CET successfully for pid:1522041
[PASS]  GET CET REG ret:0, err:0, ssp:7f57fc3ffff8
[PASS]  SET CET REG ret:0, err:0, ssp:7f57fc3ffff8
[PASS]  SET ssp -1 failed(expected) ret:-1, errno:22
[PASS]  GET xstate successfully ret:0
[PASS]  SHSTK is enabled in child process
[INFO]  Child:1522041 origin ssp:0x7f57fc400000
[INFO]  Child:1522041, ssp:0x7f57fc400000, bp,0x7ffcf32ba0f0, *bp:401dc0, *(bp+1):7f57fc43ad85
[PASS]  Disabling shadow stack succesfully
[PASS]  SHSTK_STATUS ok, feature:0 is 0, ret:0
[PASS]  Child process re-enable ssp
[PASS]  SHSTK_STATUS ok, feature:1 1st bit is 1, ret:0
[PASS]  Child process enabled wrss
[PASS]  SHSTK_STATUS ok, feature:3 2nd bit is 1, ret:0
[INFO]  Child:1522041, ssp:0x7f57fc400000, bp,0x7ffcf32ba0f0, *bp:401dc0, *(bp+1):7f57fc43ad85
[INFO]  ssp addr:0x7f57fc400000 is same as ssp_verify:0x7f57fc400000
[PASS]  Child process disable shstk successfully.
[PASS]  Parent process disable shadow stack successfully.


2. Tested fedora37 OS + Hongjiu provided user space SHSTK support Glibc:
// shstk with Glibc support:
// Related Glibc support for Fedora37:  http://gnu-4.sc.intel.com/git/?p=hjl/misc.git;a=tree;f=setup/fedora/37;h=63af84a8f28f3d0802f09266e47fb94eb5cdff26;hb=HEAD
# readelf  -n shadow_test_fork | head
readelf: Warning: Gap in build notes detected from 0x4011d7 to 0x4011e4

Displaying notes found in: .note.gnu.property
  Owner                Data size        Description
    GNU                  0x00000040       NT_GNU_PROPERTY_TYPE_0
          Properties: x86 feature: IBT, SHSTK
...
// shadow_test_fork code is in attached
// gcc -fcf-protection=full -mshstk -O0 -fno-stack-check -fno-stack-protector    shadow_test_fork.c   -o shadow_test_fork
# ./shadow_test_fork s2
[INFO]  s2: stack rbp + 1
[INFO]  do_hack() change address for return:
[INFO]  After change, rbp+1 to hacked:0x401296
Segmentation fault (core dumped)

Dmesg:
[418653.591014] shadow_test_for[16529] control protection ip:401367 sp:7fff6ed0a728 ssp:7f661265bfe0 error:1(near ret) in shadow_test_fork[401000+1000]

All above user space SHSTK tests are passed.

Many thanks Rick and all!

Thanks!
BR.
Pengfei

> -- 
> 2.17.1
>
// SPDX-License-Identifier: GPL-2.0
/*
 * Contributors:
 *      Pengfei, Xu <pengfei.xu@intel.com>
 *      - Test CET shadow stack function, should trigger #CP protection
 *      - Add the print, and show stack address and content before and after
 *        changed
 */

#define _GNU_SOURCE
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <signal.h>
#include <sched.h>
#include <immintrin.h>

static long hacked(void)
{
	printf("[INFO]\tAccess hack function\n");
	printf("[FAIL]\tpid=%d Hacked!\n", getpid());
	printf("[WARN]\tYou see this line, which means CET shstk #CP failed!\n");
	return 1;
}

/*
 * stack variable y + 1(1 means 8bytes for 64bit, 4bytes for 32bit) is bp,
 * and here use bp directly, it's bp hacked not sp hacked, so it should not
 * trigger #CP.
 */
static void stack_add1_test(unsigned long changed_bp)
{
	unsigned long *func_bp;

#ifdef __x86_64__
	asm("movq %%rbp,%0" : "=r"(func_bp));
#else
	asm("mov %%ebp,%0" : "=r"(func_bp));
#endif
	printf("[INFO]\tReal add1 function rbp content:%lx for main rbp.\n",
	       *func_bp);
	*func_bp = changed_bp;
	printf("[INFO]\tChange add1 rbp content:%lx, but right main rbp content in it!\n",
	       *func_bp);
}

/* stack base rbp + 1 addr test, which should be hacked and #CP should work */
static unsigned long stack_add2_test(void)
{
	unsigned long y;
	unsigned long *i, *j;

	i = (unsigned long *)_get_ssp();
	j = __builtin_frame_address(0);

	printf("[INFO]\tdo_hack() change address for return:\n");
	printf("[INFO]\tBefore change,y:%lx,&y:%p,j:%p,*j:%lx,*(&j+1):0x%lx, ssp:%p *ssp:0x%lx\n",
	       y, &y, j, *j, *(j+1), i, *i);

	/* j(rbp)+1 is sp address, change rbp+1 to change sp content */
	*(j + 1) = (unsigned long)hacked;

	printf("[INFO]\tAfter change, rbp+1 to hacked:0x%lx\n", *(j+1));
	printf("[INFO]\tAfter hacked  &y:%p, *j:0x%lx,*(&j+1):0x%lx\n",
	       &y, *j, *(j + 1));

	/* Debug purpose: it's not related with ret instruction in objdump. */
	return y;
}

/* stack base y + 3 addr test, which should not be hacked and #CP */
static unsigned long stack_add3_test(void)
{
	unsigned long y;

	printf("[INFO]\tdo_hack() change address for return:\n");
	printf("[INFO]\tBefore change, y:0x%lx, *(&y+2):0x%lx\n", y,
	       *((unsigned long *)&y + 2));
	*((unsigned long *)&y + 3) = (unsigned long)hacked;
	printf("[INFO]\tAfter change, *(&y+3) to change:0x%lx\n", (unsigned long)hacked);
	printf("[INFO]\tAfter change &y+3:%p,*(&x+2):0x%lx\n",
	       (unsigned long *)&y + 3, *((unsigned long *)&y + 3));
	printf("[INFO]\tAfter changed &y:%p, &y+2:%p,*(&y+2):0x%lx\n",
	       &y, (unsigned long *)&y + 2, *((unsigned long *)&y + 2));

	return y;
}

static long stack_long2_test(unsigned long i)
{
	unsigned long *p;


	printf("[INFO]\tuse rbp + long(+8bytes) size to hack:\n");
	/*
	 * Another way to read rbp
	 * asm("movq %%rbp,%0" : "=r"(p));
	 */
	p = __builtin_frame_address(0);

	printf("[INFO]\t*(p+1):%lx will be hacked\n", *(p + 1));
	*(p + 1) = (unsigned long)hacked;

	return 0;
}

/* stack base y + 2 change to random value to do shstk violation */
static unsigned long stack_random(unsigned long j)
{
	unsigned long y;
	unsigned long *p;

	y = j;
	printf("[INFO]\tSHSTK hack with random value:\n");
#ifdef __x86_64__
	asm("movq %%rbp,%0" : "=r"(p));
#else
	asm("mov %%ebp,%0" : "=r"(p));
#endif

	*(p + 1) = j;

	return y;
}

/* stack base y + 2 changed but no return */
static void stack_no_return(void)
{
	unsigned long *p;

	printf("[INFO]\tSHSTK with void no return function:\n");
#ifdef __x86_64__
	asm("movq %%rbp,%0" : "=r"(p));
#else
	asm("mov %%ebp,%0" : "=r"(p));
#endif

	*(p + 1) = (unsigned long)hacked;
}

/* buffer overflow change stack base, which should trigger #CP */
static void stack_buf_impact(void)
{
	char buffer[20];
	int overflow_num = 44;

	printf("[INFO]\tbuffer[20]:%x\n", buffer[20]);
	memset(buffer, 0, overflow_num);
	printf("[INFO]\tbuffer[44]:%x,&buffer[44]:%p\n", buffer[44], &buffer[44]);
	printf("[INFO]\tbuffer[20] after overflow:%x\n", buffer[20]);
}

/* buffer overflow not change stack base, which should not trigger #CP */
static void stack_buf_no_impact(void)
{
	char buf[20];
	int overflow_24 = 24, overflow_28 = 28;

	printf("[INFO]\tbuf[20]:%x\n", buf[20]);
#ifdef __x86_64__
	memset(buf, 0, overflow_28);
#else
	memset(buf, 0, overflow_24);
#endif
	printf("[INFO]\tbuf[20] after overflow:%x\n", buf[20]);
}

/* test hack function */
static int do_hack(void *p)
{
	/*
	 * Ret and then rip will get this value(rbp + 8 bytes in 64 bit OS)
	 * rbp(8 bytes in 64bit OS)
	 * *i, *j and so on variable content
	 */
	unsigned long *i, *j;

	i = (unsigned long *)_get_ssp();
	j = __builtin_frame_address(0);

	printf("[INFO]\tBefore: rbp+8:0x%p content=0x%lx; ssp=0x%p, ssp content=0x%lx\n",
		j + 1, *(j + 1), i, *i);
	*(j+1) = (unsigned long)hacked;
	printf("[INFO]\tAfter: rbp+8:0x%p content=0x%lx; ssp=0x%p, ssp content=0x%lx\n",
		j + 1, *(j + 1), i, *i);

	return 0;
}

/* check shadow stack wo core dump in child pid */
static void stack_wo_core(void)
{
	void *s = malloc(0x100000);

	if (fork() == 0)
		do_hack(s);
}

/* test shstk by clone way */
static int stack_clone(void)
{
	pid_t cid;

	void *child_stack = malloc(0x100000);

	if (child_stack == NULL) {
		printf("[FAIL]\tmalloc child_stack failed!\n");
		return 1;
	}

	cid = clone(
	do_hack, /* function */
	child_stack + 0x100000,
	SIGCHLD,
	0 /*arg*/
	);

	if (cid == -1) {
		printf("[FAIL]\tclone failed!\n");
		free(child_stack);
		return 1;
	}

	printf("[INFO]\tparent=%d, child=%d\n", getpid(), cid);

	if (waitpid(cid, NULL, 0) == -1) {
		printf("[FAIL]\twaitpid() failed!\n");
		return 1;
	}
	printf("[INFO]\tchild exits!\n");

	free(child_stack);
	return 0;
}

/*
 * Check shadow stack address and content and
 * rbp address and protect address content
 */
static int shadow_stack_check(void)
{
	unsigned long y;
	unsigned long *bp_a, *ssp_a;
	unsigned long long size_bp, size_ssp;

	ssp_a = (unsigned long *)_get_ssp();
	bp_a = __builtin_frame_address(0);
	size_bp = sizeof(*(bp_a + 1));
	size_ssp = sizeof(*ssp_a);

	printf("[INFO]\t&y=0x%p\n", &y);
	printf("[INFO]\tbp=%p,bp+1=%p,*(bp+1):0x%lx(size:%lld) ssp=%p *ssp=0x%lx(size:%lld)\n",
		bp_a, bp_a + 1, *(bp_a + 1), size_bp, ssp_a, *ssp_a, size_ssp);
	return 0;
}

static void usage(void)
{
	printf("Usage: [null | s1 | s2 | s3 | sl1 | sr | sn...]\n");
	printf("  null: no parm, stack add 2 test, should trigger #CP\n");
	printf("  s1: stack add 1 test\n");
	printf("  s2: stack add 2 test, should trigger #CP\n");
	printf("  s3: stack add 3 test\n");
	printf("  sl1: stack with long add 2 test\n");
	printf("  sr: stack change to random value\n");
	printf("  sn: stack change but no return\n");
	printf("  buf1: buffer overflow change stack base\n");
	printf("  buf2: buffer overflow not change stack base\n");
	printf("  snc: test shadow stack wo core dump\n");
	printf("  sc: test shadow stack by clone way\n");
	printf("  ssp: check shadow stack addr and content\n");
}

int main(int argc, char *argv[])
{
	char *parm = "";
	unsigned long a = 0, *main_rbp, fake_bp[2];

	a = rand();
	enum {
		e_s1, /* enum stack base, y + 1 */
		e_s2, /* enum stack base + 1 addr content change test */
		e_s3, /* enum stack base y + 3 */
		e_sl1, /* enum stack base with long + 2 */
		e_sr, /* enum stack base change to random value */
		e_sn, /* enum stack base changed but no return */
		e_buf1, /* buffer overflow change stack base */
		e_buf2, /* buffer overflow not change stack base */
		e_snc, /* shadow stack wo core dump */
		e_sc, /* test shstk by stack clone way */
		e_ssp /* check shadow stack addr and content */
	} option;

#ifdef __x86_64__
	asm("movq %%rbp,%0" : "=r"(main_rbp));
#else
	asm("mov %%ebp,%0" : "=r"(main_rbp));
#endif

	/* Use real main rbp address and content to make one fake bp and sp */
	fake_bp[0] = *main_rbp;
	fake_bp[1] = *(main_rbp + 1);

	if (argc == 1) {
		usage();
		stack_add2_test();
	} else {
		parm = argv[1];
		if (strcmp(argv[1], "s1") == 0)
			option = e_s1;
		else if (strcmp(argv[1], "s2") == 0)
			option = e_s2;
		else if (strcmp(argv[1], "s3") == 0)
			option = e_s3;
		else if (strcmp(argv[1], "sl1") == 0)
			option = e_sl1;
		else if (strcmp(argv[1], "sr") == 0)
			option = e_sr;
		else if (strcmp(argv[1], "sn") == 0)
			option = e_sn;
		else if (strcmp(argv[1], "buf1") == 0)
			option = e_buf1;
		else if (strcmp(argv[1], "buf2") == 0)
			option = e_buf2;
		else if (strcmp(argv[1], "snc") == 0)
			option = e_snc;
		else if (strcmp(argv[1], "sc") == 0)
			option = e_sc;
		else if (strcmp(argv[1], "ssp") == 0)
			option = e_ssp;
		else {
			usage();
			exit(1);
		}
	}

	switch (option) {
	case e_s1:
		printf("[INFO]\ts1: stack + 1\n");
		stack_add1_test((unsigned long)&fake_bp[0]);
		break;
	case e_s2:
		printf("[INFO]\ts2: stack rbp + 1\n");
		stack_add2_test();
		break;
	case e_s3:
		printf("[INFO]\ts3: stack + 3\n");
		stack_add3_test();
		break;
	case e_sl1:
		printf("[INFO]\tsl1: stack with long + 2, a:0x%lx\n", a);
		stack_long2_test(a);
		break;
	case e_sr:
		printf("[INFO]\tsr: stack changed to random value a:0x%lx\n", a);
		stack_random(a);
		break;
	case e_sn:
		printf("[INFO]\tsn: stack changed but no return\n");
		stack_no_return();
		break;
	case e_buf1:
		printf("buf1: buffer overflow change stack base\n");
		stack_buf_impact();
		break;
	case e_buf2:
		printf("[INFO]\tbuf2: buffer overflow not change stack base\n");
		stack_buf_no_impact();
		break;
	case e_snc:
		printf("[INFO]\tsnc: test shadow stack wo core dump\n");
		stack_wo_core();
		break;
	case e_sc:
		printf("[INFO]\tsc: test shstk by stack clone way\n");
		stack_clone();
		break;
	case e_ssp:
		printf("[INFO]\tssp: check shadow stack addr and content\n");
		shadow_stack_check();
		break;
	default:
		usage();
		exit(1);
	}

	printf("[RESULTS]\tParent pid=%d is done.\n", getpid());

	return 0;
}
  

On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> Hi,
> 
> This series implements Shadow Stacks for userspace using x86's Control-flow 

What is the base commit this applies on?

It ain't v6.2...

Thx.
  

On Wed, 2023-02-22 at 20:28 +0100, Borislav Petkov wrote:
> On Sat, Feb 18, 2023 at 01:13:52PM -0800, Rick Edgecombe wrote:
> > Hi,
> > 
> > This series implements Shadow Stacks for userspace using x86's
> > Control-flow 
> 
> What is the base commit this applies on?
> 
> It ain't v6.2...

It was tip/master the week I sent it out:
0a5e985fb1c8