From patchwork Mon Jul 24 12:46:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 124947 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9010:0:b0:3e4:2afc:c1 with SMTP id l16csp1800006vqg; Mon, 24 Jul 2023 06:28:45 -0700 (PDT) X-Google-Smtp-Source: APBJJlHhO/0s2cZh9/sZklRZHJL0hm6et2ium8nT5b4SbPaZ4/294abwT54tvQPX/y2CdPiKm8hG X-Received: by 2002:a17:902:a507:b0:1bb:809d:ae6a with SMTP id s7-20020a170902a50700b001bb809dae6amr7663947plq.7.1690205324665; Mon, 24 Jul 2023 06:28:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690205324; cv=none; d=google.com; s=arc-20160816; b=zB9oA60Yl4zmr7paWrUx/1Vtqx4vJRaxndEYkjYyV5V6XWHU2mw8R3WGWfjYmfBn1o UzQireANaoSPfh6QWkF+zXM90HsjqQiZZRTa6gy/9a6QH5hVX1vrdxA6stK2WeKOJ1KB 5Av0y7WgM7QGrCSHXb4dAlNETuB5LX1/9b7D3e9dTaDxHCMNMmtatteWYGsL6cBWCYm4 ybCT0fJej1CIFspWBukUe5uqsvWnT734azLLzNIhFN9NYE64J3hXn2vxT5oV7wohh4rh xuYG9IMqgNz9Nlvn1HxhADrPKRFXHV4arUTPfHqwHQWHpjRkc97x6iiQvfyFJFUu5f2+ kaUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:in-reply-to:references:message-id :content-transfer-encoding:mime-version:subject:date:from :dkim-signature; bh=GVW/Wb+UQgRbZFsiBWrQq9WDN9Jw1tFXrIhcblQEjIs=; fh=EdAFSmIgUzZTC6WCy4Jg1wBZ0/m6m2d/OjiRUJ3BsTI=; b=J1YQfgvxYUhARHPjUURQqcQr21eBaQqr+HSsorWAnpIlip3MUsLVLk9fKwX52W8xF5 0+NVNAJkXe1fTHIYqw6sK639/WVv8CnZw4JU9JVa+A8gvgkvBf+SrzlG7ta0w9R175M2 1sfF6qaxopABBIV+TN2n3lzYTM5ofIMdOVnwp/jmZMsFWtt23+woRR1Ktkd3sc8EZGL5 kgq1Q+5ElhUsQvqZDqWQeWjojDSd0+SutmeASDp0xOSVw8OIw89EYqic/EZseVUOIUfz WVZ5Fq1+RBPx1NZKcvMiBNy0tyO0A3XIfhSMvV+IvLNidcjmp6yw32w9cS7n1BAZLtaL glAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=r7HOQ9Rh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l7-20020a170902f68700b001b53c722c3fsi4401200plg.597.2023.07.24.06.28.31; Mon, 24 Jul 2023 06:28:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=r7HOQ9Rh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231160AbjGXMwG (ORCPT + 99 others); Mon, 24 Jul 2023 08:52:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230236AbjGXMvX (ORCPT ); Mon, 24 Jul 2023 08:51:23 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9E511FEE; Mon, 24 Jul 2023 05:49:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 36A2E61166; Mon, 24 Jul 2023 12:48:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5E8DC433C8; Mon, 24 Jul 2023 12:48:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1690202922; bh=sSLA8c9zD81mYMEEIJgpRALMzE62vMXK9V14WuQWjaE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=r7HOQ9Rh8UynMvFI/bPplPgBqdUDaYNqyAQEhBPj1HgLWRTSHXvaiXUNCmGS+L0PI 63oWdWdCtHcJ0Wg8CiYAF057zrhuXB4p6asxCYlJ4fpPy3q00KE3MqYVJBotQCkDAY 8SLA98IUZyEgqtd+shDVrWx3QSk3fMrumxPOMO3aPpCHEXYjjymtZZV7CiDgP351RE aVXc58tE48ZXmbuXLjwUKX0QbhbDqSWAAdB/dWzEZO8Ohtw825jHOnd7CuHzF3iDU/ D8u4nbwsp1XgIlzPZoZ7vtG3MsER7zsqUAAtT8FzL9UFaydlMubbPro61t7F/Fx3+S iOToctsnJN0IQ== From: Mark Brown Date: Mon, 24 Jul 2023 13:46:03 +0100 Subject: [PATCH v2 16/35] arm64/traps: Handle GCS exceptions MIME-Version: 1.0 Message-Id: <20230724-arm64-gcs-v2-16-dc2c1d44c2eb@kernel.org> References: <20230724-arm64-gcs-v2-0-dc2c1d44c2eb@kernel.org> In-Reply-To: <20230724-arm64-gcs-v2-0-dc2c1d44c2eb@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-099c9 X-Developer-Signature: v=1; a=openpgp-sha256; l=6026; i=broonie@kernel.org; h=from:subject:message-id; bh=sSLA8c9zD81mYMEEIJgpRALMzE62vMXK9V14WuQWjaE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBkvnKdFLOnO1lFOz6ssZ+ApC0f+pTPPIjb/1RkSS5L PckvS96JATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZL5ynQAKCRAk1otyXVSH0Jy8B/ 9oZ94IxvkzNCo7/Q1+XIbcKJMfaYEkVTNjT3TJ98fgdLpiLMUgf7ahslODb+SYX+nqOYZyBsKzAqas rrFaoS6BJUNYOpcvEWRYvGcgBM0HDY2s22q/TqWQ5PnGJxZYhsQGPwUVzuiRSw07+mPRt/YuyiWzVf VIkvlf/P0WvECMs4DUsijibnJgloq7USAiHkoEPps4BsSczcigeeJrOlLnnByQ+BwDmNnv4S/+WNzm 2Q5UrTYim+ud4gLNQ9T1apZGzrEe4tFbHmn6AOcwCeDks2VorgXUOg1VxVDNqX7cPgyME+dOICWSii udNbiRcb3gVl71CfyXcRrX6cJsTcRh X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772308738144949497 X-GMAIL-MSGID: 1772308738144949497 A new exception code is defined for GCS specific faults other than standard load/store faults, for example GCS token validation failures, add handling for this. These faults are reported to userspace as segfaults with code SEGV_CPERR (protection error), mirroring the reporting for x86 shadow stack errors. GCS faults due to memory load/store operations generate data aborts with a flag set, these will be handled separately as part of the data abort handling. Since we do not currently enable GCS for EL1 we should not get any faults there but while we're at it we wire things up there, treating any GCS fault as fatal. Signed-off-by: Mark Brown --- arch/arm64/include/asm/esr.h | 28 +++++++++++++++++++++++++++- arch/arm64/include/asm/exception.h | 2 ++ arch/arm64/kernel/entry-common.c | 23 +++++++++++++++++++++++ arch/arm64/kernel/traps.c | 11 +++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index ae35939f395b..a87a8305051f 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -51,7 +51,8 @@ #define ESR_ELx_EC_FP_EXC32 (0x28) /* Unallocated EC: 0x29 - 0x2B */ #define ESR_ELx_EC_FP_EXC64 (0x2C) -/* Unallocated EC: 0x2D - 0x2E */ +#define ESR_ELx_EC_GCS (0x2D) +/* Unallocated EC: 0x2E */ #define ESR_ELx_EC_SERROR (0x2F) #define ESR_ELx_EC_BREAKPT_LOW (0x30) #define ESR_ELx_EC_BREAKPT_CUR (0x31) @@ -382,6 +383,31 @@ #define ESR_ELx_MOPS_ISS_SRCREG(esr) (((esr) & (UL(0x1f) << 5)) >> 5) #define ESR_ELx_MOPS_ISS_SIZEREG(esr) (((esr) & (UL(0x1f) << 0)) >> 0) +/* ISS field definitions for GCS */ +#define ESR_ELx_ExType_SHIFT (20) +#define ESR_ELx_ExType_MASK GENMASK(23, 20) +#define ESR_ELx_Raddr_SHIFT (10) +#define ESR_ELx_Raddr_MASK GENMASK(14, 10) +#define ESR_ELx_Rn_SHIFT (5) +#define ESR_ELx_Rn_MASK GENMASK(9, 5) +#define ESR_ELx_Rvalue_SHIFT 5 +#define ESR_ELx_Rvalue_MASK GENMASK(9, 5) +#define ESR_ELx_IT_SHIFT (0) +#define ESR_ELx_IT_MASK GENMASK(4, 0) + +#define ESR_ELx_ExType_DATA_CHECK 0 +#define ESR_ELx_ExType_EXLOCK 1 +#define ESR_ELx_ExType_STR 2 + +#define ESR_ELx_IT_RET 0 +#define ESR_ELx_IT_GCSPOPM 1 +#define ESR_ELx_IT_RET_KEYA 2 +#define ESR_ELx_IT_RET_KEYB 3 +#define ESR_ELx_IT_GCSSS1 4 +#define ESR_ELx_IT_GCSSS2 5 +#define ESR_ELx_IT_GCSPOPCX 6 +#define ESR_ELx_IT_GCSPOPX 7 + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index ad688e157c9b..99caff458e20 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -57,6 +57,8 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr); void do_el1_undef(struct pt_regs *regs, unsigned long esr); void do_el0_bti(struct pt_regs *regs); void do_el1_bti(struct pt_regs *regs, unsigned long esr); +void do_el0_gcs(struct pt_regs *regs, unsigned long esr); +void do_el1_gcs(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 6b2e0c367702..4d86216962e5 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -400,6 +400,15 @@ static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_el1_gcs(regs, esr); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -442,6 +451,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_BTI: el1_bti(regs, esr); break; + case ESR_ELx_EC_GCS: + el1_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: @@ -621,6 +633,14 @@ static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } +static void noinstr el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_user_mode(regs); + local_daif_restore(DAIF_PROCCTX); + do_el0_gcs(regs, esr); + exit_to_user_mode(regs); +} + static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); @@ -701,6 +721,9 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_MOPS: el0_mops(regs, esr); break; + case ESR_ELx_EC_GCS: + el0_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_LOW: case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 8b70759cdbb9..65dab959f620 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -500,6 +500,16 @@ void do_el1_bti(struct pt_regs *regs, unsigned long esr) die("Oops - BTI", regs, esr); } +void do_el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + force_signal_inject(SIGSEGV, SEGV_CPERR, regs->pc, 0); +} + +void do_el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + die("Oops - GCS", regs, esr); +} + void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); @@ -884,6 +894,7 @@ static const char *esr_class_str[] = { [ESR_ELx_EC_MOPS] = "MOPS", [ESR_ELx_EC_FP_EXC32] = "FP (AArch32)", [ESR_ELx_EC_FP_EXC64] = "FP (AArch64)", + [ESR_ELx_EC_GCS] = "Guarded Control Stack", [ESR_ELx_EC_SERROR] = "SError", [ESR_ELx_EC_BREAKPT_LOW] = "Breakpoint (lower EL)", [ESR_ELx_EC_BREAKPT_CUR] = "Breakpoint (current EL)",