From patchwork Wed Oct 11 10:06:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151259 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp430161vqb; Wed, 11 Oct 2023 03:19:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGgefcdk1VPvizYvjHdarfF9amGf367YVoGdFBkWdz1SA3V/yYTztegisJBad+tF51dskkV X-Received: by 2002:a05:6a20:8f01:b0:15a:2c0b:6c81 with SMTP id b1-20020a056a208f0100b0015a2c0b6c81mr24886612pzk.3.1697019568259; Wed, 11 Oct 2023 03:19:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019568; cv=none; d=google.com; s=arc-20160816; b=bL+EAF9OOpJ3IE41e62KTF59aajSIWoofKGYuXas/FpVLRIbjygOT4tWe00a9MHxAf XM0P5CYuCJcZyVSCZO/c8HrYF4RP9RVqEVDn2SQubb59IdyGlCjfZ9sxrlwEExTpWhG5 b/VsIfbTJKW93MdWHbFh7yxwfLtLJPz4cd2Jz84px/qnrUDaIPXoStrYnwt3TlY+WbWQ 0HS7rNo7Oq9qRMnr7qDpN7dK4t34BjMQsS/NmejAA1z1p433+cgkp3kNun0D+lVx/Y6T 0BfOj5/jjulsLmclSU9A7rwTSfhEoCUJNEaxAqhZxOG6E7U1oToL327XLw9MLn46MtZ4 GCFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=YcwHAGNTdcfmK6uVzY7QZFc4S0Q4OTNs+tv+DHbv9L8=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=rYDx2F2a5QcOPQ0gTaHcL/fB9/ELWNznV38kQ0K4ezQyxwSJFz9TLa0r6eRxEJM6sa 90a3GpwyWNP9yc6YJeItR2nNB3RhTE+xz2iYqepAekRI7eB8o76PGLowR5uBhTLlGWP9 wstsCpmmcT6qTJaMyG0ijWg3mvgeOmnUVEdyM7H7JjzLGSWy2UJcUi8fMQ4FMQqd1YkQ 4amdowfjMU5557BUm0yEsCcVaxmjDqa1SCouNyBxV34wczgt9jbyNQeEE8qYY1+wSixt MnEMQWDz8MvlloZJmVIs43g+mvr/5am5HV/NpwGCTZMRfWoDCfG3Crm7BsguEE9eKWqd j5fg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id 36-20020a630f64000000b005649893593dsi13990255pgp.880.2023.10.11.03.19.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:19:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 5A5BB8020923; Wed, 11 Oct 2023 03:19:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346123AbjJKKSX (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234750AbjJKKSI (ORCPT ); Wed, 11 Oct 2023 06:18:08 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87F331993 for ; Wed, 11 Oct 2023 03:07:37 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S57g20GHHz1M92S; Wed, 11 Oct 2023 18:05:02 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:34 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 01/15] arm64: report EL1 UNDEFs better Date: Wed, 11 Oct 2023 10:06:41 +0000 Message-ID: <20231011100655.979626-2-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:19:25 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453990860453115 X-GMAIL-MSGID: 1779453990860453115 From: Mark Rutland commit b502c87d2a26c349acbc231ff2acd6f17147926b upstream. If an UNDEFINED exception is taken from EL1, and do_undefinstr() doesn't find any suitable undef_hook, it will call: BUG_ON(!user_mode(regs)) ... and the kernel will report a failure witin do_undefinstr() rather than reporting the original context that the UNDEFINED exception was taken from. The pt_regs and ESR value reported within the BUG() handler will be from within do_undefinstr() and the code dump will be for the BRK in BUG_ON(), which isn't sufficient to debug the cause of the original exception. This patch makes the reporting better by having do_undefinstr() call die() directly in this case to report the original context from which the UNDEFINED exception was taken. Prior to this patch, an undefined instruction is reported as: | kernel BUG at arch/arm64/kernel/traps.c:497! | Internal error: Oops - BUG: 0 [#1] PREEMPT SMP | Modules linked in: | CPU: 0 PID: 0 Comm: swapper Not tainted 5.19.0-rc3-00127-geff044f1b04e-dirty #3 | Hardware name: linux,dummy-virt (DT) | pstate: 000000c5 (nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) | pc : do_undefinstr+0x28c/0x2ac | lr : do_undefinstr+0x298/0x2ac | sp : ffff800009f63bc0 | x29: ffff800009f63bc0 x28: ffff800009f73c00 x27: ffff800009644a70 | x26: ffff8000096778a8 x25: 0000000000000040 x24: 0000000000000000 | x23: 00000000800000c5 x22: ffff800009894060 x21: ffff800009f63d90 | x20: 0000000000000000 x19: ffff800009f63c40 x18: 0000000000000006 | x17: 0000000000403000 x16: 00000000bfbfd000 x15: ffff800009f63830 | x14: ffffffffffffffff x13: 0000000000000000 x12: 0000000000000019 | x11: 0101010101010101 x10: 0000000000161b98 x9 : 0000000000000000 | x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0000000000000000 | x5 : ffff800009f761d0 x4 : 0000000000000000 x3 : ffff80000a2b80f8 | x2 : 0000000000000000 x1 : ffff800009f73c00 x0 : 00000000800000c5 | Call trace: | do_undefinstr+0x28c/0x2ac | el1_undef+0x2c/0x4c | el1h_64_sync_handler+0x84/0xd0 | el1h_64_sync+0x64/0x68 | setup_arch+0x550/0x598 | start_kernel+0x88/0x6ac | __primary_switched+0xb8/0xc0 | Code: 17ffff95 a9425bf5 17ffffb8 a9025bf5 (d4210000) With this patch applied, an undefined instruction is reported as: | Internal error: Oops - Undefined instruction: 0 [#1] PREEMPT SMP | Modules linked in: | CPU: 0 PID: 0 Comm: swapper Not tainted 5.19.0-rc3-00128-gf27cfcc80e52-dirty #5 | Hardware name: linux,dummy-virt (DT) | pstate: 800000c5 (Nzcv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) | pc : setup_arch+0x550/0x598 | lr : setup_arch+0x50c/0x598 | sp : ffff800009f63d90 | x29: ffff800009f63d90 x28: 0000000081000200 x27: ffff800009644a70 | x26: ffff8000096778c8 x25: 0000000000000040 x24: 0000000000000000 | x23: 0000000000000100 x22: ffff800009f69a58 x21: ffff80000a2b80b8 | x20: 0000000000000000 x19: 0000000000000000 x18: 0000000000000006 | x17: 0000000000403000 x16: 00000000bfbfd000 x15: ffff800009f63830 | x14: ffffffffffffffff x13: 0000000000000000 x12: 0000000000000019 | x11: 0101010101010101 x10: 0000000000161b98 x9 : 0000000000000000 | x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0000000000000000 | x5 : 0000000000000008 x4 : 0000000000000010 x3 : 0000000000000000 | x2 : 0000000000000000 x1 : 0000000000000000 x0 : 0000000000000000 | Call trace: | setup_arch+0x550/0x598 | start_kernel+0x88/0x6ac | __primary_switched+0xb8/0xc0 | Code: b4000080 90ffed80 912ac000 97db745f (00000000) Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Cc: Alexandru Elisei Cc: Amit Daniel Kachhap Cc: James Morse Cc: Will Deacon Reviewed-by: Anshuman Khandual Link: https://lore.kernel.org/r/20220913101732.3925290-2-mark.rutland@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/traps.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 21e69a991bc8..79e105713706 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -495,7 +495,9 @@ void do_undefinstr(struct pt_regs *regs) if (call_undef_hook(regs) == 0) return; - BUG_ON(!user_mode(regs)); + if (!user_mode(regs)) + die("Oops - Undefined instruction", regs, 0); + force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); } NOKPROBE_SYMBOL(do_undefinstr); From patchwork Wed Oct 11 10:06:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151247 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425734vqb; Wed, 11 Oct 2023 03:10:08 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFdWIsrqe3pwfpX1oYcAREXduJM0KafGZCHo3TOjoqZdzC9G89ZheqRsaB5pIhqVWmNXLb2 X-Received: by 2002:a17:90a:d10:b0:268:38a7:842e with SMTP id t16-20020a17090a0d1000b0026838a7842emr17956060pja.2.1697019008528; Wed, 11 Oct 2023 03:10:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019008; cv=none; d=google.com; s=arc-20160816; b=zqh53mVlWUVrLUUvlTf7a1XGafZQNZEn5Jq51fuhuLkv3VfUgrLbdrbCZz/bLH4Omo uxiKX45hReQWkxz+z/6SojP2t24nyeCdpHbitDnrZHHfGexRpOzWNWDSD3S+3WPjj5WP stjtFgd8Wpvvp4ObZtJEQCPpVKg3yWKgWZadScXNI7J19liaVU+J2dWUfKvrqIaSl/r0 F+6Y6U+pCyIDkzkuoIAMRcmU/T+SfvluYKCisdriKqukgKTy13WojIUe9NMmDwKmOsD7 lWGZwTXZ2K8WYzj86WOp8/T8PB1jF2cLF5yUZmx0HkncC/7gkGLBIRxoJTJnMHjE5vor x7sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=wCCZVDZ5O9059xkdpjO/JsMoGPsm9dk93FNzfnDK87M=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=TmUrwg2yfSjpQCexdd/LHFgbuNqLxaOKtL+nstwBTwqb66p5lidiMnBSwNPN8LIios Ao0v4Mr9k4+DEjC3ppEsatuGO1n3diZgObZx7zxwsjTDKWTe1yrfk6VAyIPLqYZeO5l0 8OU08DhdoizXeEjFKg9YmhNqyIzv7kC19huVgNK4EkPEe+sOe+vXLpQAM/cEPU8YW+6K r58xpPEsCLpWpdIoJ0T6v68XPCjzIM2jK6OSCJLKjQYr+EtSfGoPXGV1jZWm/WDpw2mD IbYawFjY1zdF9JAnYuRUwMvUPpWZpeBbX2x+xFhFD6DfY1A3JICRfOI4dz3eY8JhK1b+ 0IqA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id mu9-20020a17090b388900b0027ce8bab3b4si4076462pjb.113.2023.10.11.03.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id B97D680EE71F; Wed, 11 Oct 2023 03:09:55 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234480AbjJKKJX (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234758AbjJKKIl (ORCPT ); Wed, 11 Oct 2023 06:08:41 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9856199A for ; Wed, 11 Oct 2023 03:07:38 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S57g32rGqz1M9BJ; Wed, 11 Oct 2023 18:05:03 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:36 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 02/15] arm64: die(): pass 'err' as long Date: Wed, 11 Oct 2023 10:06:42 +0000 Message-ID: <20231011100655.979626-3-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:09:55 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453404388529003 X-GMAIL-MSGID: 1779453404388529003 From: Mark Rutland commit 18906ff9af6517c20763ed63dab602a4150794f7 upstream. Recently, we reworked a lot of code to consistentlt pass ESR_ELx as a 64-bit quantity. However, we missed that this can be passed into die() and __die() as the 'err' parameter where it is truncated to a 32-bit int. As notify_die() already takes 'err' as a long, this patch changes die() and __die() to also take 'err' as a long, ensuring that the full value of ESR_ELx is retained. At the same time, die() is updated to consistently log 'err' as a zero-padded 64-bit quantity. Subsequent patches will pass the ESR_ELx value to die() for a number of exceptions. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Anshuman Khandual Cc: Alexandru Elisei Cc: Amit Daniel Kachhap Cc: James Morse Cc: Will Deacon Link: https://lore.kernel.org/r/20220913101732.3925290-3-mark.rutland@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/system_misc.h | 2 +- arch/arm64/kernel/traps.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/system_misc.h b/arch/arm64/include/asm/system_misc.h index 0eb7709422e2..c34344256762 100644 --- a/arch/arm64/include/asm/system_misc.h +++ b/arch/arm64/include/asm/system_misc.h @@ -18,7 +18,7 @@ struct pt_regs; -void die(const char *msg, struct pt_regs *regs, int err); +void die(const char *msg, struct pt_regs *regs, long err); struct siginfo; void arm64_notify_die(const char *str, struct pt_regs *regs, diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 79e105713706..fcf1a306e094 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -181,12 +181,12 @@ static void dump_kernel_instr(const char *lvl, struct pt_regs *regs) #define S_SMP " SMP" -static int __die(const char *str, int err, struct pt_regs *regs) +static int __die(const char *str, long err, struct pt_regs *regs) { static int die_counter; int ret; - pr_emerg("Internal error: %s: %x [#%d]" S_PREEMPT S_SMP "\n", + pr_emerg("Internal error: %s: %016lx [#%d]" S_PREEMPT S_SMP "\n", str, err, ++die_counter); /* trap and error numbers are mostly meaningless on ARM */ @@ -207,7 +207,7 @@ static DEFINE_RAW_SPINLOCK(die_lock); /* * This function is protected against re-entrancy. */ -void die(const char *str, struct pt_regs *regs, int err) +void die(const char *str, struct pt_regs *regs, long err) { int ret; unsigned long flags; From patchwork Wed Oct 11 10:06:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151253 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429765vqb; Wed, 11 Oct 2023 03:18:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHd1KZdISPN4VhIGvkte5SQQFuN22MkbNkh4ozcP6nhGhwc57byRRcDRGfWB9A8+8IE/dcr X-Received: by 2002:a17:902:f54e:b0:1c2:c60:8387 with SMTP id h14-20020a170902f54e00b001c20c608387mr23179747plf.0.1697019518058; Wed, 11 Oct 2023 03:18:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019518; cv=none; d=google.com; s=arc-20160816; b=N3c56mMAMj5nKLz2FuUeI3hXIa4I7KgbqU6NRkcI+NrW8+Haz6wNhp3WFOCtYQIpBv H4qOefdPS7QXljW2Ux9mV/64dATjOnEvMrZHY0T1fD9l4jw0Z76fKjRW7Gl0F5iOvB3p qJyqr5ENKmHA6cTbegy2Es2u5J85OrmfvcTfacT4o5ocQ44wMBEx7l//APtHkMHXdszg 9K6VuaV21ol1mHq3WJQQv5ck8y1YVDx/OR4yXdY0oNT4xfRiRR6/i0l8HtbprXROgJby SXyZOgVderFY+bXUrHMj2jPJzc9CVXwdhPTzQ4AyA6bLTvjtXuzPTdTmX5SVRF5rRrfF QDnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=OX048BEgK/4NSNaXnRjNsnhef51QQoz4jnU1g42Odyo=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=uuMSIDJped/2+emyI3HhLc5iypxxuoXikBQsgbzxwwPfo+MzAmSAU6v7lW2rKw9lZm h+arAI5+NkzZHyBazODD6PVkamScijhjTF6zSIYM6Kj8tPyB+hWgJ1ceXgKXmO7YI8wl UefpF4ONE2XX11cm68LoF552WSxqPGhKgbHXCoUBPDWLO6URBrjo6/s+w2bauJrFpIPi aybHtxBcwvX16Nbs/MqxrYhY76Q9eLHXpwdSXSOGIYjt92b85gOHg9oNawkBHtlVSpLe weRLsCeXWgq66vWRMx4p1l9Wn8ttZcdmkCP+Wap1LfS76Kv3uZYNwunHobyO1ePp3K3t KTLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id l13-20020a170902f68d00b001c5844a4db7si15364471plg.562.2023.10.11.03.18.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id A283F80689AF; Wed, 11 Oct 2023 03:18:34 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345870AbjJKKSU (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234746AbjJKKSI (ORCPT ); Wed, 11 Oct 2023 06:18:08 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6613F19A8 for ; Wed, 11 Oct 2023 03:07:40 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57g35r48zrTN3; Wed, 11 Oct 2023 18:05:03 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:37 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 03/15] arm64: consistently pass ESR_ELx to die() Date: Wed, 11 Oct 2023 10:06:43 +0000 Message-ID: <20231011100655.979626-4-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:34 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453938259021909 X-GMAIL-MSGID: 1779453938259021909 From: Mark Rutland commit 0f2cb928a1547ae8f89e80a4b8df2c6c02ae5f96 upstream. Currently, bug_handler() and kasan_handler() call die() with '0' as the 'err' value, whereas die_kernel_fault() passes the ESR_ELx value. For consistency, this patch ensures we always pass the ESR_ELx value to die(). As this is only called for exceptions taken from kernel mode, there should be no user-visible change as a result of this patch. For UNDEFINED exceptions, I've had to modify do_undefinstr() and its callers to pass the ESR_ELx value. In all cases the ESR_ELx value had already been read and was available. Signed-off-by: Mark Rutland Cc: Mark Brown Cc: Alexandru Elisei Cc: Amit Daniel Kachhap Cc: James Morse Cc: Will Deacon Reviewed-by: Anshuman Khandual Reviewed-by: Mark Brown Link: https://lore.kernel.org/r/20220913101732.3925290-4-mark.rutland@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/exception.h | 2 +- arch/arm64/kernel/entry-common.c | 14 +++++++------- arch/arm64/kernel/traps.c | 14 +++++++------- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index 0e6535aa78c2..babbe2db8010 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -58,7 +58,7 @@ asmlinkage void call_on_irq_stack(struct pt_regs *regs, asmlinkage void asm_exit_to_user_mode(struct pt_regs *regs); void do_mem_abort(unsigned long far, unsigned long esr, struct pt_regs *regs); -void do_undefinstr(struct pt_regs *regs); +void do_undefinstr(struct pt_regs *regs, unsigned long esr); void do_bti(struct pt_regs *regs); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index fc91dad1579a..4dbdadc5daa1 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -371,11 +371,11 @@ static void noinstr el1_pc(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } -static void noinstr el1_undef(struct pt_regs *regs) +static void noinstr el1_undef(struct pt_regs *regs, unsigned long esr) { enter_from_kernel_mode(regs); local_daif_inherit(regs); - do_undefinstr(regs); + do_undefinstr(regs, esr); local_daif_mask(); exit_to_kernel_mode(regs); } @@ -417,7 +417,7 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) break; case ESR_ELx_EC_SYS64: case ESR_ELx_EC_UNKNOWN: - el1_undef(regs); + el1_undef(regs, esr); break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: @@ -554,11 +554,11 @@ static void noinstr el0_sp(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } -static void noinstr el0_undef(struct pt_regs *regs) +static void noinstr el0_undef(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_undefinstr(regs); + do_undefinstr(regs, esr); exit_to_user_mode(regs); } @@ -639,7 +639,7 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) el0_pc(regs, esr); break; case ESR_ELx_EC_UNKNOWN: - el0_undef(regs); + el0_undef(regs, esr); break; case ESR_ELx_EC_BTI: el0_bti(regs); @@ -755,7 +755,7 @@ asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_CP14_MR: case ESR_ELx_EC_CP14_LS: case ESR_ELx_EC_CP14_64: - el0_undef(regs); + el0_undef(regs, esr); break; case ESR_ELx_EC_CP15_32: case ESR_ELx_EC_CP15_64: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index fcf1a306e094..591eb1a44d30 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -486,7 +486,7 @@ void arm64_notify_segfault(unsigned long addr) force_signal_inject(SIGSEGV, code, addr, 0); } -void do_undefinstr(struct pt_regs *regs) +void do_undefinstr(struct pt_regs *regs, unsigned long esr) { /* check for AArch32 breakpoint instructions */ if (!aarch32_break_handler(regs)) @@ -496,7 +496,7 @@ void do_undefinstr(struct pt_regs *regs) return; if (!user_mode(regs)) - die("Oops - Undefined instruction", regs, 0); + die("Oops - Undefined instruction", regs, esr); force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); } @@ -755,7 +755,7 @@ void do_cp15instr(unsigned long esr, struct pt_regs *regs) hook_base = cp15_64_hooks; break; default: - do_undefinstr(regs); + do_undefinstr(regs, esr); return; } @@ -770,7 +770,7 @@ void do_cp15instr(unsigned long esr, struct pt_regs *regs) * EL0. Fall back to our usual undefined instruction handler * so that we handle these consistently. */ - do_undefinstr(regs); + do_undefinstr(regs, esr); } NOKPROBE_SYMBOL(do_cp15instr); #endif @@ -790,7 +790,7 @@ void do_sysinstr(unsigned long esr, struct pt_regs *regs) * back to our usual undefined instruction handler so that we handle * these consistently. */ - do_undefinstr(regs); + do_undefinstr(regs, esr); } NOKPROBE_SYMBOL(do_sysinstr); @@ -966,7 +966,7 @@ static int bug_handler(struct pt_regs *regs, unsigned long esr) { switch (report_bug(regs->pc, regs)) { case BUG_TRAP_TYPE_BUG: - die("Oops - BUG", regs, 0); + die("Oops - BUG", regs, esr); break; case BUG_TRAP_TYPE_WARN: @@ -1034,7 +1034,7 @@ static int kasan_handler(struct pt_regs *regs, unsigned long esr) * This is something that might be fixed at some point in the future. */ if (!recover) - die("Oops - KASAN", regs, 0); + die("Oops - KASAN", regs, esr); /* If thread survives, skip over the brk instruction and continue: */ arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); From patchwork Wed Oct 11 10:06:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151244 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425570vqb; Wed, 11 Oct 2023 03:09:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHyJay/QgxoEz4ik8lvcefhWL9oLt3b/y5OakHAW8pd1ZXfdxkbjKCOu87EAE3rtYkNwXLW X-Received: by 2002:a05:6a00:398c:b0:68a:6cbe:35a7 with SMTP id fi12-20020a056a00398c00b0068a6cbe35a7mr23382636pfb.2.1697018984612; Wed, 11 Oct 2023 03:09:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697018983; cv=none; d=google.com; s=arc-20160816; b=Rzwq2XB0pkEqXZII7jeVpN6zh0aMzlBP7laxp8+3CWEXJyVN9YD1DTn8GeZajuP9yR lKiZY9asvSy8gcQCo1jK4dfHwRs+TL5N7CNVjNzwZgbQrspo3pWUT4lTZNF7KSGJ8cmN eLApMvk/75OvUGFF09W0ITLQ4Qw9VMDU9FCqGV66Tgdw3SCd8vUV1cLD9xNMFN6SmcfE 6K2E5Shc+dtt1sIrX4DvYDzdXOtv43apv/Uq3mOY5N0YlGnNU8a1HV4HF6UDhQzXaAVv Ujh1RswmpC5Qrkz/FUjBR9aRHmLgFJr5msQiMcuw3CgD16h4QM3HqGotnd+tsdsAAuX8 7NyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=5jAsOWmAzH/2/W611Iskm1xE0mVqtuQvNa34PzpRMC4=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=fzTYuAFTDxVDJSTsxtYV9ythXa5eIfhHRrg7lBJ/YZTpVY/IvVDrS9RNLHMMQ+kiyN jH6oCQXRi/hMsFeWGnhiFd6ZXKZowIV/WZXEu79+TDN7GkTcDYd23H0ZrKs+6bx78mju VWxZ7wmnhzROgRdWzWq7UU4lM68nRW3U4sSA19I+4xkxD1vqZED4zqJqqO0aqYQeDqlP s71/JnwNrp2YZvfe10v91LMGkJVKBo2Qw/Kk6boXM97Jqr+enTjjFHWR/gmXE9r8UmRZ i6tJS/qm/5A4S0KhrGNB4ehD3uRxakBske1/Yeyc8LX7vIn1vdET7evx0Qawhb2nuDES 0yww== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id a11-20020a056a001d0b00b0068fe810e870si11589908pfx.100.2023.10.11.03.09.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:09:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 304FE8020907; Wed, 11 Oct 2023 03:09:32 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345857AbjJKKJA (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234768AbjJKKIn (ORCPT ); Wed, 11 Oct 2023 06:08:43 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50BC019AC for ; Wed, 11 Oct 2023 03:07:41 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4S57g55RL4z1M9LQ; Wed, 11 Oct 2023 18:05:05 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:38 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 04/15] arm64: rework FPAC exception handling Date: Wed, 11 Oct 2023 10:06:44 +0000 Message-ID: <20231011100655.979626-5-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:09:32 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453377985991419 X-GMAIL-MSGID: 1779453377985991419 From: Mark Rutland commit a1fafa3b24a70461bbf3e5c0770893feb0a49292 upstream. If an FPAC exception is taken from EL1, the entry code will call do_ptrauth_fault(), where due to: BUG_ON(!user_mode(regs)) ... the kernel will report a problem within do_ptrauth_fault() rather than reporting the original context the FPAC exception was taken from. The pt_regs and ESR value reported will be from within do_ptrauth_fault() and the code dump will be for the BRK in BUG_ON(), which isn't sufficient to debug the cause of the original exception. This patch makes the reporting better by having separate EL0 and EL1 FPAC exception handlers, with the latter calling die() directly to report the original context the FPAC exception was taken from. Note that we only need to prevent kprobes of the EL1 FPAC handler, since the EL0 FPAC handler cannot be called recursively. For consistency with do_el0_svc*(), I've named the split functions do_el{0,1}_fpac() rather than do_el{0,1}_ptrauth_fault(). I've also clarified the comment to not imply there are casues other than FPAC exceptions. Prior to this patch FPAC exceptions are reported as: | kernel BUG at arch/arm64/kernel/traps.c:517! | Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP | Modules linked in: | CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc3-00130-g9c8a180a1cdf-dirty #12 | Hardware name: FVP Base RevC (DT) | pstate: 00400009 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) | pc : do_ptrauth_fault+0x3c/0x40 | lr : el1_fpac+0x34/0x54 | sp : ffff80000a3bbc80 | x29: ffff80000a3bbc80 x28: ffff0008001d8000 x27: 0000000000000000 | x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 | x23: 0000000020400009 x22: ffff800008f70fa4 x21: ffff80000a3bbe00 | x20: 0000000072000000 x19: ffff80000a3bbcb0 x18: fffffbfffda37000 | x17: 3120676e696d7573 x16: 7361202c6e6f6974 x15: 0000000081a90000 | x14: 0040000000000041 x13: 0040000000000001 x12: ffff000001a90000 | x11: fffffbfffda37480 x10: 0068000000000703 x9 : 0001000080000000 | x8 : 0000000000090000 x7 : 0068000000000f03 x6 : 0060000000000783 | x5 : ffff80000a3bbcb0 x4 : ffff0008001d8000 x3 : 0000000072000000 | x2 : 0000000000000000 x1 : 0000000020400009 x0 : ffff80000a3bbcb0 | Call trace: | do_ptrauth_fault+0x3c/0x40 | el1h_64_sync_handler+0xc4/0xd0 | el1h_64_sync+0x64/0x68 | test_pac+0x8/0x10 | smp_init+0x7c/0x8c | kernel_init_freeable+0x128/0x28c | kernel_init+0x28/0x13c | ret_from_fork+0x10/0x20 | Code: 97fffe5e a8c17bfd d50323bf d65f03c0 (d4210000) With this patch applied FPAC exceptions are reported as: | Internal error: Oops - FPAC: 0000000072000000 [#1] PREEMPT SMP | Modules linked in: | CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc3-00132-g78846e1c4757-dirty #11 | Hardware name: FVP Base RevC (DT) | pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) | pc : test_pac+0x8/0x10 | lr : 0x0 | sp : ffff80000a3bbe00 | x29: ffff80000a3bbe00 x28: 0000000000000000 x27: 0000000000000000 | x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 | x23: ffff80000a2c8000 x22: 0000000000000000 x21: 0000000000000000 | x20: ffff8000099fa5b0 x19: ffff80000a007000 x18: fffffbfffda37000 | x17: 3120676e696d7573 x16: 7361202c6e6f6974 x15: 0000000081a90000 | x14: 0040000000000041 x13: 0040000000000001 x12: ffff000001a90000 | x11: fffffbfffda37480 x10: 0068000000000703 x9 : 0001000080000000 | x8 : 0000000000090000 x7 : 0068000000000f03 x6 : 0060000000000783 | x5 : ffff80000a2c6000 x4 : ffff0008001d8000 x3 : ffff800009f88378 | x2 : 0000000000000000 x1 : 0000000080210000 x0 : ffff000001a90000 | Call trace: | test_pac+0x8/0x10 | smp_init+0x7c/0x8c | kernel_init_freeable+0x128/0x28c | kernel_init+0x28/0x13c | ret_from_fork+0x10/0x20 | Code: d50323bf d65f03c0 d503233f aa1f03fe (d50323bf) Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Anshuman Khandual Cc: Alexandru Elisei Cc: Amit Daniel Kachhap Cc: James Morse Cc: Will Deacon Link: https://lore.kernel.org/r/20220913101732.3925290-5-mark.rutland@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/exception.h | 3 ++- arch/arm64/kernel/entry-common.c | 4 ++-- arch/arm64/kernel/traps.c | 16 ++++++++++------ 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index babbe2db8010..79385ae73f8d 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -71,7 +71,8 @@ void bad_el0_sync(struct pt_regs *regs, int reason, unsigned long esr); void do_cp15instr(unsigned long esr, struct pt_regs *regs); void do_el0_svc(struct pt_regs *regs); void do_el0_svc_compat(struct pt_regs *regs); -void do_ptrauth_fault(struct pt_regs *regs, unsigned long esr); +void do_el0_fpac(struct pt_regs *regs, unsigned long esr); +void do_el1_fpac(struct pt_regs *regs, unsigned long esr); void do_serror(struct pt_regs *regs, unsigned long esr); void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 4dbdadc5daa1..a4fd4de22c15 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -394,7 +394,7 @@ static void noinstr el1_fpac(struct pt_regs *regs, unsigned long esr) { enter_from_kernel_mode(regs); local_daif_inherit(regs); - do_ptrauth_fault(regs, esr); + do_el1_fpac(regs, esr); local_daif_mask(); exit_to_kernel_mode(regs); } @@ -601,7 +601,7 @@ static void noinstr el0_fpac(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_ptrauth_fault(regs, esr); + do_el0_fpac(regs, esr); exit_to_user_mode(regs); } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 591eb1a44d30..6bdbeba76fd4 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -509,16 +509,20 @@ void do_bti(struct pt_regs *regs) } NOKPROBE_SYMBOL(do_bti); -void do_ptrauth_fault(struct pt_regs *regs, unsigned long esr) +void do_el0_fpac(struct pt_regs *regs, unsigned long esr) +{ + force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); +} + +void do_el1_fpac(struct pt_regs *regs, unsigned long esr) { /* - * Unexpected FPAC exception or pointer authentication failure in - * the kernel: kill the task before it does any more harm. + * Unexpected FPAC exception in the kernel: kill the task before it + * does any more harm. */ - BUG_ON(!user_mode(regs)); - force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); + die("Oops - FPAC", regs, esr); } -NOKPROBE_SYMBOL(do_ptrauth_fault); +NOKPROBE_SYMBOL(do_el1_fpac); #define __user_cache_maint(insn, address, res) \ if (address >= user_addr_max()) { \ From patchwork Wed Oct 11 10:06:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151248 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425907vqb; Wed, 11 Oct 2023 03:10:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHSZkVPiiEq3YEF2ztdaMhnvYWsdIfeJX+qpyWX9YM6UYDKavAe0ra6tcZ5HDP3FZ9N0CZW X-Received: by 2002:a17:902:dac8:b0:1bf:1a9e:85f7 with SMTP id q8-20020a170902dac800b001bf1a9e85f7mr23720435plx.1.1697019027556; Wed, 11 Oct 2023 03:10:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019027; cv=none; d=google.com; s=arc-20160816; b=XCe51lIGCVFKviS7isb1inLCYSZTh992FeyzP3/y0HAzFxDLjLYO7THZq7ZuRG8ApG zXAzxvKIQsnYSUBo7AP6V2fcNdxHruMITGHG7ZYU+LHT3pUM++x1lu0JpZzVhPAr0S04 LzWWPj46lm1Y5mFX3fnYfb8hsglPMUTxG4I5w8uwBfcDaqcAniKnyFktQTKioucaPdwi ggMxXKEFAa28r+tMbfyjkLA4i7bdt7Cc35FYvQqO0gEIgNBkOQEpjBL0Ea3zgZWamByi ql3Ocml/1ZO69LvAQuO9kXH0bJqoPgaVn3RIN5W4PDWkLmYqNY1IKbV8ICYHU8EkaJ/y OtfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=L4nrhN7bXUNCpDivpE/MmUrZgWeCR72jnJLda6K2od8=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=y2wLpXSKw2z2eeZMSxPxZ5EEeoq1dDKQhrB173DFg27/iP/oeAu0uVpwZ2zAm6N3tM wE5qCq6UghOMfCld5oJtco3bBtQDA3C+Hc3I0vzdD4GXPXXTrXkzJbO4mZeRlwWPkajj NOT78Ky1dgv5dZmazs++QOql288fBNvk5cXZLI0J7IEhKjgifrk0lGoxSCS5qNx4bijQ YL744idENEbE2HpJfgD6cKHZQjIEuE/L5T92L7WuxBeV+8zmTz/FhNhQzyf5/rsIwHuC j97KAqxfwqXljxS04RyvYbLQw0ZXIxTgQt63QFVbmFflMGK9NlVsnxKdrsBNfwCI66Kj Mjxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id kl8-20020a170903074800b001c746bca061si3554158plb.74.2023.10.11.03.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 5F2D18157DCB; Wed, 11 Oct 2023 03:10:00 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346184AbjJKKJD (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234790AbjJKKIn (ORCPT ); Wed, 11 Oct 2023 06:08:43 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8078D19B2 for ; Wed, 11 Oct 2023 03:07:42 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57cl2998zcdGl; Wed, 11 Oct 2023 18:03:03 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:39 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 05/15] arm64: rework BTI exception handling Date: Wed, 11 Oct 2023 10:06:45 +0000 Message-ID: <20231011100655.979626-6-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:10:00 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453423974327833 X-GMAIL-MSGID: 1779453423974327833 From: Mark Rutland commit 830a2a4d853f2c4a1e4606aa03341b7f273b0e9b upstream. If a BTI exception is taken from EL1, the entry code will treat this as an unhandled exception and will panic() the kernel. This is inconsistent with the way we handle FPAC exceptions, which have a dedicated handler and only necessarily kill the thread from which the exception was taken from, and we don't log all the information that could be relevant to debug the issue. The code in do_bti() has: BUG_ON(!user_mode(regs)); ... and it seems like the intent was to call this for EL1 BTI exceptions, as with FPAC, but this was omitted due to an oversight. This patch adds separate EL0 and EL1 BTI exception handlers, with the latter calling die() directly to report the original context the BTI exception was taken from. This matches our handling of FPAC exceptions. Prior to this patch, a BTI failure is reported as: | Unhandled 64-bit el1h sync exception on CPU0, ESR 0x0000000034000002 -- BTI | CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc3-00131-g7d937ff0221d-dirty #9 | Hardware name: linux,dummy-virt (DT) | pstate: 20400809 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=-c) | pc : test_bti_callee+0x4/0x10 | lr : test_bti_caller+0x1c/0x28 | sp : ffff80000800bdf0 | x29: ffff80000800bdf0 x28: 0000000000000000 x27: 0000000000000000 | x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 | x23: ffff80000a2b8000 x22: 0000000000000000 x21: 0000000000000000 | x20: ffff8000099fa5b0 x19: ffff800009ff7000 x18: fffffbfffda37000 | x17: 3120676e696d7573 x16: 7361202c6e6f6974 x15: 0000000041a90000 | x14: 0040000000000041 x13: 0040000000000001 x12: ffff000001a90000 | x11: fffffbfffda37480 x10: 0068000000000703 x9 : 0001000040000000 | x8 : 0000000000090000 x7 : 0068000000000f03 x6 : 0060000000000f83 | x5 : ffff80000a2b6000 x4 : ffff0000028d0000 x3 : ffff800009f78378 | x2 : 0000000000000000 x1 : 0000000040210000 x0 : ffff8000080257e4 | Kernel panic - not syncing: Unhandled exception | CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc3-00131-g7d937ff0221d-dirty #9 | Hardware name: linux,dummy-virt (DT) | Call trace: | dump_backtrace.part.0+0xcc/0xe0 | show_stack+0x18/0x5c | dump_stack_lvl+0x64/0x80 | dump_stack+0x18/0x34 | panic+0x170/0x360 | arm64_exit_nmi.isra.0+0x0/0x80 | el1h_64_sync_handler+0x64/0xd0 | el1h_64_sync+0x64/0x68 | test_bti_callee+0x4/0x10 | smp_cpus_done+0xb0/0xbc | smp_init+0x7c/0x8c | kernel_init_freeable+0x128/0x28c | kernel_init+0x28/0x13c | ret_from_fork+0x10/0x20 With this patch applied, a BTI failure is reported as: | Internal error: Oops - BTI: 0000000034000002 [#1] PREEMPT SMP | Modules linked in: | CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-rc3-00132-g0ad98265d582-dirty #8 | Hardware name: linux,dummy-virt (DT) | pstate: 20400809 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=-c) | pc : test_bti_callee+0x4/0x10 | lr : test_bti_caller+0x1c/0x28 | sp : ffff80000800bdf0 | x29: ffff80000800bdf0 x28: 0000000000000000 x27: 0000000000000000 | x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000 | x23: ffff80000a2b8000 x22: 0000000000000000 x21: 0000000000000000 | x20: ffff8000099fa5b0 x19: ffff800009ff7000 x18: fffffbfffda37000 | x17: 3120676e696d7573 x16: 7361202c6e6f6974 x15: 0000000041a90000 | x14: 0040000000000041 x13: 0040000000000001 x12: ffff000001a90000 | x11: fffffbfffda37480 x10: 0068000000000703 x9 : 0001000040000000 | x8 : 0000000000090000 x7 : 0068000000000f03 x6 : 0060000000000f83 | x5 : ffff80000a2b6000 x4 : ffff0000028d0000 x3 : ffff800009f78378 | x2 : 0000000000000000 x1 : 0000000040210000 x0 : ffff800008025804 | Call trace: | test_bti_callee+0x4/0x10 | smp_cpus_done+0xb0/0xbc | smp_init+0x7c/0x8c | kernel_init_freeable+0x128/0x28c | kernel_init+0x28/0x13c | ret_from_fork+0x10/0x20 | Code: d50323bf d53cd040 d65f03c0 d503233f (d50323bf) Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Anshuman Khandual Cc: Alexandru Elisei Cc: Amit Daniel Kachhap Cc: James Morse Cc: Will Deacon Link: https://lore.kernel.org/r/20220913101732.3925290-6-mark.rutland@arm.com Signed-off-by: Catalin Marinas Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/exception.h | 3 ++- arch/arm64/kernel/entry-common.c | 14 +++++++++++++- arch/arm64/kernel/traps.c | 10 +++++++--- 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index 79385ae73f8d..81b559a780e8 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -59,7 +59,8 @@ asmlinkage void asm_exit_to_user_mode(struct pt_regs *regs); void do_mem_abort(unsigned long far, unsigned long esr, struct pt_regs *regs); void do_undefinstr(struct pt_regs *regs, unsigned long esr); -void do_bti(struct pt_regs *regs); +void do_el0_bti(struct pt_regs *regs); +void do_el1_bti(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index a4fd4de22c15..6caab4453b11 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -380,6 +380,15 @@ static void noinstr el1_undef(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_el1_bti(regs, esr); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -419,6 +428,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_UNKNOWN: el1_undef(regs, esr); break; + case ESR_ELx_EC_BTI: + el1_bti(regs, esr); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: @@ -566,7 +578,7 @@ static void noinstr el0_bti(struct pt_regs *regs) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_bti(regs); + do_el0_bti(regs); exit_to_user_mode(regs); } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 6bdbeba76fd4..ae1844e9daf0 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -502,12 +502,16 @@ void do_undefinstr(struct pt_regs *regs, unsigned long esr) } NOKPROBE_SYMBOL(do_undefinstr); -void do_bti(struct pt_regs *regs) +void do_el0_bti(struct pt_regs *regs) { - BUG_ON(!user_mode(regs)); force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); } -NOKPROBE_SYMBOL(do_bti); + +void do_el1_bti(struct pt_regs *regs, unsigned long esr) +{ + die("Oops - BTI", regs, esr); +} +NOKPROBE_SYMBOL(do_el1_bti); void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { From patchwork Wed Oct 11 10:06:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151258 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429898vqb; Wed, 11 Oct 2023 03:18:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFmgcxhJmA+Ah+POb0dfDsyJkzYh8UPWtXOGxOaOZWiLuTHhQyQewhO7ahizKe+BACM0AoH X-Received: by 2002:a05:6a20:c18f:b0:16e:26fd:7c02 with SMTP id bg15-20020a056a20c18f00b0016e26fd7c02mr12005099pzb.2.1697019536141; Wed, 11 Oct 2023 03:18:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019536; cv=none; d=google.com; s=arc-20160816; b=jOwFZ3F88D0UIuocoQe3TLpkEBhU6yjrE+aeUeFVCVRjc3OVLKDR9SPvLq6kd6mQRz I+NwLWFlNqQrWpQy70t+h8zXmyrjkikrETBWvmTd1kWRlHyWs2WSzOZOOAhmB/xMKppw hRKTsHfVEn06b8wnSyAxhiTg29kTLc7vqOA95eztDnRAWlBgFvIgkawilsnNoYnCjDYJ 3fcVgT20UpOg/VBTHTr7dzr2FDRi9sBVTaJ2RvRcDMHP1O9GW8vv2GXDQeUr3eUswvPm /al+M8pVfgX5Xp41M4UHVRjFm0HKQd8qBHg1QHkFx6T5ttqbepYcmk61BFTmrvCPI7+v zwWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=YwhA0UGDHUUz+Jl1AVcgB0PPKVS9JAUoF1GBujVmYCg=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=AnMDEBL8cik9qTVLfN0KI41VcfS9RyQd6iHF9cvxnpKiyw43igOCtTBdmayQIacMzs C87otz29mxoFLNgTPvo2dPKrYba0e385OyI5gWJeOBQpXrnWyTA9F4+RTd10B+q8pfE6 I6uRkQq3E1WyPNzb+Yu+0EjD3ayCDat4w1gHpicv7WqlDK86Vbyd1nw5TtvD+AgeurOr vnIFIrbR4kBOK1lXhrByfKFtQTx4vqQpd91TgASIGH/YLQ4ttqGwkGl7rBQttHq7LTHl r3aG7meaSuk6wMDr+reyVAiHiPcOvS4/iDQMGzJvjQQgT4iadEf/nMK3+z5mTccT0yqz ORPg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id u8-20020a1709026e0800b001c62b9d56f8si13561864plk.606.2023.10.11.03.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 578738115458; Wed, 11 Oct 2023 03:18:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346277AbjJKKSd (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234762AbjJKKSJ (ORCPT ); Wed, 11 Oct 2023 06:18:09 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE29B10C7 for ; Wed, 11 Oct 2023 03:07:43 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S57f35Mm2zVl8Q; Wed, 11 Oct 2023 18:04:11 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:40 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 06/15] arm64: allow kprobes on EL0 handlers Date: Wed, 11 Oct 2023 10:06:46 +0000 Message-ID: <20231011100655.979626-7-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:53 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453956902779467 X-GMAIL-MSGID: 1779453956902779467 From: Mark Rutland commit b3a0c010e900a9f89dcd99f10bd8f7538d21b0a9 upstream. Currently do_sysinstr() and do_cp15instr() are marked with NOKPROBE_SYMBOL(). However, these are only called for exceptions taken from EL0, and there is no risk of recursion in kprobes, so this is not necessary. Remove the NOKPROBE_SYMBOL() annotation, and rename the two functions to more clearly indicate that these are solely for exceptions taken from EL0, better matching the names used by the lower level entry points in entry-common.c. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-2-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/exception.h | 4 ++-- arch/arm64/kernel/entry-common.c | 4 ++-- arch/arm64/kernel/traps.c | 6 ++---- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index 81b559a780e8..7466239adac0 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -66,10 +66,10 @@ void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); void do_sve_acc(unsigned long esr, struct pt_regs *regs); void do_fpsimd_exc(unsigned long esr, struct pt_regs *regs); -void do_sysinstr(unsigned long esr, struct pt_regs *regs); +void do_el0_sys(unsigned long esr, struct pt_regs *regs); void do_sp_pc_abort(unsigned long addr, unsigned long esr, struct pt_regs *regs); void bad_el0_sync(struct pt_regs *regs, int reason, unsigned long esr); -void do_cp15instr(unsigned long esr, struct pt_regs *regs); +void do_el0_cp15(unsigned long esr, struct pt_regs *regs); void do_el0_svc(struct pt_regs *regs); void do_el0_svc_compat(struct pt_regs *regs); void do_el0_fpac(struct pt_regs *regs, unsigned long esr); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 6caab4453b11..31bdaccabd29 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -541,7 +541,7 @@ static void noinstr el0_sys(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_sysinstr(esr, regs); + do_el0_sys(esr, regs); exit_to_user_mode(regs); } @@ -728,7 +728,7 @@ static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_cp15instr(esr, regs); + do_el0_cp15(esr, regs); exit_to_user_mode(regs); } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index ae1844e9daf0..2ff0c8ddcacf 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -742,7 +742,7 @@ static const struct sys64_hook cp15_64_hooks[] = { {}, }; -void do_cp15instr(unsigned long esr, struct pt_regs *regs) +void do_el0_cp15(unsigned long esr, struct pt_regs *regs) { const struct sys64_hook *hook, *hook_base; @@ -780,10 +780,9 @@ void do_cp15instr(unsigned long esr, struct pt_regs *regs) */ do_undefinstr(regs, esr); } -NOKPROBE_SYMBOL(do_cp15instr); #endif -void do_sysinstr(unsigned long esr, struct pt_regs *regs) +void do_el0_sys(unsigned long esr, struct pt_regs *regs) { const struct sys64_hook *hook; @@ -800,7 +799,6 @@ void do_sysinstr(unsigned long esr, struct pt_regs *regs) */ do_undefinstr(regs, esr); } -NOKPROBE_SYMBOL(do_sysinstr); static const char *esr_class_str[] = { [0 ... ESR_ELx_EC_MAX] = "UNRECOGNIZED EC", From patchwork Wed Oct 11 10:06:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151251 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp426023vqb; Wed, 11 Oct 2023 03:10:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF0nJA+UL34YEZlogDAitB/47RigO95T5QNtBKpAGx4CMbIlwtr9TxtaAI5EKmEDUCbBwux X-Received: by 2002:a05:6a20:daa8:b0:171:947f:465b with SMTP id iy40-20020a056a20daa800b00171947f465bmr6081243pzb.4.1697019043078; Wed, 11 Oct 2023 03:10:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019043; cv=none; d=google.com; s=arc-20160816; b=0ZwsA82AyhIj4w3E1BdrH1kiXzoUzFfalzQFU7C2QR6VtNo+5rIn5pVEC7EwjPbkQL rRaRt8EPCnY7Zdp2O4/5yv3Ly9q1ShKOR8a1FxdUWtzndARmD1esjYU6B5Si70MO/dPb BjODr7mVOWEgSsiyimQ+6ad6Vxa8DuK1IGOmgOWwN1tD1UoGYGAzNsNyT26MZ4qZoho+ tq2Pd6si9HmEtIGNaeFfChR+iyaDsk6lwNIxgFogWnA1ddcombR5KbkhzIJsRdZ8mLmN erP5ktGx4aUQly8m3SHrGEQ/LbA9d16gpF3APJ7JQ2tWF+i/rR9GayOZ/zIPa/xNIejW 8CDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=sLshLNP8CY+2lMuLPwjqgvIXRjyBGHBK325dJSPVB9E=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=MFklxQMrbzDHBwVy5ZtSzbLPRw2e09Q5StH6kQAdX7WLtJHWc/1qR5JKbeTw5HGBau HV89gwAJeCnsU5UAa3fjYSIWtNnxXEo8AkYV1wdZhmZ+w9cl4+ZKzkh0rIf1ykO8EWl9 5Ea7HK30mdI/XD8I85P84ta4UcV8afe/q/OpFGB9RUwZms1WhvGLfq6+sCE1k28qFNgA VeXCb+bhxLD87/mpAz703jTXK470eRXKfCd+f/eKEuE1/tkyI/q9P4j/jD/9huLH0ilu e1ZoVrcQUYcUw578jYcUj3DeuCr1sEaUCUpU21eJAtRZFpEX5uzGSbiSg6J7WGrqnGL3 zE5g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id a26-20020a63705a000000b00573f116fba1si14164607pgn.870.2023.10.11.03.10.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 0AAB6814557C; Wed, 11 Oct 2023 03:10:17 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346277AbjJKKJU (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234796AbjJKKIn (ORCPT ); Wed, 11 Oct 2023 06:08:43 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C310319B9 for ; Wed, 11 Oct 2023 03:07:44 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S57dZ37tbzLqWT; Wed, 11 Oct 2023 18:03:46 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:41 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 07/15] arm64: split EL0/EL1 UNDEF handlers Date: Wed, 11 Oct 2023 10:06:47 +0000 Message-ID: <20231011100655.979626-8-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:10:17 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453439978237997 X-GMAIL-MSGID: 1779453439978237997 From: Mark Rutland commit 61d64a376ea80f9097e7ea599bcd68671b836dc6 upstream. In general, exceptions taken from EL1 need to be handled separately from exceptions taken from EL0, as the logic to handle the two cases can be significantly divergent, and exceptions taken from EL1 typically have more stringent requirements on locking and instrumentation. Subsequent patches will rework the way EL1 UNDEFs are handled in order to address longstanding soundness issues with instrumentation and RCU. In preparation for that rework, this patch splits the existing do_undefinstr() handler into separate do_el0_undef() and do_el1_undef() handlers. Prior to this patch, do_undefinstr() was marked with NOKPROBE_SYMBOL(), preventing instrumentation via kprobes. However, do_undefinstr() invokes other code which can be instrumented, and: * For UNDEFINED exceptions taken from EL0, there is no risk of recursion within kprobes. Therefore it is safe for do_el0_undef to be instrumented with kprobes, and it does not need to be marked with NOKPROBE_SYMBOL(). * For UNDEFINED exceptions taken from EL1, either: (a) The exception is has been taken when manipulating SSBS; these cases are limited and do not occur within code that can be invoked recursively via kprobes. Hence, in these cases instrumentation with kprobes is benign. (b) The exception has been taken for an unknown reason, as other than manipulating SSBS we do not expect to take UNDEFINED exceptions from EL1. Any handling of these exception is best-effort. ... and in either case, marking do_el1_undef() with NOKPROBE_SYMBOL() isn't sufficient to prevent recursion via kprobes as functions it calls (including die()) are instrumentable via kprobes. Hence, it's not worthwhile to mark do_el1_undef() with NOKPROBE_SYMBOL(). The same applies to do_el1_bti() and do_el1_fpac(), so their NOKPROBE_SYMBOL() annotations are also removed. Aside from the new instrumentability, there should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-3-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/exception.h | 3 ++- arch/arm64/kernel/entry-common.c | 4 ++-- arch/arm64/kernel/traps.c | 22 ++++++++++++---------- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index 7466239adac0..515ebe24fd44 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -58,7 +58,8 @@ asmlinkage void call_on_irq_stack(struct pt_regs *regs, asmlinkage void asm_exit_to_user_mode(struct pt_regs *regs); void do_mem_abort(unsigned long far, unsigned long esr, struct pt_regs *regs); -void do_undefinstr(struct pt_regs *regs, unsigned long esr); +void do_el0_undef(struct pt_regs *regs, unsigned long esr); +void do_el1_undef(struct pt_regs *regs, unsigned long esr); void do_el0_bti(struct pt_regs *regs); void do_el1_bti(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 31bdaccabd29..864423297c00 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -375,7 +375,7 @@ static void noinstr el1_undef(struct pt_regs *regs, unsigned long esr) { enter_from_kernel_mode(regs); local_daif_inherit(regs); - do_undefinstr(regs, esr); + do_el1_undef(regs, esr); local_daif_mask(); exit_to_kernel_mode(regs); } @@ -570,7 +570,7 @@ static void noinstr el0_undef(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); local_daif_restore(DAIF_PROCCTX); - do_undefinstr(regs, esr); + do_el0_undef(regs, esr); exit_to_user_mode(regs); } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 2ff0c8ddcacf..2999ccf4c117 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -486,7 +486,7 @@ void arm64_notify_segfault(unsigned long addr) force_signal_inject(SIGSEGV, code, addr, 0); } -void do_undefinstr(struct pt_regs *regs, unsigned long esr) +void do_el0_undef(struct pt_regs *regs, unsigned long esr) { /* check for AArch32 breakpoint instructions */ if (!aarch32_break_handler(regs)) @@ -495,12 +495,16 @@ void do_undefinstr(struct pt_regs *regs, unsigned long esr) if (call_undef_hook(regs) == 0) return; - if (!user_mode(regs)) - die("Oops - Undefined instruction", regs, esr); - force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); } -NOKPROBE_SYMBOL(do_undefinstr); + +void do_el1_undef(struct pt_regs *regs, unsigned long esr) +{ + if (call_undef_hook(regs) == 0) + return; + + die("Oops - Undefined instruction", regs, esr); +} void do_el0_bti(struct pt_regs *regs) { @@ -511,7 +515,6 @@ void do_el1_bti(struct pt_regs *regs, unsigned long esr) { die("Oops - BTI", regs, esr); } -NOKPROBE_SYMBOL(do_el1_bti); void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { @@ -526,7 +529,6 @@ void do_el1_fpac(struct pt_regs *regs, unsigned long esr) */ die("Oops - FPAC", regs, esr); } -NOKPROBE_SYMBOL(do_el1_fpac); #define __user_cache_maint(insn, address, res) \ if (address >= user_addr_max()) { \ @@ -763,7 +765,7 @@ void do_el0_cp15(unsigned long esr, struct pt_regs *regs) hook_base = cp15_64_hooks; break; default: - do_undefinstr(regs, esr); + do_el0_undef(regs, esr); return; } @@ -778,7 +780,7 @@ void do_el0_cp15(unsigned long esr, struct pt_regs *regs) * EL0. Fall back to our usual undefined instruction handler * so that we handle these consistently. */ - do_undefinstr(regs, esr); + do_el0_undef(regs, esr); } #endif @@ -797,7 +799,7 @@ void do_el0_sys(unsigned long esr, struct pt_regs *regs) * back to our usual undefined instruction handler so that we handle * these consistently. */ - do_undefinstr(regs, esr); + do_el0_undef(regs, esr); } static const char *esr_class_str[] = { From patchwork Wed Oct 11 10:06:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151250 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425930vqb; Wed, 11 Oct 2023 03:10:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFPT68Q5AMI9nudl5LcCPqnyWT9kUBK4EC92qa2vcSmDUJtxmRFLDtXBzjpEybf4gweC1ML X-Received: by 2002:a17:902:ce84:b0:1c3:a4f2:7c99 with SMTP id f4-20020a170902ce8400b001c3a4f27c99mr23244954plg.4.1697019030550; Wed, 11 Oct 2023 03:10:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019030; cv=none; d=google.com; s=arc-20160816; b=W6DfaFs+0bDVlmBB9nDsrXeorzKiJfMGt5Imu3Q78WQIFWCyFi3XvvQrdaAU8qDvUe TVODR6THcf2vTwHDNeMbDXO7v2GGlfat/TmK+ZTxTiDJhsbOaDlHep0qUA6PFGARrvP9 r44/GERBes1x8kAiP/5TXzd6iuz7gKfRJ1hg1w4axZJkvHr3YeqsZkRNh0AhVDCytUzX 79jHNC6sStZkbZZbdVk5R9eIc7M9ewt02mhzdm8XO0+PCH77pX0nxhCtKjE6aWnT3QRy PA3u1SypJE1KB0T/xlKtgQDzQn6gPdx+G+bBT7iIZP2MnkOpAXLBij3lIitbnK27UX2h 0Naw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=yvsBLyfpw7XYEQH/Ae+OYe8TZ7rZEB6xSI4v9wDe+e4=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=zF3H11azqSlrOQl6w56+jQZ3chG+/HdeLsePSTEqgZh5vCgp4JNfy3bWsIlAw6YmBe zW3A8nVRa/7SfvHxv3lqX1yUJ4zi/ur9uCNe1xOeMEScxwfb17NoLECBl5v33LJ/KQUY FFpnLFIJHceMrKqcb7Q+duHXc+CNvw2RP/oUL/M3GQwZY7fPrFZcIxzkQrXzH6QFRnxn U+yuuHsmBBJekpplalSk3kqrON4/j9qbId1rq5u9ccIt95SQsKtTyEkjyMqpW+dczk1E t+Sl8KUiMsJhYvxzeXgC9/1wK6H3FRANiivJHN68QC99O+oBhmvHKg3hPsgUA+TnIzL4 7tuQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id l4-20020a170903244400b001bbd639dff9si11138674pls.467.2023.10.11.03.10.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id D406E8145575; Wed, 11 Oct 2023 03:10:02 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231215AbjJKKJK (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234681AbjJKKIo (ORCPT ); Wed, 11 Oct 2023 06:08:44 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED06B19BD for ; Wed, 11 Oct 2023 03:07:45 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S57f60FpFzVkqB; Wed, 11 Oct 2023 18:04:14 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:43 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 08/15] arm64: factor out EL1 SSBS emulation hook Date: Wed, 11 Oct 2023 10:06:48 +0000 Message-ID: <20231011100655.979626-9-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:10:02 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453427440644022 X-GMAIL-MSGID: 1779453427440644022 From: Mark Rutland commit bff8f413c71ffc3cb679dbd9a5632b33af563f9f upstream. Currently call_undef_hook() is used to handle UNDEFINED exceptions from EL0 and EL1. As support for deprecated instructions may be enabled independently, the handlers for individual instructions are organised as a linked list of struct undef_hook which can be manipulated dynamically. As this can be manipulated dynamically, the list is protected with a raw_spinlock which must be acquired when handling UNDEFINED exceptions or when manipulating the list of handlers. This locking is unfortunate as it serialises handling of UNDEFINED exceptions, and requires RCU to be enabled for lockdep, requiring the use of RCU_NONIDLE() in resume path of cpu_suspend() since commit: a2c42bbabbe260b7 ("arm64: spectre: Prevent lockdep splat on v4 mitigation enable path") The list of UNDEFINED handlers largely consist of handlers for exceptions taken from EL0, and the only handler for exceptions taken from EL1 handles `MSR SSBS, #imm` on CPUs which feature PSTATE.SSBS but lack the corresponding MSR (Immediate) instruction. Other than this we never expect to take an UNDEFINED exception from EL1 in normal operation. This patch reworks do_el0_undef() to invoke the EL1 SSBS handler directly, relegating call_undef_hook() to only handle EL0 UNDEFs. This removes redundant work to iterate the list for EL1 UNDEFs, and removes the need for locking, permitting EL1 UNDEFs to be handled in parallel without contention. The RCU_NONIDLE() call in cpu_suspend() will be removed in a subsequent patch, as there are other potential issues with the use of instrumentable code and RCU in the CPU suspend code. I've tested this by forcing the detection of SSBS on a CPU that doesn't have it, and verifying that the try_emulate_el1_ssbs() callback is invoked. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-4-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/spectre.h | 2 ++ arch/arm64/kernel/proton-pack.c | 26 +++++++------------------- arch/arm64/kernel/traps.c | 15 ++++++++------- 3 files changed, 17 insertions(+), 26 deletions(-) diff --git a/arch/arm64/include/asm/spectre.h b/arch/arm64/include/asm/spectre.h index aa3d3607d5c8..db7b371b367c 100644 --- a/arch/arm64/include/asm/spectre.h +++ b/arch/arm64/include/asm/spectre.h @@ -26,6 +26,7 @@ enum mitigation_state { SPECTRE_VULNERABLE, }; +struct pt_regs; struct task_struct; /* @@ -98,5 +99,6 @@ enum mitigation_state arm64_get_spectre_bhb_state(void); bool is_spectre_bhb_affected(const struct arm64_cpu_capabilities *entry, int scope); u8 spectre_bhb_loop_affected(int scope); void spectre_bhb_enable_mitigation(const struct arm64_cpu_capabilities *__unused); +bool try_emulate_el1_ssbs(struct pt_regs *regs, u32 instr); #endif /* __ASSEMBLY__ */ #endif /* __ASM_SPECTRE_H */ diff --git a/arch/arm64/kernel/proton-pack.c b/arch/arm64/kernel/proton-pack.c index 428cfabd11c4..7515ed1f0669 100644 --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -521,10 +521,13 @@ bool has_spectre_v4(const struct arm64_cpu_capabilities *cap, int scope) return state != SPECTRE_UNAFFECTED; } -static int ssbs_emulation_handler(struct pt_regs *regs, u32 instr) +bool try_emulate_el1_ssbs(struct pt_regs *regs, u32 instr) { - if (user_mode(regs)) - return 1; + const u32 instr_mask = ~(1U << PSTATE_Imm_shift); + const u32 instr_val = 0xd500401f | PSTATE_SSBS; + + if ((instr & instr_mask) != instr_val) + return false; if (instr & BIT(PSTATE_Imm_shift)) regs->pstate |= PSR_SSBS_BIT; @@ -532,19 +535,11 @@ static int ssbs_emulation_handler(struct pt_regs *regs, u32 instr) regs->pstate &= ~PSR_SSBS_BIT; arm64_skip_faulting_instruction(regs, 4); - return 0; + return true; } -static struct undef_hook ssbs_emulation_hook = { - .instr_mask = ~(1U << PSTATE_Imm_shift), - .instr_val = 0xd500401f | PSTATE_SSBS, - .fn = ssbs_emulation_handler, -}; - static enum mitigation_state spectre_v4_enable_hw_mitigation(void) { - static bool undef_hook_registered = false; - static DEFINE_RAW_SPINLOCK(hook_lock); enum mitigation_state state; /* @@ -555,13 +550,6 @@ static enum mitigation_state spectre_v4_enable_hw_mitigation(void) if (state != SPECTRE_MITIGATED || !this_cpu_has_cap(ARM64_SSBS)) return state; - raw_spin_lock(&hook_lock); - if (!undef_hook_registered) { - register_undef_hook(&ssbs_emulation_hook); - undef_hook_registered = true; - } - raw_spin_unlock(&hook_lock); - if (spectre_v4_mitigations_off()) { sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_DSSBS); set_pstate_ssbs(1); diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 2999ccf4c117..7d9bdd79797f 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -402,12 +402,7 @@ static int call_undef_hook(struct pt_regs *regs) int (*fn)(struct pt_regs *regs, u32 instr) = NULL; void __user *pc = (void __user *)instruction_pointer(regs); - if (!user_mode(regs)) { - __le32 instr_le; - if (get_kernel_nofault(instr_le, (__force __le32 *)pc)) - goto exit; - instr = le32_to_cpu(instr_le); - } else if (compat_thumb_mode(regs)) { + if (compat_thumb_mode(regs)) { /* 16-bit Thumb instruction */ __le16 instr_le; if (get_user(instr_le, (__le16 __user *)pc)) @@ -500,9 +495,15 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr) void do_el1_undef(struct pt_regs *regs, unsigned long esr) { - if (call_undef_hook(regs) == 0) + u32 insn; + + if (aarch64_insn_read((void *)regs->pc, &insn)) + goto out_err; + + if (try_emulate_el1_ssbs(regs, insn)) return; +out_err: die("Oops - Undefined instruction", regs, esr); } From patchwork Wed Oct 11 10:06:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151249 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425909vqb; Wed, 11 Oct 2023 03:10:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFXY/SI+D8uNWl/yYD/vxChLiK+zOURytTeELXV+gkB5cOaHVC26LynFd5D/YwHzRXwqLlD X-Received: by 2002:a05:6a00:2d1b:b0:68e:3eff:e93a with SMTP id fa27-20020a056a002d1b00b0068e3effe93amr22239809pfb.2.1697019028141; Wed, 11 Oct 2023 03:10:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019028; cv=none; d=google.com; s=arc-20160816; b=DJJMMbbIg6nx5/fh1gyzvSIMyd7I9AQxpXjxO0B0NizuY+vDW9y286gwoJfBi67H9t W1XlPve1ivC6/LZjXndTD6SeyNKRHZGZDbMCrjd1G9LmmsETi8hYPnyJ2UlG7W6uhQEp s7QeJemc49oedSUFZP/zDWqFZrwhKlW5gORqWMxYTCwheOFiP7PnQKNL6YHYrhuoKPrY 3zBSy/KIv7p87XZFz7qzZGVG7gA5Nwcti2MtemGp/8IkAeX23pUNTXc3k5pyPm2DASIC N6fFWRqKXiuWB8XMaqACYl8GMmTMYD3kwhI+v3s4enZOiE40S74Fv+rB3jYvIc0bQJZW 2JOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=etkrg/9ZQmY9F+WTdhKuHEdB0/OAkMWpW1yuqjVK4sg=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=oXAn53+4r162FpEG8vpKjSVd5LqMVIZow5VIwxjavJoV/D7OJD7RT9/0H2ymqW1ICr 84a5+sDIoZAXYartVn5PbCGmRZn/mA2XdZEk58DkOspIBISo9B4JEhLc4z0Lc7l7WXwr KReemnSgaKP4wudzANRWsu68RIUFOc5CMV6lMW0RyqX2x//hIAkAqkX5STvN9Jxi95PZ oinCKfMlAvVY70C1CQYHKdd0RMrn1MF1Eth7z3cVzHIjJKsA1XuCkbfUPM2EbYkOJvBI k3kBLD+7WBCsHs2rEbVYYbcTRMSIuKrhPCel3yWLztQCCEvPE0OrMU7aYmtbawgP1UlB DLQw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id g67-20020a636b46000000b0058933a74376si13576311pgc.70.2023.10.11.03.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id CEDC680215BC; Wed, 11 Oct 2023 03:10:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346244AbjJKKJQ (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234724AbjJKKIo (ORCPT ); Wed, 11 Oct 2023 06:08:44 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E320C1BC3 for ; Wed, 11 Oct 2023 03:07:46 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57gB48CbzrTFH; Wed, 11 Oct 2023 18:05:10 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:44 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 09/15] arm64: factor insn read out of call_undef_hook() Date: Wed, 11 Oct 2023 10:06:49 +0000 Message-ID: <20231011100655.979626-10-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:10:08 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453424729291932 X-GMAIL-MSGID: 1779453424729291932 From: Mark Rutland commit dbfbd87efa79575491af0ba1a87bf567eaea6cae upstream. Subsequent patches will rework EL0 UNDEF handling, removing the need for struct undef_hook and call_undef_hook. In preparation for those changes, this patch factors the logic for reading user instructions out of call_undef_hook() and into a new user_insn_read() helper, matching the style of the existing aarch64_insn_read() helper used for reading kernel instructions. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-5-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/traps.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 7d9bdd79797f..e943b7c46d86 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -394,25 +394,22 @@ void unregister_undef_hook(struct undef_hook *hook) raw_spin_unlock_irqrestore(&undef_lock, flags); } -static int call_undef_hook(struct pt_regs *regs) +static int user_insn_read(struct pt_regs *regs, u32 *insnp) { - struct undef_hook *hook; - unsigned long flags; u32 instr; - int (*fn)(struct pt_regs *regs, u32 instr) = NULL; void __user *pc = (void __user *)instruction_pointer(regs); if (compat_thumb_mode(regs)) { /* 16-bit Thumb instruction */ __le16 instr_le; if (get_user(instr_le, (__le16 __user *)pc)) - goto exit; + return -EFAULT; instr = le16_to_cpu(instr_le); if (aarch32_insn_is_wide(instr)) { u32 instr2; if (get_user(instr_le, (__le16 __user *)(pc + 2))) - goto exit; + return -EFAULT; instr2 = le16_to_cpu(instr_le); instr = (instr << 16) | instr2; } @@ -420,10 +417,20 @@ static int call_undef_hook(struct pt_regs *regs) /* 32-bit ARM instruction */ __le32 instr_le; if (get_user(instr_le, (__le32 __user *)pc)) - goto exit; + return -EFAULT; instr = le32_to_cpu(instr_le); } + *insnp = instr; + return 0; +} + +static int call_undef_hook(struct pt_regs *regs, u32 instr) +{ + struct undef_hook *hook; + unsigned long flags; + int (*fn)(struct pt_regs *regs, u32 instr) = NULL; + raw_spin_lock_irqsave(&undef_lock, flags); list_for_each_entry(hook, &undef_hook, node) if ((instr & hook->instr_mask) == hook->instr_val && @@ -431,7 +438,7 @@ static int call_undef_hook(struct pt_regs *regs) fn = hook->fn; raw_spin_unlock_irqrestore(&undef_lock, flags); -exit: + return fn ? fn(regs, instr) : 1; } @@ -483,13 +490,19 @@ void arm64_notify_segfault(unsigned long addr) void do_el0_undef(struct pt_regs *regs, unsigned long esr) { + u32 insn; + /* check for AArch32 breakpoint instructions */ if (!aarch32_break_handler(regs)) return; - if (call_undef_hook(regs) == 0) + if (user_insn_read(regs, &insn)) + goto out_err; + + if (call_undef_hook(regs, insn) == 0) return; +out_err: force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); } From patchwork Wed Oct 11 10:06:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151257 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429890vqb; Wed, 11 Oct 2023 03:18:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE43cdxJhMKk9esyotjcqZDvs+LRDOyUAk+Z/uTOODvU5qewxfdeG30B+EKnEnCi73Y8MG4 X-Received: by 2002:a05:6a20:43a6:b0:16c:b514:a4bc with SMTP id i38-20020a056a2043a600b0016cb514a4bcmr14192740pzl.4.1697019535562; Wed, 11 Oct 2023 03:18:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019535; cv=none; d=google.com; s=arc-20160816; b=K11RWTo15f7iISACtZ3SKyjqxq51er8Fm/FKCTvy4dM2NlYWicAC3AQRAG4gfNAKE6 h+hA8QXnH3kmkwO0guIMv6WXsolvT4eonB0BiwLptdgYHTOGF6TbgPwbrlIa1ucmtth4 Zny7APIDM9WV81wJJDcbYr4m6v9ermzT40wG/l0edWCMSgiVHDRswrp4eN9NRaxVm5jb /DHjeeMMYbj7PG4x9IbX2/uwlsCMzDgmF6uATY2PTfHJoot2Ke0tu2v+yt5tI5Gq1BWo SczWjwRJYflw97a/G/sO0EsxrHZPwZY3kT3RblC5h0r4MLjUGyXZprVUJIIxinEPTN9n fnnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=DJ2jD7D/6+OeBAcWa6T9Ov60zJPYCNdHSjLEDuaLeaw=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=DFSW+bPHV/UrEatOB76O7VBrlzM/GAurcyHJn6Hy9+eQhWdNtmO7y1xro4stYA3BSC n9N1ZYAm29xrktWGY1FmEDy2FqoLStx0Lk/ZNBwUKLd0qhMioFFr9MLpl727kDjC6SR4 qz+X9fguA+7B/Z8nr4sZ7spp6qDqR+HjDCSe9HHSPc2F7P5QAvxkDTcezNcuk+CS7Y8U gpMOiB8jvGo4685GADCmLiiPW30OiiMvPbVMtM8CGHbzK7rkUzLb2pKdc5uJkXlC8Jtu BOxLjl2f+/1qUPQpbjr9WblBVoGRztSKXYQVJnlpCOzcyuAYHZYijDYmiZgxCGYcduxb Zimw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id l189-20020a6388c6000000b005775302b11csi13368966pgd.81.2023.10.11.03.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 777A181C0C92; Wed, 11 Oct 2023 03:18:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234748AbjJKKSI (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234681AbjJKKSF (ORCPT ); Wed, 11 Oct 2023 06:18:05 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2130412B for ; Wed, 11 Oct 2023 03:07:47 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57gC51XdzrTWK; Wed, 11 Oct 2023 18:05:11 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:45 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 10/15] arm64: rework EL0 MRS emulation Date: Wed, 11 Oct 2023 10:06:50 +0000 Message-ID: <20231011100655.979626-11-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:49 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453956603159099 X-GMAIL-MSGID: 1779453956603159099 From: Mark Rutland commit f5962add74b61f8ae31c6311f75ca35d7e1d2d8f upstream. On CPUs without FEAT_IDST, ID register emulation is slower than it needs to be, as all threads contend for the same lock to perform the emulation. This patch reworks the emulation to avoid this unnecessary contention. On CPUs with FEAT_IDST (which is mandatory from ARMv8.4 onwards), EL0 accesses to ID registers result in a SYS trap, and emulation of these is handled with a sys64_hook. These hooks are statically allocated, and no locking is required to iterate through the hooks and perform the emulation, allowing emulation to occur in parallel with no contention. On CPUs without FEAT_IDST, EL0 accesses to ID registers result in an UNDEFINED exception, and emulation of these accesses is handled with an undef_hook. When an EL0 MRS instruction is trapped to EL1, the kernel finds the relevant handler by iterating through all of the undef_hooks, requiring undef_lock to be held during this lookup. This locking is only required to safely traverse the list of undef_hooks (as it can be concurrently modified), and the actual emulation of the MRS does not require any mutual exclusion. This locking is an unfortunate bottleneck, especially given that MRS emulation is enabled unconditionally and is never disabled. This patch reworks the non-FEAT_IDST MRS emulation logic so that it can be invoked directly from do_el0_undef(). This removes the bottleneck, allowing MRS traps to be handled entirely in parallel, and is a stepping stone to making all of the undef_hooks lock-free. I've tested this in a 64-vCPU VM on a 64-CPU ThunderX2 host, with a benchmark which spawns a number of threads which each try to read ID_AA64ISAR0_EL1 1000000 times. This is vastly more contention than will ever be seen in realistic usage, but clearly demonstrates the removal of the bottleneck: | Threads || Time (seconds) | | || Before || After | | || Real | System || Real | System | |---------++--------+---------++--------+---------| | 1 || 0.29 | 0.20 || 0.24 | 0.12 | | 2 || 0.35 | 0.51 || 0.23 | 0.27 | | 4 || 1.08 | 3.87 || 0.24 | 0.56 | | 8 || 4.31 | 33.60 || 0.24 | 1.11 | | 16 || 9.47 | 149.39 || 0.23 | 2.15 | | 32 || 19.07 | 605.27 || 0.24 | 4.38 | | 64 || 65.40 | 3609.09 || 0.33 | 11.27 | Aside from the speedup, there should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-6-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/cpufeature.h | 3 ++- arch/arm64/kernel/cpufeature.c | 23 +++++------------------ arch/arm64/kernel/traps.c | 3 +++ 3 files changed, 10 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index a77b5f49b3a6..72aaf7b92b31 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -808,7 +808,8 @@ static inline bool system_supports_tlb_range(void) cpus_have_const_cap(ARM64_HAS_TLB_RANGE); } -extern int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt); +int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt); +bool try_emulate_mrs(struct pt_regs *regs, u32 isn); static inline u32 id_aa64mmfr0_parange_to_phys_shift(int parange) { diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index d4ee345ff429..f17d6cdea260 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -3106,35 +3106,22 @@ int do_emulate_mrs(struct pt_regs *regs, u32 sys_reg, u32 rt) return rc; } -static int emulate_mrs(struct pt_regs *regs, u32 insn) +bool try_emulate_mrs(struct pt_regs *regs, u32 insn) { u32 sys_reg, rt; + if (compat_user_mode(regs) || !aarch64_insn_is_mrs(insn)) + return false; + /* * sys_reg values are defined as used in mrs/msr instruction. * shift the imm value to get the encoding. */ sys_reg = (u32)aarch64_insn_decode_immediate(AARCH64_INSN_IMM_16, insn) << 5; rt = aarch64_insn_decode_register(AARCH64_INSN_REGTYPE_RT, insn); - return do_emulate_mrs(regs, sys_reg, rt); + return do_emulate_mrs(regs, sys_reg, rt) == 0; } -static struct undef_hook mrs_hook = { - .instr_mask = 0xffff0000, - .instr_val = 0xd5380000, - .pstate_mask = PSR_AA32_MODE_MASK, - .pstate_val = PSR_MODE_EL0t, - .fn = emulate_mrs, -}; - -static int __init enable_mrs_emulation(void) -{ - register_undef_hook(&mrs_hook); - return 0; -} - -core_initcall(enable_mrs_emulation); - enum mitigation_state arm64_get_meltdown_state(void) { if (__meltdown_safe) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index e943b7c46d86..1f9285a35560 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -499,6 +499,9 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr) if (user_insn_read(regs, &insn)) goto out_err; + if (try_emulate_mrs(regs, insn)) + return; + if (call_undef_hook(regs, insn) == 0) return; From patchwork Wed Oct 11 10:06:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151245 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425663vqb; Wed, 11 Oct 2023 03:09:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGXy6Qbmlg/nV+69Xmey5O76b0z+fqt0JXWHLXItYKuDuPreSimQSXECuDxv2aHLDYE70dj X-Received: by 2002:a05:6a20:3d85:b0:163:f945:42c4 with SMTP id s5-20020a056a203d8500b00163f94542c4mr24170731pzi.1.1697018999659; Wed, 11 Oct 2023 03:09:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697018999; cv=none; d=google.com; s=arc-20160816; b=MTk5m5h4lkRdyakBsEv4Fy3qoJmilCIkR6q5fhsgASchRW9Rysk2rxy5E7PU2KxtbP tTIFwmXubsZgvMmLUjD6t0+GH6fjhkGQQV3ZeyTP1HNX+FcypHWRvxJbSgXaDpG5ahG1 NHzJB9iXNGZSS7ZrvuvppxuWr7oImexaw2u/Vwz/9cqD6kfexSkKKdFAzxmER1JXii1S hfhKPPPuBKXo0YSM/aEHD5dZxmpyz/JNK5VY7CjDRFmPSzHezHtw3xKFfV1n1OdPQeCT /vfb4fxf9OQ6BUCmYUGxtKoy9IE3i0fweWnHUUjmbl9UKYxHtGj9n84cOVZA45uNwbCK ztwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=chC1jACEqDHsMu5zuvk6FGkyQz3LTfMlMYCsdaAzCoI=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=zUVp/3iAuIU3I+FYsK+olD16CZDhamfOF3hvuRTKR7EvYTjnGdGlnq9PGzsik309fx PrJKdUrxDJF+E28hhYf70cPUWS9qFhzOaGoX1+vd6Y9v+zTMOfETgZDQZsrp1768LEav q7oviKCFdY8R58ZvMYwDehQJKnng8d/3kZeKb9rB/2grJ0lJc12qpGRW0l2/zbyQzm8W joRjfqtYe2Yr+rcvPrEWbsdin2s6ZxrzBJGP4EG2uJ2+TCJs3ju0XH/hSI94Kw9n1nss 0sOAg/zNwGh+v7POz3KYY8CVcJQk7Se8zl3DMGm7xG7uEO9w1fk751n+2q3blvhE4ltT nYgw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id f16-20020a170902ce9000b001c755810f7esi15209588plg.499.2023.10.11.03.09.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:09:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 2D3DF8347B48; Wed, 11 Oct 2023 03:09:56 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234690AbjJKKJe (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234773AbjJKKIq (ORCPT ); Wed, 11 Oct 2023 06:08:46 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B5D41BCC for ; Wed, 11 Oct 2023 03:07:48 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57gD5wh5zrTFZ; Wed, 11 Oct 2023 18:05:12 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:46 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 11/15] arm64: armv8_deprecated: fold ops into insn_emulation Date: Wed, 11 Oct 2023 10:06:51 +0000 Message-ID: <20231011100655.979626-12-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:09:56 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453394312017417 X-GMAIL-MSGID: 1779453394312017417 From: Mark Rutland commit b4453cc8a7ebbd45436a8cd3ffeaa069ceac146f upstream. The code for emulating deprecated instructions has two related structures: struct insn_emulation_ops and struct insn_emulation, where each struct insn_emulation_ops is associated 1-1 with a struct insn_emulation. It would be simpler to combine the two into a single structure, removing the need for (unconditional) dynamic allocation at boot time, and simplifying some runtime pointer chasing. This patch merges the two structures together. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-7-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/armv8_deprecated.c | 76 ++++++++++++---------------- 1 file changed, 33 insertions(+), 43 deletions(-) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index c5da9d1e954a..494946942b7c 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -41,16 +41,12 @@ enum legacy_insn_status { INSN_OBSOLETE, }; -struct insn_emulation_ops { - const char *name; - enum legacy_insn_status status; - struct undef_hook *hooks; - int (*set_hw_mode)(bool enable); -}; - struct insn_emulation { - struct list_head node; - struct insn_emulation_ops *ops; + const char *name; + struct list_head node; + enum legacy_insn_status status; + struct undef_hook *hooks; + int (*set_hw_mode)(bool enable); int current_mode; int min; int max; @@ -61,48 +57,48 @@ static int nr_insn_emulated __initdata; static DEFINE_RAW_SPINLOCK(insn_emulation_lock); static DEFINE_MUTEX(insn_emulation_mutex); -static void register_emulation_hooks(struct insn_emulation_ops *ops) +static void register_emulation_hooks(struct insn_emulation *insn) { struct undef_hook *hook; - BUG_ON(!ops->hooks); + BUG_ON(!insn->hooks); - for (hook = ops->hooks; hook->instr_mask; hook++) + for (hook = insn->hooks; hook->instr_mask; hook++) register_undef_hook(hook); - pr_notice("Registered %s emulation handler\n", ops->name); + pr_notice("Registered %s emulation handler\n", insn->name); } -static void remove_emulation_hooks(struct insn_emulation_ops *ops) +static void remove_emulation_hooks(struct insn_emulation *insn) { struct undef_hook *hook; - BUG_ON(!ops->hooks); + BUG_ON(!insn->hooks); - for (hook = ops->hooks; hook->instr_mask; hook++) + for (hook = insn->hooks; hook->instr_mask; hook++) unregister_undef_hook(hook); - pr_notice("Removed %s emulation handler\n", ops->name); + pr_notice("Removed %s emulation handler\n", insn->name); } static void enable_insn_hw_mode(void *data) { struct insn_emulation *insn = (struct insn_emulation *)data; - if (insn->ops->set_hw_mode) - insn->ops->set_hw_mode(true); + if (insn->set_hw_mode) + insn->set_hw_mode(true); } static void disable_insn_hw_mode(void *data) { struct insn_emulation *insn = (struct insn_emulation *)data; - if (insn->ops->set_hw_mode) - insn->ops->set_hw_mode(false); + if (insn->set_hw_mode) + insn->set_hw_mode(false); } /* Run set_hw_mode(mode) on all active CPUs */ static int run_all_cpu_set_hw_mode(struct insn_emulation *insn, bool enable) { - if (!insn->ops->set_hw_mode) + if (!insn->set_hw_mode) return -EINVAL; if (enable) on_each_cpu(enable_insn_hw_mode, (void *)insn, true); @@ -126,9 +122,9 @@ static int run_all_insn_set_hw_mode(unsigned int cpu) raw_spin_lock_irqsave(&insn_emulation_lock, flags); list_for_each_entry(insn, &insn_emulation, node) { bool enable = (insn->current_mode == INSN_HW); - if (insn->ops->set_hw_mode && insn->ops->set_hw_mode(enable)) { + if (insn->set_hw_mode && insn->set_hw_mode(enable)) { pr_warn("CPU[%u] cannot support the emulation of %s", - cpu, insn->ops->name); + cpu, insn->name); rc = -EINVAL; } } @@ -145,11 +141,11 @@ static int update_insn_emulation_mode(struct insn_emulation *insn, case INSN_UNDEF: /* Nothing to be done */ break; case INSN_EMULATE: - remove_emulation_hooks(insn->ops); + remove_emulation_hooks(insn); break; case INSN_HW: if (!run_all_cpu_set_hw_mode(insn, false)) - pr_notice("Disabled %s support\n", insn->ops->name); + pr_notice("Disabled %s support\n", insn->name); break; } @@ -157,31 +153,25 @@ static int update_insn_emulation_mode(struct insn_emulation *insn, case INSN_UNDEF: break; case INSN_EMULATE: - register_emulation_hooks(insn->ops); + register_emulation_hooks(insn); break; case INSN_HW: ret = run_all_cpu_set_hw_mode(insn, true); if (!ret) - pr_notice("Enabled %s support\n", insn->ops->name); + pr_notice("Enabled %s support\n", insn->name); break; } return ret; } -static void __init register_insn_emulation(struct insn_emulation_ops *ops) +static void __init register_insn_emulation(struct insn_emulation *insn) { unsigned long flags; - struct insn_emulation *insn; - - insn = kzalloc(sizeof(*insn), GFP_KERNEL); - if (!insn) - return; - insn->ops = ops; insn->min = INSN_UNDEF; - switch (ops->status) { + switch (insn->status) { case INSN_DEPRECATED: insn->current_mode = INSN_EMULATE; /* Disable the HW mode if it was turned on at early boot time */ @@ -247,7 +237,7 @@ static void __init register_insn_emulation_sysctl(void) sysctl->mode = 0644; sysctl->maxlen = sizeof(int); - sysctl->procname = insn->ops->name; + sysctl->procname = insn->name; sysctl->data = &insn->current_mode; sysctl->extra1 = &insn->min; sysctl->extra2 = &insn->max; @@ -451,7 +441,7 @@ static struct undef_hook swp_hooks[] = { { } }; -static struct insn_emulation_ops swp_ops = { +static struct insn_emulation insn_swp = { .name = "swp", .status = INSN_OBSOLETE, .hooks = swp_hooks, @@ -538,7 +528,7 @@ static struct undef_hook cp15_barrier_hooks[] = { { } }; -static struct insn_emulation_ops cp15_barrier_ops = { +static struct insn_emulation insn_cp15_barrier = { .name = "cp15_barrier", .status = INSN_DEPRECATED, .hooks = cp15_barrier_hooks, @@ -611,7 +601,7 @@ static struct undef_hook setend_hooks[] = { {} }; -static struct insn_emulation_ops setend_ops = { +static struct insn_emulation insn_setend = { .name = "setend", .status = INSN_DEPRECATED, .hooks = setend_hooks, @@ -625,14 +615,14 @@ static struct insn_emulation_ops setend_ops = { static int __init armv8_deprecated_init(void) { if (IS_ENABLED(CONFIG_SWP_EMULATION)) - register_insn_emulation(&swp_ops); + register_insn_emulation(&insn_swp); if (IS_ENABLED(CONFIG_CP15_BARRIER_EMULATION)) - register_insn_emulation(&cp15_barrier_ops); + register_insn_emulation(&insn_cp15_barrier); if (IS_ENABLED(CONFIG_SETEND_EMULATION)) { if (system_supports_mixed_endian_el0()) - register_insn_emulation(&setend_ops); + register_insn_emulation(&insn_setend); else pr_info("setend instruction emulation is not supported on this system\n"); } From patchwork Wed Oct 11 10:06:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151255 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429811vqb; Wed, 11 Oct 2023 03:18:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEuvBb0YAu48O+BMVH54sg2T7j+cHL1aMKhsBYhski+wum2f4OtGHL2imqFF7TN+JAPUdcH X-Received: by 2002:a05:6a00:2e87:b0:690:c79c:1912 with SMTP id fd7-20020a056a002e8700b00690c79c1912mr23089075pfb.0.1697019524883; Wed, 11 Oct 2023 03:18:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019524; cv=none; d=google.com; s=arc-20160816; b=Zye/x4bKxeQ7EqxJvGNR2VXF0ylKti3q5Lc6iO98wNmhAfBdnZXiaD83rCGTf2jdpw fUatTc8Rc1iKu2yrruyOY/5UPxF5s51MtWBXODOHpqbWlEqASYS2vPGSDZo0N9om4IJ+ lMAnL+HHYRca6OwMKRLqHYvOCUmLb2TuachFMYw275ymg03vcoQ/gqKYdXIixf9fV9rR ZGTSlOhg29MUusfkWVNwtPkk6NdeCb8YNo7SOrCuoanWj6U/OIS1FEhTnVs4A2e8v7qD C0k5r93pU9/LLMbVefbQ34060VfAjVmnIktWOJI8CxE1n8K28VOz1wL33gMlYDDsw2rw sBNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=ZxzzoH26dh1AzYODI3PztvwKZzoiY9qQBnMMbD9JPPk=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=0OiyVRpBbGP3BknowMsm8gNg/7w64JzEECmliZ1j9Pbj8SkpYHwgLXAPhOG6YGedt+ CGKTjJJXTCIu7OK8RrTw/df2vDG4qw8tGdze5eut+lhyMdq8wBKt9VyUdCn21OaikhQI D4wYoO3/UW4KFnFKLfEXWbCHTHisWzOL5yNECrlbjBwYnSGw9F6cWKDsIlrlk+ZNfn9V d+uSC1etpMAish4EtQVq2JECqd171WWEG86OH1R2NaTyjTV9LzKZRhAak2byFPpSinQz wXQJcZBtAqYZ+ShJw1AIJ6YEFVxB431N0vNbLx11735von5pq9NvlKk4m6EQXuCJ6VpV hSYw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id q15-20020a056a00084f00b00690ca3d66bbsi5543581pfk.262.2023.10.11.03.18.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 0BF578115446; Wed, 11 Oct 2023 03:18:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346207AbjJKKS2 (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234690AbjJKKSG (ORCPT ); Wed, 11 Oct 2023 06:18:06 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 349551BD1 for ; Wed, 11 Oct 2023 03:07:50 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4S57dh09L3zLqZW; Wed, 11 Oct 2023 18:03:52 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:47 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 12/15] arm64: armv8_deprecated move emulation functions Date: Wed, 11 Oct 2023 10:06:52 +0000 Message-ID: <20231011100655.979626-13-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:40 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453945357034123 X-GMAIL-MSGID: 1779453945357034123 From: Mark Rutland commit 25eeac0cfe7c97ade1be07340e11e7143aab57a6 upstream. Subsequent patches will rework the logic in armv8_deprecated.c. In preparation for subsequent changes, this patch moves the emulation logic earlier in the file, and moves the infrastructure later in the file. This will make subsequent diffs simpler and easier to read. This is purely a move. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-8-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/armv8_deprecated.c | 394 +++++++++++++-------------- 1 file changed, 197 insertions(+), 197 deletions(-) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index 494946942b7c..e52ae01b4805 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -52,203 +52,6 @@ struct insn_emulation { int max; }; -static LIST_HEAD(insn_emulation); -static int nr_insn_emulated __initdata; -static DEFINE_RAW_SPINLOCK(insn_emulation_lock); -static DEFINE_MUTEX(insn_emulation_mutex); - -static void register_emulation_hooks(struct insn_emulation *insn) -{ - struct undef_hook *hook; - - BUG_ON(!insn->hooks); - - for (hook = insn->hooks; hook->instr_mask; hook++) - register_undef_hook(hook); - - pr_notice("Registered %s emulation handler\n", insn->name); -} - -static void remove_emulation_hooks(struct insn_emulation *insn) -{ - struct undef_hook *hook; - - BUG_ON(!insn->hooks); - - for (hook = insn->hooks; hook->instr_mask; hook++) - unregister_undef_hook(hook); - - pr_notice("Removed %s emulation handler\n", insn->name); -} - -static void enable_insn_hw_mode(void *data) -{ - struct insn_emulation *insn = (struct insn_emulation *)data; - if (insn->set_hw_mode) - insn->set_hw_mode(true); -} - -static void disable_insn_hw_mode(void *data) -{ - struct insn_emulation *insn = (struct insn_emulation *)data; - if (insn->set_hw_mode) - insn->set_hw_mode(false); -} - -/* Run set_hw_mode(mode) on all active CPUs */ -static int run_all_cpu_set_hw_mode(struct insn_emulation *insn, bool enable) -{ - if (!insn->set_hw_mode) - return -EINVAL; - if (enable) - on_each_cpu(enable_insn_hw_mode, (void *)insn, true); - else - on_each_cpu(disable_insn_hw_mode, (void *)insn, true); - return 0; -} - -/* - * Run set_hw_mode for all insns on a starting CPU. - * Returns: - * 0 - If all the hooks ran successfully. - * -EINVAL - At least one hook is not supported by the CPU. - */ -static int run_all_insn_set_hw_mode(unsigned int cpu) -{ - int rc = 0; - unsigned long flags; - struct insn_emulation *insn; - - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_for_each_entry(insn, &insn_emulation, node) { - bool enable = (insn->current_mode == INSN_HW); - if (insn->set_hw_mode && insn->set_hw_mode(enable)) { - pr_warn("CPU[%u] cannot support the emulation of %s", - cpu, insn->name); - rc = -EINVAL; - } - } - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); - return rc; -} - -static int update_insn_emulation_mode(struct insn_emulation *insn, - enum insn_emulation_mode prev) -{ - int ret = 0; - - switch (prev) { - case INSN_UNDEF: /* Nothing to be done */ - break; - case INSN_EMULATE: - remove_emulation_hooks(insn); - break; - case INSN_HW: - if (!run_all_cpu_set_hw_mode(insn, false)) - pr_notice("Disabled %s support\n", insn->name); - break; - } - - switch (insn->current_mode) { - case INSN_UNDEF: - break; - case INSN_EMULATE: - register_emulation_hooks(insn); - break; - case INSN_HW: - ret = run_all_cpu_set_hw_mode(insn, true); - if (!ret) - pr_notice("Enabled %s support\n", insn->name); - break; - } - - return ret; -} - -static void __init register_insn_emulation(struct insn_emulation *insn) -{ - unsigned long flags; - - insn->min = INSN_UNDEF; - - switch (insn->status) { - case INSN_DEPRECATED: - insn->current_mode = INSN_EMULATE; - /* Disable the HW mode if it was turned on at early boot time */ - run_all_cpu_set_hw_mode(insn, false); - insn->max = INSN_HW; - break; - case INSN_OBSOLETE: - insn->current_mode = INSN_UNDEF; - insn->max = INSN_EMULATE; - break; - } - - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_add(&insn->node, &insn_emulation); - nr_insn_emulated++; - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); - - /* Register any handlers if required */ - update_insn_emulation_mode(insn, INSN_UNDEF); -} - -static int emulation_proc_handler(struct ctl_table *table, int write, - void *buffer, size_t *lenp, - loff_t *ppos) -{ - int ret = 0; - struct insn_emulation *insn = container_of(table->data, struct insn_emulation, current_mode); - enum insn_emulation_mode prev_mode = insn->current_mode; - - mutex_lock(&insn_emulation_mutex); - ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos); - - if (ret || !write || prev_mode == insn->current_mode) - goto ret; - - ret = update_insn_emulation_mode(insn, prev_mode); - if (ret) { - /* Mode change failed, revert to previous mode. */ - insn->current_mode = prev_mode; - update_insn_emulation_mode(insn, INSN_UNDEF); - } -ret: - mutex_unlock(&insn_emulation_mutex); - return ret; -} - -static void __init register_insn_emulation_sysctl(void) -{ - unsigned long flags; - int i = 0; - struct insn_emulation *insn; - struct ctl_table *insns_sysctl, *sysctl; - - insns_sysctl = kcalloc(nr_insn_emulated + 1, sizeof(*sysctl), - GFP_KERNEL); - if (!insns_sysctl) - return; - - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_for_each_entry(insn, &insn_emulation, node) { - sysctl = &insns_sysctl[i]; - - sysctl->mode = 0644; - sysctl->maxlen = sizeof(int); - - sysctl->procname = insn->name; - sysctl->data = &insn->current_mode; - sysctl->extra1 = &insn->min; - sysctl->extra2 = &insn->max; - sysctl->proc_handler = emulation_proc_handler; - i++; - } - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); - - register_sysctl("abi", insns_sysctl); -} - /* * Implement emulation of the SWP/SWPB instructions using load-exclusive and * store-exclusive. @@ -608,6 +411,203 @@ static struct insn_emulation insn_setend = { .set_hw_mode = setend_set_hw_mode, }; +static LIST_HEAD(insn_emulation); +static int nr_insn_emulated __initdata; +static DEFINE_RAW_SPINLOCK(insn_emulation_lock); +static DEFINE_MUTEX(insn_emulation_mutex); + +static void register_emulation_hooks(struct insn_emulation *insn) +{ + struct undef_hook *hook; + + BUG_ON(!insn->hooks); + + for (hook = insn->hooks; hook->instr_mask; hook++) + register_undef_hook(hook); + + pr_notice("Registered %s emulation handler\n", insn->name); +} + +static void remove_emulation_hooks(struct insn_emulation *insn) +{ + struct undef_hook *hook; + + BUG_ON(!insn->hooks); + + for (hook = insn->hooks; hook->instr_mask; hook++) + unregister_undef_hook(hook); + + pr_notice("Removed %s emulation handler\n", insn->name); +} + +static void enable_insn_hw_mode(void *data) +{ + struct insn_emulation *insn = (struct insn_emulation *)data; + if (insn->set_hw_mode) + insn->set_hw_mode(true); +} + +static void disable_insn_hw_mode(void *data) +{ + struct insn_emulation *insn = (struct insn_emulation *)data; + if (insn->set_hw_mode) + insn->set_hw_mode(false); +} + +/* Run set_hw_mode(mode) on all active CPUs */ +static int run_all_cpu_set_hw_mode(struct insn_emulation *insn, bool enable) +{ + if (!insn->set_hw_mode) + return -EINVAL; + if (enable) + on_each_cpu(enable_insn_hw_mode, (void *)insn, true); + else + on_each_cpu(disable_insn_hw_mode, (void *)insn, true); + return 0; +} + +/* + * Run set_hw_mode for all insns on a starting CPU. + * Returns: + * 0 - If all the hooks ran successfully. + * -EINVAL - At least one hook is not supported by the CPU. + */ +static int run_all_insn_set_hw_mode(unsigned int cpu) +{ + int rc = 0; + unsigned long flags; + struct insn_emulation *insn; + + raw_spin_lock_irqsave(&insn_emulation_lock, flags); + list_for_each_entry(insn, &insn_emulation, node) { + bool enable = (insn->current_mode == INSN_HW); + if (insn->set_hw_mode && insn->set_hw_mode(enable)) { + pr_warn("CPU[%u] cannot support the emulation of %s", + cpu, insn->name); + rc = -EINVAL; + } + } + raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); + return rc; +} + +static int update_insn_emulation_mode(struct insn_emulation *insn, + enum insn_emulation_mode prev) +{ + int ret = 0; + + switch (prev) { + case INSN_UNDEF: /* Nothing to be done */ + break; + case INSN_EMULATE: + remove_emulation_hooks(insn); + break; + case INSN_HW: + if (!run_all_cpu_set_hw_mode(insn, false)) + pr_notice("Disabled %s support\n", insn->name); + break; + } + + switch (insn->current_mode) { + case INSN_UNDEF: + break; + case INSN_EMULATE: + register_emulation_hooks(insn); + break; + case INSN_HW: + ret = run_all_cpu_set_hw_mode(insn, true); + if (!ret) + pr_notice("Enabled %s support\n", insn->name); + break; + } + + return ret; +} + +static void __init register_insn_emulation(struct insn_emulation *insn) +{ + unsigned long flags; + + insn->min = INSN_UNDEF; + + switch (insn->status) { + case INSN_DEPRECATED: + insn->current_mode = INSN_EMULATE; + /* Disable the HW mode if it was turned on at early boot time */ + run_all_cpu_set_hw_mode(insn, false); + insn->max = INSN_HW; + break; + case INSN_OBSOLETE: + insn->current_mode = INSN_UNDEF; + insn->max = INSN_EMULATE; + break; + } + + raw_spin_lock_irqsave(&insn_emulation_lock, flags); + list_add(&insn->node, &insn_emulation); + nr_insn_emulated++; + raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); + + /* Register any handlers if required */ + update_insn_emulation_mode(insn, INSN_UNDEF); +} + +static int emulation_proc_handler(struct ctl_table *table, int write, + void *buffer, size_t *lenp, + loff_t *ppos) +{ + int ret = 0; + struct insn_emulation *insn = container_of(table->data, struct insn_emulation, current_mode); + enum insn_emulation_mode prev_mode = insn->current_mode; + + mutex_lock(&insn_emulation_mutex); + ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos); + + if (ret || !write || prev_mode == insn->current_mode) + goto ret; + + ret = update_insn_emulation_mode(insn, prev_mode); + if (ret) { + /* Mode change failed, revert to previous mode. */ + insn->current_mode = prev_mode; + update_insn_emulation_mode(insn, INSN_UNDEF); + } +ret: + mutex_unlock(&insn_emulation_mutex); + return ret; +} + +static void __init register_insn_emulation_sysctl(void) +{ + unsigned long flags; + int i = 0; + struct insn_emulation *insn; + struct ctl_table *insns_sysctl, *sysctl; + + insns_sysctl = kcalloc(nr_insn_emulated + 1, sizeof(*sysctl), + GFP_KERNEL); + if (!insns_sysctl) + return; + + raw_spin_lock_irqsave(&insn_emulation_lock, flags); + list_for_each_entry(insn, &insn_emulation, node) { + sysctl = &insns_sysctl[i]; + + sysctl->mode = 0644; + sysctl->maxlen = sizeof(int); + + sysctl->procname = insn->name; + sysctl->data = &insn->current_mode; + sysctl->extra1 = &insn->min; + sysctl->extra2 = &insn->max; + sysctl->proc_handler = emulation_proc_handler; + i++; + } + raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); + + register_sysctl("abi", insns_sysctl); +} + /* * Invoked as core_initcall, which guarantees that the instruction * emulation is ready for userspace. From patchwork Wed Oct 11 10:06:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151252 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429695vqb; Wed, 11 Oct 2023 03:18:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFkxg/cP7cdMgmK4sVbYn97UhsO5OuB0IBvVRCjw08i0OiklJJ9fZHxIjoX+F9rpthVLrGJ X-Received: by 2002:a05:6602:140c:b0:79f:a8c2:290d with SMTP id t12-20020a056602140c00b0079fa8c2290dmr27177364iov.0.1697019508680; Wed, 11 Oct 2023 03:18:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019508; cv=none; d=google.com; s=arc-20160816; b=oXAfnU7ZuRgy9IR3pWoVaS+8o5sLFYIS957cM/OHqrWDx9/bvRVExyuf1HAqtZnCSl lH7amxNH6BHRTMB6z6aNECm5YXs/5xQ7BXGtjtZdEIHbMP7uGAdsYFTnvF/AXVeqDkHU zgDmfo2OZv1AMFNLUyfUGML29jKaorC9WC0i2jFGJdOjsGyDCFg0xiVO17FnCYVReSUT PYQSgYhYCOz4RHofgXtYOUIpNdweGmiFNUJXAEMNwtNmbHuQk6tcjQZnBYCyUy7PuDVY Guw1SvAbqktdEDU4NrvUPr6PjIBAoUDWbmW7wQ+Kh87NlcSY+XpnJOoZWjfOzYjKhIGE hnqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Hrq3y4xRbvvO2iWiNmHAgoCoLU44DYyAdlVskUoJz3Q=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=NipZ9W28Km2mwTEUdpuww3Nv9NHMtfD+1xvhTf6jsQHfumq8G3utYBXHtkWSiCPxob 8G/IbxGQk26AhwZGzDnQv4tti4WW6Uz1Q4YKnJVa976NbEU+a5AxIZPq+azgKUiU8452 2AAxPRWuWK+ZqX0NJyoKNibTHpoomM1c/kYl+lPPLJ2Bx+VtI5ScgjU8Vc6gOOVZ2b9f tOY48rW9rPaug6TNFP4XvouMr1BJwUPffSgmR7VRg2DtJhuTfSRZ9kyEZKNV6uC2anVy v+ikbyBykSKpgMvt3LjDkCpv7xSntoJY+DFQid3wfkEE0rSPg6iNHEa7iqsmuXRTY1tg H3wQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id q8-20020a056a00084800b0069338b22bfdsi12152348pfk.205.2023.10.11.03.18.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 6317B8115443; Wed, 11 Oct 2023 03:18:26 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234583AbjJKKSN (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234720AbjJKKSG (ORCPT ); Wed, 11 Oct 2023 06:18:06 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 497201BDF for ; Wed, 11 Oct 2023 03:07:51 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4S57cw1m5Qzcd4S; Wed, 11 Oct 2023 18:03:12 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:48 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 13/15] arm64: armv8_deprecated: move aarch32 helper earlier Date: Wed, 11 Oct 2023 10:06:53 +0000 Message-ID: <20231011100655.979626-14-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:26 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453928208442333 X-GMAIL-MSGID: 1779453928208442333 From: Mark Rutland commit 0c5f416219da3795dc8b33e5bb7865a6b3c4e55c upstream. Subsequent patches will rework the logic in armv8_deprecated.c. In preparation for subsequent changes, this patch moves some shared logic earlier in the file. This will make subsequent diffs simpler and easier to read. At the same time, drop the `__kprobes` annotation from aarch32_check_condition(), as this is only used for traps from compat userspace, and has no risk of recursion within kprobes. As this is the last kprobes annotation in armve8_deprecated.c, we no longer need to include . Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-9-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/armv8_deprecated.c | 39 ++++++++++++++-------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index e52ae01b4805..6555bceff54b 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -17,7 +17,6 @@ #include #include #include -#include #define CREATE_TRACE_POINTS #include "trace-events-emulation.h" @@ -52,6 +51,25 @@ struct insn_emulation { int max; }; +#define ARM_OPCODE_CONDTEST_FAIL 0 +#define ARM_OPCODE_CONDTEST_PASS 1 +#define ARM_OPCODE_CONDTEST_UNCOND 2 + +#define ARM_OPCODE_CONDITION_UNCOND 0xf + +static unsigned int aarch32_check_condition(u32 opcode, u32 psr) +{ + u32 cc_bits = opcode >> 28; + + if (cc_bits != ARM_OPCODE_CONDITION_UNCOND) { + if ((*aarch32_opcode_cond_checks[cc_bits])(psr)) + return ARM_OPCODE_CONDTEST_PASS; + else + return ARM_OPCODE_CONDTEST_FAIL; + } + return ARM_OPCODE_CONDTEST_UNCOND; +} + /* * Implement emulation of the SWP/SWPB instructions using load-exclusive and * store-exclusive. @@ -138,25 +156,6 @@ static int emulate_swpX(unsigned int address, unsigned int *data, return res; } -#define ARM_OPCODE_CONDTEST_FAIL 0 -#define ARM_OPCODE_CONDTEST_PASS 1 -#define ARM_OPCODE_CONDTEST_UNCOND 2 - -#define ARM_OPCODE_CONDITION_UNCOND 0xf - -static unsigned int __kprobes aarch32_check_condition(u32 opcode, u32 psr) -{ - u32 cc_bits = opcode >> 28; - - if (cc_bits != ARM_OPCODE_CONDITION_UNCOND) { - if ((*aarch32_opcode_cond_checks[cc_bits])(psr)) - return ARM_OPCODE_CONDTEST_PASS; - else - return ARM_OPCODE_CONDTEST_FAIL; - } - return ARM_OPCODE_CONDTEST_UNCOND; -} - /* * swp_handler logs the id of calling process, dissects the instruction, sanity * checks the memory location, calls emulate_swpX for the actual operation and From patchwork Wed Oct 11 10:06:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151256 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp429841vqb; Wed, 11 Oct 2023 03:18:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH2Ye+Es2Kast54sqCZZDQ7XQScjQZ1AifkXOEPWRyaqbNdEg6cG8e4nEKH06ARR2fJ+XtL X-Received: by 2002:a05:6359:639d:b0:149:cbfd:c663 with SMTP id sg29-20020a056359639d00b00149cbfdc663mr16146970rwb.2.1697019527643; Wed, 11 Oct 2023 03:18:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019527; cv=none; d=google.com; s=arc-20160816; b=s0Urr5JfSL51GRyJR7xCCboCFOI+IcLoKgL5ZgP7BkQshVcXqv27ZBgoHJxi6VSr6J N3o1KX0xoJlx2qtLoMHArmuxoY8O1NNYpnzDj+XrAnqB8yPmTjnWXP/XRFF1HxqRepqS 3vrwTbNTVL1JU3HMzhHfbpeT5mnbePIv2w13is6XkmYU/CRaZapfR1GtS+ZGArVmMpjL BEL03EBXDgX71tbT8SCHP7KJkbWGBD9QPcjW8hXKB0BGRRmFEKJGSPHjOshVLunX6Cmd y21jUXKvXP4n/1V7ChdVcxJG+DnfIbiyMxkVcYwGSWAjjJZq6OwygR7LgJHIMne2TcPc 71zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=PYQM1YxiXvB7S2QJH/aRYj2s8fkD0v0mJFeUBXg6xgc=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=vyTqHzxJOAAGhtV2FEH522XRk856thldCy8imQDsCx79uxTJZI8VR3N6Wo3yjgRqAt RVk0ELoT7d3RxA6Ap8kJvu91tFKEDpxKTnGRQzQLaflgiSeHD02ZL2eILpxZIfIP/CnU zcielOCzO3o96D+t87J5jnsxzLlMhXKu+A6P9w+uYUEfXWt8SUNUVEfmwuk5Zv+bS4PJ gSmi7cbW+1VAdS3fLo+mRS2H9ckQ75ELeR9Ea3i5dBkPSQdrGD1fIGIEpC1cFwAWkzQ6 8SU6M+tEuqbeq5oog9G/G5LNtdMYDw2rLbC6q01L69aPm7Iaf41iNJYxEIe0J3fIcrrB PkLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id be11-20020a056a001f0b00b0068beea8071esi11405182pfb.221.2023.10.11.03.18.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:18:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 8746D81B8019; Wed, 11 Oct 2023 03:18:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345954AbjJKKSm (ORCPT + 18 others); Wed, 11 Oct 2023 06:18:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234641AbjJKKSL (ORCPT ); Wed, 11 Oct 2023 06:18:11 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF4BC186 for ; Wed, 11 Oct 2023 03:07:52 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4S57dh4NJ9zkYB0; Wed, 11 Oct 2023 18:03:52 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:49 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 14/15] arm64: armv8_deprecated: rework deprected instruction handling Date: Wed, 11 Oct 2023 10:06:54 +0000 Message-ID: <20231011100655.979626-15-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:18:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453948165018653 X-GMAIL-MSGID: 1779453948165018653 From: Mark Rutland commit 124c49b1b5d947b7180c5d6cbb09ddf76ea45ea2 upstream. Support for deprecated instructions can be enabled or disabled at runtime. To handle this, the code in armv8_deprecated.c registers and unregisters undef_hooks, and makes cross CPU calls to configure HW support. This is rather complicated, and the synchronization required to make this safe ends up serializing the handling of instructions which have been trapped. This patch simplifies the deprecated instruction handling by removing the dynamic registration and unregistration, and changing the trap handling code to determine whether a handler should be invoked. This removes the need for dynamic list management, and simplifies the locking requirements, making it possible to handle trapped instructions entirely in parallel. Where changing the emulation state requires a cross-call, this is serialized by locally disabling interrupts, ensuring that the CPU is not left in an inconsistent state. To simplify sysctl management, each insn_emulation is given a separate sysctl table, permitting these to be registered separately. The core sysctl code will iterate over all of these when walking sysfs. I've tested this with userspace programs which use each of the deprecated instructions, and I've concurrently modified the support level for each of the features back-and-forth between HW and emulated to check that there are no spurious SIGILLs sent to userspace when the support level is changed. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: James Morse Cc: Joey Gouly Cc: Peter Zijlstra Cc: Will Deacon Link: https://lore.kernel.org/r/20221019144123.612388-10-mark.rutland@arm.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/include/asm/traps.h | 19 +- arch/arm64/kernel/armv8_deprecated.c | 287 ++++++++++++++------------- arch/arm64/kernel/traps.c | 40 +--- 3 files changed, 154 insertions(+), 192 deletions(-) diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h index 6e5826470bea..1f361e2da516 100644 --- a/arch/arm64/include/asm/traps.h +++ b/arch/arm64/include/asm/traps.h @@ -13,17 +13,16 @@ struct pt_regs; -struct undef_hook { - struct list_head node; - u32 instr_mask; - u32 instr_val; - u64 pstate_mask; - u64 pstate_val; - int (*fn)(struct pt_regs *regs, u32 instr); -}; +#ifdef CONFIG_ARMV8_DEPRECATED +bool try_emulate_armv8_deprecated(struct pt_regs *regs, u32 insn); +#else +static inline bool +try_emulate_armv8_deprecated(struct pt_regs *regs, u32 insn) +{ + return false; +} +#endif /* CONFIG_ARMV8_DEPRECATED */ -void register_undef_hook(struct undef_hook *hook); -void unregister_undef_hook(struct undef_hook *hook); void force_signal_inject(int signal, int code, unsigned long address, unsigned long err); void arm64_notify_segfault(unsigned long addr); void arm64_force_sig_fault(int signo, int code, unsigned long far, const char *str); diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index 6555bceff54b..758008020b59 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -38,17 +38,24 @@ enum insn_emulation_mode { enum legacy_insn_status { INSN_DEPRECATED, INSN_OBSOLETE, + INSN_UNAVAILABLE, }; struct insn_emulation { const char *name; - struct list_head node; enum legacy_insn_status status; - struct undef_hook *hooks; + bool (*try_emulate)(struct pt_regs *regs, + u32 insn); int (*set_hw_mode)(bool enable); + int current_mode; int min; int max; + + /* + * sysctl for this emulation + a sentinal entry. + */ + struct ctl_table sysctl[2]; }; #define ARM_OPCODE_CONDTEST_FAIL 0 @@ -70,6 +77,7 @@ static unsigned int aarch32_check_condition(u32 opcode, u32 psr) return ARM_OPCODE_CONDTEST_UNCOND; } +#ifdef CONFIG_SWP_EMULATION /* * Implement emulation of the SWP/SWPB instructions using load-exclusive and * store-exclusive. @@ -228,28 +236,27 @@ static int swp_handler(struct pt_regs *regs, u32 instr) return 0; } -/* - * Only emulate SWP/SWPB executed in ARM state/User mode. - * The kernel must be SWP free and SWP{B} does not exist in Thumb. - */ -static struct undef_hook swp_hooks[] = { - { - .instr_mask = 0x0fb00ff0, - .instr_val = 0x01000090, - .pstate_mask = PSR_AA32_MODE_MASK, - .pstate_val = PSR_AA32_MODE_USR, - .fn = swp_handler - }, - { } -}; +static bool try_emulate_swp(struct pt_regs *regs, u32 insn) +{ + /* SWP{B} only exists in ARM state and does not exist in Thumb */ + if (!compat_user_mode(regs) || compat_thumb_mode(regs)) + return false; + + if ((insn & 0x0fb00ff0) != 0x01000090) + return false; + + return swp_handler(regs, insn) == 0; +} static struct insn_emulation insn_swp = { .name = "swp", .status = INSN_OBSOLETE, - .hooks = swp_hooks, + .try_emulate = try_emulate_swp, .set_hw_mode = NULL, }; +#endif /* CONFIG_SWP_EMULATION */ +#ifdef CONFIG_CP15_BARRIER_EMULATION static int cp15barrier_handler(struct pt_regs *regs, u32 instr) { perf_sw_event(PERF_COUNT_SW_EMULATION_FAULTS, 1, regs, regs->pc); @@ -312,31 +319,29 @@ static int cp15_barrier_set_hw_mode(bool enable) return 0; } -static struct undef_hook cp15_barrier_hooks[] = { - { - .instr_mask = 0x0fff0fdf, - .instr_val = 0x0e070f9a, - .pstate_mask = PSR_AA32_MODE_MASK, - .pstate_val = PSR_AA32_MODE_USR, - .fn = cp15barrier_handler, - }, - { - .instr_mask = 0x0fff0fff, - .instr_val = 0x0e070f95, - .pstate_mask = PSR_AA32_MODE_MASK, - .pstate_val = PSR_AA32_MODE_USR, - .fn = cp15barrier_handler, - }, - { } -}; +static bool try_emulate_cp15_barrier(struct pt_regs *regs, u32 insn) +{ + if (!compat_user_mode(regs) || compat_thumb_mode(regs)) + return false; + + if ((insn & 0x0fff0fdf) == 0x0e070f9a) + return cp15barrier_handler(regs, insn) == 0; + + if ((insn & 0x0fff0fff) == 0x0e070f95) + return cp15barrier_handler(regs, insn) == 0; + + return false; +} static struct insn_emulation insn_cp15_barrier = { .name = "cp15_barrier", .status = INSN_DEPRECATED, - .hooks = cp15_barrier_hooks, + .try_emulate = try_emulate_cp15_barrier, .set_hw_mode = cp15_barrier_set_hw_mode, }; +#endif /* CONFIG_CP15_BARRIER_EMULATION */ +#ifdef CONFIG_SETEND_EMULATION static int setend_set_hw_mode(bool enable) { if (!cpu_supports_mixed_endian_el0()) @@ -384,61 +389,41 @@ static int t16_setend_handler(struct pt_regs *regs, u32 instr) return rc; } -static struct undef_hook setend_hooks[] = { - { - .instr_mask = 0xfffffdff, - .instr_val = 0xf1010000, - .pstate_mask = PSR_AA32_MODE_MASK, - .pstate_val = PSR_AA32_MODE_USR, - .fn = a32_setend_handler, - }, - { - /* Thumb mode */ - .instr_mask = 0xfffffff7, - .instr_val = 0x0000b650, - .pstate_mask = (PSR_AA32_T_BIT | PSR_AA32_MODE_MASK), - .pstate_val = (PSR_AA32_T_BIT | PSR_AA32_MODE_USR), - .fn = t16_setend_handler, - }, - {} -}; +static bool try_emulate_setend(struct pt_regs *regs, u32 insn) +{ + if (compat_thumb_mode(regs) && + (insn & 0xfffffff7) == 0x0000b650) + return t16_setend_handler(regs, insn) == 0; + + if (compat_user_mode(regs) && + (insn & 0xfffffdff) == 0xf1010000) + return a32_setend_handler(regs, insn) == 0; + + return false; +} static struct insn_emulation insn_setend = { .name = "setend", .status = INSN_DEPRECATED, - .hooks = setend_hooks, + .try_emulate = try_emulate_setend, .set_hw_mode = setend_set_hw_mode, }; +#endif /* CONFIG_SETEND_EMULATION */ + +static struct insn_emulation *insn_emulations[] = { +#ifdef CONFIG_SWP_EMULATION + &insn_swp, +#endif +#ifdef CONFIG_CP15_BARRIER_EMULATION + &insn_cp15_barrier, +#endif +#ifdef CONFIG_SETEND_EMULATION + &insn_setend, +#endif +}; -static LIST_HEAD(insn_emulation); -static int nr_insn_emulated __initdata; -static DEFINE_RAW_SPINLOCK(insn_emulation_lock); static DEFINE_MUTEX(insn_emulation_mutex); -static void register_emulation_hooks(struct insn_emulation *insn) -{ - struct undef_hook *hook; - - BUG_ON(!insn->hooks); - - for (hook = insn->hooks; hook->instr_mask; hook++) - register_undef_hook(hook); - - pr_notice("Registered %s emulation handler\n", insn->name); -} - -static void remove_emulation_hooks(struct insn_emulation *insn) -{ - struct undef_hook *hook; - - BUG_ON(!insn->hooks); - - for (hook = insn->hooks; hook->instr_mask; hook++) - unregister_undef_hook(hook); - - pr_notice("Removed %s emulation handler\n", insn->name); -} - static void enable_insn_hw_mode(void *data) { struct insn_emulation *insn = (struct insn_emulation *)data; @@ -473,20 +458,27 @@ static int run_all_cpu_set_hw_mode(struct insn_emulation *insn, bool enable) */ static int run_all_insn_set_hw_mode(unsigned int cpu) { + int i; int rc = 0; unsigned long flags; - struct insn_emulation *insn; - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_for_each_entry(insn, &insn_emulation, node) { - bool enable = (insn->current_mode == INSN_HW); + /* + * Disable IRQs to serialize against an IPI from + * run_all_cpu_set_hw_mode(), ensuring the HW is programmed to the most + * recent enablement state if the two race with one another. + */ + local_irq_save(flags); + for (i = 0; i < ARRAY_SIZE(insn_emulations); i++) { + struct insn_emulation *insn = insn_emulations[i]; + bool enable = READ_ONCE(insn->current_mode) == INSN_HW; if (insn->set_hw_mode && insn->set_hw_mode(enable)) { pr_warn("CPU[%u] cannot support the emulation of %s", cpu, insn->name); rc = -EINVAL; } } - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); + local_irq_restore(flags); + return rc; } @@ -499,7 +491,6 @@ static int update_insn_emulation_mode(struct insn_emulation *insn, case INSN_UNDEF: /* Nothing to be done */ break; case INSN_EMULATE: - remove_emulation_hooks(insn); break; case INSN_HW: if (!run_all_cpu_set_hw_mode(insn, false)) @@ -511,7 +502,6 @@ static int update_insn_emulation_mode(struct insn_emulation *insn, case INSN_UNDEF: break; case INSN_EMULATE: - register_emulation_hooks(insn); break; case INSN_HW: ret = run_all_cpu_set_hw_mode(insn, true); @@ -523,34 +513,6 @@ static int update_insn_emulation_mode(struct insn_emulation *insn, return ret; } -static void __init register_insn_emulation(struct insn_emulation *insn) -{ - unsigned long flags; - - insn->min = INSN_UNDEF; - - switch (insn->status) { - case INSN_DEPRECATED: - insn->current_mode = INSN_EMULATE; - /* Disable the HW mode if it was turned on at early boot time */ - run_all_cpu_set_hw_mode(insn, false); - insn->max = INSN_HW; - break; - case INSN_OBSOLETE: - insn->current_mode = INSN_UNDEF; - insn->max = INSN_EMULATE; - break; - } - - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_add(&insn->node, &insn_emulation); - nr_insn_emulated++; - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); - - /* Register any handlers if required */ - update_insn_emulation_mode(insn, INSN_UNDEF); -} - static int emulation_proc_handler(struct ctl_table *table, int write, void *buffer, size_t *lenp, loff_t *ppos) @@ -568,7 +530,7 @@ static int emulation_proc_handler(struct ctl_table *table, int write, ret = update_insn_emulation_mode(insn, prev_mode); if (ret) { /* Mode change failed, revert to previous mode. */ - insn->current_mode = prev_mode; + WRITE_ONCE(insn->current_mode, prev_mode); update_insn_emulation_mode(insn, INSN_UNDEF); } ret: @@ -576,21 +538,34 @@ static int emulation_proc_handler(struct ctl_table *table, int write, return ret; } -static void __init register_insn_emulation_sysctl(void) +static void __init register_insn_emulation(struct insn_emulation *insn) { - unsigned long flags; - int i = 0; - struct insn_emulation *insn; - struct ctl_table *insns_sysctl, *sysctl; + struct ctl_table *sysctl; + + insn->min = INSN_UNDEF; + + switch (insn->status) { + case INSN_DEPRECATED: + insn->current_mode = INSN_EMULATE; + /* Disable the HW mode if it was turned on at early boot time */ + run_all_cpu_set_hw_mode(insn, false); + insn->max = INSN_HW; + break; + case INSN_OBSOLETE: + insn->current_mode = INSN_UNDEF; + insn->max = INSN_EMULATE; + break; + case INSN_UNAVAILABLE: + insn->current_mode = INSN_UNDEF; + insn->max = INSN_UNDEF; + break; + } - insns_sysctl = kcalloc(nr_insn_emulated + 1, sizeof(*sysctl), - GFP_KERNEL); - if (!insns_sysctl) - return; + /* Program the HW if required */ + update_insn_emulation_mode(insn, INSN_UNDEF); - raw_spin_lock_irqsave(&insn_emulation_lock, flags); - list_for_each_entry(insn, &insn_emulation, node) { - sysctl = &insns_sysctl[i]; + if (insn->status != INSN_UNAVAILABLE) { + sysctl = &insn->sysctl[0]; sysctl->mode = 0644; sysctl->maxlen = sizeof(int); @@ -600,11 +575,34 @@ static void __init register_insn_emulation_sysctl(void) sysctl->extra1 = &insn->min; sysctl->extra2 = &insn->max; sysctl->proc_handler = emulation_proc_handler; - i++; + + register_sysctl("abi", sysctl); } - raw_spin_unlock_irqrestore(&insn_emulation_lock, flags); +} + +bool try_emulate_armv8_deprecated(struct pt_regs *regs, u32 insn) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(insn_emulations); i++) { + struct insn_emulation *ie = insn_emulations[i]; - register_sysctl("abi", insns_sysctl); + if (ie->status == INSN_UNAVAILABLE) + continue; + + /* + * A trap may race with the mode being changed + * INSN_EMULATE<->INSN_HW. Try to emulate the instruction to + * avoid a spurious UNDEF. + */ + if (READ_ONCE(ie->current_mode) == INSN_UNDEF) + continue; + + if (ie->try_emulate(regs, insn)) + return true; + } + + return false; } /* @@ -613,24 +611,27 @@ static void __init register_insn_emulation_sysctl(void) */ static int __init armv8_deprecated_init(void) { - if (IS_ENABLED(CONFIG_SWP_EMULATION)) - register_insn_emulation(&insn_swp); + int i; - if (IS_ENABLED(CONFIG_CP15_BARRIER_EMULATION)) - register_insn_emulation(&insn_cp15_barrier); +#ifdef CONFIG_SETEND_EMULATION + if (!system_supports_mixed_endian_el0()) { + insn_setend.status = INSN_UNAVAILABLE; + pr_info("setend instruction emulation is not supported on this system\n"); + } - if (IS_ENABLED(CONFIG_SETEND_EMULATION)) { - if (system_supports_mixed_endian_el0()) - register_insn_emulation(&insn_setend); - else - pr_info("setend instruction emulation is not supported on this system\n"); +#endif + for (i = 0; i < ARRAY_SIZE(insn_emulations); i++) { + struct insn_emulation *ie = insn_emulations[i]; + + if (ie->status == INSN_UNAVAILABLE) + continue; + + register_insn_emulation(ie); } cpuhp_setup_state_nocalls(CPUHP_AP_ARM64_ISNDEP_STARTING, "arm64/isndep:starting", run_all_insn_set_hw_mode, NULL); - register_insn_emulation_sysctl(); - return 0; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 1f9285a35560..c71074cb2bef 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -373,27 +373,6 @@ void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) regs->pstate &= ~PSR_BTYPE_MASK; } -static LIST_HEAD(undef_hook); -static DEFINE_RAW_SPINLOCK(undef_lock); - -void register_undef_hook(struct undef_hook *hook) -{ - unsigned long flags; - - raw_spin_lock_irqsave(&undef_lock, flags); - list_add(&hook->node, &undef_hook); - raw_spin_unlock_irqrestore(&undef_lock, flags); -} - -void unregister_undef_hook(struct undef_hook *hook) -{ - unsigned long flags; - - raw_spin_lock_irqsave(&undef_lock, flags); - list_del(&hook->node); - raw_spin_unlock_irqrestore(&undef_lock, flags); -} - static int user_insn_read(struct pt_regs *regs, u32 *insnp) { u32 instr; @@ -425,23 +404,6 @@ static int user_insn_read(struct pt_regs *regs, u32 *insnp) return 0; } -static int call_undef_hook(struct pt_regs *regs, u32 instr) -{ - struct undef_hook *hook; - unsigned long flags; - int (*fn)(struct pt_regs *regs, u32 instr) = NULL; - - raw_spin_lock_irqsave(&undef_lock, flags); - list_for_each_entry(hook, &undef_hook, node) - if ((instr & hook->instr_mask) == hook->instr_val && - (regs->pstate & hook->pstate_mask) == hook->pstate_val) - fn = hook->fn; - - raw_spin_unlock_irqrestore(&undef_lock, flags); - - return fn ? fn(regs, instr) : 1; -} - void force_signal_inject(int signal, int code, unsigned long address, unsigned long err) { const char *desc; @@ -502,7 +464,7 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr) if (try_emulate_mrs(regs, insn)) return; - if (call_undef_hook(regs, insn) == 0) + if (try_emulate_armv8_deprecated(regs, insn)) return; out_err: From patchwork Wed Oct 11 10:06:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jinjie Ruan X-Patchwork-Id: 151246 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp425709vqb; Wed, 11 Oct 2023 03:10:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEa8d0IAg2lAzP7WWV1uRYzPvFpy3/GkdSrSYD8ZCwXZ/SyhhuS7TMnOaesISQCwXiNpbdW X-Received: by 2002:a05:6870:bacf:b0:1e9:8ab9:11ca with SMTP id js15-20020a056870bacf00b001e98ab911camr2058879oab.3.1697019004983; Wed, 11 Oct 2023 03:10:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697019004; cv=none; d=google.com; s=arc-20160816; b=meFvav4SS0tXwaDEffo/MJBh5l79BOAeGdpNcPIjkzxtKGNoPyx2TSLYpdrjc7/M1n 6qBMRnsPBy8b6HPumEuqhQE2IybMpT2MWy1aSAiymnDh9c2QectIdXWA3E+ajWSW80PV k8b5zZI0vUw1VBziNrzGdOxK9KbxryLo/ta/tLFWRh8kkTKyltMFvNiS8wjtuKGRXcwI 5s45rOnWta/w572r/KrvhULfn05XySr9esoNZQnMLmbiBjN0jf2YwzsEBLc0NuXt43cg OWklR/7z0cEpRO4Cu81Q5sRE6cSpDz9G4tAYcv4oZEI8J8GbZ6yp2ydxvhFyCuWM/UHU lpRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=b1ZGj5bvhlBNvB6FtugnEJAOspiyWfICIFsEuq7YmMk=; fh=zwYgZI7fViiWMszSXUp9QDpMCZZ/fkBX1HIXF8Q+kVM=; b=bfn4huid6NX/mO5dp6thjCdJHi7N4mGPLLWG/UdfsUhrRrRnnRmijjDIJIPuRj9drk Z3tfwYQpCGqGXA+bDUsbCKxnHpB2R71pXX/pfV5H1NPPtT+kMGDKKqifFMJYpVEx4OCU bNechMtJyMUQ2PEFjkawl/b4odS1CK8rhQttYnSCJAqeKvab86Dt30j25bWlkANojRLL 3sEd/mnLVrhiusidEtdEXm0e069y/xbdNuSQnE5J8UIjgC6MEAUyc47pqG8i/A9oo4VL zTcX4/CAA5QR+ntrUDrC0kjQH9S4kbGWsEJYrDIQbeUWs0ntVnYTAInudmt3u5aiI7rE ydrw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id bv193-20020a632eca000000b00578d3f8d4d4si13805920pgb.448.2023.10.11.03.10.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 03:10:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id A790B822740D; Wed, 11 Oct 2023 03:10:02 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234757AbjJKKJj (ORCPT + 18 others); Wed, 11 Oct 2023 06:09:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234817AbjJKKIt (ORCPT ); Wed, 11 Oct 2023 06:08:49 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB5871BFB for ; Wed, 11 Oct 2023 03:07:54 -0700 (PDT) Received: from kwepemi500008.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4S57dk41hpzkYBH; Wed, 11 Oct 2023 18:03:54 +0800 (CST) Received: from huawei.com (10.67.174.55) by kwepemi500008.china.huawei.com (7.221.188.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 11 Oct 2023 18:07:51 +0800 From: Jinjie Ruan To: , , , , , , , , , , , , , , , , , CC: Subject: [PATCH v5.15 15/15] arm64: armv8_deprecated: fix unused-function error Date: Wed, 11 Oct 2023 10:06:55 +0000 Message-ID: <20231011100655.979626-16-ruanjinjie@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231011100655.979626-1-ruanjinjie@huawei.com> References: <20231011100655.979626-1-ruanjinjie@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.55] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemi500008.china.huawei.com (7.221.188.139) X-CFilter-Loop: Reflected X-Spam-Status: No, score=2.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Wed, 11 Oct 2023 03:10:02 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779453400247322652 X-GMAIL-MSGID: 1779453400247322652 From: Ren Zhijie commit 223d3a0d30b6e9f979f5642e430e1753d3e29f89 upstream. If CONFIG_SWP_EMULATION is not set and CONFIG_CP15_BARRIER_EMULATION is not set, aarch64-linux-gnu complained about unused-function : arch/arm64/kernel/armv8_deprecated.c:67:21: error: ‘aarch32_check_condition’ defined but not used [-Werror=unused-function] static unsigned int aarch32_check_condition(u32 opcode, u32 psr) ^~~~~~~~~~~~~~~~~~~~~~~ cc1: all warnings being treated as errors To fix this warning, modify aarch32_check_condition() with __maybe_unused. Fixes: 0c5f416219da ("arm64: armv8_deprecated: move aarch32 helper earlier") Signed-off-by: Ren Zhijie Acked-by: Mark Rutland Link: https://lore.kernel.org/r/20221124022429.19024-1-renzhijie2@huawei.com Signed-off-by: Will Deacon Signed-off-by: Jinjie Ruan --- arch/arm64/kernel/armv8_deprecated.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index 758008020b59..91eabe56093d 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -64,7 +64,7 @@ struct insn_emulation { #define ARM_OPCODE_CONDITION_UNCOND 0xf -static unsigned int aarch32_check_condition(u32 opcode, u32 psr) +static unsigned int __maybe_unused aarch32_check_condition(u32 opcode, u32 psr) { u32 cc_bits = opcode >> 28;