From patchwork Wed Jun 14 04:52:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Filippov X-Patchwork-Id: 107711 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1029639vqr; Tue, 13 Jun 2023 23:14:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4hn6Z9IIQucjwZlAZmgp/9FF06OMAa7N55xbEtY1YkkVwiJqTNfJALBAy7a667TR60XWA5 X-Received: by 2002:a17:902:830c:b0:1b5:79b:8b14 with SMTP id bd12-20020a170902830c00b001b5079b8b14mr88584plb.26.1686723277139; Tue, 13 Jun 2023 23:14:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686723277; cv=none; d=google.com; s=arc-20160816; b=NqXBbSPUe5TWoPB+QcHMYs04WnB5uIiF426ROM0cri55u4JE95HXYd6Vih54MXiU9t CdldGWRev2gp4WsCHYOR3ODpS3JpDB4Ry5ixwNt4k39GOCzbFtBL2IS8UchzdTt2JBwN /+t/3bRO7hH17EtlhPpX0ui8/Gipy4TwD5EXnurpcHzMwFkiZ97a3Qj2WAp9yL3ydx0B 6V9ag81lHF4sih2pq91f0l7MJKrnVToKoyYv1n+vJjmrYCNIqriUCwrgDFMSbXTd6Jhb plKNrCct07ZT45MDjOu3rcw1KIvd4Q2FToIDPry6/fiEdITinMohWvoXm1D4Vu82p2MP Xw7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wqw+pfXvRxFyKtNsvJYgxpMTw9Cw0YTqu9wRT0MEL7w=; b=wu0y+zQV4RILhIsvgDQSdlXcz/PqsQ4qxjcudbgl+hJW/iro2fI/iLVt1bf2qiR2qi UYiUUlGLjNCYU+txvAn1tnXNsGd94pOTGS3CHlS/yeV+jX3xk/xQKJ9bMVSCQkNTI2WZ GaPpDpeUiUNKbf/LbI1hH4E0qx6w1E80cpq/R2YHM9ZrEQDtDuFOJUNVNgkf0r6TLaq6 30BOXlbK/zr/vWYiZL1Pk8nVASVMmGSoRK5hHaffofltAKF4uHSMurj+yfXHO/CHSf3Q 43OSkpdr8aSwN7L8jUb34mYFv04zpOJ6i7IL7zkSufXCvQqLyZxmER9p/TVXOjTv6IvN UR2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=OimPmjXC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p15-20020a170902e74f00b001b3d1203160si4907987plf.650.2023.06.13.23.14.21; Tue, 13 Jun 2023 23:14:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=OimPmjXC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242531AbjFNEw3 (ORCPT + 99 others); Wed, 14 Jun 2023 00:52:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233244AbjFNEwY (ORCPT ); Wed, 14 Jun 2023 00:52:24 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF21C184 for ; Tue, 13 Jun 2023 21:52:23 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1b3c0c47675so28067395ad.1 for ; Tue, 13 Jun 2023 21:52:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686718343; x=1689310343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wqw+pfXvRxFyKtNsvJYgxpMTw9Cw0YTqu9wRT0MEL7w=; b=OimPmjXCG8okhtcLCLa7VYDFJqQQBCmFVW5QxbTi/cal0RAl1RvumolnmY/ZWjUCoV oAMg/p+5sJnSfGLa1Xgim8Gjoj6vSVlhLWg6RI+cpiA//YXzuXnOnMsIoCQmW9CiYUFp f9G02GCCyRDuSQ3CiBOUAfoeGzBbmOsTYD4KmHXoOej4GW5tb3SbIBXbqncciwlS2+lz q6ej+NWHVzvPRmC6PhlhVWpjst8vogmOy6yvHFUh6xELCDDufLxDg1YLhsYI6YQ+xCnx pFuK5gzixEZjso6CajRcYB+wVKQUCqd8aA3ecRaRhDJ4Huuklx9j7Bv1V/tmNfrDHg8o B7OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686718343; x=1689310343; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wqw+pfXvRxFyKtNsvJYgxpMTw9Cw0YTqu9wRT0MEL7w=; b=GZdUsxgXwBpnYT8zKdC81WTNcAPV/XirzjdS4TK9UYDI/va4OsLGSQ/zsdDpG+TfTJ znkjJ5i4v2U5aOAq1YIS43be4qZbKyjhlrb8/57AKBPSfwqkqXMwb0NGn2zN9PXcg9IQ Xq9NgVUfc6L399Y6cXkobOMGhYMfijDW81mLVkLry/wT/LZEwPV5k0ZtcCpAvYdIq3C7 Bd7c8A830uRCGzOnKKwzBTKxhT/tXRLCxJgAZrPSIdN/3u4Sf1aRvOCQDTNVG7QHXc1A ZBeOLXW53ISob8/YdFf7ZYYZRKuodehLlZJpX8K7usQO/bdk/HGsrlGJlmirToCwsFH3 9JgQ== X-Gm-Message-State: AC+VfDx9T7UFFJICE3RvNZLi3KKqoDpU5WGzYexTSll7lEiauekZUQRG sypD3x46SA7J4QtesU4oPT2dGqjl/EQ= X-Received: by 2002:a17:902:e842:b0:1b2:4fea:c62 with SMTP id t2-20020a170902e84200b001b24fea0c62mr12789996plg.30.1686718342949; Tue, 13 Jun 2023 21:52:22 -0700 (PDT) Received: from octofox.hsd1.ca.comcast.net ([2601:641:401:1d20:bf5f:2e99:78f2:6007]) by smtp.gmail.com with ESMTPSA id z18-20020a170903019200b001acad86ebc5sm11089790plg.33.2023.06.13.21.52.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 21:52:22 -0700 (PDT) From: Max Filippov To: linux-kernel@vger.kernel.org Cc: Chris Zankel , Max Filippov Subject: [PATCH 1/4] xtensa: move early_trap_init from kasan_early_init to init_arch Date: Tue, 13 Jun 2023 21:52:09 -0700 Message-Id: <20230614045212.2534746-2-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230614045212.2534746-1-jcmvbkbc@gmail.com> References: <20230614045212.2534746-1-jcmvbkbc@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, FROM_LOCAL_NOVOWEL,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768657546805186483?= X-GMAIL-MSGID: =?utf-8?q?1768657546805186483?= There may be other users for the early traps besides KASAN. Move call to the early_trap_init from kasan_early_init. Protect init_exc_table initializer with ifdef to make sure it builds on noMMU configurations. Signed-off-by: Max Filippov --- arch/xtensa/include/asm/traps.h | 2 ++ arch/xtensa/kernel/setup.c | 6 ++++++ arch/xtensa/mm/kasan_init.c | 2 -- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/xtensa/include/asm/traps.h b/arch/xtensa/include/asm/traps.h index 6f74ccc0c7ea..acffb02f8760 100644 --- a/arch/xtensa/include/asm/traps.h +++ b/arch/xtensa/include/asm/traps.h @@ -64,8 +64,10 @@ void do_unhandled(struct pt_regs *regs); static inline void __init early_trap_init(void) { static struct exc_table init_exc_table __initdata = { +#ifdef CONFIG_MMU .fast_kernel_handler[EXCCAUSE_DTLB_MISS] = fast_second_level_miss, +#endif }; xtensa_set_sr(&init_exc_table, excsave1); } diff --git a/arch/xtensa/kernel/setup.c b/arch/xtensa/kernel/setup.c index 94aafa19771b..5c0c0fcac144 100644 --- a/arch/xtensa/kernel/setup.c +++ b/arch/xtensa/kernel/setup.c @@ -47,6 +47,7 @@ #include #include #include +#include #if defined(CONFIG_VGA_CONSOLE) || defined(CONFIG_DUMMY_CONSOLE) struct screen_info screen_info = { @@ -242,6 +243,11 @@ void __init early_init_devtree(void *params) void __init init_arch(bp_tag_t *bp_start) { + /* Initialize basic exception handling if configuration may need it */ + + if (IS_ENABLED(CONFIG_KASAN)) + early_trap_init(); + /* Initialize MMU. */ init_mmu(); diff --git a/arch/xtensa/mm/kasan_init.c b/arch/xtensa/mm/kasan_init.c index 1fef24db2ff6..f00d122aa806 100644 --- a/arch/xtensa/mm/kasan_init.c +++ b/arch/xtensa/mm/kasan_init.c @@ -14,7 +14,6 @@ #include #include #include -#include void __init kasan_early_init(void) { @@ -31,7 +30,6 @@ void __init kasan_early_init(void) BUG_ON(!pmd_none(*pmd)); set_pmd(pmd, __pmd((unsigned long)kasan_early_shadow_pte)); } - early_trap_init(); } static void __init populate(void *start, void *end) From patchwork Wed Jun 14 04:52:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Filippov X-Patchwork-Id: 107700 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1005320vqr; Tue, 13 Jun 2023 22:07:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6mw//iRpKiuXTz2sFID9X5O2ZZCnZCoBSXxTIe/uGLRIncSAHdQ2qAenNL9sFa7zO++qun X-Received: by 2002:a05:620a:2483:b0:761:f5d2:16ec with SMTP id i3-20020a05620a248300b00761f5d216ecmr5661379qkn.66.1686719278997; Tue, 13 Jun 2023 22:07:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686719278; cv=none; d=google.com; s=arc-20160816; b=DkMNnoqsr6+KwgX3WUm2CrRWbk+bNgwlFRDJzIV4NqMXgogFhw/75EPMnFtnNCwTsw eUiHaCQuSgVg7ieHJ+gATQ5dkkrqaaGDKwsFhtzBwBswcaALZet7tkp5R/anNvB4FeGL yVFCBx0lRquIQwQXvMjlvttd/VI6On+bsVpkaHvsOnTj8dFlMFpJsf34W2w53xUr7qQi VQwqCtZXXbKbKea0GQp/pYQBWfStCTbTzvZJHCklhSygjkipWvmIG0asJAqmafCJtktm 5kNbE3QBnHoB4+gBKh9f75ZrKqRD49LyRbFuUGZVOvzNcVHMGCBr2xz/6jMHMRcdUEd0 ND2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FulhVEhNpgfCsNB5OSDOju9ICmBQY+Gxi28mpx6VJCw=; b=CUiJ2RSKSmhz7vXIOCruhvuQ3FL+e4ne9EvxAcFNR2ssgxTZ5WMlCrrk8QIVmqrEFH PxHGLfyOVTBn/69HMmbjsed56bv1/WPEEYF8QYTCpacilvY/EX4DnVAyWf4gy4N+ptdp WzenUrA4VTHbtpKcAFprJIFHkLcWVkfJ24Jp78yQEDPtXongu0E8bylrdO9lwq1egUUH ZwrCYXl+KnLhpkp73DpnY44Ytg6hvwMCv85SKXsL422v2OEQGASaRisxMn7gJOKhe0Jv 1BJZVCR3p4LD1JKDac4gwxBa3LnPu3QjZUMjT4LN3qEj24e8Ende7OX1ecU8vAzFqcdj ok8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=OOuNLK6Y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z17-20020aa79f91000000b0064e1b65f01dsi9942829pfr.237.2023.06.13.22.07.43; Tue, 13 Jun 2023 22:07:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=OOuNLK6Y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233244AbjFNEwg (ORCPT + 99 others); Wed, 14 Jun 2023 00:52:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231720AbjFNEw0 (ORCPT ); Wed, 14 Jun 2023 00:52:26 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80A3E183 for ; Tue, 13 Jun 2023 21:52:25 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1b3d8aa2c17so15801295ad.3 for ; Tue, 13 Jun 2023 21:52:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686718344; x=1689310344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FulhVEhNpgfCsNB5OSDOju9ICmBQY+Gxi28mpx6VJCw=; b=OOuNLK6YyThTUcwakap3WAuddmLrp02i1QJxCPts/TtJZm+X6kxsHz2AfPBhSuCPwy vd+G35pGrtwiFx1eBmfziFzW+UsVhi3pX8JeDHjfX0S7jX/WGcUcxKtKgzOCohWw0bOj xq70p5LPAknA2/z/ia8VQ+d+3DyShd/6PgxRZIV2scoQk8zmdD2wRHhHUqzmtYM4i97Q MHTLTczfFsD0SwsxxSnXDQFtKYagpW0hK0gsrWlc8ZYsqa/l5Z+AmWUdaaih64ytmodV Vq+WCuROxSClr5AcnrRVOlxVh4SOQ+OF3Um7z7SzPmVEwG5rme/REUNbvJ30AvPKbaFQ bonA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686718344; x=1689310344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FulhVEhNpgfCsNB5OSDOju9ICmBQY+Gxi28mpx6VJCw=; b=EOWY/GyDX4RP6QQi7Mgk7t/M/4l+NlfUY86lB9WZtx+tq0H2ggccOKmO0IKtCfO4rt e71LNBaI1UTQqwUbq4yG1829Di7MSHOQcRiNEDXUOkmFPUH9ky5WBGT26/QpFU01i7Yw LGHxT5jw2sSp9pSd2FHPg5qRMfDeavnytyAm8nalw9eNSzVb+drLpyaKbLWSalg7p/3t 8keeZrkLELlQAIuu+8yCmokIgkrVNFDVy4UNkssaERGOtjnVnKneGG9nRFT/x5/7ApXB EDSZaTMQvX4t7v+sKh/wzW166RKeQS/yHeVp+zBVRVFAb0c0dScT6bG83XgEVLjx8qxl /0hw== X-Gm-Message-State: AC+VfDxVIMTOQdwxMho/r0cMKGO0mZGwooTXzqyXWFmcIN5e8Fzc3+B2 /MmYkVXaTLuVYAbL/6kigSTdtbwtaAM= X-Received: by 2002:a17:903:187:b0:1b3:f8db:6f0e with SMTP id z7-20020a170903018700b001b3f8db6f0emr2192157plg.43.1686718343714; Tue, 13 Jun 2023 21:52:23 -0700 (PDT) Received: from octofox.hsd1.ca.comcast.net ([2601:641:401:1d20:bf5f:2e99:78f2:6007]) by smtp.gmail.com with ESMTPSA id z18-20020a170903019200b001acad86ebc5sm11089790plg.33.2023.06.13.21.52.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 21:52:23 -0700 (PDT) From: Max Filippov To: linux-kernel@vger.kernel.org Cc: Chris Zankel , Max Filippov Subject: [PATCH 2/4] xtensa: always install slow handler for unaligned access exception Date: Tue, 13 Jun 2023 21:52:10 -0700 Message-Id: <20230614045212.2534746-3-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230614045212.2534746-1-jcmvbkbc@gmail.com> References: <20230614045212.2534746-1-jcmvbkbc@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, FROM_LOCAL_NOVOWEL,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768653354344386346?= X-GMAIL-MSGID: =?utf-8?q?1768653354344386346?= Currently slow handler for unaligned access exception is not installed when CPU has hardware support for unaligned access. However some opcodes (e.g. l32ai, s32ri, s32c1i) would still raise unaligned access exception even on such CPUs. In that case instead of SIGBUS and a diagnostic entry in the kernel log the faulting process would receive SIGILL. Always install slow handler for unaligned access exception to fix that. Signed-off-by: Max Filippov --- arch/xtensa/kernel/traps.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/xtensa/kernel/traps.c b/arch/xtensa/kernel/traps.c index a92c8593d4f1..f447262468c5 100644 --- a/arch/xtensa/kernel/traps.c +++ b/arch/xtensa/kernel/traps.c @@ -54,9 +54,7 @@ static void do_interrupt(struct pt_regs *regs); #if XTENSA_FAKE_NMI static void do_nmi(struct pt_regs *regs); #endif -#if XCHAL_UNALIGNED_LOAD_EXCEPTION || XCHAL_UNALIGNED_STORE_EXCEPTION static void do_unaligned_user(struct pt_regs *regs); -#endif static void do_multihit(struct pt_regs *regs); #if XTENSA_HAVE_COPROCESSORS static void do_coprocessor(struct pt_regs *regs); @@ -102,9 +100,9 @@ static dispatch_init_table_t __initdata dispatch_init_table[] = { #ifdef CONFIG_XTENSA_UNALIGNED_USER { EXCCAUSE_UNALIGNED, USER, fast_unaligned }, #endif -{ EXCCAUSE_UNALIGNED, 0, do_unaligned_user }, { EXCCAUSE_UNALIGNED, KRNL, fast_unaligned }, #endif +{ EXCCAUSE_UNALIGNED, 0, do_unaligned_user }, #ifdef CONFIG_MMU { EXCCAUSE_ITLB_MISS, 0, do_page_fault }, { EXCCAUSE_ITLB_MISS, USER|KRNL, fast_second_level_miss}, @@ -363,7 +361,6 @@ static void do_div0(struct pt_regs *regs) * accesses causes from user space. */ -#if XCHAL_UNALIGNED_LOAD_EXCEPTION || XCHAL_UNALIGNED_STORE_EXCEPTION static void do_unaligned_user(struct pt_regs *regs) { __die_if_kernel("Unhandled unaligned exception in kernel", @@ -375,7 +372,6 @@ static void do_unaligned_user(struct pt_regs *regs) task_pid_nr(current), regs->pc); force_sig_fault(SIGBUS, BUS_ADRALN, (void *) regs->excvaddr); } -#endif #if XTENSA_HAVE_COPROCESSORS static void do_coprocessor(struct pt_regs *regs) From patchwork Wed Jun 14 04:52:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Filippov X-Patchwork-Id: 107707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1022370vqr; Tue, 13 Jun 2023 22:58:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ktktfpNLiS6ZCXcWk21beIkYETCWzjC9fytoYPgKPDBl0GsADrNggJQxq1DxLO8nWtnBv X-Received: by 2002:a17:907:2682:b0:978:6634:d05c with SMTP id bn2-20020a170907268200b009786634d05cmr16679300ejc.21.1686722329162; Tue, 13 Jun 2023 22:58:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686722329; cv=none; d=google.com; s=arc-20160816; b=T2C2lxBbJCv49k0jTnKd/IZ3tkGyy6tE27BH3I6s3mQDUvptq8HflWtvmdcwJp6dTZ Hbzr3LMRT+3bpoOFCnrpE/XNkzfJSjmAHWOrVgCKWT2GlsrAnCx3y7kIJ2R4ofje8RCw WlEPGAdvKWnK9DVLUkFxS5OpEbhxzO7HxVcS5+r7xGkgwfSfJQYg77S/unW3eztHItnG 4y38ojHBZhxDtg0CxAZDJ7FtfU4EriqvsURQeZi70XpJRpmPvX3spDwryf1JyObEU/yw Xauo/riWPqeOP2F6TtHqTT7uCdI9OmtlZU2ssvrZHo9NUslma5f1h+/2Hl/6j6MYVX6c e1lQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pnBq6jRAi07bdIevMbcQiNQG6VP5ua0fb3T92QcFjPs=; b=ja46Q+azkxkltjUVJRbNOaJ45CSZz6xj52+jQ2cNxWg9duChxL0t0ac7bV8+0/FdA8 BSug5h+XmiZPYxSeyIr8FGUcxA470l+pdrpcp3OAICIGzHEOeTv7WSx6lE8VwmhDXXlH Tx6HymTgUSjUQeL+L2ZvV+Hh0rjZCZC2ArCGsIRaKi4S69Mp56Ku5gWzbamP2oIrfp/s IFGUQzAytDm9Bnqp3UQoQZjlI7sxKX3QluS9qs6cRNpnIMQtqs2FmuAUS0pKDX/8WeVX NaxoMvBM8JviGjV1xom28NruPOUWpoc2EODyAVusaWYg3G2uqsLIl/NPgFuL5fpBKQ9M VeBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=q4Qd2Gri; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ba8-20020a0564021ac800b00514911f9e5fsi8245722edb.242.2023.06.13.22.58.25; Tue, 13 Jun 2023 22:58:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=q4Qd2Gri; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231720AbjFNEwm (ORCPT + 99 others); Wed, 14 Jun 2023 00:52:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233971AbjFNEw0 (ORCPT ); Wed, 14 Jun 2023 00:52:26 -0400 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 796E311B for ; Tue, 13 Jun 2023 21:52:25 -0700 (PDT) Received: by mail-ot1-x32c.google.com with SMTP id 46e09a7af769-6b1576c2f6eso4003419a34.3 for ; Tue, 13 Jun 2023 21:52:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686718344; x=1689310344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pnBq6jRAi07bdIevMbcQiNQG6VP5ua0fb3T92QcFjPs=; b=q4Qd2GriVxvaFyVITi6RJ6+CVtI4pvoQ+jKl6u4SGlIskewJ9JwJGVynq4S10NA8Em lJgfr1wErvaPDDB/JUzRIBCMLLtEYLq2LgksDSz/6cFxC2sqT/5WvlcoBXn+VoIDmkUy a46SOnH3Kf9TyfnaSHLiD2l2eMIbdZjABJgpLgKCDfBI/oaJf629gRdZdOVNEkAHr6f0 yQNolxLyzUb9IISIPk5P78vglcFDxTmGG1gCmA7I3hskveftmQbQMYQ7DIPtrT/xnVqL NrVx1mcNcvUKkHLPoW4fmQDxne0kKPYPlGUC0dbvjcSKQpDheb23T56H3iSCC5ez6fB5 gYKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686718344; x=1689310344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pnBq6jRAi07bdIevMbcQiNQG6VP5ua0fb3T92QcFjPs=; b=PknN9UrcfKH8OzXT/FYLOHZjGAOhMsD1OYYep882V5bWjUUG6WXi3vWqPlcbU/WACL earnYqvUCFtD/MZtz20oEHffn87QR3j1fDBLQ83ll8OklOQTxuqoKriLTrSQyczJQBUr c50R+ByAsSRI1hFu5bKe+Iph/Qicf0qwTIbEqM5vmzA+8iypcJWDa27PVPjhMEQmTPik a5MiZxvI4LgCiEhxI16UwFkUUZC5rehlc69RqVPhZH7/UJ0KJPlC7rAHGCQ6iG/iYIH/ SzFrhPuXykSZvIkYVqV57fqnju1yLyXpk4w7cjIiRtzL2Qd+vMuVhJhX49YX+O1eltl+ Ttpg== X-Gm-Message-State: AC+VfDwMj4US1co++ZLUmJe1slsAFGWuPGLfOnCh5WM4lm0iQp6heLjO NLBnNtt/BR+xbrXIZliE8c6S0EXHtws= X-Received: by 2002:a05:6358:cb13:b0:129:c374:5f82 with SMTP id gr19-20020a056358cb1300b00129c3745f82mr7663211rwb.21.1686718344324; Tue, 13 Jun 2023 21:52:24 -0700 (PDT) Received: from octofox.hsd1.ca.comcast.net ([2601:641:401:1d20:bf5f:2e99:78f2:6007]) by smtp.gmail.com with ESMTPSA id z18-20020a170903019200b001acad86ebc5sm11089790plg.33.2023.06.13.21.52.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 21:52:24 -0700 (PDT) From: Max Filippov To: linux-kernel@vger.kernel.org Cc: Chris Zankel , Max Filippov Subject: [PATCH 3/4] xtensa: rearrange unaligned exception handler Date: Tue, 13 Jun 2023 21:52:11 -0700 Message-Id: <20230614045212.2534746-4-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230614045212.2534746-1-jcmvbkbc@gmail.com> References: <20230614045212.2534746-1-jcmvbkbc@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, FROM_LOCAL_NOVOWEL,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768656553351721513?= X-GMAIL-MSGID: =?utf-8?q?1768656553351721513?= - extract initialization part of the exception handler into a separate function. - use single label for invalid instruction instead of two labels, one for load and one for store, at one place. - use sext instruction for sign extension when available. - store SAR on the stack instead of in a0. - replace numeric labels for load and store writeback with .Lload_w and .Lstore_w respectively. Signed-off-by: Max Filippov --- arch/xtensa/kernel/align.S | 171 +++++++++++++++++++------------------ 1 file changed, 89 insertions(+), 82 deletions(-) diff --git a/arch/xtensa/kernel/align.S b/arch/xtensa/kernel/align.S index d062c732ef18..bcbd7962a684 100644 --- a/arch/xtensa/kernel/align.S +++ b/arch/xtensa/kernel/align.S @@ -157,58 +157,7 @@ .literal_position ENTRY(fast_unaligned) - /* Note: We don't expect the address to be aligned on a word - * boundary. After all, the processor generated that exception - * and it would be a hardware fault. - */ - - /* Save some working register */ - - s32i a4, a2, PT_AREG4 - s32i a5, a2, PT_AREG5 - s32i a6, a2, PT_AREG6 - s32i a7, a2, PT_AREG7 - s32i a8, a2, PT_AREG8 - - rsr a0, depc - s32i a0, a2, PT_AREG2 - s32i a3, a2, PT_AREG3 - - rsr a3, excsave1 - movi a4, fast_unaligned_fixup - s32i a4, a3, EXC_TABLE_FIXUP - - /* Keep value of SAR in a0 */ - - rsr a0, sar - rsr a8, excvaddr # load unaligned memory address - - /* Now, identify one of the following load/store instructions. - * - * The only possible danger of a double exception on the - * following l32i instructions is kernel code in vmalloc - * memory. The processor was just executing at the EPC_1 - * address, and indeed, already fetched the instruction. That - * guarantees a TLB mapping, which hasn't been replaced by - * this unaligned exception handler that uses only static TLB - * mappings. However, high-level interrupt handlers might - * modify TLB entries, so for the generic case, we register a - * TABLE_FIXUP handler here, too. - */ - - /* a3...a6 saved on stack, a2 = SP */ - - /* Extract the instruction that caused the unaligned access. */ - - rsr a7, epc1 # load exception address - movi a3, ~3 - and a3, a3, a7 # mask lower bits - - l32i a4, a3, 0 # load 2 words - l32i a5, a3, 4 - - __ssa8 a7 - __src_b a4, a4, a5 # a4 has the instruction + call0 .Lsave_and_load_instruction /* Analyze the instruction (load or store?). */ @@ -249,7 +198,7 @@ ENTRY(fast_unaligned) addi a7, a7, 2 # increment PC (assume 16-bit insn) extui a5, a4, INSN_OP0, 4 - _beqi a5, OP0_L32I_N, 1f # l32i.n: jump + _beqi a5, OP0_L32I_N, .Lload_w# l32i.n: jump addi a7, a7, 1 #else @@ -257,21 +206,24 @@ ENTRY(fast_unaligned) #endif extui a5, a4, INSN_OP1, 4 - _beqi a5, OP1_L32I, 1f # l32i: jump + _beqi a5, OP1_L32I, .Lload_w # l32i: jump extui a3, a3, 0, 16 # extract lower 16 bits - _beqi a5, OP1_L16UI, 1f + _beqi a5, OP1_L16UI, .Lload_w addi a5, a5, -OP1_L16SI - _bnez a5, .Linvalid_instruction_load + _bnez a5, .Linvalid_instruction /* sign extend value */ - +#if XCHAL_HAVE_SEXT + sext a3, a3, 15 +#else slli a3, a3, 16 srai a3, a3, 16 +#endif /* Set target register. */ -1: +.Lload_w: extui a4, a4, INSN_T, 4 # extract target register movi a5, .Lload_table addx8 a4, a4, a5 @@ -297,28 +249,27 @@ ENTRY(fast_unaligned) mov a15, a3 ; _j .Lexit; .align 8 .Lstore_table: - l32i a3, a2, PT_AREG0; _j 1f; .align 8 - mov a3, a1; _j 1f; .align 8 # fishy?? - l32i a3, a2, PT_AREG2; _j 1f; .align 8 - l32i a3, a2, PT_AREG3; _j 1f; .align 8 - l32i a3, a2, PT_AREG4; _j 1f; .align 8 - l32i a3, a2, PT_AREG5; _j 1f; .align 8 - l32i a3, a2, PT_AREG6; _j 1f; .align 8 - l32i a3, a2, PT_AREG7; _j 1f; .align 8 - l32i a3, a2, PT_AREG8; _j 1f; .align 8 - mov a3, a9 ; _j 1f; .align 8 - mov a3, a10 ; _j 1f; .align 8 - mov a3, a11 ; _j 1f; .align 8 - mov a3, a12 ; _j 1f; .align 8 - mov a3, a13 ; _j 1f; .align 8 - mov a3, a14 ; _j 1f; .align 8 - mov a3, a15 ; _j 1f; .align 8 + l32i a3, a2, PT_AREG0; _j .Lstore_w; .align 8 + mov a3, a1; _j .Lstore_w; .align 8 # fishy?? + l32i a3, a2, PT_AREG2; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG3; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG4; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG5; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG6; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG7; _j .Lstore_w; .align 8 + l32i a3, a2, PT_AREG8; _j .Lstore_w; .align 8 + mov a3, a9 ; _j .Lstore_w; .align 8 + mov a3, a10 ; _j .Lstore_w; .align 8 + mov a3, a11 ; _j .Lstore_w; .align 8 + mov a3, a12 ; _j .Lstore_w; .align 8 + mov a3, a13 ; _j .Lstore_w; .align 8 + mov a3, a14 ; _j .Lstore_w; .align 8 + mov a3, a15 ; _j .Lstore_w; .align 8 /* We cannot handle this exception. */ .extern _kernel_exception -.Linvalid_instruction_load: -.Linvalid_instruction_store: +.Linvalid_instruction: movi a4, 0 rsr a3, excsave1 @@ -326,6 +277,7 @@ ENTRY(fast_unaligned) /* Restore a4...a8 and SAR, set SP, and jump to default exception. */ + l32i a0, a2, PT_SAR l32i a8, a2, PT_AREG8 l32i a7, a2, PT_AREG7 l32i a6, a2, PT_AREG6 @@ -343,8 +295,8 @@ ENTRY(fast_unaligned) 2: movi a0, _user_exception jx a0 -1: # a7: instruction pointer, a4: instruction, a3: value - + # a7: instruction pointer, a4: instruction, a3: value +.Lstore_w: movi a6, 0 # mask: ffffffff:00000000 #if XCHAL_HAVE_DENSITY @@ -361,7 +313,7 @@ ENTRY(fast_unaligned) extui a5, a4, INSN_OP1, 4 # extract OP1 _beqi a5, OP1_S32I, 1f # jump if 32 bit store - _bnei a5, OP1_S16I, .Linvalid_instruction_store + _bnei a5, OP1_S16I, .Linvalid_instruction movi a5, -1 __extl a3, a3 # get 16-bit value @@ -434,6 +386,7 @@ ENTRY(fast_unaligned) /* Restore working register */ + l32i a0, a2, PT_SAR l32i a8, a2, PT_AREG8 l32i a7, a2, PT_AREG7 l32i a6, a2, PT_AREG6 @@ -448,6 +401,59 @@ ENTRY(fast_unaligned) l32i a2, a2, PT_AREG2 rfe + .align 4 +.Lsave_and_load_instruction: + + /* Save some working register */ + + s32i a3, a2, PT_AREG3 + s32i a4, a2, PT_AREG4 + s32i a5, a2, PT_AREG5 + s32i a6, a2, PT_AREG6 + s32i a7, a2, PT_AREG7 + s32i a8, a2, PT_AREG8 + + rsr a4, depc + s32i a4, a2, PT_AREG2 + + rsr a5, sar + s32i a5, a2, PT_SAR + + rsr a3, excsave1 + movi a4, fast_unaligned_fixup + s32i a4, a3, EXC_TABLE_FIXUP + + rsr a8, excvaddr # load unaligned memory address + + /* Now, identify one of the following load/store instructions. + * + * The only possible danger of a double exception on the + * following l32i instructions is kernel code in vmalloc + * memory. The processor was just executing at the EPC_1 + * address, and indeed, already fetched the instruction. That + * guarantees a TLB mapping, which hasn't been replaced by + * this unaligned exception handler that uses only static TLB + * mappings. However, high-level interrupt handlers might + * modify TLB entries, so for the generic case, we register a + * TABLE_FIXUP handler here, too. + */ + + /* a3...a6 saved on stack, a2 = SP */ + + /* Extract the instruction that caused the unaligned access. */ + + rsr a7, epc1 # load exception address + movi a3, ~3 + and a3, a3, a7 # mask lower bits + + l32i a4, a3, 0 # load 2 words + l32i a5, a3, 4 + + __ssa8 a7 + __src_b a4, a4, a5 # a4 has the instruction + + ret + ENDPROC(fast_unaligned) ENTRY(fast_unaligned_fixup) @@ -459,10 +465,11 @@ ENTRY(fast_unaligned_fixup) l32i a7, a2, PT_AREG7 l32i a6, a2, PT_AREG6 l32i a5, a2, PT_AREG5 - l32i a4, a2, PT_AREG4 + l32i a4, a2, PT_SAR l32i a0, a2, PT_AREG2 - xsr a0, depc # restore depc and a0 - wsr a0, sar + wsr a4, sar + wsr a0, depc # restore depc and a0 + l32i a4, a2, PT_AREG4 rsr a0, exccause s32i a0, a2, PT_DEPC # mark as a regular exception From patchwork Wed Jun 14 04:52:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Filippov X-Patchwork-Id: 107713 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1029942vqr; Tue, 13 Jun 2023 23:15:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6xjEnjdIPmeoiX/3B0Ctow6yyp5extNOcDZsmMXfIgHoY9zr3evU1Gfkvhg1l224TaFpA1 X-Received: by 2002:a05:6a00:a23:b0:656:c971:951 with SMTP id p35-20020a056a000a2300b00656c9710951mr1047902pfh.8.1686723323014; Tue, 13 Jun 2023 23:15:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686723322; cv=none; d=google.com; s=arc-20160816; b=xRh1IJ2XB/DA8uhZ5TDKw1tv5x2P4WVC35XK1RSlQorflQiPnMmk3PJu5dhe96/qC4 H4VoShE0KfgVF/96EWmc37cettmdB9f/5TcN2Bv6Xr/f/G2dz2E+NAAJ/qpXXh55auDz T891gXAmesf+7oYCMSeZMKaXGqXCN5c2ZNXz6AULkIZPTnTVjP2aFDF5w04eIKNsZKju YuOEYYXoAGMi6UMPX2e9uR7B62aRIV7f29Z+Rew594KZTKqeoOhvFAO7gVHSV8IiKTU1 53ippIIdQ55soWVtOW/Tc6fb7wCY/uNXHP6XpDfopT7sDfvKMVbdZ8D2m5xCmgvOQCBT kC8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zOGUIeRe2rK2jH6MjOQ7yvAIrs8mgJudUR0sm9NyIA8=; b=yTOGdWibXHhWxwPhZUiuovAHbZrA/YrAzUJ8uxnlcvd4rfM6ukJvUOy3oi43j7ymwu 9N5s5sacWJt/2vkEmP0OnpPqsKDOXio1yQlkJjOTatfSvPeD7vb/wNKbzUi3yjlavOPJ etG4jycBLAoOSFCYbbUmcpJDQIkLl0hMERJSJbtv/MRZnysVmJz+zaNfylF1v+Vcj36Y MNrFy2C2lXW37zgJzckW1FQFpiTKiaeYVdOFIaRrz5qTfyT7D0oDxevdUIt1hcceaDYT C+iGZUOJtxcrDEAUQCjqhIYyfvMo7weo07ZtpZcu6pYm52+uQDWK+wy/fNu4d52iqF3u rAWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=Qn7pJFh6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bv128-20020a632e86000000b0054f93b05b51si4527701pgb.96.2023.06.13.23.15.03; Tue, 13 Jun 2023 23:15:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=Qn7pJFh6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242656AbjFNEwq (ORCPT + 99 others); Wed, 14 Jun 2023 00:52:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242338AbjFNEw1 (ORCPT ); Wed, 14 Jun 2023 00:52:27 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 590AC122 for ; Tue, 13 Jun 2023 21:52:26 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1b4f8523197so3476125ad.1 for ; Tue, 13 Jun 2023 21:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686718345; x=1689310345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zOGUIeRe2rK2jH6MjOQ7yvAIrs8mgJudUR0sm9NyIA8=; b=Qn7pJFh6xO4WWSsEvgjH1opYxxWqsZzkWMnhskuVes2RbFb0G9Db8wOg5c2t/g5uV4 l+NB+sdWccozmqfgrIO0cQ2bCbjPfwynIP1bNggehSck+yXsTOzslUxqhzpCC8N9/Zvv vs/GQ05QI7fbfZPOc654z8tZvRRpy+oaY5eScgIzuU4ACqnVCe2fyuSddgZ8WZkrVY4l J/S/xzfgMwIQA7uWtnDEMyHMDrAC3EIUv81GD70LHmdZgRlGkVnFkBoloBHfeFuLh1hb xr/6tszPWnfn/rtetdX4ptDoVRPcHUOF851waRJ+d4elXbzRgvCxOZbaNtExexFZvahN AHCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686718345; x=1689310345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zOGUIeRe2rK2jH6MjOQ7yvAIrs8mgJudUR0sm9NyIA8=; b=MRU/e0Lecz2uKRvyEXvQUGoLDwlTvTeQOFo3bDsQDCgoE2iUYdayYHYVYgkza+s0nq kdqPx5yd/N048QPYVCk48UufISypUeR86OOICKd51HhKhwiOLh8qPSXwobZkfxNABVvG Grg8AnD6/7xvSzdh7wDfy/s06aySWn9AwQ3xAsf2XMBltp2yz1ddvrHoxoRrzV5GdIOU LhvmU7gda8iWt/S0p+YEHrpw9DXq+vMibeJKjcJqjGImJG5xO0s14zjy2QDTEZ7x8N2R H56TaaDxwUvvrteFoD5Sq0fFhG28BtaKE8rtQ9zGpt67YWQjSlTDhxm2OGKC0wLem7pN hRaQ== X-Gm-Message-State: AC+VfDze5Fgkm9KF4pR702GaEC1ZgDaoBnhNvG6rcJ0Azb2bs/XruNi+ cdu8fao8IZVzAkRdfR78oxajt+Akrao= X-Received: by 2002:a17:903:1d1:b0:1a1:d54b:71df with SMTP id e17-20020a17090301d100b001a1d54b71dfmr792990plh.0.1686718345433; Tue, 13 Jun 2023 21:52:25 -0700 (PDT) Received: from octofox.hsd1.ca.comcast.net ([2601:641:401:1d20:bf5f:2e99:78f2:6007]) by smtp.gmail.com with ESMTPSA id z18-20020a170903019200b001acad86ebc5sm11089790plg.33.2023.06.13.21.52.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 21:52:24 -0700 (PDT) From: Max Filippov To: linux-kernel@vger.kernel.org Cc: Chris Zankel , Max Filippov Subject: [PATCH 4/4] xtensa: add load/store exception handler Date: Tue, 13 Jun 2023 21:52:12 -0700 Message-Id: <20230614045212.2534746-5-jcmvbkbc@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230614045212.2534746-1-jcmvbkbc@gmail.com> References: <20230614045212.2534746-1-jcmvbkbc@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, FROM_LOCAL_NOVOWEL,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768655326078611829?= X-GMAIL-MSGID: =?utf-8?q?1768657595385015975?= Memory attached to instruction bus of the xtensa CPU is only accessible for a limited subset of opcodes. Other opcodes generate an exception with the load/store error cause code. This property complicates use of such systems. Provide a handler that recognizes and transparently fixes such exceptions. The following opcodes are recognized when used outside of FLIX bundles: l32i, l32i.n, l16ui, l16si, l8ui. Signed-off-by: Max Filippov --- arch/xtensa/Kconfig | 12 ++++ arch/xtensa/include/asm/traps.h | 5 ++ arch/xtensa/kernel/align.S | 109 ++++++++++++++++++++++++++++---- arch/xtensa/kernel/setup.c | 3 +- arch/xtensa/kernel/traps.c | 21 +++++- 5 files changed, 136 insertions(+), 14 deletions(-) diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index b1189f085a68..c9cb0cc78ebb 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -209,6 +209,18 @@ config XTENSA_UNALIGNED_USER Say Y here to enable unaligned memory access in user space. +config XTENSA_LOAD_STORE + bool "Load/store exception handler for memory only readable with l32" + help + The Xtensa architecture only allows reading memory attached to its + instruction bus with l32r and l32i instructions, all other + instructions raise an exception with the LoadStoreErrorCause code. + This makes it hard to use some configurations, e.g. store string + literals in FLASH memory attached to the instruction bus. + + Say Y here to enable exception handler that allows transparent + byte and 2-byte access to memory attached to instruction bus. + config HAVE_SMP bool "System Supports SMP (MX)" depends on XTENSA_VARIANT_CUSTOM diff --git a/arch/xtensa/include/asm/traps.h b/arch/xtensa/include/asm/traps.h index acffb02f8760..212c3b9ff407 100644 --- a/arch/xtensa/include/asm/traps.h +++ b/arch/xtensa/include/asm/traps.h @@ -47,6 +47,7 @@ __init trap_set_handler(int cause, xtensa_exception_handler *handler); asmlinkage void fast_illegal_instruction_user(void); asmlinkage void fast_syscall_user(void); asmlinkage void fast_alloca(void); +asmlinkage void fast_load_store(void); asmlinkage void fast_unaligned(void); asmlinkage void fast_second_level_miss(void); asmlinkage void fast_store_prohibited(void); @@ -64,6 +65,10 @@ void do_unhandled(struct pt_regs *regs); static inline void __init early_trap_init(void) { static struct exc_table init_exc_table __initdata = { +#ifdef CONFIG_XTENSA_LOAD_STORE + .fast_kernel_handler[EXCCAUSE_LOAD_STORE_ERROR] = + fast_load_store, +#endif #ifdef CONFIG_MMU .fast_kernel_handler[EXCCAUSE_DTLB_MISS] = fast_second_level_miss, diff --git a/arch/xtensa/kernel/align.S b/arch/xtensa/kernel/align.S index bcbd7962a684..20d6b4961001 100644 --- a/arch/xtensa/kernel/align.S +++ b/arch/xtensa/kernel/align.S @@ -22,7 +22,17 @@ #include #include -#if XCHAL_UNALIGNED_LOAD_EXCEPTION || XCHAL_UNALIGNED_STORE_EXCEPTION +#if XCHAL_UNALIGNED_LOAD_EXCEPTION || defined CONFIG_XTENSA_LOAD_STORE +#define LOAD_EXCEPTION_HANDLER +#endif + +#if XCHAL_UNALIGNED_STORE_EXCEPTION || defined LOAD_EXCEPTION_HANDLER +#define ANY_EXCEPTION_HANDLER +#endif + +#if XCHAL_HAVE_WINDOWED +#define UNALIGNED_USER_EXCEPTION +#endif /* First-level exception handler for unaligned exceptions. * @@ -58,10 +68,6 @@ * BE shift left / mask 0 0 X X */ -#if XCHAL_HAVE_WINDOWED -#define UNALIGNED_USER_EXCEPTION -#endif - #if XCHAL_HAVE_BE #define HWORD_START 16 @@ -103,7 +109,7 @@ * * 23 0 * ----------------------------- - * res 0000 0010 + * L8UI xxxx xxxx 0000 ssss tttt 0010 * L16UI xxxx xxxx 0001 ssss tttt 0010 * L32I xxxx xxxx 0010 ssss tttt 0010 * XXX 0011 ssss tttt 0010 @@ -128,9 +134,11 @@ #define OP0_L32I_N 0x8 /* load immediate narrow */ #define OP0_S32I_N 0x9 /* store immediate narrow */ +#define OP0_LSAI 0x2 /* load/store */ #define OP1_SI_MASK 0x4 /* OP1 bit set for stores */ #define OP1_SI_BIT 2 /* OP1 bit number for stores */ +#define OP1_L8UI 0x0 #define OP1_L32I 0x2 #define OP1_L16UI 0x1 #define OP1_L16SI 0x9 @@ -155,8 +163,73 @@ */ .literal_position +#ifdef CONFIG_XTENSA_LOAD_STORE +ENTRY(fast_load_store) + + call0 .Lsave_and_load_instruction + + /* Analyze the instruction (load or store?). */ + + extui a0, a4, INSN_OP0, 4 # get insn.op0 nibble + +#if XCHAL_HAVE_DENSITY + _beqi a0, OP0_L32I_N, 1f # L32I.N, jump +#endif + bnei a0, OP0_LSAI, .Linvalid_instruction + /* 'store indicator bit' set, jump */ + bbsi.l a4, OP1_SI_BIT + INSN_OP1, .Linvalid_instruction + +1: + movi a3, ~3 + and a3, a3, a8 # align memory address + + __ssa8 a8 + +#ifdef CONFIG_MMU + /* l32e can't be used here even when it's available. */ + /* TODO access_ok(a3) could be used here */ + j .Linvalid_instruction +#endif + l32i a5, a3, 0 + l32i a6, a3, 4 + __src_b a3, a5, a6 # a3 has the data word + +#if XCHAL_HAVE_DENSITY + addi a7, a7, 2 # increment PC (assume 16-bit insn) + _beqi a0, OP0_L32I_N, .Lload_w# l32i.n: jump + addi a7, a7, 1 +#else + addi a7, a7, 3 +#endif + + extui a5, a4, INSN_OP1, 4 + _beqi a5, OP1_L32I, .Lload_w + bnei a5, OP1_L8UI, .Lload16 + extui a3, a3, 0, 8 + j .Lload_w + +ENDPROC(fast_load_store) +#endif + +/* + * Entry condition: + * + * a0: trashed, original value saved on stack (PT_AREG0) + * a1: a1 + * a2: new stack pointer, original in DEPC + * a3: a3 + * depc: a2, original value saved on stack (PT_DEPC) + * excsave_1: dispatch table + * + * PT_DEPC >= VALID_DOUBLE_EXCEPTION_ADDRESS: double exception, DEPC + * < VALID_DOUBLE_EXCEPTION_ADDRESS: regular exception + */ + +#ifdef ANY_EXCEPTION_HANDLER ENTRY(fast_unaligned) +#if XCHAL_UNALIGNED_LOAD_EXCEPTION || XCHAL_UNALIGNED_STORE_EXCEPTION + call0 .Lsave_and_load_instruction /* Analyze the instruction (load or store?). */ @@ -171,12 +244,17 @@ ENTRY(fast_unaligned) /* 'store indicator bit' not set, jump */ _bbci.l a4, OP1_SI_BIT + INSN_OP1, .Lload +#endif +#if XCHAL_UNALIGNED_STORE_EXCEPTION + /* Store: Jump to table entry to get the value in the source register.*/ .Lstore:movi a5, .Lstore_table # table extui a6, a4, INSN_T, 4 # get source register addx8 a5, a6, a5 jx a5 # jump into table +#endif +#if XCHAL_UNALIGNED_LOAD_EXCEPTION /* Load: Load memory address. */ @@ -207,7 +285,9 @@ ENTRY(fast_unaligned) extui a5, a4, INSN_OP1, 4 _beqi a5, OP1_L32I, .Lload_w # l32i: jump - +#endif +#ifdef LOAD_EXCEPTION_HANDLER +.Lload16: extui a3, a3, 0, 16 # extract lower 16 bits _beqi a5, OP1_L16UI, .Lload_w addi a5, a5, -OP1_L16SI @@ -247,7 +327,8 @@ ENTRY(fast_unaligned) mov a13, a3 ; _j .Lexit; .align 8 mov a14, a3 ; _j .Lexit; .align 8 mov a15, a3 ; _j .Lexit; .align 8 - +#endif +#if XCHAL_UNALIGNED_STORE_EXCEPTION .Lstore_table: l32i a3, a2, PT_AREG0; _j .Lstore_w; .align 8 mov a3, a1; _j .Lstore_w; .align 8 # fishy?? @@ -265,7 +346,9 @@ ENTRY(fast_unaligned) mov a3, a13 ; _j .Lstore_w; .align 8 mov a3, a14 ; _j .Lstore_w; .align 8 mov a3, a15 ; _j .Lstore_w; .align 8 +#endif +#ifdef ANY_EXCEPTION_HANDLER /* We cannot handle this exception. */ .extern _kernel_exception @@ -294,6 +377,8 @@ ENTRY(fast_unaligned) 2: movi a0, _user_exception jx a0 +#endif +#if XCHAL_UNALIGNED_STORE_EXCEPTION # a7: instruction pointer, a4: instruction, a3: value .Lstore_w: @@ -358,7 +443,8 @@ ENTRY(fast_unaligned) #else s32i a6, a4, 4 #endif - +#endif +#ifdef ANY_EXCEPTION_HANDLER .Lexit: #if XCHAL_HAVE_LOOPS rsr a4, lend # check if we reached LEND @@ -453,7 +539,7 @@ ENTRY(fast_unaligned) __src_b a4, a4, a5 # a4 has the instruction ret - +#endif ENDPROC(fast_unaligned) ENTRY(fast_unaligned_fixup) @@ -490,5 +576,4 @@ ENTRY(fast_unaligned_fixup) jx a0 ENDPROC(fast_unaligned_fixup) - -#endif /* XCHAL_UNALIGNED_LOAD_EXCEPTION || XCHAL_UNALIGNED_STORE_EXCEPTION */ +#endif diff --git a/arch/xtensa/kernel/setup.c b/arch/xtensa/kernel/setup.c index 5c0c0fcac144..08d9cb5cbd41 100644 --- a/arch/xtensa/kernel/setup.c +++ b/arch/xtensa/kernel/setup.c @@ -245,7 +245,8 @@ void __init init_arch(bp_tag_t *bp_start) { /* Initialize basic exception handling if configuration may need it */ - if (IS_ENABLED(CONFIG_KASAN)) + if (IS_ENABLED(CONFIG_KASAN) || + IS_ENABLED(CONFIG_XTENSA_LOAD_STORE)) early_trap_init(); /* Initialize MMU. */ diff --git a/arch/xtensa/kernel/traps.c b/arch/xtensa/kernel/traps.c index f447262468c5..5fc19aa5ba5d 100644 --- a/arch/xtensa/kernel/traps.c +++ b/arch/xtensa/kernel/traps.c @@ -54,6 +54,9 @@ static void do_interrupt(struct pt_regs *regs); #if XTENSA_FAKE_NMI static void do_nmi(struct pt_regs *regs); #endif +#ifdef CONFIG_XTENSA_LOAD_STORE +static void do_load_store(struct pt_regs *regs); +#endif static void do_unaligned_user(struct pt_regs *regs); static void do_multihit(struct pt_regs *regs); #if XTENSA_HAVE_COPROCESSORS @@ -89,7 +92,10 @@ static dispatch_init_table_t __initdata dispatch_init_table[] = { { EXCCAUSE_SYSTEM_CALL, USER, fast_syscall_user }, { EXCCAUSE_SYSTEM_CALL, 0, system_call }, /* EXCCAUSE_INSTRUCTION_FETCH unhandled */ -/* EXCCAUSE_LOAD_STORE_ERROR unhandled*/ +#ifdef CONFIG_XTENSA_LOAD_STORE +{ EXCCAUSE_LOAD_STORE_ERROR, USER|KRNL, fast_load_store }, +{ EXCCAUSE_LOAD_STORE_ERROR, 0, do_load_store }, +#endif { EXCCAUSE_LEVEL1_INTERRUPT, 0, do_interrupt }, #ifdef SUPPORT_WINDOWED { EXCCAUSE_ALLOCA, USER|KRNL, fast_alloca }, @@ -354,6 +360,19 @@ static void do_div0(struct pt_regs *regs) force_sig_fault(SIGFPE, FPE_INTDIV, (void __user *)regs->pc); } +#ifdef CONFIG_XTENSA_LOAD_STORE +static void do_load_store(struct pt_regs *regs) +{ + __die_if_kernel("Unhandled load/store exception in kernel", + regs, SIGKILL); + + pr_info_ratelimited("Load/store error to %08lx in '%s' (pid = %d, pc = %#010lx)\n", + regs->excvaddr, current->comm, + task_pid_nr(current), regs->pc); + force_sig_fault(SIGBUS, BUS_ADRERR, (void *)regs->excvaddr); +} +#endif + /* * Handle unaligned memory accesses from user space. Kill task. *