From patchwork Tue Oct 17 16:59:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Poimboeuf X-Patchwork-Id: 154396 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4273013vqb; Tue, 17 Oct 2023 10:00:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFoCKMWxZixQNT8yS3wnZXEA0m/uratF1nSvYl4IlOj2CRQkenqqdNoLxnNdqBh4yg/NsjA X-Received: by 2002:a05:6359:740b:b0:166:d97d:c5c3 with SMTP id va11-20020a056359740b00b00166d97dc5c3mr2389558rwb.1.1697562009771; Tue, 17 Oct 2023 10:00:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697562009; cv=none; d=google.com; s=arc-20160816; b=RFjRKaVbMGCU8ssb47p+CP/KPRa06ryXLtUeaydECFoLe95+1WD81VnrxJBl3scRb3 M+zpUAiY7KGBdQGnMx9wjIJz7XrqwImVcmqNjtT4l9Fos2I6hnYv4/tP4ej2b7z2aj13 ivHxcYOtIjaSbDgdQ5ayhFVSEOE8oAcOqtfyyFDAXgT6f5MDG6h0xL3fvBxewODc+0j4 1Roh7lWbZUKPIFFohL/ROFnZE5mwngsXJ2NYHIoLiYJPUwQLWFvffCvjBhghzk/aRO9v 1xfvZUIjeGi6G/PRK8A99cVqIqoudDxXl+PoVJJIKofFlfTQngQ5BOfc+6javd4ZMSGD imwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=JeCZbZ3DwY9q17J2YSdD84Jr4tl60Dp++f2Q2RZKsbA=; fh=MsYuFCl+NMe1wZd+UJTSltvRq+SUlh3mktW9eHCqbUM=; b=yD7uK5dxYyg3/vxToixZgcs6wyZ8i9bB7LzpTB0mrry8x0EXWH5qi8c6W8ozSrzgSR Z2Fc9C/HWEpQs0NYdD8A3OD3SqH9Z75+ki+VOca9cJQwgBKZRQ7qyU5oDiTuFygTsfuJ 9dFV3o9KZN4STIqrQvXWtQPvHAJC7yytIE6Tl4uX5Y4x9xPOswcWry5iXk05hQDrSUIc XFLLY2y6vMFLSgYAVTS8TjW2yCf3GVofvUNue2/I9ORXf8Nqa/gotf84XxgqGZ8dMo03 sR82GOtrNxcE5hERcZSYFOgAh+epTsLCXq9uYDrT2SzQnT/iS++OEUrH4XFcQkvXuIke cklQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nc9rTl4P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id s39-20020a056a0017a700b006b5c34a0c8csi2097905pfg.251.2023.10.17.10.00.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 10:00:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nc9rTl4P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id BAEC5802EAD9; Tue, 17 Oct 2023 10:00:03 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343650AbjJQQ7w (ORCPT + 20 others); Tue, 17 Oct 2023 12:59:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232593AbjJQQ7u (ORCPT ); Tue, 17 Oct 2023 12:59:50 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E854AAB; Tue, 17 Oct 2023 09:59:48 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B373C433C7; Tue, 17 Oct 2023 16:59:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697561988; bh=geQXKRC2Jvb636FBZET9oMZ/p3D7F0Z9S0YyA2XEv1I=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nc9rTl4PB8xxDCMVkyK3df1khEKShyJ57b6NDmta0G2eJU2ZvclP44qA/XiusW0S5 D97OnJZLlKyg2zbwoGWPg/rWGjKiNb1x1F9oe2tXplNWGfr4dPS7Q5573kYwlt5er8 B/2QMqCcZFyNoFMP2NVxLLJBEPLbasPIapHFS3r/2Y6mlFhOGrgVgbOb2yjqfnixnE T/YFYT+970Fl3Yk//lRN4LSeVvjJUtrtQ8U+VMJfvbfUQ4rv7kk6sP2dZMn1Uoi2KS AtX9jeMJCLQGrQds3DPLPzAeOgk8FbMvPkvnif0xghx5KCMhapfwIhjffbMKd38kEi jiQdkUSdfBRtA== Date: Tue, 17 Oct 2023 09:59:46 -0700 From: Josh Poimboeuf To: Nathan Chancellor Cc: "Kaplan, David" , Borislav Petkov , "linux-kernel@vger.kernel.org" , "linux-tip-commits@vger.kernel.org" , Ingo Molnar , "Peter Zijlstra (Intel)" , "x86@kernel.org" , "llvm@lists.linux.dev" Subject: [PATCH] x86/srso: Fix panic in return thunk during boot Message-ID: <20231017165946.v4i2d4exyqwqq3bx@treble> References: <20231012141031.GHZSf+V1NjjUJTc9a9@fat_crate.local> <169713303534.3135.10558074245117750218.tip-bot2@tip-bot2> <20231016211040.GA3789555@dev-arch.thelio-3990X> <20231016212944.GGZS2rSCbIsViqZBDe@fat_crate.local> <20231016214810.GA3942238@dev-arch.thelio-3990X> <20231017052834.v53regh66hspv45n@treble> <20231017153222.GA707258@dev-arch.thelio-3990X> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231017153222.GA707258@dev-arch.thelio-3990X> X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 17 Oct 2023 10:00:03 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780022781790257369 X-GMAIL-MSGID: 1780022781790257369 Enabling CONFIG_KCSAN causes a panic during boot due to an "invalid opcode" in __x86_return_thunk(): invalid opcode: 0000 [#1] PREEMPT SMP NOPTI CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.6.0-rc2-00316-g91174087dcc7 #1 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-14-g1e1da7a96300-prebuilt.qemu.org 04/01/2014 RIP: 0010:__x86_return_thunk+0x0/0x10 Code: e8 01 00 00 00 cc e8 01 00 00 00 cc 48 81 c4 80 00 00 00 65 48 c7 04 25 d0 ac 02 00 ff ff ff ff c3 cc 0f 1f 84 00 00 00 00 00 <0f> 0b cc cc cc cc cc cc cc cc cc cc cc cc cc cc e9 db 8c 8e fe 0f RSP: 0018:ffffaef1c0013ed0 EFLAGS: 00010246 RAX: ffffffffa0e80eb0 RBX: ffffffffa0f05240 RCX: 0001ffffffffffff RDX: 0000000000000551 RSI: ffffffffa0dcc64e RDI: ffffffffa0f05238 RBP: ffff8f93c11708e0 R08: ffffffffa1387280 R09: 0000000000000000 R10: 0000000000000282 R11: 0001ffffa0f05238 R12: 0000000000000002 R13: 0000000000000282 R14: 0000000000000001 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff8f93df000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8f93d6c01000 CR3: 0000000015c2e000 CR4: 0000000000350ef0 The panic is triggered by the UD2 instruction which gets patched into __x86_return_thunk() when alternatives are applied. After that point, the default return thunk should no longer be used. As David Kaplan describes, the issue is caused by a couple of KCSAN-generated constructors which aren't processed by objtool: "When KCSAN is enabled, GCC generates lots of constructor functions named _sub_I_00099_0 which call __tsan_init and then return. The returns in these are generally annotated normally by objtool and fixed up at runtime. But objtool runs on vmlinux.o and vmlinux.o does not include a couple of object files that are in vmlinux, like init/version-timestamp.o and .vmlinux.export.o, both of which contain _sub_I_00099_0 functions. As a result, the returns in these functions are not annotated, and the panic occurs when we call one of them in do_ctors and it uses the default return thunk. This difference can be seen by counting the number of these functions in the object files: $ objdump -d vmlinux.o|grep -c "<_sub_I_00099_0>:" 2601 $ objdump -d vmlinux|grep -c "<_sub_I_00099_0>:" 2603 If these functions are only run during kernel boot, there is no speculation concern." Fix it by disabling KCSAN on version-timestamp.o and .vmlinux.export.o so the extra functions don't get generated. KASAN and GCOV are already disabled for those files. Fixes: 91174087dcc7 ("x86/retpoline: Ensure default return thunk isn't used at runtime") Reported-by: Nathan Chancellor Closes: https://lore.kernel.org/lkml/20231016214810.GA3942238@dev-arch.thelio-3990X/ Debugged-by: David Kaplan Tested-by: Nathan Chancellor Reviewed-by: Nick Desaulniers Acked-by: Marco Elver Signed-off-by: Josh Poimboeuf --- init/Makefile | 1 + scripts/Makefile.vmlinux | 1 + 2 files changed, 2 insertions(+) diff --git a/init/Makefile b/init/Makefile index ec557ada3c12..cbac576c57d6 100644 --- a/init/Makefile +++ b/init/Makefile @@ -60,4 +60,5 @@ include/generated/utsversion.h: FORCE $(obj)/version-timestamp.o: include/generated/utsversion.h CFLAGS_version-timestamp.o := -include include/generated/utsversion.h KASAN_SANITIZE_version-timestamp.o := n +KCSAN_SANITIZE_version-timestamp.o := n GCOV_PROFILE_version-timestamp.o := n diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index 3cd6ca15f390..c9f3e03124d7 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -19,6 +19,7 @@ quiet_cmd_cc_o_c = CC $@ ifdef CONFIG_MODULES KASAN_SANITIZE_.vmlinux.export.o := n +KCSAN_SANITIZE_.vmlinux.export.o := n GCOV_PROFILE_.vmlinux.export.o := n targets += .vmlinux.export.o vmlinux: .vmlinux.export.o