From patchwork Mon Dec 19 14:35:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 34567 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2424312wrn; Mon, 19 Dec 2022 06:36:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf7jTwHntQSsWrYgOHtlcxCip+j2YvrJqfO0JtjjXCyKlXKtks3Y9FatuHC8WWoaTy0S0Qb6 X-Received: by 2002:a05:6402:28cb:b0:467:57ad:a1b2 with SMTP id ef11-20020a05640228cb00b0046757ada1b2mr20343619edb.21.1671460576401; Mon, 19 Dec 2022 06:36:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671460576; cv=none; d=google.com; s=arc-20160816; b=zK7bOM2d3kKhkowzepAwAzCa6iZpPoe+O5SxEFvcz0AhxjhdwEFWb9z/FHDGfeEyxJ 02xR+ATdpnH5tnIIB/MgGiFY/geaZTvvaUM/RpAZSPIX56zKPELC2sXwCXR4OqDMdvRl sR0kppfkOUYqfVGRBobyH8t7T+hb3yWLeARRBJ12BkGLnm2jRZoJzEfmCZHPJNTqSXR9 cN6z9J0EsS34pEl4KRzg2H2mrhO+32TBYCrBx+N4j6PCnDi/bChofCPtVLVcVmHu/F59 uN7NQEca7XvKSO6F+mRGNTTCJo6xkowgEDNGtUUTmGgFCJK9DfHJx3bKLJSCgFB7nOZA xK0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=SlTyr4G09bR2ij8WsoTipvwNmvTJWo6vY9CuPHyzn5c=; b=eVZUsq8RDq3rIPW9fkIiMMc+NbowZw79kxUDq+ZmE3/96qSF1whpHHwj07gI2DsB0e 3ccpCTvoK86f9X6SOT7F3u56nvrZcOE8e3oyq+I3v/PGk9Df+SUTT8KyId2t2DTAqndt kB+4dcxJZRg6UoXQFpdEmNkiVX/Y/Jx1IhZVlpgM3lp+RBNB2ZPYp52Y2nWLqv3eon5h dm2XTrLR4gw33m3eCu1IWLFaxVZXO9iIHYgyF1050u3xzShPwS/QZxzUy0qbranc1jke e4byuvllUc4CY0pkZWIo4vE1CeI3Bo4qrgtmnHNs0Ca5c4/1J2x3goW7Pd0XCPTGg5RJ CXzw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=mzUZIqGk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r11-20020a05640251cb00b0046b1c4b8199si10626322edd.82.2022.12.19.06.35.53; Mon, 19 Dec 2022 06:36:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=mzUZIqGk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232277AbiLSOf2 (ORCPT + 99 others); Mon, 19 Dec 2022 09:35:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232292AbiLSOfR (ORCPT ); Mon, 19 Dec 2022 09:35:17 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96F6FBAA; Mon, 19 Dec 2022 06:35:16 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4E61BB80D1F; Mon, 19 Dec 2022 14:35:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56F3AC433D2; Mon, 19 Dec 2022 14:35:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671460514; bh=CX+kGVYhEAgwOekYRSqXxT9FLRH2V059fUkrpfVmAU0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mzUZIqGk0Qpyqy6ed2SvJAwBxMg5R0na6yBpRjjha/hhZFe54tf6YG6deFhSdfs/j k8jkOxEM9ai2AmBoNXNFy3SRs887HOK9WwPjpDp+euHmyoqGkNkIPy3KtiyJMFF+Fu TpMOiwkBsqyKZsvez6JPl4nAyRFN9OnW7lIwMWifNRUgTWHH0daRDZK39rGTp8S99P 5JDh2dNR/bucDggHUJcE905yDr/CGK58hPy5UwggzZgWRVY/3KTreX4rST+kKZfDZs EY1H2E22G/O4jqjG+25ydmwb1qUdj5BhkDuKNrNkqPWqDzQCx8BOptn+7++ZNAtaCl jceaziZCHtQug== From: "Masami Hiramatsu (Google)" To: x86@kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Ingo Molnar , Suleiman Souhlal , bpf , linux-kernel@vger.kernel.org, Borislav Petkov , Peter Zijlstra , Josh Poimboeuf , Nadav Amit Subject: [PATCH -tip v4 1/2] x86/kprobes: Fix kprobes instruction boudary check with CONFIG_RETHUNK Date: Mon, 19 Dec 2022 23:35:10 +0900 Message-Id: <167146051026.1374301.392728975473572291.stgit@devnote3> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <167146050052.1374301.10407562178447545337.stgit@devnote3> References: <167146050052.1374301.10407562178447545337.stgit@devnote3> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752653445566526835?= X-GMAIL-MSGID: =?utf-8?q?1752653445566526835?= From: Masami Hiramatsu (Google) Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after RET instruction, kprobes always failes to check the probed instruction boundary by decoding the function body if the probed address is after such sequence. (Note that some conditional code blocks will be placed after function return, if compiler decides it is not on the hot path.) This is because kprobes expects kgdb puts the INT3 as a software breakpoint and it will replace the original instruction. But these INT3 are not such purpose, it doesn't need to recover the original instruction. To avoid this issue, kprobes checks whether the INT3 is owned by kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction. Signed-off-by: Masami Hiramatsu (Google) Suggested-by: Peter Zijlstra Fixes: e463a09af2f0 ("x86: Add straight-line-speculation mitigation") Cc: stable@vger.kernel.org --- arch/x86/kernel/kprobes/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 66299682b6b7..b36f3c367cb2 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -281,12 +282,15 @@ static int can_probe(unsigned long paddr) if (ret < 0) return 0; +#ifdef CONFIG_KGDB /* - * Another debugging subsystem might insert this breakpoint. - * In that case, we can't recover it. + * If there is a dynamically installed kgdb sw breakpoint, + * this function should not be probed. */ - if (insn.opcode.bytes[0] == INT3_INSN_OPCODE) + if (insn.opcode.bytes[0] == INT3_INSN_OPCODE && + kgdb_has_hit_break(addr)) return 0; +#endif addr += insn.length; } From patchwork Mon Dec 19 14:35:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 34568 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2424751wrn; Mon, 19 Dec 2022 06:37:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf6a19xk/Wsbw8kJoQVRHxM+3ODPLAaT6o5PvjbLLjLFHPVTBYCw462oS3K9+5TESnRBwVBG X-Received: by 2002:a17:906:714b:b0:7c1:6986:2b7c with SMTP id z11-20020a170906714b00b007c169862b7cmr29541586ejj.8.1671460620653; Mon, 19 Dec 2022 06:37:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671460620; cv=none; d=google.com; s=arc-20160816; b=rHd4VuY4Dc2+uPJL0bx33qMbuB/OxXul/xfrzP6To73y05M752gBCwgOSrqE0MeQbc Z79CwVZr1suc290WyrWBIWUZHxngFG3ASbR4PIeAJ5QfQsXa4HDRtw5LMSHYXiPGmdZ5 0AfAUhPngIFTRXU9kEIttH28e1xHBJpiSpRfjfrACpTl3/Ir5DNym5oDStGRD682fyyx +kUdFpGPg0BUwksPyycpYb0T86bKyjb0Scu4zVWThNwU0V6qlpdxGWE56WwSPLHlAccO 2T5UH/b7nqc92pmlauyhnNuQ5cLDnlcGuCcLvcv/lB6ElAUxG8mrbMjZ6VoPF4AytIRw 2+nA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=WjW6DugKoOVwvf3KAAWEERzMl5ttO562mxtqKpDUCb0=; b=QyJxBkZ80n1WXxNqeDa0GBLU/5U+sn4qklucpUDxCs4xUWeMQNd4dhMHiUWbnpP4iz P8mAT8f80jkOcjUvEAvyqxSxkEMVlWN4LPn3dbJVYw90kcrCnbgfZl41yomSeVibO1CW JFlD7rrD7vYqIt3ybTs7KmP/GMvl4zoH24CYvRqwII0M9Pi2benHNuQ9RMlVCDVU2uEc o3cGO5bNPitDkiatL0gB2J1KdDEIcUcQ4p5vZAseWx9eU/xFtchDNwdiA0PIrpdXswKx MspUYRjNsOMOnCwFJyb5bhy8SlGa65oR6wFGLraXWYno8yoB2zpXLOyv56ca2NBEQpTr 6vbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="VKmyNk/G"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fq32-20020a1709069da000b007c040bc1679si6982297ejc.237.2022.12.19.06.36.37; Mon, 19 Dec 2022 06:37:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="VKmyNk/G"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232418AbiLSOfw (ORCPT + 99 others); Mon, 19 Dec 2022 09:35:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232350AbiLSOfo (ORCPT ); Mon, 19 Dec 2022 09:35:44 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1E94BAA; Mon, 19 Dec 2022 06:35:25 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5AC48B80E47; Mon, 19 Dec 2022 14:35:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5AABEC433F1; Mon, 19 Dec 2022 14:35:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1671460523; bh=EnQDCn6+DxnGRVKzzZhsuaX7OEoKlrHh00hQOt+anfY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VKmyNk/GF2R5MK37W+xC38LM2qUuhTaDb71eCE6vtpI9dfYwRmRU7RYbCmx58dvAG QwEL1oxHSD9VkHZXbP7ljFFQyJNunM7ngpwu/p0KeBbFv05ogAC55baunB1j6psxqa IqS6WMY0krA6NnijbgbFlRqbHcOpHebwOBNY8qSq6JtME6suudObOr+8ghvO+vWR2h sVu+Neo38H+y9nuEGTg6MCDBD5vV8BPAYUu7G4+0Fzgq2g759rIyz1FghlzegeG6pN MbvmHiczzYmBqjxAVq96B+VGYT27+QIJH7Lon6pZjIu0GFyCjzkaA9sgINx9xLdLOv k/c4A9txaI27A== From: "Masami Hiramatsu (Google)" To: x86@kernel.org Cc: Steven Rostedt , Masami Hiramatsu , Ingo Molnar , Suleiman Souhlal , bpf , linux-kernel@vger.kernel.org, Borislav Petkov , Peter Zijlstra , Josh Poimboeuf , Nadav Amit Subject: [PATCH -tip v4 2/2] x86/kprobes: Fix optprobe optimization check with CONFIG_RETHUNK Date: Mon, 19 Dec 2022 23:35:19 +0900 Message-Id: <167146051929.1374301.7419382929328081706.stgit@devnote3> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <167146050052.1374301.10407562178447545337.stgit@devnote3> References: <167146050052.1374301.10407562178447545337.stgit@devnote3> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752653491815967296?= X-GMAIL-MSGID: =?utf-8?q?1752653491815967296?= From: Masami Hiramatsu (Google) Since the CONFIG_RETHUNK and CONFIG_SLS will use INT3 for stopping speculative execution after function return, kprobe jump optimization always fails on the functions with such INT3 inside the function body. (It already checks the INT3 padding between functions, but not inside the function) To avoid this issue, as same as kprobes, check whether the INT3 comes from kgdb or not, and if so, stop decoding and make it fail. The other INT3 will come from CONFIG_RETHUNK/CONFIG_SLS and those can be treated as a one-byte instruction. Signed-off-by: Masami Hiramatsu (Google) Suggested-by: Peter Zijlstra Fixes: e463a09af2f0 ("x86: Add straight-line-speculation mitigation") Cc: stable@vger.kernel.org --- arch/x86/kernel/kprobes/opt.c | 28 ++++++++-------------------- 1 file changed, 8 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index e6b8c5362b94..e57e07b0edb6 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -279,19 +280,6 @@ static int insn_is_indirect_jump(struct insn *insn) return ret; } -static bool is_padding_int3(unsigned long addr, unsigned long eaddr) -{ - unsigned char ops; - - for (; addr < eaddr; addr++) { - if (get_kernel_nofault(ops, (void *)addr) < 0 || - ops != INT3_INSN_OPCODE) - return false; - } - - return true; -} - /* Decode whole function to ensure any instructions don't jump into target */ static int can_optimize(unsigned long paddr) { @@ -334,15 +322,15 @@ static int can_optimize(unsigned long paddr) ret = insn_decode_kernel(&insn, (void *)recovered_insn); if (ret < 0) return 0; - +#ifdef CONFIG_KGDB /* - * In the case of detecting unknown breakpoint, this could be - * a padding INT3 between functions. Let's check that all the - * rest of the bytes are also INT3. + * If there is a dynamically installed kgdb sw breakpoint, + * this function should not be probed. */ - if (insn.opcode.bytes[0] == INT3_INSN_OPCODE) - return is_padding_int3(addr, paddr - offset + size) ? 1 : 0; - + if (insn.opcode.bytes[0] == INT3_INSN_OPCODE && + kgdb_has_hit_break(addr)) + return 0; +#endif /* Recover address */ insn.kaddr = (void *)addr; insn.next_byte = (void *)(addr + insn.length);