From patchwork Sun Nov 5 21:34:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 161786 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp2314685vqu; Sun, 5 Nov 2023 13:38:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IEY2lrjva8dJDlhWx2rCsQ22ThMgO7oFCotYcW41CmFzmDvk0/XjOKkt8Ofyff12wHkbRt7 X-Received: by 2002:a05:6358:710:b0:169:abed:1fc3 with SMTP id e16-20020a056358071000b00169abed1fc3mr13557212rwj.0.1699220299379; Sun, 05 Nov 2023 13:38:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699220299; cv=none; d=google.com; s=arc-20160816; b=k8ZUOSYMuzwCWi1VSBZa4X+IIRkOkwyvPZ6/n/tA5DkEMD3SojKNZrleM6njCZOuTR 1/obmnCxFgb3xnaaEACuGb3Nzcij0wLzQOdkbN7xLi5Wg9OkdM0z1brzU4hsG6plsK6o 5WR5Ob8GgOcmuhlcBO7qEieQWYOUBHXLjrMOst04oWy+k+/+iqorVSRVNIMQQGUBsGac wxtryHsYu1jLNUXP7RkTZOrFjAdxbm3/ePQSdOvON368Ify6zq/oREpydNnOmzkxYDxt +8o2czmeuIr+5lATdcEMjglT+i7NxoxRy4waoONrfSdycYUTHRWo4vNSQdz/d14pvXS6 W5pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=diDfHIDfGzS7c9D5uSx32GZFst0D3sZaoS/knnykPTI=; fh=sWK7YOurtkTyaE7aaZtytOmSvUDe8vhU+5hRY6IWLIo=; b=hYqTg6K/5uYwl5+XkhdzLs4dweCtauIpOsBtKaDlRXogotPmDdAFgEkWu6YQPdhISR PmHEokC/TVq+UUi0tavsrQUn4xd5CYzkIGUXnIg/7MirlAFgnPOA8HN6YxQ8xeqoIhVf AuwiNJt6Ehsb2EdNGDmTZ8j0jkgua+6tHqfBpsbLTte88DT7o3/JNDl+J0zrfhuZt6c4 yoknL+kJCSCtudF/QwG7mPN3fKCU3lHPkZgBl0s++bzkFDT4zpA+Cq7KSeaL7o8mUSVV pJ/AnId8ldzb5vUtIPljzfYk2EYuiB0hCWEJ5qcc7ud09bqxNwmFua8TSOJsdwCMoEKL YPcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="aD2/r9nX"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id a24-20020a17090a8c1800b002803298b98asi6744053pjo.43.2023.11.05.13.38.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:38:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="aD2/r9nX"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 1FF0480620B4; Sun, 5 Nov 2023 13:38:14 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229740AbjKEVhv (ORCPT + 34 others); Sun, 5 Nov 2023 16:37:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229489AbjKEVhq (ORCPT ); Sun, 5 Nov 2023 16:37:46 -0500 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30D02CF for ; Sun, 5 Nov 2023 13:37:44 -0800 (PST) Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5437d60fb7aso6367781a12.3 for ; Sun, 05 Nov 2023 13:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699220262; x=1699825062; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=diDfHIDfGzS7c9D5uSx32GZFst0D3sZaoS/knnykPTI=; b=aD2/r9nXgwm+9xCoRq8kBwt492FoNlaj7r7AGqiuV6bMETq969Sr1i5qjv3RP/WHKo iTty/iZ18yEw03/6NubkAL9iMpVxH9Y09SJOqbWfm4oi/hD9eIusJAT8hoqrOooc0Luw t0FEefT89M0LRk34JVa3TBisoe4c/m3hbBPu9El4R5uaY65tcolH2MMuvjI1XlpfHs8K GelxEleqG/n/TbyXiSVD7Y/waIH24M5VqZXijqIqvXarMKlz368/pWXdUIhIvX36gfcQ 66x2bsrYXoGqTFptMN1aWJVopMvuVLCrSAXUKCVcUi/VLFghmuDsXuARZ+8h4JYk9PX6 k/GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699220262; x=1699825062; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=diDfHIDfGzS7c9D5uSx32GZFst0D3sZaoS/knnykPTI=; b=kHrx7E5deiGBGRlWS67A5GfzOXpLF2oxXA3Fqty1pDjf/r9suxoKvwuh4G/GEHch7l c4Sq5uScR3AGJCwUMBdipYszwkkWiiEGWWJ3LH3vq0pBNmOmvs1tVLXjsmBzwHx/kXph 4VPPbjCTzxhwBbZt5DlVzHsPxoHNyzGzE21bIm+aEQtwLuEX6e4HxdFze9Nv6dlCsce0 wd6E7jKYwxKWcE+G882qZd/SH1iBTMXgaj6FpyyMKxlX1atkYql1gImWWhYycPS74Z6E G5usL9L12vd2hVB7CA5sWpKv4lvg2jGFMTdAk/II0mDGbMYO2o8+7p210jHFEg1Ou7jQ UyIg== X-Gm-Message-State: AOJu0Yz+azVQ4zc/hXNh0ZN+piGyIjUZcEZcvgEy3zhcTG3LkTps33LV Ho6rZBA9kEorYrhLOUsMd4c= X-Received: by 2002:a17:907:96a7:b0:9c3:b3cb:29b8 with SMTP id hd39-20020a17090796a700b009c3b3cb29b8mr13422705ejc.0.1699220262378; Sun, 05 Nov 2023 13:37:42 -0800 (PST) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id g23-20020a1709064e5700b009c5c5c2c5a4sm3386559ejw.219.2023.11.05.13.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:37:42 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra Subject: [PATCH -tip v2 1/3] x86/percpu: Define PER_CPU_VAR macro also for !__ASSEMBLY__ Date: Sun, 5 Nov 2023 22:34:35 +0100 Message-ID: <20231105213731.1878100-2-ubizjak@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231105213731.1878100-1-ubizjak@gmail.com> References: <20231105213731.1878100-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 05 Nov 2023 13:38:14 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781761625014853971 X-GMAIL-MSGID: 1781761625014853971 Some C source files define 'asm' statements that use PER_CPU_VAR, so make PER_CPU_VAR macro available also without __ASSEMBLY__. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: "H. Peter Anvin" Cc: Peter Zijlstra Signed-off-by: Uros Bizjak --- v2: Make PER_CPU_VAR macro available in 'asm' statements instead of moving 'asm' statements to *.S assembly files. --- arch/x86/include/asm/percpu.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index b86b27d15e52..0f12b2004b94 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -84,10 +84,15 @@ }) #endif /* CONFIG_USE_X86_SEG_SUPPORT */ +#define PER_CPU_VAR(var) %__percpu_seg:(var)__percpu_rel + #else /* CONFIG_SMP */ #define __percpu_seg_override #define __percpu_prefix "" #define __force_percpu_prefix "" + +#define PER_CPU_VAR(var) (var)__percpu_rel + #endif /* CONFIG_SMP */ #define __my_cpu_type(var) typeof(var) __percpu_seg_override From patchwork Sun Nov 5 21:34:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 161788 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp2314752vqu; Sun, 5 Nov 2023 13:38:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IHlYNI6pL353IhXPA1J+k5G+ze3ykDMxlJCeBZSVbWmsXOXP0EIErRJUKCLDxJ8Ge/5kWQ5 X-Received: by 2002:a17:90a:fb94:b0:280:4796:59b8 with SMTP id cp20-20020a17090afb9400b00280479659b8mr13075728pjb.9.1699220311810; Sun, 05 Nov 2023 13:38:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699220311; cv=none; d=google.com; s=arc-20160816; b=aRCoHngk5XyMyx6Y57OVwnL+3j6748t3IqtLdiT4gnSaK9W16KH9mVltVJcpm66uRw T8mTDORXXmSSPZ52QlYb1eHJrqAxTHVBFPd7DIInD3G6T6jtD2S++RCd8ZNFLpsiJJk2 XR552KFMbwjlOxVh0VKzODkHXBGI4pM6FR3AGziVQGcCJ+L7Pkf1PC8kUNmgnUerzFOf 5Q0R52cRwbPuISoCzUPehyv7xP9KqMDSM9jDP1a9ReN2zUP3vy+W8DOcIHcA28PDH92H 9ZwKUhO9YEzHt/I/6jWleKSQSTDAzDSxjvwB8t+LUYQKFRrPdsLl9nNFVUEBxsB8D0im R/MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QWoTXFWh8QhoMIOAEXcP9Z3H1Z7HK9iPUfGCUZ9d8xA=; fh=sWK7YOurtkTyaE7aaZtytOmSvUDe8vhU+5hRY6IWLIo=; b=VCMSCgBx2C6G6jyus1li2iiJ9Sqk4klqVLCTAY+dwRBglVAIY7Ob6VzEaIObCEa07E 2Y4YC4atz1yXLCAkh1ZvGJQZFVoMXLYhD2LUDuj7Y9jy7o8tRzI9UAReoZAHYbWOKfa4 vgHaYyj+nOKRTTFUhwaxBlHc47oOCTgnOpeaBkbUDBgeeCm4zXHdMgq1bVOjr/JwDDuC isZJWLm6fhyc8lQAraQRN0o/Zr3WQ2G+7+k+gyV3qb1PIG9ZaAEaFIyhRRTRGMmnlpJV TsEUp053+LTwhfSSonB9ZED4jaGQAUy0FxyZ9IbvUErLEw7q+JQmkj/9ti559edO4Ok6 dxfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kfP3BLMj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id l73-20020a63914c000000b005b99683f21fsi6477446pge.760.2023.11.05.13.38.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:38:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kfP3BLMj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 4CA458072147; Sun, 5 Nov 2023 13:38:05 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229735AbjKEVhx (ORCPT + 34 others); Sun, 5 Nov 2023 16:37:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229592AbjKEVhs (ORCPT ); Sun, 5 Nov 2023 16:37:48 -0500 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FDEFCC for ; Sun, 5 Nov 2023 13:37:45 -0800 (PST) Received: by mail-ed1-x529.google.com with SMTP id 4fb4d7f45d1cf-5437269a661so9636179a12.0 for ; Sun, 05 Nov 2023 13:37:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699220264; x=1699825064; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QWoTXFWh8QhoMIOAEXcP9Z3H1Z7HK9iPUfGCUZ9d8xA=; b=kfP3BLMj31VbHR5yLGP2TgI5Qrcfh6F268pJAznApGtPiHxDAO1AdFewCgqoNsYK+2 q+jdoou+hqBYW0Hc2sNBeWIMZmq4FTly6B7XQgbEa/pSgkWrsbgjoAMjlm5namqLS4vL A0Kfizk2wAlOlT1WVH5ur6lXFp9P6dEZ9o+eFEUdeZcvrvP4ZWt5MMeJaA008BeIJrFt MFQfbNL4TXSuxjf5kVMb4H7ap1P0LVGLuxmqePzhaTt7eTsHBJLp9BDoCMgZ1RUZuXWx nwjJxBrVgGWNsuFYPUHXk4RAbolnnhHj0BxFEUmjEmKk7MbTmQdidIFinJC/sY1wmE+t oCvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699220264; x=1699825064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QWoTXFWh8QhoMIOAEXcP9Z3H1Z7HK9iPUfGCUZ9d8xA=; b=hddOKAB9W31mHWdDXU3IHKZL9wmi7fnwoPw7P3nGwwSzcFS9fvs78xS1muBxSjJVLt n+uGSC94pg6GmmuMpgApi51/pCzBdXZ3C9NILofahsskTESM8Uy4MQCVWxduvLgm+jaK JK8/TsPrsGxd7g5a51AVNgvlnoomJVOCOiS7lD35HGB8/jrbJgqS+R2czHv3zgQlA1tn z6yKUq6kM4a4d+LzCfe2K4f90z9GJq5jB3TUtiztHo3lzFb1oZx7Y77zM4+YfqB02+P4 ZmyPUW2eFAMPmEVpbBnSuhsQzlGHc1Nkhss8+qSxJf09JzIO5Lk3TRIdWAAj6Bvty9w8 Ux0w== X-Gm-Message-State: AOJu0Yw1rsz2gXcHfU9GLT6BZQ0ndrVKCTtby3JGr1QSykk46Hrh9QJU xEE+SpBxqua0jCupCkmy+6Y= X-Received: by 2002:a17:907:e8d:b0:9de:9ecb:eb03 with SMTP id ho13-20020a1709070e8d00b009de9ecbeb03mr4469048ejc.33.1699220263848; Sun, 05 Nov 2023 13:37:43 -0800 (PST) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id g23-20020a1709064e5700b009c5c5c2c5a4sm3386559ejw.219.2023.11.05.13.37.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:37:42 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra Subject: [PATCH -tip v2 2/3] x86/callthunks: Handle %rip-relative relocations in call thunk template Date: Sun, 5 Nov 2023 22:34:36 +0100 Message-ID: <20231105213731.1878100-3-ubizjak@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231105213731.1878100-1-ubizjak@gmail.com> References: <20231105213731.1878100-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 05 Nov 2023 13:38:05 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781761637640312142 X-GMAIL-MSGID: 1781761637640312142 Contrary to alternatives, relocations are currently not supported in call thunk templates. Re-use the existing infrastructure from alternative.c to allow %rip-relative relocations when copying call thunk template from its storage location. The patch allows unification of ASM_INCREMENT_CALL_DEPTH, which already uses PER_CPU_VAR macro, with INCREMENT_CALL_DEPTH, used in call thunk template, which is currently limited to use absolute address. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: "H. Peter Anvin" Cc: Peter Zijlstra Signed-off-by: Uros Bizjak --- v2: Reuse existing relocation infrastructure from alternative.c. --- arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c | 3 +-- arch/x86/kernel/callthunks.c | 32 ++++++++++++++++++++++------ 3 files changed, 28 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index 29832c338cdc..ba8d900f3ebe 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -18,6 +18,8 @@ static inline void apply_paravirt(struct paravirt_patch_site *start, #define __parainstructions_end NULL #endif +void apply_relocation(u8 *buf, size_t len, u8 *dest, u8 *src, size_t src_len); + /* * Currently, the max observed size in the kernel code is * JUMP_LABEL_NOP_SIZE/RELATIVEJUMP_SIZE, which are 5. diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 73be3931e4f0..66140c54d4f6 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -325,8 +325,7 @@ bool need_reloc(unsigned long offset, u8 *src, size_t src_len) return (target < src || target > src + src_len); } -static void __init_or_module noinline -apply_relocation(u8 *buf, size_t len, u8 *dest, u8 *src, size_t src_len) +void apply_relocation(u8 *buf, size_t len, u8 *dest, u8 *src, size_t src_len) { int prev, target = 0; diff --git a/arch/x86/kernel/callthunks.c b/arch/x86/kernel/callthunks.c index e9ad518a5003..ef9c04707b3c 100644 --- a/arch/x86/kernel/callthunks.c +++ b/arch/x86/kernel/callthunks.c @@ -24,6 +24,8 @@ static int __initdata_or_module debug_callthunks; +#define MAX_PATCH_LEN (255-1) + #define prdbg(fmt, args...) \ do { \ if (debug_callthunks) \ @@ -179,10 +181,15 @@ static const u8 nops[] = { static void *patch_dest(void *dest, bool direct) { unsigned int tsize = SKL_TMPL_SIZE; + u8 insn_buff[MAX_PATCH_LEN]; u8 *pad = dest - tsize; + memcpy(insn_buff, skl_call_thunk_template, tsize); + apply_relocation(insn_buff, tsize, pad, + skl_call_thunk_template, tsize); + /* Already patched? */ - if (!bcmp(pad, skl_call_thunk_template, tsize)) + if (!bcmp(pad, insn_buff, tsize)) return pad; /* Ensure there are nops */ @@ -192,9 +199,9 @@ static void *patch_dest(void *dest, bool direct) } if (direct) - memcpy(pad, skl_call_thunk_template, tsize); + memcpy(pad, insn_buff, tsize); else - text_poke_copy_locked(pad, skl_call_thunk_template, tsize, true); + text_poke_copy_locked(pad, insn_buff, tsize, true); return pad; } @@ -291,20 +298,27 @@ void *callthunks_translate_call_dest(void *dest) static bool is_callthunk(void *addr) { unsigned int tmpl_size = SKL_TMPL_SIZE; - void *tmpl = skl_call_thunk_template; + u8 insn_buff[MAX_PATCH_LEN]; unsigned long dest; + u8 *pad; dest = roundup((unsigned long)addr, CONFIG_FUNCTION_ALIGNMENT); if (!thunks_initialized || skip_addr((void *)dest)) return false; - return !bcmp((void *)(dest - tmpl_size), tmpl, tmpl_size); + *pad = dest - tmpl_size; + + memcpy(insn_buff, skl_call_thunk_template, tmpl_size); + apply_relocation(insn_buff, tmpl_size, pad, + skl_call_thunk_template, tmpl_size); + + return !bcmp(pad, insn_buff, tmpl_size); } int x86_call_depth_emit_accounting(u8 **pprog, void *func) { unsigned int tmpl_size = SKL_TMPL_SIZE; - void *tmpl = skl_call_thunk_template; + u8 insn_buff[MAX_PATCH_LEN]; if (!thunks_initialized) return 0; @@ -313,7 +327,11 @@ int x86_call_depth_emit_accounting(u8 **pprog, void *func) if (func && is_callthunk(func)) return 0; - memcpy(*pprog, tmpl, tmpl_size); + memcpy(insn_buff, skl_call_thunk_template, tmpl_size); + apply_relocation(insn_buff, tmpl_size, *pprog, + skl_call_thunk_template, tmpl_size); + + memcpy(*pprog, insn_buff, tmpl_size); *pprog += tmpl_size; return tmpl_size; } From patchwork Sun Nov 5 21:34:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 161787 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp2314731vqu; Sun, 5 Nov 2023 13:38:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IHjMxG9Kuj2YTgp4wXqFgUFIvvjlNqgy07/DIlH1MuvvJh/t35Xo6CGC+aSk8JcF5ECOrlb X-Received: by 2002:a05:6358:6f9a:b0:168:e843:9000 with SMTP id s26-20020a0563586f9a00b00168e8439000mr30766199rwn.26.1699220305915; Sun, 05 Nov 2023 13:38:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699220305; cv=none; d=google.com; s=arc-20160816; b=MqiIsllbgq6p+tFXj4HiHzYUOOdT4QBEoYrtBt9nWNO6FpBqRsdmpkz7HZFBqqFc5t 50d3jqMbKPrKQmxRCcxMEXKEcpG7MDzqfcISpd0DSYf0HnKCLPaLAHoZ8dJLa1bkHnx9 S0AAZB3Fe+T3mbv4VepldvSpdY9ss28ryjQPgmmJny5DbyxVDeXyNzBSwRX04DrDcvvv qzu702Iq8gauwZyjSdo/bUQcxufsAn8K9Rndvt5/yoZZBkauuxtNfQe332vllTQN/ujt 35OUREHfZpR8EveFcj+ik9Ld2B6ApzBk7gYOAbfGlI8jy5gfgBtcVG9Kmoiq7isoBk80 oxiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tgAqePGVhUCu3U56kTFLSJugpfg6qt/lSZ7uytS2Gf4=; fh=sWK7YOurtkTyaE7aaZtytOmSvUDe8vhU+5hRY6IWLIo=; b=voGiebTCu4qZPFwrN5I1N0EEUC42rFb8OSNHlfRdUGQd/72Cw/TzWSRAvOyaIs1m6I LP/z7Okr9IxRmnibDpXEmjdQhhSAL3d2AMehje/SJGzCx9C5xCybPgP9D7E8JvPjj3Ex iDE3GweXICgL0aN4Tj+f31UctwBeRCTThpeJy3Ji/ykPgAI9wlzIdrezMn2chuzIkU8X J8Mg1qn0lX/sNYvJ8tJEkDp8WyU3PiCjMQagew9p3+HxI09Nwf8jn72lxRmByiEbiRHK qs/rWp2F6dhbV9I3RikQsOeRIZT0l+RtnRg0eEYlOlIn/VeKJ0X+ypK5pbHOx5SSmrhT zygg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=beN4WwOD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id cu20-20020a17090afa9400b00277387bc11dsi6461624pjb.188.2023.11.05.13.38.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:38:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=beN4WwOD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 3ACE780620A9; Sun, 5 Nov 2023 13:38:23 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229893AbjKEVh4 (ORCPT + 34 others); Sun, 5 Nov 2023 16:37:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229698AbjKEVht (ORCPT ); Sun, 5 Nov 2023 16:37:49 -0500 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51066CF for ; Sun, 5 Nov 2023 13:37:46 -0800 (PST) Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-9ae2cc4d17eso586104166b.1 for ; Sun, 05 Nov 2023 13:37:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699220265; x=1699825065; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tgAqePGVhUCu3U56kTFLSJugpfg6qt/lSZ7uytS2Gf4=; b=beN4WwOD5FVwnUp8UIdN4MM/fbIPFd7Z5y1dDuuF0fgIrBdq3bhAMHdF+l77ltYJ79 x+18nDszzq9cFePvqiBWqncGzsDff8Bkr/JkODmWZPu0qXo24n1luWcSSUdcHK7G4gWz 60guZ8EIP0tyCm184ZwbH/2RPcEL4AMOvux0RdpVMA3di1iznqUOfYBI1Ew82ylHlxAT sA5y+7TU2UpTNLsbo1ETR7KqvDuw5lv/6ET20VDmIQ9M4p6F0M6cuXdyeEopYuI+nZ6F 5AlbWtqIlLdP9Q/JGbouIMRtXu7Bj68kT2Xo5iw/GHG0uU5arY5SUk36coK98WSnEp4H ZnSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699220265; x=1699825065; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tgAqePGVhUCu3U56kTFLSJugpfg6qt/lSZ7uytS2Gf4=; b=MQB8K0xi0UqBO8/s4FORXUbpeXLwXzw9w+BIB0CJKJuDUceyxmukLxyz5Ih1yIxY76 oT+9w5w93ESp7v7ZbI/zAPNcDBoxp8aADVDyH1gXibJFbwOgH9jzy7ZzY58t4hKihR00 FucmyLBnrtJshwIIzWvAq/MPn6Pqngm6qTCjEZtWl8kLxM+HvYiKsUphT3KTpt0r1uUM 2KJjl3msn0bL/tY1emwHxe3ydD2q/55DgHbQlLnYPrYcEEzRdzxbxC6Kn9LPBY0+keWr l/Aq0kCL3KcKLm7RpzW1TOO2lSlNVZLYrO5n8thmbhZF5P72mDBFyJGXuG11QXp5Un/K 6qZA== X-Gm-Message-State: AOJu0Yz5jicpIgAD27qIxQ2t3QCoQG94a250AZCQgVpfgpUIEH79GSTk U0OJ37vW+Iuo325+X3ZHBd8= X-Received: by 2002:a17:907:c313:b0:9bd:e3ae:ef57 with SMTP id tl19-20020a170907c31300b009bde3aeef57mr12829693ejc.60.1699220264570; Sun, 05 Nov 2023 13:37:44 -0800 (PST) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id g23-20020a1709064e5700b009c5c5c2c5a4sm3386559ejw.219.2023.11.05.13.37.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Nov 2023 13:37:44 -0800 (PST) From: Uros Bizjak To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra Subject: [PATCH -tip v2 3/3] x86/callthunks: Fix and unify call thunks assembly snippets Date: Sun, 5 Nov 2023 22:34:37 +0100 Message-ID: <20231105213731.1878100-4-ubizjak@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231105213731.1878100-1-ubizjak@gmail.com> References: <20231105213731.1878100-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 05 Nov 2023 13:38:23 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781761631325160358 X-GMAIL-MSGID: 1781761631325160358 Currently thunk debug macros explicitly define %gs: segment register prefix for their percpu variables. This is not compatible with !CONFIG_SMP, which requires non-prefixed percpu variables. Fix call thunks debug macros to use PER_CPU_VAR macro from percpu.h to conditionally use %gs: segment register prefix, depending on CONFIG_SMP. Finally, unify ASM_ prefixed assembly macros with their non-prefixed variants. With support of %rip-relative relocations in place, call thunk templates allow %rip-relative addressing, so unified assembly snippet can be used everywhere. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: "H. Peter Anvin" Cc: Peter Zijlstra Signed-off-by: Uros Bizjak --- arch/x86/include/asm/nospec-branch.h | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index f93e9b96927a..6f677be6bdb9 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -59,13 +59,13 @@ #ifdef CONFIG_CALL_THUNKS_DEBUG # define CALL_THUNKS_DEBUG_INC_CALLS \ - incq %gs:__x86_call_count; + incq PER_CPU_VAR(__x86_call_count); # define CALL_THUNKS_DEBUG_INC_RETS \ - incq %gs:__x86_ret_count; + incq PER_CPU_VAR(__x86_ret_count); # define CALL_THUNKS_DEBUG_INC_STUFFS \ - incq %gs:__x86_stuffs_count; + incq PER_CPU_VAR(__x86_stuffs_count); # define CALL_THUNKS_DEBUG_INC_CTXSW \ - incq %gs:__x86_ctxsw_count; + incq PER_CPU_VAR(__x86_ctxsw_count); #else # define CALL_THUNKS_DEBUG_INC_CALLS # define CALL_THUNKS_DEBUG_INC_RETS @@ -80,9 +80,6 @@ #define CREDIT_CALL_DEPTH \ movq $-1, PER_CPU_VAR(pcpu_hot + X86_call_depth); -#define ASM_CREDIT_CALL_DEPTH \ - movq $-1, PER_CPU_VAR(pcpu_hot + X86_call_depth); - #define RESET_CALL_DEPTH \ xor %eax, %eax; \ bts $63, %rax; \ @@ -95,20 +92,14 @@ CALL_THUNKS_DEBUG_INC_CALLS #define INCREMENT_CALL_DEPTH \ - sarq $5, %gs:pcpu_hot + X86_call_depth; \ - CALL_THUNKS_DEBUG_INC_CALLS - -#define ASM_INCREMENT_CALL_DEPTH \ sarq $5, PER_CPU_VAR(pcpu_hot + X86_call_depth); \ CALL_THUNKS_DEBUG_INC_CALLS #else #define CREDIT_CALL_DEPTH -#define ASM_CREDIT_CALL_DEPTH #define RESET_CALL_DEPTH -#define INCREMENT_CALL_DEPTH -#define ASM_INCREMENT_CALL_DEPTH #define RESET_CALL_DEPTH_FROM_CALL +#define INCREMENT_CALL_DEPTH #endif /* @@ -158,7 +149,7 @@ jnz 771b; \ /* barrier for jnz misprediction */ \ lfence; \ - ASM_CREDIT_CALL_DEPTH \ + CREDIT_CALL_DEPTH \ CALL_THUNKS_DEBUG_INC_CTXSW #else /* @@ -311,7 +302,7 @@ .macro CALL_DEPTH_ACCOUNT #ifdef CONFIG_CALL_DEPTH_TRACKING ALTERNATIVE "", \ - __stringify(ASM_INCREMENT_CALL_DEPTH), X86_FEATURE_CALL_DEPTH + __stringify(INCREMENT_CALL_DEPTH), X86_FEATURE_CALL_DEPTH #endif .endm