From patchwork Sun Nov 6 09:51:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Mailhol X-Patchwork-Id: 16107 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1417050wru; Sun, 6 Nov 2022 01:55:32 -0800 (PST) X-Google-Smtp-Source: AMsMyM4h22HpC9jv0W8uHCLvx2wdT3Ri4p73VCcI13FcEHcDjLMqX14G9+tKSn/yjCZFmuWVaL9p X-Received: by 2002:a17:903:1c2:b0:187:feb:1f31 with SMTP id e2-20020a17090301c200b001870feb1f31mr41495192plh.92.1667728531935; Sun, 06 Nov 2022 01:55:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1667728531; cv=none; d=google.com; s=arc-20160816; b=kTg99YZvd/BZVdYErxM7tYQM5M0ThxvTD+kYydqTkfxXM+Jk23NVm19zGvPi42qiwB HKiV6c6rEpyII+Ln3qTkJwRPhCU492YT+3FIw0e4sBTVkQfIvGhbSGyXV4SFM4SuNFDi udkhFSkEkI3Zxx1q4GwJBid7pcrera1xPL7fY6slyOIsafvOBFarT8f8Se+H7utAh1Fh Smd7Ujr4jlXbynWk4fMAsE/GeQc51FZQNTTVtYIaByJKjvHsiP+4cGFDH1VLMQA+g3+y 10h+jcYS4hjUHzGOlo1Cb0UupIOtG/N57UwPjVYNTbt4lumdtin90RmA9XogXZSpV7M/ exQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=ypcKLldwiqaDsOlKIEGYcVxJUId8HFUjb6S7/qxsdjE=; b=bYeSm4JAYX/h4ZDGhn/5ZBzjaxba0iYHcVUCAHrqRCFOBiiUSDVNqniDp+RcOkSUJC Gd2L9pICcqIaB3VOL7eG8k46J0Tl7rGrjP/HclUjQwE8gyHgEGiuLkz5qCWq93Pz4esa W2o3HXc39nb8dCOrh0CGIGKPsc9eVFFKQS8RwQIpO6zBhwN7oilpgt7U8fFyKqq9Sd1n wjiemQC5UBf7V5z+0WoZ6z0aFsyXPQVuqecyPWuLPEQtQYehwHPKtFCwuNpqfddiQEXC 0FzMxu1ZtPxTzySmYjf7sWBAhcE3aO0secQ0Yb/PwlktE0uewmqDgLb25i3IVucin3p6 85iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=QBgr9zaT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a17090a550500b0020388f845e9si9922369pji.52.2022.11.06.01.55.19; Sun, 06 Nov 2022 01:55:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=QBgr9zaT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229850AbiKFJvf (ORCPT + 99 others); Sun, 6 Nov 2022 04:51:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229829AbiKFJva (ORCPT ); Sun, 6 Nov 2022 04:51:30 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 538E9E0B2 for ; Sun, 6 Nov 2022 01:51:29 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id q9so8152824pfg.5 for ; Sun, 06 Nov 2022 01:51:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=ypcKLldwiqaDsOlKIEGYcVxJUId8HFUjb6S7/qxsdjE=; b=QBgr9zaTfSggETOEfREZkmAC9yEJR6EOPOcX9uReV/JQPsZ/IU1xIcz14lf13MYlNJ PfwH/wZiLZ5HW/VSUBBAnIoxDaFNoBfpgHqP0OVPOwkagdtbatdoSACMxEfkwNSKPJJa +xD8mX/j+P4RRrKXr/QJ890acISwlsaHC5764DhakcUjQq7x97TjCgS78/quvz1p6KQo HgQj+z0vW2mini8Xj7P4O+0NAk9EQpcan1J+VKPTPWs/rNorAxtHQClBp01tB0ztr1+w ts2D7OIpGVyaIsjN11sEzKqqhHOl41ldAWwnqa2GlIlGUjMuTJJLiXpNwcW5VwhZhura Bg3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ypcKLldwiqaDsOlKIEGYcVxJUId8HFUjb6S7/qxsdjE=; b=1AcSIxkuym5G0ZfPnd6viV/bM5ZaKaCRh4Jufx8s/U6wXDY7OyusCXklLV8S/mRtAo XKaKaTeCp4lt0JJnARdRyp970aDZwMJ09P6Bpwz9sKq+4rYVNInFsUxp0l+oIWPR9OKg qsaimri9sAgTZpsTRZf/mYiWFuXLY5sQ0u/lq7apzE2QzeMoCOiuwm0M+sHi934pdWy4 JlhCaF/UtWODMhDfDhpYZ+htom8064JuA12JHGOd1iawOZbRxJxgF0RASMM80nneCkyr lmc84dDfIwkKNDzI47EBQk2CtcznXRigUZF9X7MNb/9dD02RVJCx+7y8QYmsRVA8gyjm UXlQ== X-Gm-Message-State: ACrzQf0M0em6zFr95qm2SthdvyL9/gDMM3MuBHz9QO0Nfn/pY/6dmmEk QYYjVLF9cSic04k5x3UV8oM= X-Received: by 2002:a63:6bc5:0:b0:460:bd9a:64b8 with SMTP id g188-20020a636bc5000000b00460bd9a64b8mr39144875pgc.257.1667728288714; Sun, 06 Nov 2022 01:51:28 -0800 (PST) Received: from localhost.localdomain (124x33x176x97.ap124.ftth.ucom.ne.jp. [124.33.176.97]) by smtp.gmail.com with ESMTPSA id b5-20020aa79505000000b0056bfd4a2702sm2368677pfp.45.2022.11.06.01.51.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Nov 2022 01:51:28 -0800 (PST) Sender: Vincent Mailhol From: Vincent Mailhol To: x86@kernel.org, Ingo Molnar , Borislav Petkov Cc: Nick Desaulniers , Thomas Gleixner , linux-kernel@vger.kernel.org, Yury Norov , llvm@lists.linux.dev, Vincent Mailhol , Borislav Petkov Subject: [PATCH v1 2/2] x86/asm/bitops: Use __builtin_clz*() to evaluate constant expressions Date: Sun, 6 Nov 2022 18:51:06 +0900 Message-Id: <20221106095106.849154-3-mailhol.vincent@wanadoo.fr> X-Mailer: git-send-email 2.37.4 In-Reply-To: <20221106095106.849154-1-mailhol.vincent@wanadoo.fr> References: <20221106095106.849154-1-mailhol.vincent@wanadoo.fr> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748740112614044567?= X-GMAIL-MSGID: =?utf-8?q?1748740112614044567?= GCC and clang offers the __builtin_clz(x) and __builtin_clzll(x) functions which return the number of leading zero bits in x. c.f. [1]. By a simple subtraction, we can derive below equivalences: * For fls: Aside of the x = 0 special case, fls(x) is equivalent to BITS_PER_TYPE(x) - __builtin_clz(x). * For fls64: Aside of the x = 0 special case, fls64(x) is equivalent to BITS_PER_TYPE(x) - __builtin_clzll(x). __builtin_clzll() takes an unsigned long long as argument. We choose this version because BITS_PER_LONG_LONG is defined as 64 bits for all architecture making this flavor the most portable one. A BUILD_BUG_ON() safety net is added. When used on constant expressions, the compiler is only able to fold the builtin version (c.f. [2]). However, for non constant expressions, the kernel inline assembly results in better code for both GCC and clang. Use __builtin_constant_p() to select between the kernel's fls()/fls64() __builtin_clz()/__builtin_clzll() depending on whether the argument is constant or not. While renaming fls64() to variable_fls64(), change the argument type from __64 to u64 because we are not in an uapi header. [1] Built-in Functions Provided by GCC: https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html [2] commit 146034fed6ee ("x86/asm/bitops: Use __builtin_ffs() to evaluate constant expressions") CC: Borislav Petkov CC: Nick Desaulniers CC: Yury Norov Signed-off-by: Vincent Mailhol --- arch/x86/include/asm/bitops.h | 57 ++++++++++++++++++++++++----------- 1 file changed, 40 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h index a31453d7686d..58fb2fc49760 100644 --- a/arch/x86/include/asm/bitops.h +++ b/arch/x86/include/asm/bitops.h @@ -333,18 +333,15 @@ static __always_inline int variable_ffs(int x) */ #define ffs(x) (__builtin_constant_p(x) ? __builtin_ffs(x) : variable_ffs(x)) -/** - * fls - find last set bit in word - * @x: the word to search - * - * This is defined in a similar way as the libc and compiler builtin - * ffs, but returns the position of the most significant set bit. - * - * fls(value) returns 0 if value is 0 or the position of the last - * set bit if value is nonzero. The last (most significant) bit is - * at position 32. - */ -static __always_inline int fls(unsigned int x) +static __always_inline int constant_fls(unsigned int x) +{ + if (!x) + return 0; + + return BITS_PER_TYPE(x) - __builtin_clz(x); +} + +static __always_inline int variable_fls(unsigned int x) { int r; @@ -375,18 +372,30 @@ static __always_inline int fls(unsigned int x) } /** - * fls64 - find last set bit in a 64-bit word + * fls - find last set bit in word * @x: the word to search * * This is defined in a similar way as the libc and compiler builtin - * ffsll, but returns the position of the most significant set bit. + * ffs, but returns the position of the most significant set bit. * - * fls64(value) returns 0 if value is 0 or the position of the last + * fls(value) returns 0 if value is 0 or the position of the last * set bit if value is nonzero. The last (most significant) bit is - * at position 64. + * at position 32. */ +#define fls(x) (__builtin_constant_p(x) ? constant_fls(x) : variable_fls(x)) + #ifdef CONFIG_X86_64 -static __always_inline int fls64(__u64 x) +static __always_inline int constant_fls64(u64 x) +{ + BUILD_BUG_ON(sizeof(unsigned long long) != sizeof(x)); + + if (!x) + return 0; + + return BITS_PER_TYPE(x) - __builtin_clzll(x); +} + +static __always_inline int variable_fls64(u64 x) { int bitpos = -1; /* @@ -399,6 +408,20 @@ static __always_inline int fls64(__u64 x) : "rm" (x)); return bitpos + 1; } + +/** + * fls64 - find last set bit in a 64-bit word + * @x: the word to search + * + * This is defined in a similar way as the libc and compiler builtin + * ffsll, but returns the position of the most significant set bit. + * + * fls64(value) returns 0 if value is 0 or the position of the last + * set bit if value is nonzero. The last (most significant) bit is + * at position 64. + */ +#define fls64(x) \ + (__builtin_constant_p(x) ? constant_fls64(x) : variable_fls64(x)) #else #include #endif