From patchwork Wed Feb 15 20:49:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57719 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp410673wrn; Wed, 15 Feb 2023 12:51:12 -0800 (PST) X-Google-Smtp-Source: AK7set8/VrWQCBILwXsnQOOPGidvSsNbI9PvCgPv44yLOp8v+9pfHTTI+Pv8VikCUS1qvw0JgVqz X-Received: by 2002:a05:6402:1517:b0:4ac:d3bc:cb0d with SMTP id f23-20020a056402151700b004acd3bccb0dmr3938069edw.3.1676494271956; Wed, 15 Feb 2023 12:51:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494271; cv=none; d=google.com; s=arc-20160816; b=EdjZSdfttjJePFWbM33TpiJPPdBdywzhuVZfJpCjozxEWvGRwoBED4iP2+AEgDKk1d a7OGLrrO0Liq+V7SfucYW/LErZH2Sl8LviwLvng0w80RdtgBPo+yL+1OcdqbrGu6WU3K 16ACuMIUrV0tAtdoYX5B6BFWf9xnDfNeURpVXxkxBSg0dQArtWdHiEigBr41om19byqk UkeY59BybvBgmicwNVTHZFZ8f0BtCExcWWsTvqiinp6/XPhP8R/84UT5qdF+TUiQKQcM aigz1O/e7hRAdP6hauq7vpVLsOT8JumUZrI1sbJDZ9dglzNKgunPeqtP3oy7Wt/tATL0 ZJjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=adsKN4WaAWVPV63CWQYL28a1za4vKMQ5h3svhN7NFJM=; b=aqnIuHlJBnW92ydNfAgpa3JSB7G6DZML5D7A0e8kp3i6RZwBiUPM1+Dj1f1fm0qWu/ WOc3gLNO903O4gpPvBVzFFtRbPcwCv8aWFBAS8HJSTXwk+UauFnkH03f+53sF9kwGk9E UklVWSO+EY96Eb+OpJ5ejweJM8QIBEDpjHxE50DhRJK79ijennHe5oYioWpjBVnQiLr+ YNuQl2NgyBqNQvCRuhxH+tGoPgPxdQKK8gphb4NfazOGzuOhuPGmAnXLY4lhU/CM+Fue 6+eJTCu+E1ZB6FBb5NjXPEfB5qji81R5k8NEmcywOqIMW0K0Gvi3jyHKgdXp+7TUlzQ7 XZnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RvXTEAFr; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id i4-20020aa7c704000000b004acb603a70esi13644318edq.230.2023.02.15.12.51.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:51:11 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RvXTEAFr; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C10AF3881D07 for ; Wed, 15 Feb 2023 20:50:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C10AF3881D07 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494227; bh=adsKN4WaAWVPV63CWQYL28a1za4vKMQ5h3svhN7NFJM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=RvXTEAFrdAorgKKlACICQE/MSSbe56R749YaiEfU20hny2Qc1DgzHl5n/JvKHSnnt PxhhbN8u4l3w3kmTSP0HYGn0+8XgvcqTXNbsHqTWvJAmdhpJX41Sloc9mbV9OlTxwi a+4Y8HNfzA84EiZhUjqH25nnCR/TRPFvSSaLc9wI= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id A1DDC3857437; Wed, 15 Feb 2023 20:49:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A1DDC3857437 Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id D69A3202AD62; Wed, 15 Feb 2023 21:49:37 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id NONIUnXqFl2R; Wed, 15 Feb 2023 21:49:37 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id B5F33202AD51; Wed, 15 Feb 2023 21:49:37 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:49:37 +0100 To: , Subject: [PATCH 1/7] libstdc++: Ensure __builtin_constant_p isn't lost on the way Date: Wed, 15 Feb 2023 21:49:36 +0100 Message-ID: <21793652.EfDdHjke4D@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvEX7.Campus.gsi.de (10.10.4.97) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931657532942977?= X-GMAIL-MSGID: =?utf-8?q?1757931657532942977?= The more expensive code path should only be taken if it can be optimized away. Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * include/experimental/bits/simd.h (_SimdWrapper::_M_is_constprop_none_of) (_SimdWrapper::_M_is_constprop_all_of): Return false unless the computed result still satisfies __builtin_constant_p. --- libstdc++-v3/include/experimental/bits/simd.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/include/experimental/bits/simd.h b/libstdc++-v3/include/experimental/bits/simd.h index e76f4781fa6..3de966bbf22 100644 --- a/libstdc++-v3/include/experimental/bits/simd.h +++ b/libstdc++-v3/include/experimental/bits/simd.h @@ -2673,7 +2673,8 @@ template else __execute_n_times<_Width>( [&](auto __i) { __r &= _M_data[__i.value] == _Tp(); }); - return __r; + if (__builtin_constant_p(__r)) + return __r; } return false; } @@ -2693,7 +2694,8 @@ template else __execute_n_times<_Width>( [&](auto __i) { __r &= _M_data[__i.value] == ~_Tp(); }); - return __r; + if (__builtin_constant_p(__r)) + return __r; } return false; } From patchwork Wed Feb 15 20:49:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57723 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp412296wrn; Wed, 15 Feb 2023 12:54:58 -0800 (PST) X-Google-Smtp-Source: AK7set/IGz0OY6/1gBMG1wGcnxggI9x2l2VGcUP+EYIrFlbCWySt/Hz6Hg1sKOhqMMuE+BM7bRgX X-Received: by 2002:a05:6402:703:b0:4ac:cb71:42e with SMTP id w3-20020a056402070300b004accb71042emr3546679edx.12.1676494498521; Wed, 15 Feb 2023 12:54:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494498; cv=none; d=google.com; s=arc-20160816; b=sBD5CvKZUeR/YDW0FWe1JIuV33sBdN6kmhkjiz91ioeu5BFnlCVwDLiZtmLuYglH60 DufTWHzsXXC7GE2gD6XWQwVzQXIwQv7iq5Muj6lTJFYb3+hDoJiQKkM2bKvnSejTo3+E c9Qer2aA7Qbw+WIXSgeqRZ4YlzlHu7cYfvZp8jctvKFcqT5XzFVQWF8FtJFTGUhTY16m WVXNP4tsaBU9LLqSAGXCa/OQVNOLMb2rout7/Vsgx/+Hf1hHIjqxS0mPc2FpKdI0wN+v J3sZ7h9YeBNQBbzaeS5zpfi4ZbixCuANvOR8hGkBzTcTQ4AM1JV+1vfXIk0pEiUlXr4D sTvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=ByTV4Z627m/HqGZnpcru6mxG8XCOmh+bzuRkbD/a/mI=; b=gjcT57ny6AT1ac3tpW58y3B4712ekaJuBxLhFgYVzmEPsOiGm3GQJmYC73XObn3fH4 bLIQYJHookZiNin5yHQZ+72sE8ygReJ86fO/1NHBCPw5qb6Sfw07se95uG24w2XxJKoA KkBpp8lBDK7kmzL5sEejcB5lo/Y25BTO8Ij2PYHPVY8Q7sykCbmwAa9ibBHk2HJAKdUY mnlwKyRkzOX71WpAUOY3GwZ/9UEdJNGxj/LGGqiHGITCik50Sw51MBJiFZX6DpSveuY5 t5mIRKzmWF5g0t9U9YCbNIpjvEjQFNh4vvw0QahE2yO6b1HyX/XyKfeUMOxdCHjAiT0K jqxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Ggq6ALyF; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id gh13-20020a1709073c0d00b008b14ef753b8si638622ejc.731.2023.02.15.12.54.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:54:58 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Ggq6ALyF; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7E863392AC3E for ; Wed, 15 Feb 2023 20:51:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 7E863392AC3E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494317; bh=ByTV4Z627m/HqGZnpcru6mxG8XCOmh+bzuRkbD/a/mI=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Ggq6ALyFc1AZWN0mqMGrEG6vYu63EL/ioePpjeSTkX/X5o7rDUJkbzq7pPDYzI41p P3fzv2OCIaA46AqKpqR/SPJsSwyCc7DoijckvJnhmCLkao/0vOIPhLE5etctJ2WLN/ VHbBGSjfAmxT1WVONCDn8bL68MWKCNOY2tSMOgUc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id 995333854839; Wed, 15 Feb 2023 20:49:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 995333854839 Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id B77AB202AD6D; Wed, 15 Feb 2023 21:49:52 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id y_Hz73CkP1iX; Wed, 15 Feb 2023 21:49:52 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id 81988202AD51; Wed, 15 Feb 2023 21:49:52 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:49:52 +0100 To: , Subject: [PATCH 2/7] libstdc++: Annotate most lambdas with always_inline Date: Wed, 15 Feb 2023 21:49:46 +0100 Message-ID: <9083131.CDJkKcVGEf@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvEX7.Campus.gsi.de (10.10.4.97) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931895215494813?= X-GMAIL-MSGID: =?utf-8?q?1757931895215494813?= All of the annotated lambdas are simply a necessary means for implementing these functions and should never result in an actual function call. Many of these lambdas would go away if C++ had better language support for packs. Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: PR libstdc++/108030 * include/experimental/bits/simd_detail.h: Define _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA. * include/experimental/bits/simd.h: Annotate lambdas with _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA. * include/experimental/bits/simd_builtin.h: Ditto. * include/experimental/bits/simd_converter.h: Ditto. * include/experimental/bits/simd_fixed_size.h: Ditto. * include/experimental/bits/simd_math.h: Ditto. * include/experimental/bits/simd_neon.h: Ditto. * include/experimental/bits/simd_x86.h: Ditto. --- libstdc++-v3/include/experimental/bits/simd.h | 239 ++++++------ .../include/experimental/bits/simd_builtin.h | 351 ++++++++++-------- .../experimental/bits/simd_converter.h | 22 +- .../include/experimental/bits/simd_detail.h | 3 + .../experimental/bits/simd_fixed_size.h | 265 ++++++------- .../include/experimental/bits/simd_math.h | 52 +-- .../include/experimental/bits/simd_neon.h | 14 +- .../include/experimental/bits/simd_x86.h | 122 +++--- 8 files changed, 575 insertions(+), 493 deletions(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/include/experimental/bits/simd.h b/libstdc++-v3/include/experimental/bits/simd.h index 3de966bbf22..ffe72fa6ccf 100644 --- a/libstdc++-v3/include/experimental/bits/simd.h +++ b/libstdc++-v3/include/experimental/bits/simd.h @@ -609,28 +609,34 @@ template operator&(_Ip __rhs) const { return __generate_from_n_evaluations<_Np, _Ip>( - [&](auto __i) { return __rhs._M_data[__i] & _M_data[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __rhs._M_data[__i] & _M_data[__i]; + }); } _GLIBCXX_SIMD_INTRINSIC constexpr _Ip operator|(_Ip __rhs) const { return __generate_from_n_evaluations<_Np, _Ip>( - [&](auto __i) { return __rhs._M_data[__i] | _M_data[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __rhs._M_data[__i] | _M_data[__i]; + }); } _GLIBCXX_SIMD_INTRINSIC constexpr _Ip operator^(_Ip __rhs) const { return __generate_from_n_evaluations<_Np, _Ip>( - [&](auto __i) { return __rhs._M_data[__i] ^ _M_data[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __rhs._M_data[__i] ^ _M_data[__i]; + }); } _GLIBCXX_SIMD_INTRINSIC constexpr _Ip operator~() const { return __generate_from_n_evaluations<_Np, _Ip>( - [&](auto __i) { return ~_M_data[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return ~_M_data[__i]; }); } }; return _Ip{}; @@ -1391,7 +1397,7 @@ template operator^=(const _BitMask& __b) & noexcept { __execute_n_times<_S_array_size>( - [&](auto __i) { _M_bits[__i] ^= __b._M_bits[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { _M_bits[__i] ^= __b._M_bits[__i]; }); return *this; } @@ -1399,7 +1405,7 @@ template operator|=(const _BitMask& __b) & noexcept { __execute_n_times<_S_array_size>( - [&](auto __i) { _M_bits[__i] |= __b._M_bits[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { _M_bits[__i] |= __b._M_bits[__i]; }); return *this; } @@ -1407,7 +1413,7 @@ template operator&=(const _BitMask& __b) & noexcept { __execute_n_times<_S_array_size>( - [&](auto __i) { _M_bits[__i] &= __b._M_bits[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { _M_bits[__i] &= __b._M_bits[__i]; }); return *this; } @@ -1797,8 +1803,9 @@ template __vector_broadcast(_Tp __x) { return __call_with_n_evaluations<_Np>( - [](auto... __xx) { return __vector_type_t<_Tp, _Np>{__xx...}; }, - [&__x](int) { return __x; }); + [](auto... __xx) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __vector_type_t<_Tp, _Np>{__xx...}; + }, [&__x](int) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x; }); } // }}} @@ -2205,7 +2212,7 @@ template ( - __x, [](auto... __entries) { + __x, [](auto... __entries) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return reinterpret_cast<_R>(_Up{__entries...}); }); } @@ -2607,7 +2614,7 @@ template _GLIBCXX_SIMD_INTRINSIC constexpr _SimdWrapper(initializer_list<_Tp> __init) : _Base(__generate_from_n_evaluations<_Width, _BuiltinType>( - [&](auto __i) { return __init.begin()[__i.value]; })) {} + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __init.begin()[__i.value]; })) {} _GLIBCXX_SIMD_INTRINSIC constexpr _SimdWrapper() = default; _GLIBCXX_SIMD_INTRINSIC constexpr _SimdWrapper(const _SimdWrapper&) @@ -2632,10 +2639,9 @@ template _GLIBCXX_SIMD_INTRINSIC constexpr operator _SimdTuple<_Tp, _As...>() const { - const auto& dd = _M_data; // workaround for GCC7 ICE - return __generate_from_n_evaluations>([&]( - auto __i) constexpr { return dd[int(__i)]; }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return _M_data[int(__i)]; }); } _GLIBCXX_SIMD_INTRINSIC constexpr operator const _BuiltinType&() const @@ -3192,21 +3198,19 @@ template { return __x; } template - _GLIBCXX_SIMD_INTRINSIC auto + _GLIBCXX_SIMD_INTRINSIC fixed_size_simd<_Tp, simd_size_v<_Tp, _Ap>> to_fixed_size(const simd<_Tp, _Ap>& __x) { - return simd<_Tp, simd_abi::fixed_size>>([&__x]( - auto __i) constexpr { return __x[__i]; }); + using _Rp = fixed_size_simd<_Tp, simd_size_v<_Tp, _Ap>>; + return _Rp([&__x](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }); } template - _GLIBCXX_SIMD_INTRINSIC auto + _GLIBCXX_SIMD_INTRINSIC fixed_size_simd_mask<_Tp, simd_size_v<_Tp, _Ap>> to_fixed_size(const simd_mask<_Tp, _Ap>& __x) { - constexpr int _Np = simd_mask<_Tp, _Ap>::size(); - fixed_size_simd_mask<_Tp, _Np> __r; - __execute_n_times<_Np>([&](auto __i) constexpr { __r[__i] = __x[__i]; }); - return __r; + return {__private_init, + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }}; } // to_native {{{2 @@ -3225,7 +3229,9 @@ template enable_if_t<(_Np == native_simd_mask<_Tp>::size()), native_simd_mask<_Tp>> to_native(const fixed_size_simd_mask<_Tp, _Np>& __x) { - return native_simd_mask<_Tp>([&](auto __i) constexpr { return __x[__i]; }); + return native_simd_mask<_Tp>( + __private_init, + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }); } // to_compatible {{{2 @@ -3242,7 +3248,10 @@ template _GLIBCXX_SIMD_INTRINSIC enable_if_t<(_Np == simd_mask<_Tp>::size()), simd_mask<_Tp>> to_compatible(const simd_mask<_Tp, simd_abi::fixed_size<_Np>>& __x) - { return simd_mask<_Tp>([&](auto __i) constexpr { return __x[__i]; }); } + { + return simd_mask<_Tp>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }); + } // masked assignment [simd_mask.where] {{{1 @@ -3400,9 +3409,9 @@ template _Impl::template _S_masked_cassign( \ __data(_M_k), __data(_M_value), \ __to_value_type_or_member_type<_Tp>(static_cast<_Up&&>(__x)), \ - [](auto __impl, auto __lhs, auto __rhs) constexpr { \ - return __impl.__name(__lhs, __rhs); \ - }); \ + [](auto __impl, auto __lhs, auto __rhs) \ + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { return __impl.__name(__lhs, __rhs); }); \ } \ static_assert(true) _GLIBCXX_SIMD_OP_(+, _S_plus); @@ -3899,12 +3908,11 @@ template >([&]( - auto __i) constexpr { - return _V([&](auto __j) constexpr { - return __x[__i * _V::size() + __j]; - }); - }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return _V([&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return __x[__i * _V::size() + __j]; }); + }); } else if constexpr ( __is_fixed_size_abi_v<_Ap> @@ -3917,41 +3925,40 @@ template * const __element_ptr = reinterpret_cast*>(&__data(__x)); - return __generate_from_n_evaluations>([&]( - auto __i) constexpr { - return _V(__element_ptr + __i * _V::size(), vector_aligned); - }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return _V(__element_ptr + __i * _V::size(), vector_aligned); }); #else const auto& __xx = __data(__x); - return __generate_from_n_evaluations>([&]( - auto __i) constexpr { - [[maybe_unused]] constexpr size_t __offset - = decltype(__i)::value * _V::size(); - return _V([&](auto __j) constexpr { - constexpr _SizeConstant<__j + __offset> __k; - return __xx[__k]; - }); - }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + [[maybe_unused]] constexpr size_t __offset + = decltype(__i)::value * _V::size(); + return _V([&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr _SizeConstant<__j + __offset> __k; + return __xx[__k]; + }); + }); #endif } else if constexpr (is_same_v) { // normally memcpy should work here as well - return __generate_from_n_evaluations>([&]( - auto __i) constexpr { return __x[__i]; }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }); } else { - return __generate_from_n_evaluations>([&]( - auto __i) constexpr { - if constexpr (__is_fixed_size_abi_v) - return _V([&](auto __j) constexpr { - return __x[__i * _V::size() + __j]; - }); - else - return _V(__private_init, - __extract_part(__data(__x))); - }); + return __generate_from_n_evaluations>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (__is_fixed_size_abi_v) + return _V([&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __x[__i * _V::size() + __j]; + }); + else + return _V(__private_init, + __extract_part(__data(__x))); + }); } } @@ -3975,22 +3982,22 @@ template >([&]( - auto __i) constexpr { - constexpr size_t __offset = __i * _V::size(); - return _V(__bitset_init, (__bits >> __offset).to_ullong()); - }); + return __generate_from_n_evaluations<_Parts, array<_V, _Parts>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr size_t __offset = __i * _V::size(); + return _V(__bitset_init, (__bits >> __offset).to_ullong()); + }); } else { - return __generate_from_n_evaluations<_Parts, array<_V, _Parts>>([&]( - auto __i) constexpr { - constexpr size_t __offset = __i * _V::size(); - return _V( - __private_init, [&](auto __j) constexpr { - return __x[__j + __offset]; - }); - }); + return __generate_from_n_evaluations<_Parts, array<_V, _Parts>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr size_t __offset = __i * _V::size(); + return _V(__private_init, + [&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __x[__j + __offset]; + }); + }); } } @@ -4008,12 +4015,14 @@ template using _V = __deduced_simd<_Tp, _N0>; if (__x._M_is_constprop()) - return __generate_from_n_evaluations([&]( - auto __i) constexpr { - using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; - constexpr size_t __offset = _SL::_S_before(__i); - return _Vi([&](auto __j) constexpr { return __x[__offset + __j]; }); - }); + return __generate_from_n_evaluations( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; + constexpr size_t __offset = _SL::_S_before(__i); + return _Vi([&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __x[__offset + __j]; + }); + }); else if constexpr (_Np == _N0) { static_assert(sizeof...(_Sizes) == 1); @@ -4080,28 +4089,28 @@ template #ifdef _GLIBCXX_SIMD_USE_ALIASING_LOADS const __may_alias<_Tp>* const __element_ptr = reinterpret_cast*>(&__x); - return __generate_from_n_evaluations([&]( - auto __i) constexpr { - using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; - constexpr size_t __offset = _SL::_S_before(__i); - constexpr size_t __base_align = alignof(simd<_Tp, _Ap>); - constexpr size_t __a - = __base_align - ((__offset * sizeof(_Tp)) % __base_align); - constexpr size_t __b = ((__a - 1) & __a) ^ __a; - constexpr size_t __alignment = __b == 0 ? __a : __b; - return _Vi(__element_ptr + __offset, overaligned<__alignment>); - }); + return __generate_from_n_evaluations( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; + constexpr size_t __offset = _SL::_S_before(__i); + constexpr size_t __base_align = alignof(simd<_Tp, _Ap>); + constexpr size_t __a + = __base_align - ((__offset * sizeof(_Tp)) % __base_align); + constexpr size_t __b = ((__a - 1) & __a) ^ __a; + constexpr size_t __alignment = __b == 0 ? __a : __b; + return _Vi(__element_ptr + __offset, overaligned<__alignment>); + }); #else - return __generate_from_n_evaluations([&]( - auto __i) constexpr { - using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; - const auto& __xx = __data(__x); - using _Offset = decltype(_SL::_S_before(__i)); - return _Vi([&](auto __j) constexpr { - constexpr _SizeConstant<_Offset::value + __j> __k; - return __xx[__k]; - }); - }); + return __generate_from_n_evaluations( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + using _Vi = __deduced_simd<_Tp, _SL::_S_at(__i)>; + const auto& __xx = __data(__x); + using _Offset = decltype(_SL::_S_before(__i)); + return _Vi([&](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr _SizeConstant<_Offset::value + __j> __k; + return __xx[__k]; + }); + }); #endif } @@ -4143,8 +4152,9 @@ template return simd_cast<_Rp>(__xs...); else if ((... && __xs._M_is_constprop())) return simd<_Tp, - simd_abi::deduce_t<_Tp, (simd_size_v<_Tp, _As> + ...)>>([&]( - auto __i) constexpr { return __subscript_in_pack<__i>(__xs...); }); + simd_abi::deduce_t<_Tp, (simd_size_v<_Tp, _As> + ...)>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return __subscript_in_pack<__i>(__xs...); }); else { _Rp __r{}; @@ -4160,9 +4170,10 @@ template _GLIBCXX_SIMD_CONSTEXPR __deduced_simd<_Tp, simd_size_v<_Tp, _Abi> * _Np> concat(const array, _Np>& __x) { - return __call_with_subscripts<_Np>(__x, [](const auto&... __xs) { - return concat(__xs...); - }); + return __call_with_subscripts<_Np>( + __x, [](const auto&... __xs) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return concat(__xs...); + }); } // }}} @@ -4695,7 +4706,7 @@ template simd_mask(_PrivateInit, _Fp&& __gen) : _M_data() { - __execute_n_times([&](auto __i) constexpr { + __execute_n_times([&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { _Impl::_S_set(_M_data, __i, __gen(__i)); }); } @@ -4881,7 +4892,9 @@ template if (__builtin_is_constant_evaluated() || __k._M_is_constprop()) { const int __r = __call_with_subscripts>( - __k, [](auto... __elements) { return ((__elements != 0) + ...); }); + __k, [](auto... __elements) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return ((__elements != 0) + ...); + }); if (__builtin_is_constant_evaluated() || __builtin_constant_p(__r)) return __r; } @@ -4896,8 +4909,11 @@ template { constexpr size_t _Np = simd_size_v<_Tp, _Abi>; const size_t _Idx = __call_with_n_evaluations<_Np>( - [](auto... __indexes) { return std::min({__indexes...}); }, - [&](auto __i) { return __k[__i] ? +__i : _Np; }); + [](auto... __indexes) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::min({__indexes...}); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __k[__i] ? +__i : _Np; + }); if (_Idx >= _Np) __invoke_ub("find_first_set(empty mask) is UB"); if (__builtin_constant_p(_Idx)) @@ -4914,8 +4930,11 @@ template { constexpr size_t _Np = simd_size_v<_Tp, _Abi>; const int _Idx = __call_with_n_evaluations<_Np>( - [](auto... __indexes) { return std::max({__indexes...}); }, - [&](auto __i) { return __k[__i] ? int(__i) : -1; }); + [](auto... __indexes) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::max({__indexes...}); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __k[__i] ? int(__i) : -1; + }); if (_Idx < 0) __invoke_ub("find_first_set(empty mask) is UB"); if (__builtin_constant_p(_Idx)) diff --git a/libstdc++-v3/include/experimental/bits/simd_builtin.h b/libstdc++-v3/include/experimental/bits/simd_builtin.h index 8851da69800..792439a81bf 100644 --- a/libstdc++-v3/include/experimental/bits/simd_builtin.h +++ b/libstdc++-v3/include/experimental/bits/simd_builtin.h @@ -194,8 +194,11 @@ template > using _Up = decltype(__w); return __intrin_bitcast<_Tp>( __call_with_n_evaluations<(sizeof(_Tp) - __shift) / __chunksize>( - [](auto... __chunks) { return _Up{__chunks...}; }, - [&](auto __i) { return __w[__shift / __chunksize + __i]; })); + [](auto... __chunks) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return _Up{__chunks...}; + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __w[__shift / __chunksize + __i]; + })); } } @@ -225,7 +228,9 @@ template // by _Total"); if (__x._M_is_constprop()) return __generate_from_n_evaluations<__return_size, _R>( - [&](auto __i) { return __x[__values_to_skip + __i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __x[__values_to_skip + __i]; + }); if constexpr (_Index == 0 && _Total == 1) return __x; else if constexpr (_Index == 0) @@ -570,7 +575,9 @@ template >( - [&](auto __i) { return static_cast<_To>(__v[__i + _Offset]); }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return static_cast<_To>(__v[__i + _Offset]); + }); } else { @@ -611,13 +618,14 @@ template (__vv); }; [[maybe_unused]] const auto __vi = __to_intrin(__v); - auto&& __make_array = [](auto __x0, [[maybe_unused]] auto __x1) { - if constexpr (_Np == 1) - return _R{__intrin_bitcast<_To>(__x0)}; - else - return _R{__intrin_bitcast<_To>(__x0), - __intrin_bitcast<_To>(__x1)}; - }; + auto&& __make_array + = [](auto __x0, [[maybe_unused]] auto __x1) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (_Np == 1) + return _R{__intrin_bitcast<_To>(__x0)}; + else + return _R{__intrin_bitcast<_To>(__x0), + __intrin_bitcast<_To>(__x1)}; + }; if constexpr (_Np == 0) return _R{}; @@ -642,7 +650,7 @@ template , _Np>( __adjust(_SizeConstant<_Np * 4>(), __v)); return __generate_from_n_evaluations<_Np, _R>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __vector_convert<_To>(__as_wrapper(__ints[__i])); }); } @@ -687,36 +695,40 @@ template (_mm_unpacklo_epi16(__vv[1], __vv[1])), __vector_bitcast(_mm_unpackhi_epi16(__vv[1], __vv[1]))}; if constexpr (sizeof(_ToT) == 4) - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - return __vector_convert<_To>( - _SimdWrapper(__vvvv[__i] >> 24)); - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __vector_convert<_To>( + _SimdWrapper(__vvvv[__i] >> 24)); + }); else if constexpr (is_integral_v<_ToT>) - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - const auto __signbits = __to_intrin(__vvvv[__i / 2] >> 31); - const auto __sx32 = __to_intrin(__vvvv[__i / 2] >> 24); - return __vector_bitcast<_ToT>( - __i % 2 == 0 ? _mm_unpacklo_epi32(__sx32, __signbits) - : _mm_unpackhi_epi32(__sx32, __signbits)); - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + const auto __signbits = __to_intrin(__vvvv[__i / 2] >> 31); + const auto __sx32 = __to_intrin(__vvvv[__i / 2] >> 24); + return __vector_bitcast<_ToT>( + __i % 2 == 0 ? _mm_unpacklo_epi32(__sx32, __signbits) + : _mm_unpackhi_epi32(__sx32, __signbits)); + }); else - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - const _SimdWrapper __int4 = __vvvv[__i / 2] >> 24; - return __vector_convert<_To>( - __i % 2 == 0 ? __int4 - : _SimdWrapper( - _mm_unpackhi_epi64(__to_intrin(__int4), - __to_intrin(__int4)))); - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + const _SimdWrapper __int4 = __vvvv[__i / 2] >> 24; + return __vector_convert<_To>( + __i % 2 == 0 ? __int4 + : _SimdWrapper( + _mm_unpackhi_epi64(__to_intrin(__int4), + __to_intrin(__int4)))); + }); } else if constexpr (sizeof(_FromT) == 1 && sizeof(_ToT) == 4) { const auto __shorts = __convert_all<__vector_type16_t< conditional_t, short, unsigned short>>>( __adjust(_SizeConstant<(_Np + 1) / 2 * 8>(), __v)); - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - return __convert_all<_To>(__shorts[__i / 2])[__i % 2]; - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __convert_all<_To>(__shorts[__i / 2])[__i % 2]; + }); } else if constexpr (sizeof(_FromT) == 2 && sizeof(_ToT) == 8 && is_signed_v<_FromT> && is_integral_v<_ToT>) @@ -736,9 +748,10 @@ template ( _mm_unpackhi_epi32(_mm_srai_epi32(__vv[1], 16), _mm_srai_epi32(__vv[1], 31)))}; - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - return __vector_bitcast<_ToT>(__vvvv[__i]); - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __vector_bitcast<_ToT>(__vvvv[__i]); + }); } else if constexpr (sizeof(_FromT) <= 2 && sizeof(_ToT) == 8) { @@ -747,9 +760,10 @@ template || is_floating_point_v<_ToT>, int, unsigned int>>>( __adjust(_SizeConstant<(_Np + 1) / 2 * 4>(), __v)); - return __generate_from_n_evaluations<_Np, _R>([&](auto __i) { - return __convert_all<_To>(__ints[__i / 2])[__i % 2]; - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __convert_all<_To>(__ints[__i / 2])[__i % 2]; + }); } else __assert_unreachable<_To>(); @@ -779,14 +793,14 @@ template (__v))}; else - return __generate_from_n_evaluations<_Np, _R>([&]( - auto __i) constexpr { - auto __part - = __extract_part<__i * _ToVT::_S_full_size + _Offset, - _FromVT::_S_partial_width, - _ToVT::_S_full_size>(__v); - return __vector_convert<_To>(__part); - }); + return __generate_from_n_evaluations<_Np, _R>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + auto __part + = __extract_part<__i * _ToVT::_S_full_size + _Offset, + _FromVT::_S_partial_width, + _ToVT::_S_full_size>(__v); + return __vector_convert<_To>(__part); + }); } else if constexpr (_Offset == 0) return array<_To, 1>{__vector_convert<_To>(__v)}; @@ -1017,8 +1031,9 @@ template else { constexpr auto __size = _S_size<_Tp>; - _GLIBCXX_SIMD_USE_CONSTEXPR auto __r = __generate_vector<_UV>( - [](auto __i) constexpr { return __i < __size ? -1 : 0; }); + _GLIBCXX_SIMD_USE_CONSTEXPR auto __r + = __generate_vector<_UV>([](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return __i < __size ? -1 : 0; }); return __r; } } @@ -1208,7 +1223,7 @@ template if constexpr (is_integral_v) return __x | __generate_vector<_Tp, _S_full_size<_Tp>>( - [](auto __i) -> _Tp { + [](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> _Tp { if (__i < _Np) return 0; else @@ -1348,26 +1363,27 @@ struct _CommonImplBuiltin } else { - __execute_n_times<__div_roundup(_Np, 4)>([&](auto __i) { - constexpr int __offset = __i * 4; - constexpr int __remaining = _Np - __offset; - if constexpr (__remaining > 4 && __remaining <= 7) - { - const _ULLong __bool7 - = (__x.template _M_extract<__offset>()._M_to_bits() - * 0x40810204081ULL) - & 0x0101010101010101ULL; - _S_store<__remaining>(__bool7, __mem + __offset); - } - else if constexpr (__remaining >= 4) - { - int __bits = __x.template _M_extract<__offset>()._M_to_bits(); - if constexpr (__remaining > 7) - __bits &= 0xf; - const int __bool4 = (__bits * 0x204081) & 0x01010101; - _S_store<4>(__bool4, __mem + __offset); - } - }); + __execute_n_times<__div_roundup(_Np, 4)>( + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr int __offset = __i * 4; + constexpr int __remaining = _Np - __offset; + if constexpr (__remaining > 4 && __remaining <= 7) + { + const _ULLong __bool7 + = (__x.template _M_extract<__offset>()._M_to_bits() + * 0x40810204081ULL) + & 0x0101010101010101ULL; + _S_store<__remaining>(__bool7, __mem + __offset); + } + else if constexpr (__remaining >= 4) + { + int __bits = __x.template _M_extract<__offset>()._M_to_bits(); + if constexpr (__remaining > 7) + __bits &= 0xf; + const int __bool4 = (__bits * 0x204081) & 0x01010101; + _S_store<4>(__bool4, __mem + __offset); + } + }); } } @@ -1434,13 +1450,13 @@ template inline static constexpr _SimdMember<_Tp> _S_generator(_Fp&& __gen, _TypeTag<_Tp>) { - return __generate_vector<_Tp, _S_full_size<_Tp>>([&]( - auto __i) constexpr { - if constexpr (__i < _S_size<_Tp>) - return __gen(__i); - else - return 0; - }); + return __generate_vector<_Tp, _S_full_size<_Tp>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (__i < _S_size<_Tp>) + return __gen(__i); + else + return 0; + }); } // _S_load {{{2 @@ -1455,10 +1471,10 @@ template : 16; constexpr size_t __bytes_to_load = sizeof(_Up) * _Np; if constexpr (sizeof(_Up) > 8) - return __generate_vector<_Tp, _SimdMember<_Tp>::_S_full_size>([&]( - auto __i) constexpr { - return static_cast<_Tp>(__i < _Np ? __mem[__i] : 0); - }); + return __generate_vector<_Tp, _SimdMember<_Tp>::_S_full_size>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return static_cast<_Tp>(__i < _Np ? __mem[__i] : 0); + }); else if constexpr (is_same_v<_Up, _Tp>) return _CommonImpl::template _S_load<_Tp, _S_full_size<_Tp>, _Np * sizeof(_Tp)>(__mem); @@ -1470,13 +1486,12 @@ template constexpr size_t __n_loads = __bytes_to_load / __max_load_size; constexpr size_t __elements_per_load = _Np / __n_loads; return __call_with_n_evaluations<__n_loads>( - [](auto... __uncvted) { - return __convert<_SimdMember<_Tp>>(__uncvted...); - }, - [&](auto __i) { - return _CommonImpl::template _S_load<_Up, __elements_per_load>( - __mem + __i * __elements_per_load); - }); + [](auto... __uncvted) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __convert<_SimdMember<_Tp>>(__uncvted...); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return _CommonImpl::template _S_load<_Up, __elements_per_load>( + __mem + __i * __elements_per_load); + }); } else if constexpr (__bytes_to_load % (__max_load_size / 2) == 0 && __max_load_size > 16) @@ -1485,20 +1500,19 @@ template = __bytes_to_load / (__max_load_size / 2); constexpr size_t __elements_per_load = _Np / __n_loads; return __call_with_n_evaluations<__n_loads>( - [](auto... __uncvted) { - return __convert<_SimdMember<_Tp>>(__uncvted...); - }, - [&](auto __i) { - return _CommonImpl::template _S_load<_Up, __elements_per_load>( - __mem + __i * __elements_per_load); - }); + [](auto... __uncvted) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __convert<_SimdMember<_Tp>>(__uncvted...); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return _CommonImpl::template _S_load<_Up, __elements_per_load>( + __mem + __i * __elements_per_load); + }); } else // e.g. int[] -> return __call_with_subscripts( - __mem, make_index_sequence<_Np>(), [](auto... __args) { - return __vector_type_t<_Tp, _S_full_size<_Tp>>{ - static_cast<_Tp>(__args)...}; - }); + __mem, make_index_sequence<_Np>(), + [](auto... __args) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __vector_type_t<_Tp, _S_full_size<_Tp>>{static_cast<_Tp>(__args)...}; + }); } // _S_masked_load {{{2 @@ -1507,9 +1521,10 @@ template _S_masked_load(_SimdWrapper<_Tp, _Np> __merge, _MaskMember<_Tp> __k, const _Up* __mem) noexcept { - _BitOps::_S_bit_iteration(_MaskImpl::_S_to_bits(__k), [&](auto __i) { - __merge._M_set(__i, static_cast<_Tp>(__mem[__i])); - }); + _BitOps::_S_bit_iteration(_MaskImpl::_S_to_bits(__k), + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __merge._M_set(__i, static_cast<_Tp>(__mem[__i])); + }); return __merge; } @@ -1523,7 +1538,7 @@ template constexpr size_t __max_store_size = _SuperImpl::template _S_max_store_size<_Up>; if constexpr (sizeof(_Up) > 8) - __execute_n_times<_Np>([&](auto __i) constexpr { + __execute_n_times<_Np>([&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __mem[__i] = __v[__i]; }); else if constexpr (is_same_v<_Up, _Tp>) @@ -1540,9 +1555,10 @@ template using _V = __vector_type_t<_Up, __vsize>; const array<_V, __stores> __converted = __convert_all<_V, __stores>(__v); - __execute_n_times<__full_stores>([&](auto __i) constexpr { - _CommonImpl::_S_store(__converted[__i], __mem + __i * __vsize); - }); + __execute_n_times<__full_stores>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + _CommonImpl::_S_store(__converted[__i], __mem + __i * __vsize); + }); if constexpr (__full_stores < __stores) _CommonImpl::template _S_store<(_Np - __full_stores * __vsize) * sizeof(_Up)>( @@ -1557,7 +1573,8 @@ template _MaskMember<_Tp> __k) { _BitOps::_S_bit_iteration( - _MaskImpl::_S_to_bits(__k), [&](auto __i) constexpr { + _MaskImpl::_S_to_bits(__k), + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __mem[__i] = __v[__i]; }); } @@ -1579,7 +1596,7 @@ template _Up> || (is_integral_v<_Tp> && is_integral_v<_Up> && sizeof(_Tp) == sizeof(_Up))) { // bitwise or no conversion, reinterpret: - const _MaskMember<_Up> __kk = [&]() { + const _MaskMember<_Up> __kk = [&]() _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr (__is_bitmask_v) return _MaskMember<_Up>(__k._M_data); else @@ -1618,7 +1635,7 @@ template constexpr size_t _NParts = _S_full_size<_Tp> / _UW_size; const array<_UV, _NAllStores> __converted = __convert_all<_UV, _NAllStores>(__v); - __execute_n_times<_NFullStores>([&](auto __i) { + __execute_n_times<_NFullStores>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { _SuperImpl::_S_masked_store_nocvt( _UW(__converted[__i]), __mem + __i * _UW_size, _UAbi::_MaskImpl::template _S_convert< @@ -1637,10 +1654,10 @@ template } } else - _BitOps::_S_bit_iteration( - _MaskImpl::_S_to_bits(__k), [&](auto __i) constexpr { - __mem[__i] = static_cast<_Up>(__v[__i]); - }); + _BitOps::_S_bit_iteration(_MaskImpl::_S_to_bits(__k), + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __mem[__i] = static_cast<_Up>(__v[__i]); + }); } // _S_complement {{{2 @@ -1932,7 +1949,9 @@ template static _Tp _S_##__name(const _Tp& __x, const _More&... __more) \ { \ return __generate_vector<_Tp>( \ - [&](auto __i) { return __name(__x[__i], __more[__i]...); }); \ + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \ + return __name(__x[__i], __more[__i]...); \ + }); \ } #define _GLIBCXX_SIMD_MATH_FALLBACK_MASKRET(__name) \ @@ -1941,23 +1960,25 @@ template const _More&... __more) \ { \ return __generate_vector<_Tp>( \ - [&](auto __i) { return __name(__x[__i], __more[__i]...); }); \ - } - -#define _GLIBCXX_SIMD_MATH_FALLBACK_FIXEDRET(_RetTp, __name) \ - template \ - static auto _S_##__name(const _Tp& __x, const _More&... __more) \ - { \ - return __fixed_size_storage_t<_RetTp, \ - _VectorTraits<_Tp>::_S_partial_width>:: \ - _S_generate([&](auto __meta) constexpr { \ - return __meta._S_generator( \ - [&](auto __i) { \ - return __name(__x[__meta._S_offset + __i], \ - __more[__meta._S_offset + __i]...); \ - }, \ - static_cast<_RetTp*>(nullptr)); \ - }); \ + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \ + return __name(__x[__i], __more[__i]...); \ + }); \ + } + +#define _GLIBCXX_SIMD_MATH_FALLBACK_FIXEDRET(_RetTp, __name) \ + template \ + static auto _S_##__name(const _Tp& __x, const _More&... __more) \ + { \ + return __fixed_size_storage_t<_RetTp, \ + _VectorTraits<_Tp>::_S_partial_width>:: \ + _S_generate([&](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \ + return __meta._S_generator( \ + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \ + return __name(__x[__meta._S_offset + __i], \ + __more[__meta._S_offset + __i]...); \ + }, \ + static_cast<_RetTp*>(nullptr)); \ + }); \ } _GLIBCXX_SIMD_MATH_FALLBACK(acos) @@ -2010,7 +2031,7 @@ template _S_remquo(const _Tp __x, const _Tp __y, __fixed_size_storage_t* __z) { - return __generate_vector<_Tp>([&](auto __i) { + return __generate_vector<_Tp>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { int __tmp; auto __r = remquo(__x[__i], __y[__i], &__tmp); __z->_M_set(__i, __tmp); @@ -2423,7 +2444,7 @@ template #endif // _GLIBCXX_SIMD_X86INTRIN else if constexpr (__fixed_size_storage_t::_S_tuple_size == 1) return {__call_with_subscripts<_Np>(__vector_bitcast<_LLong>(__tmp), - [](auto... __l) { + [](auto... __l) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __make_wrapper(__l...); })}; else @@ -2554,13 +2575,13 @@ struct _MaskImplBuiltinMixin _S_to_maskvector(_BitMask<_Np, _Sanitized> __x) { static_assert(is_same_v<_Up, __int_for_sizeof_t<_Up>>); - return __generate_vector<__vector_type_t<_Up, _ToN>>([&]( - auto __i) constexpr { - if constexpr (__i < _Np) - return __x[__i] ? ~_Up() : _Up(); - else - return _Up(); - }); + return __generate_vector<__vector_type_t<_Up, _ToN>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (__i < _Np) + return __x[__i] ? ~_Up() : _Up(); + else + return _Up(); + }); } template (__y); else */ { - return __generate_vector<__vector_type_t<_Up, _ToN>>([&]( - auto __i) constexpr { - if constexpr (__i < _Np) - return _Up(__x[__i.value]); - else - return _Up(); - }); + return __generate_vector<__vector_type_t<_Up, _ToN>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (__i < _Np) + return _Up(__x[__i.value]); + else + return _Up(); + }); } } } @@ -2625,7 +2646,9 @@ struct _MaskImplBuiltinMixin = __vector_bitcast<_Up>(__x) >> (sizeof(_Up) * __CHAR_BIT__ - 1); _ULLong __r = 0; __execute_n_times<_Np>( - [&](auto __i) { __r |= _ULLong(__bools[__i.value]) << __i; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __r |= _ULLong(__bools[__i.value]) << __i; + }); return __r; } @@ -2677,9 +2700,10 @@ template return __bools > 0; } else - return __generate_vector<_I, _S_size<_Tp>>([&](auto __i) constexpr { - return __mem[__i] ? ~_I() : _I(); - }); + return __generate_vector<_I, _S_size<_Tp>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __mem[__i] ? ~_I() : _I(); + }); } // }}} @@ -2752,7 +2776,7 @@ template // AVX(2) has 32/64 bit maskload, but nothing at 8 bit granularity auto __tmp = __wrapper_bitcast<__int_for_sizeof_t<_Tp>>(__merge); _BitOps::_S_bit_iteration(_SuperImpl::_S_to_bits(__mask), - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __tmp._M_set(__i, -__mem[__i]); }); __merge = __wrapper_bitcast<_Tp>(__tmp); @@ -2764,7 +2788,7 @@ template _GLIBCXX_SIMD_INTRINSIC static void _S_store(_SimdWrapper<_Tp, _Np> __v, bool* __mem) noexcept { - __execute_n_times<_Np>([&](auto __i) constexpr { + __execute_n_times<_Np>([&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __mem[__i] = __v[__i]; }); } @@ -2775,10 +2799,10 @@ template _S_masked_store(const _SimdWrapper<_Tp, _Np> __v, bool* __mem, const _SimdWrapper<_Tp, _Np> __k) noexcept { - _BitOps::_S_bit_iteration( - _SuperImpl::_S_to_bits(__k), [&](auto __i) constexpr { - __mem[__i] = __v[__i]; - }); + _BitOps::_S_bit_iteration(_SuperImpl::_S_to_bits(__k), + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __mem[__i] = __v[__i]; + }); } // _S_from_bitmask{{{2 @@ -2845,7 +2869,7 @@ template { __k = __generate_from_n_evaluations<_Np, __vector_type_t<_Tp, _Np>>( - [&](auto __j) { + [&](auto __j) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if (__i == __j) return _Tp(-__x); else @@ -2890,7 +2914,8 @@ template { return __call_with_subscripts( __data(__k), make_index_sequence<_S_size<_Tp>>(), - [](const auto... __ent) constexpr { return (... && !(__ent == 0)); }); + [](const auto... __ent) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return (... && !(__ent == 0)); }); } // }}} @@ -2901,7 +2926,8 @@ template { return __call_with_subscripts( __data(__k), make_index_sequence<_S_size<_Tp>>(), - [](const auto... __ent) constexpr { return (... || !(__ent == 0)); }); + [](const auto... __ent) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return (... || !(__ent == 0)); }); } // }}} @@ -2912,7 +2938,8 @@ template { return __call_with_subscripts( __data(__k), make_index_sequence<_S_size<_Tp>>(), - [](const auto... __ent) constexpr { return (... && (__ent == 0)); }); + [](const auto... __ent) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return (... && (__ent == 0)); }); } // }}} diff --git a/libstdc++-v3/include/experimental/bits/simd_converter.h b/libstdc++-v3/include/experimental/bits/simd_converter.h index 00b91c099ee..3160e251632 100644 --- a/libstdc++-v3/include/experimental/bits/simd_converter.h +++ b/libstdc++-v3/include/experimental/bits/simd_converter.h @@ -121,7 +121,7 @@ template { return __call_with_subscripts( __x, make_index_sequence<_Np>(), - [](auto... __values) constexpr->_Ret { + [](auto... __values) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> _Ret { return __make_simd_tuple<_To, decltype((void) __values, simd_abi::scalar())...>( static_cast<_To>(__values)...); @@ -233,7 +233,9 @@ template static_assert(_Ret::_FirstAbi::template _S_is_partial<_To>); return _Ret{__generate_from_n_evaluations< _Np, typename _VectorTraits::type>( - [&](auto __i) { return static_cast<_To>(__x[__i]); })}; + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return static_cast<_To>(__x[__i]); + })}; } else { @@ -241,7 +243,7 @@ template constexpr auto __n = __div_roundup(_Ret::_S_first_size, _Arg::_S_first_size); return __call_with_n_evaluations<__n>( - [&__x](auto... __uncvted) { + [&__x](auto... __uncvted) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { // assuming _Arg Abi tags for all __i are _Arg::_FirstAbi _SimdConverter<_From, typename _Arg::_FirstAbi, _To, typename _Ret::_FirstAbi> @@ -255,8 +257,9 @@ template _From, simd_abi::fixed_size<_Np - _Ret::_S_first_size>, _To, simd_abi::fixed_size<_Np - _Ret::_S_first_size>>()( __simd_tuple_pop_front<_Ret::_S_first_size>(__x))}; - }, - [&__x](auto __i) { return __get_tuple_at<__i>(__x); }); + }, [&__x](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __get_tuple_at<__i>(__x); + }); } } }; @@ -322,13 +325,14 @@ template return __vector_convert<__vector_type_t<_To, _Np>>(__x.first); else if constexpr (_Arg::_S_is_homogeneous) return __call_with_n_evaluations<_Arg::_S_tuple_size>( - [](auto... __members) { + [](auto... __members) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr ((is_convertible_v && ...)) return __vector_type_t<_To, _Np>{static_cast<_To>(__members)...}; else return __vector_convert<__vector_type_t<_To, _Np>>(__members...); - }, - [&](auto __i) { return __get_tuple_at<__i>(__x); }); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __get_tuple_at<__i>(__x); + }); else if constexpr (__fixed_size_storage_t<_To, _Np>::_S_tuple_size == 1) { _SimdConverter<_From, simd_abi::fixed_size<_Np>, _To, @@ -340,7 +344,7 @@ template { const _SimdWrapper<_From, _Np> __xv = __generate_from_n_evaluations<_Np, __vector_type_t<_From, _Np>>( - [&](auto __i) { return __x[__i]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }); return __vector_convert<__vector_type_t<_To, _Np>>(__xv); } } diff --git a/libstdc++-v3/include/experimental/bits/simd_detail.h b/libstdc++-v3/include/experimental/bits/simd_detail.h index 8cabc504863..a0ad10efe0f 100644 --- a/libstdc++-v3/include/experimental/bits/simd_detail.h +++ b/libstdc++-v3/include/experimental/bits/simd_detail.h @@ -262,6 +262,7 @@ #define _GLIBCXX_SIMD_INTRINSIC \ [[__gnu__::__always_inline__, __gnu__::__artificial__]] inline #define _GLIBCXX_SIMD_ALWAYS_INLINE [[__gnu__::__always_inline__]] inline +#define _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA __attribute__((__always_inline__)) #define _GLIBCXX_SIMD_IS_UNLIKELY(__x) __builtin_expect(__x, 0) #define _GLIBCXX_SIMD_IS_LIKELY(__x) __builtin_expect(__x, 1) @@ -294,6 +295,8 @@ #ifdef _GLIBCXX_SIMD_NO_ALWAYS_INLINE #undef _GLIBCXX_SIMD_ALWAYS_INLINE #define _GLIBCXX_SIMD_ALWAYS_INLINE inline +#undef _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA +#define _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA #undef _GLIBCXX_SIMD_INTRINSIC #define _GLIBCXX_SIMD_INTRINSIC inline #endif diff --git a/libstdc++-v3/include/experimental/bits/simd_fixed_size.h b/libstdc++-v3/include/experimental/bits/simd_fixed_size.h index 9ecc8e521ca..3ac6eaa3f6b 100644 --- a/libstdc++-v3/include/experimental/bits/simd_fixed_size.h +++ b/libstdc++-v3/include/experimental/bits/simd_fixed_size.h @@ -434,14 +434,15 @@ template if constexpr (is_same_v<_SimdTuple, __remove_cvref_t<_Tup>>) return __tup.first; else if (__builtin_is_constant_evaluated()) - return __fixed_size_storage_t<_TupT, _S_first_size>::_S_generate([&]( - auto __meta) constexpr { - return __meta._S_generator( - [&](auto __i) constexpr { return __tup[__i]; }, - static_cast<_TupT*>(nullptr)); + return __fixed_size_storage_t<_TupT, _S_first_size>::_S_generate( + [&](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __meta._S_generator( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __tup[__i]; + }, static_cast<_TupT*>(nullptr)); }); else - return [&]() { + return [&]() { // not always_inline; allow the compiler to decide __fixed_size_storage_t<_TupT, _S_first_size> __r; __builtin_memcpy(__r._M_as_charptr(), __tup._M_as_charptr(), sizeof(__r)); @@ -515,12 +516,11 @@ template negation>>>) ) { // need to write back at least one of __more after calling __fun - auto&& __first = [&](auto... __args) constexpr - { + auto&& __first = [&](auto... __args) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { auto __r = __fun(__tuple_element_meta<_Tp, _Abi0, 0>(), first, __args...); [[maybe_unused]] auto&& __ignore_me = {( - [](auto&& __dst, const auto& __src) { + [](auto&& __dst, const auto& __src) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr (is_assignable_v) { @@ -530,8 +530,7 @@ template }(static_cast<_More&&>(__more), __args), 0)...}; return __r; - } - (_M_extract_argument(__more)...); + }(_M_extract_argument(__more)...); if constexpr (_S_tuple_size == 1) return {__first}; else @@ -776,18 +775,18 @@ template sizeof...(_VX) == 0, "An array of scalars must be the last argument to __to_simd_tuple"); return __call_with_subscripts( - __from, - make_index_sequence<_NV>(), [&](const auto... __args) constexpr { - return __simd_tuple_concat( - _SimdTuple<_Tp, simd_abi::scalar>{__args}..., _SimdTuple<_Tp>()); - }); + __from, make_index_sequence<_NV>(), + [&](const auto... __args) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __simd_tuple_concat( + _SimdTuple<_Tp, simd_abi::scalar>{__args}..., _SimdTuple<_Tp>()); + }); } else return __call_with_subscripts( - __from, - make_index_sequence<_NV>(), [&](const auto... __args) constexpr { - return __to_simd_tuple<_Tp, _Np>(__args..., __fromX...); - }); + __from, make_index_sequence<_NV>(), + [&](const auto... __args) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __to_simd_tuple<_Tp, _Np>(__args..., __fromX...); + }); } template @@ -841,7 +840,7 @@ template ) return {__generate_from_n_evaluations<_R::_S_first_size, typename _R::_FirstType>( - [&](auto __i) { return __x[__i]; }), + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __x[__i]; }), __optimize_simd_tuple( __simd_tuple_pop_front<_R::_S_first_size>(__x))}; else if constexpr (is_same_v<_A0, _A1> @@ -994,10 +993,11 @@ template (element_ptr, element_aligned)); #else [[maybe_unused]] constexpr size_t __offset = __values_to_skip; - return __as_vector(simd<_Tp, _RetAbi>([&](auto __i) constexpr { - constexpr _SizeConstant<__i + __offset> __k; - return __x[__k]; - })); + return __as_vector(simd<_Tp, _RetAbi>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + constexpr _SizeConstant<__i + __offset> __k; + return __x[__k]; + })); #endif } @@ -1286,9 +1286,10 @@ template template static constexpr inline _SimdMember<_Tp> _S_broadcast(_Tp __x) noexcept { - return _SimdMember<_Tp>::_S_generate([&](auto __meta) constexpr { - return __meta._S_broadcast(__x); - }); + return _SimdMember<_Tp>::_S_generate( + [&](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __meta._S_broadcast(__x); + }); } // _S_generator {{{2 @@ -1296,14 +1297,15 @@ template static constexpr inline _SimdMember<_Tp> _S_generator(_Fp&& __gen, _TypeTag<_Tp>) { - return _SimdMember<_Tp>::_S_generate([&__gen](auto __meta) constexpr { - return __meta._S_generator( - [&](auto __i) constexpr { - return __i < _Np ? __gen(_SizeConstant<__meta._S_offset + __i>()) - : 0; - }, - _TypeTag<_Tp>()); - }); + return _SimdMember<_Tp>::_S_generate( + [&__gen](auto __meta) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __meta._S_generator( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __i < _Np ? __gen(_SizeConstant<__meta._S_offset + __i>()) + : 0; + }, + _TypeTag<_Tp>()); + }); } // _S_load {{{2 @@ -1311,9 +1313,10 @@ template static inline _SimdMember<_Tp> _S_load(const _Up* __mem, _TypeTag<_Tp>) noexcept { - return _SimdMember<_Tp>::_S_generate([&](auto __meta) { - return __meta._S_load(&__mem[__meta._S_offset], _TypeTag<_Tp>()); - }); + return _SimdMember<_Tp>::_S_generate( + [&](auto __meta) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __meta._S_load(&__mem[__meta._S_offset], _TypeTag<_Tp>()); + }); } // _S_masked_load {{{2 @@ -1323,7 +1326,7 @@ template const _MaskMember __bits, const _Up* __mem) noexcept { auto __merge = __old; - __for_each(__merge, [&](auto __meta, auto& __native) { + __for_each(__merge, [&](auto __meta, auto& __native) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if (__meta._S_submask(__bits).any()) #pragma GCC diagnostic push // __mem + __mem._S_offset could be UB ([expr.add]/4.3, but it punts @@ -1344,7 +1347,7 @@ template static inline void _S_store(const _SimdMember<_Tp>& __v, _Up* __mem, _TypeTag<_Tp>) noexcept { - __for_each(__v, [&](auto __meta, auto __native) { + __for_each(__v, [&](auto __meta, auto __native) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __meta._S_store(__native, &__mem[__meta._S_offset], _TypeTag<_Tp>()); }); } @@ -1355,7 +1358,7 @@ template _Up* __mem, const _MaskMember __bits) noexcept { - __for_each(__v, [&](auto __meta, auto __native) { + __for_each(__v, [&](auto __meta, auto __native) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if (__meta._S_submask(__bits).any()) #pragma GCC diagnostic push // __mem + __mem._S_offset could be UB ([expr.add]/4.3, but it punts @@ -1376,7 +1379,7 @@ template { _MaskMember __bits = 0; __for_each( - __x, [&__bits](auto __meta, auto __native) constexpr { + __x, [&__bits](auto __meta, auto __native) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __bits |= __meta._S_mask_to_shifted_ullong(__meta._S_negate(__native)); }); @@ -1414,7 +1417,7 @@ template { const auto& __x2 = __call_with_n_evaluations< __div_roundup(_Tup::_S_tuple_size, 2)>( - [](auto __first_simd, auto... __remaining) { + [](auto __first_simd, auto... __remaining) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr (sizeof...(__remaining) == 0) return __first_simd; else @@ -1428,7 +1431,7 @@ template __make_simd_tuple(__first_simd, __remaining...)); } }, - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { auto __left = __tup.template _M_simd_at<2 * __i>(); if constexpr (2 * __i + 1 == _Tup::_S_tuple_size) return __left; @@ -1444,7 +1447,9 @@ template _GLIBCXX_SIMD_USE_CONSTEXPR_API typename _LT::mask_type __k( __private_init, - [](auto __j) constexpr { return __j < _RT::size(); }); + [](auto __j) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __j < _RT::size(); + }); _LT __ext_right = __left; where(__k, __ext_right) = __proposed::resizing_simd_cast<_LT>(__right); @@ -1464,7 +1469,7 @@ template const _SimdTuple<_Tp, _As...>& __b) { return __a._M_apply_per_chunk( - [](auto __impl, auto __aa, auto __bb) constexpr { + [](auto __impl, auto __aa, auto __bb) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __impl._S_min(__aa, __bb); }, __b); @@ -1476,7 +1481,7 @@ template const _SimdTuple<_Tp, _As...>& __b) { return __a._M_apply_per_chunk( - [](auto __impl, auto __aa, auto __bb) constexpr { + [](auto __impl, auto __aa, auto __bb) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __impl._S_max(__aa, __bb); }, __b); @@ -1487,9 +1492,10 @@ template static inline constexpr _SimdTuple<_Tp, _As...> _S_complement(const _SimdTuple<_Tp, _As...>& __x) noexcept { - return __x._M_apply_per_chunk([](auto __impl, auto __xx) constexpr { - return __impl._S_complement(__xx); - }); + return __x._M_apply_per_chunk( + [](auto __impl, auto __xx) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __impl._S_complement(__xx); + }); } // _S_unary_minus {{{2 @@ -1497,23 +1503,24 @@ template static inline constexpr _SimdTuple<_Tp, _As...> _S_unary_minus(const _SimdTuple<_Tp, _As...>& __x) noexcept { - return __x._M_apply_per_chunk([](auto __impl, auto __xx) constexpr { - return __impl._S_unary_minus(__xx); - }); + return __x._M_apply_per_chunk( + [](auto __impl, auto __xx) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __impl._S_unary_minus(__xx); + }); } // arithmetic operators {{{2 -#define _GLIBCXX_SIMD_FIXED_OP(name_, op_) \ - template \ - static inline constexpr _SimdTuple<_Tp, _As...> name_( \ - const _SimdTuple<_Tp, _As...>& __x, const _SimdTuple<_Tp, _As...>& __y)\ - { \ - return __x._M_apply_per_chunk( \ - [](auto __impl, auto __xx, auto __yy) constexpr { \ - return __impl.name_(__xx, __yy); \ - }, \ - __y); \ +#define _GLIBCXX_SIMD_FIXED_OP(name_, op_) \ + template \ + static inline constexpr _SimdTuple<_Tp, _As...> name_( \ + const _SimdTuple<_Tp, _As...>& __x, const _SimdTuple<_Tp, _As...>& __y) \ + { \ + return __x._M_apply_per_chunk( \ + [](auto __impl, auto __xx, auto __yy) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { \ + return __impl.name_(__xx, __yy); \ + }, \ + __y); \ } _GLIBCXX_SIMD_FIXED_OP(_S_plus, +) @@ -1532,18 +1539,20 @@ template static inline constexpr _SimdTuple<_Tp, _As...> _S_bit_shift_left(const _SimdTuple<_Tp, _As...>& __x, int __y) { - return __x._M_apply_per_chunk([__y](auto __impl, auto __xx) constexpr { - return __impl._S_bit_shift_left(__xx, __y); - }); + return __x._M_apply_per_chunk( + [__y](auto __impl, auto __xx) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __impl._S_bit_shift_left(__xx, __y); + }); } template static inline constexpr _SimdTuple<_Tp, _As...> _S_bit_shift_right(const _SimdTuple<_Tp, _As...>& __x, int __y) { - return __x._M_apply_per_chunk([__y](auto __impl, auto __xx) constexpr { - return __impl._S_bit_shift_right(__xx, __y); - }); + return __x._M_apply_per_chunk( + [__y](auto __impl, auto __xx) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __impl._S_bit_shift_right(__xx, __y); + }); } // math {{{2 @@ -1557,35 +1566,40 @@ template { \ if constexpr (is_same_v<_Tp, _RetTp>) \ return __x._M_apply_per_chunk( \ - [](auto __impl, auto __xx) constexpr { \ - using _V = typename decltype(__impl)::simd_type; \ - return __data(__name(_V(__private_init, __xx))); \ - }); \ + [](auto __impl, auto __xx) \ + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { \ + using _V = typename decltype(__impl)::simd_type; \ + return __data(__name(_V(__private_init, __xx))); \ + }); \ else \ return __optimize_simd_tuple( \ - __x.template _M_apply_r<_RetTp>([](auto __impl, auto __xx) { \ - return __impl._S_##__name(__xx); \ - })); \ + __x.template _M_apply_r<_RetTp>( \ + [](auto __impl, auto __xx) \ + _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { return __impl._S_##__name(__xx); })); \ } \ else if constexpr ( \ is_same_v< \ _Tp, \ _RetTp> && (... && is_same_v<_SimdTuple<_Tp, _As...>, _More>) ) \ return __x._M_apply_per_chunk( \ - [](auto __impl, auto __xx, auto... __pack) constexpr { \ - using _V = typename decltype(__impl)::simd_type; \ - return __data(__name(_V(__private_init, __xx), \ - _V(__private_init, __pack)...)); \ - }, \ - __more...); \ + [](auto __impl, auto __xx, auto... __pack) \ + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { \ + using _V = typename decltype(__impl)::simd_type; \ + return __data(__name(_V(__private_init, __xx), \ + _V(__private_init, __pack)...)); \ + }, __more...); \ else if constexpr (is_same_v<_Tp, _RetTp>) \ return __x._M_apply_per_chunk( \ - [](auto __impl, auto __xx, auto... __pack) constexpr { \ - using _V = typename decltype(__impl)::simd_type; \ - return __data(__name(_V(__private_init, __xx), \ - __autocvt_to_simd(__pack)...)); \ - }, \ - __more...); \ + [](auto __impl, auto __xx, auto... __pack) \ + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { \ + using _V = typename decltype(__impl)::simd_type; \ + return __data(__name(_V(__private_init, __xx), \ + __autocvt_to_simd(__pack)...)); \ + }, __more...); \ else \ __assert_unreachable<_Tp>(); \ } @@ -1657,10 +1671,10 @@ template __fixed_size_storage_t::_S_size()>* __z) { return __x._M_apply_per_chunk( - [](auto __impl, const auto __xx, const auto __yy, auto& __zz) { - return __impl._S_remquo(__xx, __yy, &__zz); - }, - __y, *__z); + [](auto __impl, const auto __xx, const auto __yy, auto& __zz) + _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { return __impl._S_remquo(__xx, __yy, &__zz); }, + __y, *__z); } template @@ -1669,12 +1683,10 @@ template __fixed_size_storage_t& __exp) noexcept { return __x._M_apply_per_chunk( - [](auto __impl, const auto& __a, auto& __b) { - return __data( - frexp(typename decltype(__impl)::simd_type(__private_init, __a), - __autocvt_to_simd(__b))); - }, - __exp); + [](auto __impl, const auto& __a, auto& __b) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __data(frexp(typename decltype(__impl)::simd_type(__private_init, __a), + __autocvt_to_simd(__b))); + }, __exp); } #define _GLIBCXX_SIMD_TEST_ON_TUPLE_(name_) \ @@ -1700,7 +1712,7 @@ template _S_increment(_SimdTuple<_Ts...>& __x) { __for_each( - __x, [](auto __meta, auto& native) constexpr { + __x, [](auto __meta, auto& native) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __meta._S_increment(native); }); } @@ -1710,7 +1722,7 @@ template _S_decrement(_SimdTuple<_Ts...>& __x) { __for_each( - __x, [](auto __meta, auto& native) constexpr { + __x, [](auto __meta, auto& native) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __meta._S_decrement(native); }); } @@ -1722,11 +1734,10 @@ template __cmp(const _SimdTuple<_Tp, _As...>& __x, \ const _SimdTuple<_Tp, _As...>& __y) \ { \ - return _M_test( \ - [](auto __impl, auto __xx, auto __yy) constexpr { \ - return __impl.__cmp(__xx, __yy); \ - }, \ - __x, __y); \ + return _M_test([](auto __impl, auto __xx, auto __yy) \ + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA \ + { return __impl.__cmp(__xx, __yy); }, \ + __x, __y); \ } _GLIBCXX_SIMD_CMP_OPERATIONS(_S_equal_to) @@ -1753,12 +1764,13 @@ template _S_masked_assign(const _MaskMember __bits, _SimdTuple<_Tp, _As...>& __lhs, const __type_identity_t<_SimdTuple<_Tp, _As...>>& __rhs) { - __for_each( - __lhs, __rhs, - [&](auto __meta, auto& __native_lhs, auto __native_rhs) constexpr { - __meta._S_masked_assign(__meta._S_make_mask(__bits), __native_lhs, - __native_rhs); - }); + __for_each(__lhs, __rhs, + [&](auto __meta, auto& __native_lhs, auto __native_rhs) + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { + __meta._S_masked_assign(__meta._S_make_mask(__bits), __native_lhs, + __native_rhs); + }); } // Optimization for the case where the RHS is a scalar. No need to broadcast @@ -1769,7 +1781,7 @@ template const __type_identity_t<_Tp> __rhs) { __for_each( - __lhs, [&](auto __meta, auto& __native_lhs) constexpr { + __lhs, [&](auto __meta, auto& __native_lhs) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __meta._S_masked_assign(__meta._S_make_mask(__bits), __native_lhs, __rhs); }); @@ -1782,12 +1794,13 @@ template const _SimdTuple<_Tp, _As...>& __rhs, _Op __op) { - __for_each( - __lhs, __rhs, - [&](auto __meta, auto& __native_lhs, auto __native_rhs) constexpr { - __meta.template _S_masked_cassign(__meta._S_make_mask(__bits), - __native_lhs, __native_rhs, __op); - }); + __for_each(__lhs, __rhs, + [&](auto __meta, auto& __native_lhs, auto __native_rhs) + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + { + __meta.template _S_masked_cassign(__meta._S_make_mask(__bits), + __native_lhs, __native_rhs, __op); + }); } // Optimization for the case where the RHS is a scalar. No need to broadcast @@ -1798,7 +1811,7 @@ template const _Tp& __rhs, _Op __op) { __for_each( - __lhs, [&](auto __meta, auto& __native_lhs) constexpr { + __lhs, [&](auto __meta, auto& __native_lhs) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __meta.template _S_masked_cassign(__meta._S_make_mask(__bits), __native_lhs, __rhs, __op); }); @@ -1899,7 +1912,7 @@ template // _Np _UShort, _UInt, _ULLong, float, and double can be more efficient. _ULLong __r = 0; using _Vs = __fixed_size_storage_t<_UChar, _Np>; - __for_each(_Vs{}, [&](auto __meta, auto) { + __for_each(_Vs{}, [&](auto __meta, auto) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __r |= __meta._S_mask_to_shifted_ullong( __meta._S_mask_impl._S_load(&__mem[__meta._S_offset], _SizeConstant<__meta._S_size()>())); @@ -1912,9 +1925,10 @@ template _MaskMember __mask, const bool* __mem) noexcept { - _BitOps::_S_bit_iteration(__mask.to_ullong(), [&](auto __i) { - __merge.set(__i, __mem[__i]); - }); + _BitOps::_S_bit_iteration(__mask.to_ullong(), + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __merge.set(__i, __mem[__i]); + }); return __merge; } @@ -1932,7 +1946,8 @@ template static inline void _S_masked_store(const _MaskMember __v, bool* __mem, const _MaskMember __k) noexcept { - _BitOps::_S_bit_iteration(__k, [&](auto __i) { __mem[__i] = __v[__i]; }); + _BitOps::_S_bit_iteration( + __k, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __mem[__i] = __v[__i]; }); } // logical and bitwise operators {{{2 diff --git a/libstdc++-v3/include/experimental/bits/simd_math.h b/libstdc++-v3/include/experimental/bits/simd_math.h index 2aff8ff5fa4..c20315e4e30 100644 --- a/libstdc++-v3/include/experimental/bits/simd_math.h +++ b/libstdc++-v3/include/experimental/bits/simd_math.h @@ -788,7 +788,7 @@ template // __exponent(__x) returns the exponent value (bias removed) as // simd<_Up> with integral _Up - auto&& __exponent = [](const _V& __v) { + auto&& __exponent = [](const _V& __v) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { using namespace std::experimental::__proposed; using _IV = rebind_simd_t< conditional_t, _V>; @@ -931,7 +931,7 @@ template { return {__private_init, __data(__arg0)._M_apply_per_chunk( - [&](auto __impl, const auto&... __inner) { + [&](auto __impl, const auto&... __inner) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { using _V = typename decltype(__impl)::simd_type; return __data(__apply(_V(__private_init, __inner)...)); }, @@ -1092,8 +1092,9 @@ _GLIBCXX_SIMD_CVTING2(hypot) if constexpr (__is_fixed_size_abi_v<_Abi> && _V::size() > 1) { return __fixed_size_apply>( - [](auto __a, auto __b, auto __c) { return hypot(__a, __b, __c); }, - __x, __y, __z); + [](auto __a, auto __b, auto __c) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return hypot(__a, __b, __c); + }, __x, __y, __z); } else { @@ -1380,9 +1381,9 @@ template const fixed_size_simd>& __m, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>([&](auto __i) { - return std::assoc_laguerre(__n[__i], __m[__i], __x[__i]); - }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::assoc_laguerre(__n[__i], __m[__i], __x[__i]); + }); } template @@ -1391,9 +1392,9 @@ template const fixed_size_simd>& __m, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>([&](auto __i) { - return std::assoc_legendre(__n[__i], __m[__i], __x[__i]); - }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::assoc_legendre(__n[__i], __m[__i], __x[__i]); + }); } _GLIBCXX_SIMD_MATH_CALL2_(beta, _Tp) @@ -1414,8 +1415,9 @@ template hermite(const fixed_size_simd>& __n, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>( - [&](auto __i) { return std::hermite(__n[__i], __x[__i]); }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::hermite(__n[__i], __x[__i]); + }); } template @@ -1423,8 +1425,9 @@ template laguerre(const fixed_size_simd>& __n, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>( - [&](auto __i) { return std::laguerre(__n[__i], __x[__i]); }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::laguerre(__n[__i], __x[__i]); + }); } template @@ -1432,8 +1435,9 @@ template legendre(const fixed_size_simd>& __n, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>( - [&](auto __i) { return std::legendre(__n[__i], __x[__i]); }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::legendre(__n[__i], __x[__i]); + }); } _GLIBCXX_SIMD_MATH_CALL_(riemann_zeta) @@ -1443,8 +1447,9 @@ template sph_bessel(const fixed_size_simd>& __n, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>( - [&](auto __i) { return std::sph_bessel(__n[__i], __x[__i]); }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::sph_bessel(__n[__i], __x[__i]); + }); } template @@ -1453,9 +1458,9 @@ template const fixed_size_simd>& __m, const simd<_Tp, _Abi>& theta) { - return simd<_Tp, _Abi>([&](auto __i) { - return std::assoc_legendre(__l[__i], __m[__i], theta[__i]); - }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::assoc_legendre(__l[__i], __m[__i], theta[__i]); + }); } template @@ -1463,8 +1468,9 @@ template sph_neumann(const fixed_size_simd>& __n, const simd<_Tp, _Abi>& __x) { - return simd<_Tp, _Abi>( - [&](auto __i) { return std::sph_neumann(__n[__i], __x[__i]); }); + return simd<_Tp, _Abi>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return std::sph_neumann(__n[__i], __x[__i]); + }); } // }}} diff --git a/libstdc++-v3/include/experimental/bits/simd_neon.h b/libstdc++-v3/include/experimental/bits/simd_neon.h index 8429c252196..7e4cb17b205 100644 --- a/libstdc++-v3/include/experimental/bits/simd_neon.h +++ b/libstdc++-v3/include/experimental/bits/simd_neon.h @@ -61,7 +61,7 @@ template _S_masked_load(_SimdWrapper<_Tp, _Np> __merge, _MaskMember<_Tp> __k, const _Up* __mem) noexcept { - __execute_n_times<_Np>([&](auto __i) { + __execute_n_times<_Np>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if (__k[__i] != 0) __merge._M_set(__i, static_cast<_Tp>(__mem[__i])); }); @@ -75,7 +75,7 @@ template _S_masked_store_nocvt(_SimdWrapper<_Tp, _Np> __v, _Tp* __mem, _MaskMember<_Tp> __k) { - __execute_n_times<_Np>([&](auto __i) { + __execute_n_times<_Np>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if (__k[__i] != 0) __mem[__i] = __v[__i]; }); @@ -286,7 +286,7 @@ struct _MaskImplNeonMixin { constexpr auto __bitsel = __generate_from_n_evaluations<16, __vector_type_t<_I, 16>>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_I>( __i < _Np ? (__i < 8 ? 1 << __i : 1 << (__i - 8)) : 0); }); @@ -306,7 +306,7 @@ struct _MaskImplNeonMixin { constexpr auto __bitsel = __generate_from_n_evaluations<8, __vector_type_t<_I, 8>>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_I>(__i < _Np ? 1 << __i : 0); }); __asint &= __bitsel; @@ -322,7 +322,7 @@ struct _MaskImplNeonMixin { constexpr auto __bitsel = __generate_from_n_evaluations<4, __vector_type_t<_I, 4>>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_I>(__i < _Np ? 1 << __i : 0); }); __asint &= __bitsel; @@ -346,7 +346,7 @@ struct _MaskImplNeonMixin { constexpr auto __bitsel = __generate_from_n_evaluations<8, __vector_type_t<_I, 8>>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_I>(__i < _Np ? 1 << __i : 0); }); __asint &= __bitsel; @@ -361,7 +361,7 @@ struct _MaskImplNeonMixin { constexpr auto __bitsel = __generate_from_n_evaluations<4, __vector_type_t<_I, 4>>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_I>(__i < _Np ? 1 << __i : 0); }); __asint &= __bitsel; diff --git a/libstdc++-v3/include/experimental/bits/simd_x86.h b/libstdc++-v3/include/experimental/bits/simd_x86.h index 0f4aa95e1a4..60e80d394ba 100644 --- a/libstdc++-v3/include/experimental/bits/simd_x86.h +++ b/libstdc++-v3/include/experimental/bits/simd_x86.h @@ -537,16 +537,17 @@ struct _CommonImplX86 : _CommonImplBuiltin _S_store_bool_array(const _BitMask<_Np, _Sanitized> __x, bool* __mem) { if constexpr (__have_avx512bw_vl) // don't care for BW w/o VL - _S_store<_Np>(1 & __vector_bitcast<_UChar, _Np>([=]() constexpr { - if constexpr (_Np <= 16) - return _mm_movm_epi8(__x._M_to_bits()); - else if constexpr (_Np <= 32) - return _mm256_movm_epi8(__x._M_to_bits()); - else if constexpr (_Np <= 64) - return _mm512_movm_epi8(__x._M_to_bits()); - else - __assert_unreachable<_SizeConstant<_Np>>(); - }()), + _S_store<_Np>(1 & __vector_bitcast<_UChar, _Np>( + [=]() constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (_Np <= 16) + return _mm_movm_epi8(__x._M_to_bits()); + else if constexpr (_Np <= 32) + return _mm256_movm_epi8(__x._M_to_bits()); + else if constexpr (_Np <= 64) + return _mm512_movm_epi8(__x._M_to_bits()); + else + __assert_unreachable<_SizeConstant<_Np>>(); + }()), __mem); else if constexpr (__have_bmi2) { @@ -554,7 +555,7 @@ struct _CommonImplX86 : _CommonImplBuiltin _S_store<_Np>(_pdep_u32(__x._M_to_bits(), 0x01010101U), __mem); else __execute_n_times<__div_roundup(_Np, sizeof(size_t))>( - [&](auto __i) { + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { constexpr size_t __offset = __i * sizeof(size_t); constexpr int __todo = std::min(sizeof(size_t), _Np - __offset); if constexpr (__todo == 1) @@ -575,7 +576,7 @@ struct _CommonImplX86 : _CommonImplBuiltin }); } else if constexpr (__have_sse2 && _Np > 7) - __execute_n_times<__div_roundup(_Np, 16)>([&](auto __i) { + __execute_n_times<__div_roundup(_Np, 16)>([&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { constexpr int __offset = __i * 16; constexpr int __todo = std::min(16, int(_Np) - __offset); const int __bits = __x.template _M_extract<__offset>()._M_to_bits(); @@ -765,9 +766,10 @@ struct _CommonImplX86 : _CommonImplBuiltin static_assert(is_same_v<_Tp, _Tp> && __have_avx512f); if (__k._M_is_constprop() && __at0._M_is_constprop() && __at1._M_is_constprop()) - return __generate_from_n_evaluations<_Np, - __vector_type_t<_Tp, _Np>>([&]( - auto __i) constexpr { return __k[__i] ? __at1[__i] : __at0[__i]; }); + return __generate_from_n_evaluations<_Np, __vector_type_t<_Tp, _Np>>( + [&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __k[__i] ? __at1[__i] : __at0[__i]; + }); else if constexpr (sizeof(__at0) == 64 || (__have_avx512vl && sizeof(__at0) >= 16)) return _S_blend_avx512(__k._M_data, __at0._M_data, __at1._M_data); @@ -994,9 +996,8 @@ template } else _BitOps::_S_bit_iteration(_MaskImpl::_S_to_bits(__k), - [&](auto __i) { - __merge._M_set(__i, static_cast<_Tp>( - __mem[__i])); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + __merge._M_set(__i, static_cast<_Tp>(__mem[__i])); }); } /* Very uncertain, that the following improves anything. Needs @@ -1417,11 +1418,12 @@ template const auto __yf = __convert_all<_FloatV, __n_floatv>( _Abi::__make_padding_nonzero(__as_vector(__y))); return __call_with_n_evaluations<__n_floatv>( - [](auto... __quotients) { + [](auto... __quotients) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __vector_convert<_R>(__quotients...); }, - [&__xf, - &__yf](auto __i) -> _SimdWrapper<_Float, __n_intermediate> { + [&__xf, &__yf](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA + -> _SimdWrapper<_Float, __n_intermediate> + { #if !defined __clang__ && __GCC_IEC_559 == 0 // If -freciprocal-math is active, using the `/` operator is // incorrect because it may be translated to an imprecise @@ -1980,7 +1982,7 @@ template { auto __mask = __vector_bitcast<_UChar>( __vector_bitcast<_UShort>(__iy) << 5); - auto __maskl = [&]() { + auto __maskl = [&]() _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return __to_intrin(__vector_bitcast<_UShort>(__mask) << 8); }; auto __xh = __vector_bitcast(__ix); @@ -2067,19 +2069,20 @@ template } //}}} else if constexpr (sizeof(_Up) == 2 && sizeof(__x) >= 4) //{{{ { - [[maybe_unused]] auto __blend_0xaa = [](auto __a, auto __b) { - if constexpr (sizeof(__a) == 16) - return _mm_blend_epi16(__to_intrin(__a), __to_intrin(__b), - 0xaa); - else if constexpr (sizeof(__a) == 32) - return _mm256_blend_epi16(__to_intrin(__a), __to_intrin(__b), - 0xaa); - else if constexpr (sizeof(__a) == 64) - return _mm512_mask_blend_epi16(0xaaaa'aaaaU, __to_intrin(__a), - __to_intrin(__b)); - else - __assert_unreachable(); - }; + [[maybe_unused]] auto __blend_0xaa + = [](auto __a, auto __b) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + if constexpr (sizeof(__a) == 16) + return _mm_blend_epi16(__to_intrin(__a), __to_intrin(__b), + 0xaa); + else if constexpr (sizeof(__a) == 32) + return _mm256_blend_epi16(__to_intrin(__a), __to_intrin(__b), + 0xaa); + else if constexpr (sizeof(__a) == 64) + return _mm512_mask_blend_epi16(0xaaaa'aaaaU, __to_intrin(__a), + __to_intrin(__b)); + else + __assert_unreachable(); + }; if constexpr (__have_avx512bw_vl && sizeof(_Tp) <= 16) return __intrin_bitcast<_V>(is_signed_v<_Up> ? _mm_srav_epi16(__ix, __iy) @@ -2136,9 +2139,10 @@ template { auto __k = __vector_bitcast<_UShort>(__iy) << 11; auto __x128 = __vector_bitcast<_Up>(__ix); - auto __mask = [](__vector_type16_t<_UShort> __kk) { - return __vector_bitcast(__kk) < 0; - }; + auto __mask + = [](__vector_type16_t<_UShort> __kk) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return __vector_bitcast(__kk) < 0; + }; // do __x128 = 0 where __y[4] is set __x128 = __mask(__k) ? decltype(__x128)() : __x128; // do __x128 =>> 8 where __y[3] is set @@ -2178,7 +2182,7 @@ template } else { - auto __shift = [](auto __a, auto __b) { + auto __shift = [](auto __a, auto __b) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr (is_signed_v<_Up>) return _mm_sra_epi32(__a, __b); else @@ -3492,7 +3496,7 @@ struct _MaskImplX86Mixin return _S_to_maskvector<_Up, _ToN>(__k); else if (__x._M_is_constprop() || __builtin_is_constant_evaluated()) return __generate_from_n_evaluations( - [&](auto __i) -> _Up { return -__x[__i.value]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> _Up { return -__x[__i.value]; }); else if constexpr (sizeof(_Up) == 1) { if constexpr (sizeof(_UI) == 16) @@ -3737,9 +3741,9 @@ struct _MaskImplX86Mixin else if constexpr (__bits_per_element >= _ToN) { constexpr auto __bitmask - = __generate_vector<_V>([](auto __i) constexpr->_UpUInt { - return __i < _ToN ? 1ull << __i : 0; - }); + = __generate_vector<_V>([](auto __i) + constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> _UpUInt + { return __i < _ToN ? 1ull << __i : 0; }); const auto __bits = __vector_broadcast<_ToN, _UpUInt>(__k) & __bitmask; if constexpr (__bits_per_element > _ToN) @@ -3750,11 +3754,11 @@ struct _MaskImplX86Mixin else { const _V __tmp - = __generate_vector<_V>([&](auto __i) constexpr { + = __generate_vector<_V>([&](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_UpUInt>( __k >> (__bits_per_element * (__i / __bits_per_element))); }) - & __generate_vector<_V>([](auto __i) constexpr { + & __generate_vector<_V>([](auto __i) constexpr _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { return static_cast<_UpUInt>(1ull << (__i % __bits_per_element)); }); // mask bit index @@ -3790,7 +3794,7 @@ struct _MaskImplX86Mixin const auto __y = __vector_bitcast<__int_for_sizeof_t<_Tp>>(__x); return __generate_from_n_evaluations>( - [&](auto __i) -> _Up { return __y[__i.value]; }); + [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> _Up { return __y[__i.value]; }); } using _To = __vector_type_t<_Up, _ToN>; [[maybe_unused]] constexpr size_t _FromN = _Np; @@ -4125,8 +4129,11 @@ struct _MaskImplX86Mixin { const auto __bools = -__x._M_data; const _ULLong __k = __call_with_n_evaluations<_Np>( - [](auto... __bits) { return (__bits | ...); }, - [&](auto __i) { return _ULLong(__bools[+__i]) << __i; }); + [](auto... __bits) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return (__bits | ...); + }, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { + return _ULLong(__bools[+__i]) << __i; + }); if (__builtin_is_constant_evaluated() || __builtin_constant_p(__k)) return __k; @@ -4282,13 +4289,14 @@ template static_assert(is_same_v<_Tp, __int_for_sizeof_t<_Tp>>); if constexpr (__have_avx512bw) { - const auto __to_vec_or_bits = [](auto __bits) -> decltype(auto) { - if constexpr (__is_avx512_abi<_Abi>()) - return __bits; - else - return _S_to_maskvector<_Tp>( - _BitMask<_S_size<_Tp>>(__bits)._M_sanitized()); - }; + const auto __to_vec_or_bits + = [](auto __bits) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA -> decltype(auto) { + if constexpr (__is_avx512_abi<_Abi>()) + return __bits; + else + return _S_to_maskvector<_Tp>( + _BitMask<_S_size<_Tp>>(__bits)._M_sanitized()); + }; if constexpr (_S_size<_Tp> <= 16 && __have_avx512vl) { @@ -4475,7 +4483,7 @@ template } else { - _BitOps::_S_bit_iteration(__mask, [&](auto __i) { + _BitOps::_S_bit_iteration(__mask, [&](auto __i) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { __merge._M_set(__i, __mem[__i]); }); return __merge; @@ -4554,7 +4562,7 @@ template { if constexpr (__have_avx512bw_vl) _CommonImplX86::_S_store<_Np>( - __vector_bitcast([](auto __data) { + __vector_bitcast([](auto __data) _GLIBCXX_SIMD_ALWAYS_INLINE_LAMBDA { if constexpr (_Np <= 16) return _mm_maskz_set1_epi8(__data, 1); else if constexpr (_Np <= 32) From patchwork Wed Feb 15 20:49:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57720 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp411017wrn; Wed, 15 Feb 2023 12:51:53 -0800 (PST) X-Google-Smtp-Source: AK7set/UHBFi9t/LGMKeUR/28wi7KKeEZvFGEaAjJKXgtKlkPdC4yB3BhhkFEKYoArmgluS6oPVT X-Received: by 2002:a17:906:3943:b0:87b:1be:a8c2 with SMTP id g3-20020a170906394300b0087b01bea8c2mr3769681eje.73.1676494313251; Wed, 15 Feb 2023 12:51:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494313; cv=none; d=google.com; s=arc-20160816; b=Tm1+hMMxzhwx6HChIsnkJOMj0SFc93a+SdCPtN8LApoQhRex3Wj2uUr8Rjjn/V1a7S petH25vA1MwlmOxBcx8pVIQK2MHdDMp7sarOEgESAt9Yx9FLxd6Y6BPH1k3mT2dCO0Iq BZPFRqJAcfMVpDfAP10lwMObc69/qwZc3VppSbBxhve8R9n5pQx/ExBgboQNNIeleMH9 nxCUWIVl//gYxvlY5h/Oz76M8RIVr9rzuGoMIMOSNDC/fZEeNzPKPmNwviTt9tyw5Xxl YyF55y3nzDn/KeAzjRCS7JZoEK8uiTN2uSsvYrLdBPO0Y/9Lwk3MXbnjWZHeu4AMrOEm +Ylg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=RKMR1CgsLWDVCTqHddpi88DArRB+troce0vZ9rDhHMA=; b=KqEEF4QD3AO5+OcbCoRlbUv65cOVWiA49aT3jB8m3WfgSvrUNQKaLHsYG1fkqe+bgO +oOMfTwpolLhYYwSe87tMPIMT2CJ+qlL5qNlYJea6MesNQ5J2CGHzvyGK/dChAME8JkV brTNDqGgHMnrkxeVpE9PYlftxQ4kzObnj8eO6MUtgvR+6tI4Nz/r8AYUrZY1DmXKevBb s98tTDsXPnfrTpVqq/RV4cHXoOsfR5NhG3JtXGRkDLT84yBxvsQ8d/FycrYg9oUj5oCO NeFmIynFgcqIfhGtOXFW4hD4Pu0Ir/F6izS4f+IjZyybGkJLvoxmKgIi1V91K0CTo2C1 BP3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=FPtBLm2h; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id 28-20020a170906005c00b0087bdb2f681csi25924320ejg.19.2023.02.15.12.51.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:51:53 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=FPtBLm2h; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 681C23881D05 for ; Wed, 15 Feb 2023 20:50:45 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 681C23881D05 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494245; bh=RKMR1CgsLWDVCTqHddpi88DArRB+troce0vZ9rDhHMA=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=FPtBLm2hgFpg3YVa4F+jslaNc+RscXZ0fVXByJS/v8FTmfq7YaQ04RNZjQCNL7Q6Y +S5L7PY0BfNH3ND55MoDz3UCuL2jr7LtQrWT2HNRER0q6iJWWgH/uGF+HBh2BJIcWz 0KUSvDb4DA6GvBWetA4uIAnRmbvIkoTh8jlVTtH0= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id ED1AC385DC3C; Wed, 15 Feb 2023 20:49:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org ED1AC385DC3C Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id 21D60202AD5E; Wed, 15 Feb 2023 21:49:54 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id wMMxygg0FUnz; Wed, 15 Feb 2023 21:49:54 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id 04AD4202AD51; Wed, 15 Feb 2023 21:49:54 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:49:52 +0100 To: , Subject: [PATCH 3/7] libstdc++: Document timeout and timeout-factor of simd tests Date: Wed, 15 Feb 2023 21:49:51 +0100 Message-ID: <8239109.NyiUUSuA9g@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvEX7.Campus.gsi.de (10.10.4.97) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931700759290818?= X-GMAIL-MSGID: =?utf-8?q?1757931700759290818?= Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * testsuite/experimental/simd/README.md: Document the timeout and timeout-factor directives. Minor typo fixed. --- libstdc++-v3/testsuite/experimental/simd/README.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/testsuite/experimental/simd/README.md b/libstdc++-v3/testsuite/experimental/simd/README.md index b82453df403..ef8b7c33de7 100644 --- a/libstdc++-v3/testsuite/experimental/simd/README.md +++ b/libstdc++-v3/testsuite/experimental/simd/README.md @@ -139,7 +139,13 @@ allowed_distance)` macros. test then shows as "XFAIL: ...". If the test passes, the test shows "XPASS: ...". -All patterns are matched via +* `timeout: ` + Set the timeout of this test to `` seconds. + +* `timeout-factor: ` + Multiply the default timeout with ``. + +All patterns except `timeout` and `timeout-factor` are matched via ```sh case '' in ) @@ -147,7 +153,7 @@ case '' in ;; esac ``` -The `` is implicitly adds a `*` wildcard before and after the +The `` implicitly adds a `*` wildcard before and after the pattern. Thus, the `CXXFLAGS` pattern matches a substring and all other patterns require a full match. From patchwork Wed Feb 15 20:49:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57724 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp412495wrn; Wed, 15 Feb 2023 12:55:25 -0800 (PST) X-Google-Smtp-Source: AK7set+bg6oAaea8/SoRTlAUSnrXIRcAfvo6ZObRBdqdpo802SSoKrtuJd/bLz6aF4eeLCn3vqIm X-Received: by 2002:a05:6402:520d:b0:4ad:23f:90a7 with SMTP id s13-20020a056402520d00b004ad023f90a7mr691722edd.13.1676494525166; Wed, 15 Feb 2023 12:55:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494525; cv=none; d=google.com; s=arc-20160816; b=0L+BGGiuIhR5QqhpUD+4U82e8UbYIR2tWVj4wSb8hM23jEnzw64tlLyJwLOl7eHWn4 xaOIJBQO4pGspzBz1NfNwbfdCwPOPvBINsury76bBeLA/s9Mg7QRo2Ip84qZXJKGbT8V vKAtKuZZWx+hUg6ld+h6N9Jf7VagYXdkfaxzOCCqKXbrjj8E7P+q4asWB0oCZ7orqInv 0pLJ+g4d5CoTl4hc3dOtyONeFDvcHnZ1r/ALglxHvUs5yKuzlYtjCXVw0kVKYnn07ZW0 c6INfgqsaMulOWIm2QcS2qB0Qt05CPdOUuvBhxgER946eL1kh4qwQ+rba8QVaCk42WdV cqfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=yO0RWBBguC9OfzKKQ+R4DfuL1QB7i1AEz3QtG8HG0DU=; b=QziskhG4suDcubpfLeIEvo8Q7McNM8U0od+4+XNG6dkJuIuIHnjNHzUQn4C5XpO03l ytXpYrJLKu9UtzB3nH4arsQFfKztdLec/VooxuTJELSwBHFMv0XdhSPFn1ufDfuwEJRr 7LMbvDHcxYRMYQlI/zaQaw/rEfap2BO9ZKdK5ZibHAlylB86eeAI7wpZrE4QGU65OqX+ UZ/nXh0eEFhCbZObQGpiqM/YOe+fTy/Vc//f3OZ+Hz0pujftd3VY4Tky6dJ352mv3OLV Bt9RcT6zNNfoTZ3utlzSnf5VJUe8NyoRhfQURFlRzRyPaZ7OvTIGBrgoAtZQbCPXbOfH SawA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=w2g5RpDV; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id n22-20020a056402061600b004acbdf2bc05si14334091edv.511.2023.02.15.12.55.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:55:25 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=w2g5RpDV; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 23B4D3945C11 for ; Wed, 15 Feb 2023 20:52:08 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 23B4D3945C11 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494328; bh=yO0RWBBguC9OfzKKQ+R4DfuL1QB7i1AEz3QtG8HG0DU=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=w2g5RpDV7jw5dWxXGy6Yu9lI+7k5KOATRA1di5Vzlp2NFQOKZ2AjGtjK2j2cfBc1D swKjf7AP/OwCfMap0NJTMzjW4uFpeqYhPl2aT4a40e3YEoDUZa0GxBbunrVhUKKagh 1xBCXiZkQ3IjF6uVIT669yLN2t+MIQqrq9B6Ttok= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id EC83F3864812; Wed, 15 Feb 2023 20:50:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EC83F3864812 Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id 17338202AD5E; Wed, 15 Feb 2023 21:50:00 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 4KIe8TTjc7Rh; Wed, 15 Feb 2023 21:49:59 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id A2B8F202AD51; Wed, 15 Feb 2023 21:49:59 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:49:59 +0100 To: , Subject: [PATCH 4/7] libstdc++: Use a PCH to speed up check-simd Date: Wed, 15 Feb 2023 21:49:58 +0100 Message-ID: <22986674.6Emhk5qWAg@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvex5.Campus.gsi.de (10.10.4.95) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931923302546361?= X-GMAIL-MSGID: =?utf-8?q?1757931923302546361?= Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * testsuite/experimental/simd/generate_makefile.sh: Generate and pre-compile pch.h, which includes all headers that do not depend on command-line macros. * testsuite/experimental/simd/tests/bits/conversions.h: Add include guard. (genHalfBits): Simplify. * testsuite/experimental/simd/tests/bits/make_vec.h: Add include guard. (make_alternating_mask): Moved from mask_loadstore. * testsuite/experimental/simd/tests/bits/mathreference.h: Add include guard. * testsuite/experimental/simd/tests/bits/test_values.h: Ditto. * testsuite/experimental/simd/tests/mask_loadstore.cc (make_mask, make_alternating_mask): Removed. * testsuite/experimental/simd/tests/mask_reductions.cc: Ditto. * testsuite/experimental/simd/tests/operators.cc (genHalfBits): Removed. * testsuite/experimental/simd/tests/abs.cc: Only include bits/main.h. * testsuite/experimental/simd/tests/algorithms.cc: Ditto. * testsuite/experimental/simd/tests/broadcast.cc: Ditto. * testsuite/experimental/simd/tests/casts.cc: Ditto. * testsuite/experimental/simd/tests/fpclassify.cc: Ditto. * testsuite/experimental/simd/tests/frexp.cc: Ditto. * testsuite/experimental/simd/tests/generator.cc: Ditto. * testsuite/experimental/simd/tests/hypot3_fma.cc: Ditto. * testsuite/experimental/simd/tests/integer_operators.cc: Ditto. * testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc: Ditto. * testsuite/experimental/simd/tests/loadstore.cc: Ditto. * testsuite/experimental/simd/tests/logarithm.cc: Ditto. * testsuite/experimental/simd/tests/mask_broadcast.cc: Ditto. * testsuite/experimental/simd/tests/mask_implicit_cvt.cc: Ditto. * testsuite/experimental/simd/tests/mask_operator_cvt.cc: Ditto. * testsuite/experimental/simd/tests/mask_operators.cc: Ditto. * testsuite/experimental/simd/tests/math_1arg.cc: Ditto. * testsuite/experimental/simd/tests/math_2arg.cc: Ditto. * testsuite/experimental/simd/tests/operator_cvt.cc: Ditto. * testsuite/experimental/simd/tests/reductions.cc: Ditto. * testsuite/experimental/simd/tests/remqo.cc: Ditto. * testsuite/experimental/simd/tests/sincos.cc: Ditto. * testsuite/experimental/simd/tests/split_concat.cc: Ditto. * testsuite/experimental/simd/tests/trigonometric.cc: Ditto. * testsuite/experimental/simd/tests/trunc_ceil_floor.cc: Ditto. * testsuite/experimental/simd/tests/where.cc: Ditto. --- .../experimental/simd/generate_makefile.sh | 24 ++++- .../testsuite/experimental/simd/tests/abs.cc | 4 +- .../experimental/simd/tests/algorithms.cc | 3 +- .../simd/tests/bits/conversions.h | 25 ++---- .../experimental/simd/tests/bits/main.h | 87 +++++++++++++++++++ .../experimental/simd/tests/bits/make_vec.h | 10 +++ .../simd/tests/bits/mathreference.h | 3 + .../simd/tests/bits/test_values.h | 6 ++ .../experimental/simd/tests/bits/verify.h | 64 -------------- .../experimental/simd/tests/broadcast.cc | 3 +- .../experimental/simd/tests/casts.cc | 4 +- .../experimental/simd/tests/fpclassify.cc | 4 +- .../experimental/simd/tests/frexp.cc | 4 +- .../experimental/simd/tests/generator.cc | 3 +- .../experimental/simd/tests/hypot3_fma.cc | 4 +- .../simd/tests/integer_operators.cc | 5 +- .../simd/tests/ldexp_scalbn_scalbln_modf.cc | 4 +- .../experimental/simd/tests/loadstore.cc | 4 +- .../experimental/simd/tests/logarithm.cc | 5 +- .../experimental/simd/tests/mask_broadcast.cc | 3 +- .../simd/tests/mask_conversions.cc | 2 +- .../simd/tests/mask_implicit_cvt.cc | 3 +- .../experimental/simd/tests/mask_loadstore.cc | 29 +------ .../simd/tests/mask_operator_cvt.cc | 3 +- .../experimental/simd/tests/mask_operators.cc | 3 +- .../simd/tests/mask_reductions.cc | 30 +------ .../experimental/simd/tests/math_1arg.cc | 3 +- .../experimental/simd/tests/math_2arg.cc | 4 +- .../experimental/simd/tests/operator_cvt.cc | 3 +- .../experimental/simd/tests/operators.cc | 14 +-- .../experimental/simd/tests/reductions.cc | 4 +- .../experimental/simd/tests/remqo.cc | 4 +- .../testsuite/experimental/simd/tests/simd.cc | 2 +- .../experimental/simd/tests/sincos.cc | 6 +- .../experimental/simd/tests/split_concat.cc | 4 +- .../experimental/simd/tests/splits.cc | 2 +- .../experimental/simd/tests/trigonometric.cc | 4 +- .../simd/tests/trunc_ceil_floor.cc | 3 +- .../experimental/simd/tests/where.cc | 4 +- 39 files changed, 170 insertions(+), 226 deletions(-) create mode 100644 libstdc++-v3/testsuite/experimental/simd/tests/bits/ main.h -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh b/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh index ce5162a6ceb..9fb33e4cbd2 100755 --- a/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh +++ b/libstdc++-v3/testsuite/experimental/simd/generate_makefile.sh @@ -75,6 +75,7 @@ while [ $# -gt 0 ]; do done mkdir -p "$dst" +pch="$dst/pch.h" dst="$dst/Makefile" if [ -f "$dst" ]; then echo "Error: $dst already exists. Aborting." 1>&2 @@ -84,9 +85,23 @@ fi CXX="$1" shift +cat >> "$pch" <> "$dst" -echo CXXFLAGS = "$@" "\$(test_flags)" >> "$dst" +echo CXXFLAGS = "$@" -include pch.h "\$(test_flags)" >> "$dst" [ -n "$sim" ] && echo "export GCC_TEST_SIMULATOR = $sim" >> "$dst" cat >> "$dst" <. // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" #include // abs & sqrt #include // integer abs -#include "bits/test_values.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/algorithms.cc b/libstdc++-v3/testsuite/experimental/simd/tests/algorithms.cc index dac0981622e..6242eb89337 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/algorithms.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/algorithms.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/conversions.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/conversions.h index 2b12de6b4b2..7bd70cea53b 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/conversions.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/conversions.h @@ -15,6 +15,8 @@ // with this library; see the file COPYING3. If not see // . +#ifndef SIMD_TESTS_BITS_CONVERSIONS_H_ +#define SIMD_TESTS_BITS_CONVERSIONS_H_ #include // is_conversion_undefined @@ -86,22 +88,12 @@ template template constexpr T genHalfBits() - { return std::__finite_max_v >> (std::__digits_v / 2); } - -template <> - constexpr long double - genHalfBits() - { return 0; } - -template <> - constexpr double - genHalfBits() - { return 0; } - -template <> - constexpr float - genHalfBits() - { return 0; } + { + if constexpr (std::is_floating_point_v) + return 0; + else + return std::__finite_max_v >> (std::__digits_v / 2); + } template constexpr U @@ -182,3 +174,4 @@ template operator[](size_t i) const { return cvt_input_data[i]; } }; +#endif // SIMD_TESTS_BITS_CONVERSIONS_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/main.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/main.h new file mode 100644 index 00000000000..270b433aa17 --- /dev/null +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/main.h @@ -0,0 +1,87 @@ +// Copyright (C) 2020-2023 Free Software Foundation, Inc. +// +// This file is part of the GNU ISO C++ Library. This library is free +// software; you can redistribute it and/or modify it under the +// terms of the GNU General Public License as published by the +// Free Software Foundation; either version 3, or (at your option) +// any later version. +// +// This library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this library; see the file COPYING3. If not see +// . + +#ifndef TESTS_BITS_MAIN_H_ +#define TESTS_BITS_MAIN_H_ + +#include "verify.h" + +template + void + iterate_abis() + { + using namespace std::experimental::parallelism_v2; +#ifndef EXTENDEDTESTS + invoke_test>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 0 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 1 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 2 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 3 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 4 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 5 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 6 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 7 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#elif EXTENDEDTESTS == 8 + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); + invoke_test>>(int()); +#endif + } + +int main() +{ + iterate_abis<_GLIBCXX_SIMD_TESTTYPE>(); + return 0; +} + +#endif // TESTS_BITS_MAIN_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/make_vec.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/make_vec.h index 82e63d18ad1..3744965f439 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/make_vec.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/make_vec.h @@ -15,6 +15,8 @@ // with this library; see the file COPYING3. If not see // . +#ifndef SIMD_TESTS_BITS_MAKE_VEC_H_ +#define SIMD_TESTS_BITS_MAKE_VEC_H_ #include template @@ -36,6 +38,13 @@ template } } +template + M + make_alternating_mask() + { + return make_mask({false, true}); + } + template inline V make_vec(const std::initializer_list &init, @@ -57,3 +66,4 @@ template base += inc; } } +#endif // SIMD_TESTS_BITS_MAKE_VEC_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/mathreference.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/mathreference.h index 1553aa7bb51..87c4df7b176 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/mathreference.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/mathreference.h @@ -15,6 +15,8 @@ // with this library; see the file COPYING3. If not see // . +#ifndef SIMD_TESTS_BITS_MATHREFERENCE_H_ +#define SIMD_TESTS_BITS_MATHREFERENCE_H_ #include #include #include @@ -158,3 +160,4 @@ template > } return data; } +#endif // SIMD_TESTS_BITS_MATHREFERENCE_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/test_values.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/test_values.h index ccbe19de4b0..c3bd082f709 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/test_values.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/test_values.h @@ -15,6 +15,11 @@ // with this library; see the file COPYING3. If not see // . +#ifndef SIMD_TESTS_BITS_TEST_VALUES_H_ +#define SIMD_TESTS_BITS_TEST_VALUES_H_ + +#include "verify.h" + #include #include #include @@ -381,3 +386,4 @@ template #endif #define MAKE_TESTER(name_) MAKE_TESTER_2(name_, std::name_) +#endif // SIMD_TESTS_BITS_TEST_VALUES_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h index e60ff93d2e6..2ab3ad3fa8c 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h @@ -286,68 +286,4 @@ template __builtin_fprintf(stderr, "PASS: %s\n", __PRETTY_FUNCTION__); } -template - void - iterate_abis() - { - using namespace std::experimental::parallelism_v2; -#ifndef EXTENDEDTESTS - invoke_test>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 0 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 1 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 2 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 3 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 4 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 5 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 6 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 7 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#elif EXTENDEDTESTS == 8 - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); - invoke_test>>(int()); -#endif - } - -int main() -{ - iterate_abis<_GLIBCXX_SIMD_TESTTYPE>(); - return 0; -} - #endif // TESTS_BITS_VERIFY_H_ diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/broadcast.cc b/libstdc++-v3/testsuite/experimental/simd/tests/broadcast.cc index 7e06c6632d8..6265e31b079 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/broadcast.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/broadcast.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" enum unscoped_enum { foo }; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/casts.cc b/libstdc++-v3/testsuite/experimental/simd/tests/casts.cc index 022a583c331..8e8d828c756 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/casts.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/casts.cc @@ -16,9 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/conversions.h" +#include "bits/main.h" using std::experimental::simd_cast; using std::experimental::static_simd_cast; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/fpclassify.cc b/libstdc++-v3/testsuite/experimental/simd/tests/fpclassify.cc index 453f17f256e..00c608f9530 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/fpclassify.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/fpclassify.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" #include template diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/frexp.cc b/libstdc++-v3/testsuite/experimental/simd/tests/frexp.cc index 5e04f4f2927..f6a47cedd13 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/frexp.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/frexp.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/generator.cc b/libstdc++-v3/testsuite/experimental/simd/tests/generator.cc index b582c04067b..001625cfd26 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/generator.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/generator.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template struct call_generator diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/hypot3_fma.cc b/libstdc++-v3/testsuite/experimental/simd/tests/hypot3_fma.cc index b5e3749ebc4..6f0d75ac007 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/hypot3_fma.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/hypot3_fma.cc @@ -18,9 +18,7 @@ // only: float|double|ldouble * * * // skip: ldouble * powerpc64* * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" // 3-arg std::hypot needs to be fixed, this is a better reference: template diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/integer_operators.cc b/libstdc++-v3/testsuite/experimental/simd/tests/integer_operators.cc index e3718ce40a3..7a2bc085e49 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/integer_operators.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/integer_operators.cc @@ -16,9 +16,8 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/make_vec.h" -#include "bits/metahelpers.h" +// timeout-factor: 2 +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc b/libstdc++-v3/testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc index 1943c360077..0fb1338fc04 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/ldexp_scalbn_scalbln_modf.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/loadstore.cc b/libstdc++-v3/testsuite/experimental/simd/tests/loadstore.cc index f354eff9139..f7933007cc7 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/loadstore.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/loadstore.cc @@ -17,9 +17,7 @@ // expensive: * [1-9] * * // timeout-factor: 2 -#include "bits/verify.h" -#include "bits/make_vec.h" -#include "bits/conversions.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/logarithm.cc b/libstdc++-v3/testsuite/experimental/simd/tests/logarithm.cc index d7f4297c094..31ad1499e00 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/logarithm.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/logarithm.cc @@ -17,10 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/mathreference.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_broadcast.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_broadcast.cc index 0ed56e2d2ee..2f601368775 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_broadcast.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_broadcast.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_conversions.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_conversions.cc index 13afd69348e..8cdd7b74b44 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_conversions.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_conversions.cc @@ -16,7 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" +#include "bits/main.h" namespace stdx = std::experimental; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_implicit_cvt.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_implicit_cvt.cc index 0003db3114d..a7b6c92e2a0 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_implicit_cvt.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_implicit_cvt.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template constexpr bool assign_should_work diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_loadstore.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_loadstore.cc index b9f3cef021d..d8e58a0032b 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_loadstore.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_loadstore.cc @@ -16,34 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" - -// simd_mask generator functions -template - M - make_mask(const std::initializer_list& init) - { - std::size_t i = 0; - M r = {}; - for (;;) - { - for (bool x : init) - { - r[i] = x; - if (++i == M::size()) - { - return r; - } - } - } - } - -template - M - make_alternating_mask() - { - return make_mask({false, true}); - } +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_operator_cvt.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_operator_cvt.cc index b91832a431c..1c7ccab5836 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_operator_cvt.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_operator_cvt.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" using schar = signed char; using uchar = unsigned char; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_operators.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_operators.cc index b42520202a0..70b82a514bb 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_operators.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_operators.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/mask_reductions.cc b/libstdc++-v3/testsuite/experimental/simd/tests/mask_reductions.cc index b863f2d5c0c..1e9ffb7aca5 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/mask_reductions.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/mask_reductions.cc @@ -16,35 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" - -// simd_mask generator functions -template - M - make_mask(const std::initializer_list& init) - { - std::size_t i = 0; - M r = {}; - for (;;) - { - for (bool x : init) - { - r[i] = x; - if (++i == M::size()) - { - return r; - } - } - } - } - -template - M - make_alternating_mask() - { - return make_mask({false, true}); - } +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/math_1arg.cc b/libstdc++-v3/testsuite/experimental/simd/tests/math_1arg.cc index db401c124f2..50a740dec81 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/math_1arg.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/math_1arg.cc @@ -17,8 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/math_2arg.cc b/libstdc++-v3/testsuite/experimental/simd/tests/math_2arg.cc index 08b36c60331..1abf18128fb 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/math_2arg.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/math_2arg.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/operator_cvt.cc b/libstdc++-v3/testsuite/experimental/simd/tests/operator_cvt.cc index 921917e4d88..c1acfdf10e0 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/operator_cvt.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/operator_cvt.cc @@ -16,8 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" +#include "bits/main.h" // type with sizeof(char) but different signedness using xchar = std::conditional_t, schar, uchar>; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/operators.cc b/libstdc++-v3/testsuite/experimental/simd/tests/operators.cc index 6d482201fab..c4d91fa5d2b 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/operators.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/operators.cc @@ -16,19 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/make_vec.h" -#include "bits/test_values.h" - -template - constexpr T - genHalfBits() - { - if constexpr (std::is_floating_point_v) - return 0; - else - return std::__finite_max_v >> (std::__digits_v / 2); - } +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/reductions.cc b/libstdc++-v3/testsuite/experimental/simd/tests/reductions.cc index 62072e72155..0c4c79feb20 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/reductions.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/reductions.cc @@ -16,9 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" #include template diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/remqo.cc b/libstdc++-v3/testsuite/experimental/simd/tests/remqo.cc index 8c76f66b842..bc9f94d8265 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/remqo.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/remqo.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/simd.cc b/libstdc++-v3/testsuite/experimental/simd/tests/simd.cc index 14fc71ac1a4..b729c97bec9 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/simd.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/simd.cc @@ -16,7 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/sincos.cc b/libstdc++-v3/testsuite/experimental/simd/tests/sincos.cc index 16637f50cfc..2e81a8e5585 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/sincos.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/sincos.cc @@ -18,11 +18,7 @@ // only: float|double|ldouble * * * // xfail: run * * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/mathreference.h" -#include "bits/simd_view.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/split_concat.cc b/libstdc++-v3/testsuite/experimental/simd/tests/split_concat.cc index 85ca7ae8199..102229b70f1 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/split_concat.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/split_concat.cc @@ -16,9 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/conversions.h" +#include "bits/main.h" using std::experimental::simd_cast; diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/splits.cc b/libstdc++-v3/testsuite/experimental/simd/tests/splits.cc index 94c82ecc3c3..f87bce8a7dc 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/splits.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/splits.cc @@ -16,7 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/trigonometric.cc b/libstdc++-v3/testsuite/experimental/simd/tests/trigonometric.cc index 8e16ce4a1cf..a678ff48fee 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/trigonometric.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/trigonometric.cc @@ -17,9 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/metahelpers.h" -#include "bits/test_values.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/trunc_ceil_floor.cc b/libstdc++-v3/testsuite/experimental/simd/tests/trunc_ceil_floor.cc index 51575df48cb..ecbc1481600 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/trunc_ceil_floor.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/trunc_ceil_floor.cc @@ -17,8 +17,7 @@ // only: float|double|ldouble * * * // expensive: * [1-9] * * -#include "bits/test_values.h" -#include "bits/verify.h" +#include "bits/main.h" template void diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/where.cc b/libstdc++-v3/testsuite/experimental/simd/tests/where.cc index 79cc949c64e..b6f6d333191 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/where.cc +++ b/libstdc++-v3/testsuite/experimental/simd/tests/where.cc @@ -16,9 +16,7 @@ // . // expensive: * [1-9] * * -#include "bits/verify.h" -#include "bits/make_vec.h" -#include "bits/metahelpers.h" +#include "bits/main.h" template struct Convertible From patchwork Wed Feb 15 20:50:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57722 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp412112wrn; Wed, 15 Feb 2023 12:54:34 -0800 (PST) X-Google-Smtp-Source: AK7set+rtru+Q4c1ifjXvTBSgm3kAHcwYMnAAaIsN8l3lhpbGl4y+HCiOYokAxC66BOTAneMwQI7 X-Received: by 2002:a05:6402:120e:b0:4aa:cc30:b858 with SMTP id c14-20020a056402120e00b004aacc30b858mr3656084edw.7.1676494474508; Wed, 15 Feb 2023 12:54:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494474; cv=none; d=google.com; s=arc-20160816; b=suzec752wzpTA6PbnJQKfa1lb5bfs5zZCgHX3h42/k5/3m5c01XpfQ8BPy6aEGUZCp lqU/e2Zg3fgeFhdZ+T9M87GOS9ErUKPb+P/onTlvFdsoC+XoKARqlRDaBptfdTaBNAbn gFDqbKj0c2EQRUB4/d/wBFNx/YOaw9hGJeJZe/D4kDfWaRdTkrIfA5Q7qQASurdi8bZD 0aBjt/Jgxtrm9WrYEJqvjaGa34nWcwrfP3i01KTYbR9ECaNDanHlb25G5bm1GWGKRmcC +j4hiPiNM3rg4plDFxcj9Y/NuxeBemLACZzH6UP76d2RXOPJl3mil08vmGPrz/XWvTk+ LmaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=FMbQ4gXlR8gn4jmIJV1fyqEBI5FeuQa1oYVmjgAF0Nk=; b=lWNJH9nRA87TRYyLGMcVbQQK9nTA3pl9WrztpBU9PXnjmhlOzWEaUkSKu6/PlAR/8r mNSKoZJDL/rFi9e8QaJkeWOD9eVo2f/TZmi4tHbiLJlEmwCkYZ3svqIEmQDiN1E3sjn2 XveBTlIMxi3udGh1P4NuR9YiB+ZcMvVdoe1JmntFT50lqD8PGYZejcbIOHngV7z3VjQ6 qfUMpMeZnR8dyNQ7Kzb8bkNbkT5B3UNvA8y/TYtUCeJPNIuWJ8q1w8nkHgAiO134wioZ WdOfAjk+BGJ2QyM1oGKSkTbSUvm0FEWd0MGlQstlZCGpnAnZI11+G7sa4ukPcUGrN6D4 FTXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=HinUQ85h; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id z5-20020aa7cf85000000b004ab4bb74439si18726975edx.411.2023.02.15.12.54.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:54:34 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=HinUQ85h; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6097538A9098 for ; Wed, 15 Feb 2023 20:51:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6097538A9098 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494308; bh=FMbQ4gXlR8gn4jmIJV1fyqEBI5FeuQa1oYVmjgAF0Nk=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=HinUQ85hlkA/Y5lGCGTpGU1XszjkRr1r4eo3d0k/8VLB90TQmMZjZPwjEWtmG+duA 2p7GXevLlI9FUXQbjwliPSmFtrULXRwurjjQZk938Lo4cwbdRqIuQICbpXORW6gPGn OpamKBvctgrmUP0RFbI+CGvhS9x0S86ks3N7Hl24= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id C10AF385B539; Wed, 15 Feb 2023 20:50:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C10AF385B539 Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id 0066F202AD62; Wed, 15 Feb 2023 21:50:04 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 70R2bMxHF-pw; Wed, 15 Feb 2023 21:50:03 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id DA401202AD51; Wed, 15 Feb 2023 21:50:03 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:50:03 +0100 To: , Subject: [PATCH 5/7] libstdc++: printf format string fix in testsuite Date: Wed, 15 Feb 2023 21:50:03 +0100 Message-ID: <3557908.R56niFO833@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvEX6.Campus.gsi.de (10.10.4.96) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931870270567080?= X-GMAIL-MSGID: =?utf-8?q?1757931870270567080?= Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * testsuite/experimental/simd/tests/bits/verify.h (verify::verify): Use %zx for size_t in format string. --- libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h b/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h index 2ab3ad3fa8c..01ad50bd01a 100644 --- a/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h +++ b/libstdc++-v3/testsuite/experimental/simd/tests/bits/verify.h @@ -137,7 +137,7 @@ public: { if (m_failed) [&] { - __builtin_fprintf(stderr, "%s:%d: (%s):\nInstruction Pointer: %x\n" + __builtin_fprintf(stderr, "%s:%d: (%s):\nInstruction Pointer: %zx\n" "Assertion '%s' failed.\n", file, line, func, m_ip, cond); (print(extra_info, int()), ...); From patchwork Wed Feb 15 20:50:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57725 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp413258wrn; Wed, 15 Feb 2023 12:57:35 -0800 (PST) X-Google-Smtp-Source: AK7set/UEhEd39E73b0rTG4sx2nhxeEbshnpz6aPpmLX3JihrtHGZ9rsA0VWzrZ3Z20JHQpTILnv X-Received: by 2002:a17:906:79d0:b0:860:c12c:14f9 with SMTP id m16-20020a17090679d000b00860c12c14f9mr3629602ejo.40.1676494655079; Wed, 15 Feb 2023 12:57:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494655; cv=none; d=google.com; s=arc-20160816; b=aEnBw1Ww6JiuEjpNRsgHyqsh5VWdP3iicLSb+RHWyA9VpeIcOYA+2ho6A0Iykj0JAt D/khohvOg+v5cVnS2Ff2x4SYtk81F+EX2Iu1Vacrnx8/Q0+EV9WYk8dx5pI3KZxiRR5o 9sOG4TXjBR08j324rIAZjYdZeBs+TGJQi8Cuu84JjHBL6Ja7PjSS/hqTaky0bMJlxGHE fPORAPrAiK/Wr0kuccnIpIuwyO0h91LaQjq0ju3swIz0G05M5GeCFvZBlg1jeJjuVZGR VZwHkGH3XVi1+U86Ua2xrAS0Uz81b3c1vrY1mingezGJ4OsVPOLkszLrxiDKzurW7uA0 /exA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=vsiWBVaqTW36WJF2uQ+UG7ny5T2JvptPguE6Kpkcc4E=; b=eJa0FGn3y7vG5f0eFPcrva1fqPhEcsRXWQUOpaP6RSFvLDy21NJcJsK5t0ilbQd/OC n59Sarj76hbXPWVhDC0+5BiIB/OQzJExv6dfnZx+q6pMGPvhWu1Yh9GfnkEY6vClmbPM pOScoj4xmRX75UDUT7FLzjktL07tP1MeoJv1NZ+M7aCpOALcSPY+gko4UIXMlDUrckv6 4LhAg7gS4gkEtMcLGCVdDGk3mdsrZXX5HJPRMkO5qj6Nu5fgRTTm03lA7yVd3XG6Ef4c Q8TqqF7OR1h95uH/lNShVNG6E1BBdfaZBxHJgVzp50u70GnGBhrw5+XgbLowaQhLYYW6 wPVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=r1kuiMi1; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id 1-20020a170906100100b008b12654c973si8836393ejm.269.2023.02.15.12.57.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:57:35 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=r1kuiMi1; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3AE833882FC6 for ; Wed, 15 Feb 2023 20:53:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3AE833882FC6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494424; bh=vsiWBVaqTW36WJF2uQ+UG7ny5T2JvptPguE6Kpkcc4E=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=r1kuiMi1xLCAZvpYnOQ4XEuwBgDEbWbHqpwjY//U2YriB+Pvf6agG81SvIz79ENC/ Uq6tE+rUxhsSXUlGpJa5QrvyQ/ttm0uhG8vI4ULnG7z2Plt0A35kxov+M5H/LUFv9M 4VY9xy5lSGFNhBQlL5k3CC2UhFR05hpY6m+ujgWs= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id 6655B38493FD; Wed, 15 Feb 2023 20:50:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6655B38493FD Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id 9F9C6202AD5E; Wed, 15 Feb 2023 21:50:08 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id d0jQKvauu6A0; Wed, 15 Feb 2023 21:50:08 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id 80100202AD51; Wed, 15 Feb 2023 21:50:08 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:50:08 +0100 To: , Subject: [PATCH 6/7] libstdc++: Fix incorrect __builtin_is_constant_evaluated calls Date: Wed, 15 Feb 2023 21:50:07 +0100 Message-ID: <45331655.fMDQidcC6G@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvex5.Campus.gsi.de (10.10.4.95) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757932059606470788?= X-GMAIL-MSGID: =?utf-8?q?1757932059606470788?= Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * include/experimental/bits/simd_x86.h (_SimdImplX86::_S_not_equal_to, _SimdImplX86::_S_less) (_SimdImplX86::_S_less_equal): Do not call __builtin_is_constant_evaluated in constexpr-if. --- .../include/experimental/bits/simd_x86.h | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/include/experimental/bits/simd_x86.h b/libstdc++-v3/include/experimental/bits/simd_x86.h index 60e80d394ba..dcfdc2a9496 100644 --- a/libstdc++-v3/include/experimental/bits/simd_x86.h +++ b/libstdc++-v3/include/experimental/bits/simd_x86.h @@ -2344,15 +2344,16 @@ template else __assert_unreachable<_Tp>(); } // }}} - else if constexpr (!__builtin_is_constant_evaluated() // {{{ - && sizeof(__x) == 8) + else if (__builtin_is_constant_evaluated()) + return _Base::_S_not_equal_to(__x, __y); + else if constexpr (sizeof(__x) == 8) { const auto __r128 = __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__x) != __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__y); _MaskMember<_Tp> __r64; __builtin_memcpy(&__r64._M_data, &__r128, sizeof(__r64)); return __r64; - } // }}} + } else return _Base::_S_not_equal_to(__x, __y); } @@ -2451,15 +2452,16 @@ template else __assert_unreachable<_Tp>(); } // }}} - else if constexpr (!__builtin_is_constant_evaluated() // {{{ - && sizeof(__x) == 8) + else if (__builtin_is_constant_evaluated()) + return _Base::_S_less(__x, __y); + else if constexpr (sizeof(__x) == 8) { const auto __r128 = __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__x) < __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__y); _MaskMember<_Tp> __r64; __builtin_memcpy(&__r64._M_data, &__r128, sizeof(__r64)); return __r64; - } // }}} + } else return _Base::_S_less(__x, __y); } @@ -2558,15 +2560,16 @@ template else __assert_unreachable<_Tp>(); } // }}} - else if constexpr (!__builtin_is_constant_evaluated() // {{{ - && sizeof(__x) == 8) + else if (__builtin_is_constant_evaluated()) + return _Base::_S_less_equal(__x, __y); + else if constexpr (sizeof(__x) == 8) { const auto __r128 = __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__x) <= __vector_bitcast<_Tp, 16 / sizeof(_Tp)>(__y); _MaskMember<_Tp> __r64; __builtin_memcpy(&__r64._M_data, &__r128, sizeof(__r64)); return __r64; - } // }}} + } else return _Base::_S_less_equal(__x, __y); } From patchwork Wed Feb 15 20:50:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthias Kretz X-Patchwork-Id: 57721 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp411346wrn; Wed, 15 Feb 2023 12:52:40 -0800 (PST) X-Google-Smtp-Source: AK7set82kGfDxGFg6OkM+te5QojuHWVVc1scKSb1ONzKdngLpJs0ZXSaGNzWIV8B4WwrAhUMsKIx X-Received: by 2002:a17:906:4acb:b0:889:5ca0:146e with SMTP id u11-20020a1709064acb00b008895ca0146emr3810864ejt.16.1676494360396; Wed, 15 Feb 2023 12:52:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676494360; cv=none; d=google.com; s=arc-20160816; b=bozaRMP89jZGRW6b/WhL/LijREdKnEHctRZlZXFII8CZlEN6dQm5KvNbTQaCJdZxTb R03BecxmfhsygFmqVRkr5dD3PfhZk5mq8xSrW1f/26BpE9cQ2geqouFxmyt2X1mPiuO5 9gcg622cIddR4zBEzT3kbql8oPXnVyhIT6Dj1XuUBgkg+9k+hkhdU4sueONmtutMNubs +pLWzDW9NCU1DHj3J5ZFRD6mob+HTnoleVRHhFNXuYw+OFYZIBydU0w/deJw+gmm+mny pG01HEBLCv7o+CjVD+6hFlBGa9zZXWjtG2nwiBzF9dtkgPwbXqpoH3lCx1uOC9Vj6Wvh hi1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :organization:message-id:date:subject:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=YmvGXuxNH4WYfSY4GgDQkl79cFoGV8RuGzSqQlQwNqQ=; b=wag4tiydCfeN06smxGwTPb732h+f5ABTQrsZ+jVjczAylwRM+k7CF7j7K6ALKd9Ukt cABPKK5gIZi0MowJZ5rSYtLcBC8dwO0MuMDTl9KRakrSoaOqelwfwa9k4sMMGNBHrZiU stbRaPjldm6O0QaN9WPYJ4KH5WWV2lDveIcoHn4pzVugW+5mzULGvXvUHt54wGYp/AE8 L2a2Je/vQo2IUuE6TXXaf2d8nvDR82mfRA1/AiccRChpRiSZg6M2Hx0IQnwgegXYTvJO 26Et74PHJlt78BUZw1aJbBTvoM3hX09MKCiTrjURollKYZkh2WtxXkngCIuAa9zOQxKf 9mhw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=R4ZOFTki; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id mr40-20020a17090782a800b008b149d060d0si2219796ejc.12.2023.02.15.12.52.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Feb 2023 12:52:40 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=R4ZOFTki; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CCB7D3858434 for ; Wed, 15 Feb 2023 20:51:02 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CCB7D3858434 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1676494262; bh=YmvGXuxNH4WYfSY4GgDQkl79cFoGV8RuGzSqQlQwNqQ=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=R4ZOFTki52nHBUUQAJvHk1tveHaDoa+pQLesXYmqAL347Wf+D8ryQAOymqNsKUrVg HsgKuk09bMTweCBliPWWEKY+AA1tKx9LYNZCq7DIgb5SNkJyRP625+w+V9lxHnBXBm GdzDJss45IskKU9kyxfHex1t3aFma8aM4XIerBMg= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from lxmtout2.gsi.de (lxmtout2.gsi.de [140.181.3.112]) by sourceware.org (Postfix) with ESMTPS id 4183A384F035; Wed, 15 Feb 2023 20:50:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4183A384F035 Received: from localhost (localhost [127.0.0.1]) by lxmtout2.gsi.de (Postfix) with ESMTP id 6A917202AD5E; Wed, 15 Feb 2023 21:50:12 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at lxmtout2.gsi.de Received: from lxmtout2.gsi.de ([127.0.0.1]) by localhost (lxmtout2.gsi.de [127.0.0.1]) (amavisd-new, port 10024) with LMTP id b636al7Edxgt; Wed, 15 Feb 2023 21:50:12 +0100 (CET) Received: from srvEX6.campus.gsi.de (unknown [10.10.4.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lxmtout2.gsi.de (Postfix) with ESMTPS id 4FE47202AD51; Wed, 15 Feb 2023 21:50:12 +0100 (CET) Received: from minbar.localnet (140.181.3.12) by srvEX6.campus.gsi.de (10.10.4.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.21; Wed, 15 Feb 2023 21:50:12 +0100 To: , Subject: [PATCH 7/7] libstdc++: Fix incorrect function call in -ffast-math optimization Date: Wed, 15 Feb 2023 21:50:11 +0100 Message-ID: <111787281.nniJfEyVGO@minbar> Organization: GSI Helmholtz Centre for Heavy Ion Research In-Reply-To: <3238840.44csPzL39Z@minbar> References: <3238840.44csPzL39Z@minbar> MIME-Version: 1.0 X-Originating-IP: [140.181.3.12] X-ClientProxiedBy: srvex5.Campus.gsi.de (10.10.4.95) To srvEX6.campus.gsi.de (10.10.4.96) X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Matthias Kretz via Gcc-patches From: Matthias Kretz Reply-To: Matthias Kretz Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757931750429010333?= X-GMAIL-MSGID: =?utf-8?q?1757931750429010333?= Signed-off-by: Matthias Kretz libstdc++-v3/ChangeLog: * include/experimental/bits/simd_math.h (__hypot): Bitcasting between scalars requires the __bit_cast helper function instead of simd_bit_cast. --- libstdc++-v3/include/experimental/bits/simd_math.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- ────────────────────────────────────────────────────────────────────────── Dr. Matthias Kretz https://mattkretz.github.io GSI Helmholtz Centre for Heavy Ion Research https://gsi.de stdₓ::simd ────────────────────────────────────────────────────────────────────────── diff --git a/libstdc++-v3/include/experimental/bits/simd_math.h b/libstdc++-v3/include/experimental/bits/simd_math.h index c20315e4e30..c91f05fceb3 100644 --- a/libstdc++-v3/include/experimental/bits/simd_math.h +++ b/libstdc++-v3/include/experimental/bits/simd_math.h @@ -1010,7 +1010,7 @@ template using _IV = rebind_simd_t<_Ip, _V>; const auto __as_int = simd_bit_cast<_IV>(__hi_exp); const _V __scale - = simd_bit_cast<_V>(2 * simd_bit_cast<_Ip>(_Tp(1)) - __as_int); + = simd_bit_cast<_V>(2 * __bit_cast<_Ip>(_Tp(1)) - __as_int); #else const _V __scale = (__hi_exp ^ __inf) * _Tp(.5); #endif @@ -1181,7 +1181,7 @@ _GLIBCXX_SIMD_CVTING2(hypot) using _IV = rebind_simd_t<_Ip, _V>; const auto __as_int = simd_bit_cast<_IV>(__hi_exp); const _V __scale - = simd_bit_cast<_V>(2 * simd_bit_cast<_Ip>(_Tp(1)) - __as_int); + = simd_bit_cast<_V>(2 * __bit_cast<_Ip>(_Tp(1)) - __as_int); #else const _V __scale = (__hi_exp ^ __inf) * _Tp(.5); #endif