From patchwork Tue Oct 18 09:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jiang, Haochen" X-Patchwork-Id: 4043 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1853567wrs; Tue, 18 Oct 2022 02:14:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4n6U/dU0tYu+OFlgB4aZTMIkLqLGLfVMwveFeOhwwLNFUe7QqeFfOeWvfVaB3RAJKXZw44 X-Received: by 2002:a05:6402:3890:b0:45c:2b5:b622 with SMTP id fd16-20020a056402389000b0045c02b5b622mr1717346edb.69.1666084458885; Tue, 18 Oct 2022 02:14:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666084458; cv=none; d=google.com; s=arc-20160816; b=vFPwg0/8bx+0Bd5pQCMgV8pfH4AT5JYbv7z24v8r70/vQlXtjdR9L0BsYWt6qRy3PI SRxvYQfuKGOXhbDnm1PcXSjrIfzgFZHoyOkj90oKvx743zHZBsrmBrxIc0K70ArKDUl7 XSNvkQMSQECx8rBUz2pElFDMR8tgkJRDX9/c88QB7HWvWq4ZFov1OKNkQaCaJ3kI4AXH uvXjEzR+rs65Xx/Egs2yRjLH2CtDj51ZMTvfPKApi/JQqcGA0R9Yfdg1seCT4YJuXVfu YXq/HT0qmoOMpbHBSu6aX+g3Ryc0M8e9W8eZQRJ+HK2kvm5UHZ3+GCavvgNVSsK2rE6w Vc+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :references:in-reply-to:message-id:date:subject:to:dmarc-filter :delivered-to:dkim-signature:dkim-filter; bh=hpzqxTOOpnoji4c6Hlf9YSY43lKxV4FuJu0zc72lWQM=; b=0v66PwH5FvW/6jgEnVEyH3ToYiPQgYUfxLZOu8HYGTS7ocPIWI4rELPPV7jRytUu6k MrvYOHbLFQT40JzJuBgE1W3t50GCyRDnFmduTa4YcG94ohQ1a0SnEuVzux7mZJ8v56Ln I/Ln5D9clykWjK9JZTw6C6X3NzoGorWsAfRlItNKaAfyMaYG8ztqcejgKNTnT6H3V7Ow v31yiDZIxFKFPoAUSwi9jVl6mde4MawNBT1l9d8XuLHsUvN3seq/xjM9CXKAy/1glMd9 g1oVbS6Y0rogpslSiyPaL55RCe8gjX3oSRZpFCPZvRG5QNMRaXS2irQngVAzXGjmGRv6 sanA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=xOR2pB7C; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id f7-20020a056402354700b004573107a5basi12523686edd.352.2022.10.18.02.14.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Oct 2022 02:14:18 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=xOR2pB7C; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5D168385735A for ; Tue, 18 Oct 2022 09:14:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5D168385735A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666084443; bh=hpzqxTOOpnoji4c6Hlf9YSY43lKxV4FuJu0zc72lWQM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=xOR2pB7CMLps1fcF3+7uwioEEZJLhlojnCD1M0sozrcqA9MEZZivqcPZL1ohux7OP MVrtjqNd7G/uYxY9Ew933hMLTsQzDPETTXWor81x1S/5+RjkC+MqFhKmARaq8vsTB8 B9XHIlWTteS5RFI4NoZKjB+ZhNSeNLIj6egzCfbc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by sourceware.org (Postfix) with ESMTPS id 388A53858D3C for ; Tue, 18 Oct 2022 09:13:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 388A53858D3C X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="307133442" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="307133442" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Oct 2022 02:12:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="631031290" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="631031290" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by fmsmga007.fm.intel.com with ESMTP; 18 Oct 2022 02:12:51 -0700 Received: from shliclel320.sh.intel.com (shliclel320.sh.intel.com [10.239.240.127]) by shvmail03.sh.intel.com (Postfix) with ESMTP id 73DC61009810; Tue, 18 Oct 2022 17:12:50 +0800 (CST) To: gcc-patches@gcc.gnu.org Subject: [PATCH v2] Support Intel AVX-VNNI-INT8 Date: Tue, 18 Oct 2022 17:10:50 +0800 Message-Id: <20221018091050.82778-1-haochen.jiang@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: References: X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_NUMSUBJECT, KAM_SHORT, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Haochen Jiang via Gcc-patches From: "Jiang, Haochen" Reply-To: Haochen Jiang Cc: hongtao.liu@intel.com Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746648984697081656?= X-GMAIL-MSGID: =?utf-8?q?1747016177358067374?= From: Kong Lingling Hi all, This is our v2 patch on AVX-VNNI-INT8. This main change in this patch is to rename the previous UNSPEC_VPMADDxxx things to new vnni style. Ok for trunk? BRs, Haochen gcc/ChangeLog * common/config/i386/cpuinfo.h (get_available_features): Detect avxvnniint8. * common/config/i386/i386-common.cc (OPTION_MASK_ISA2_AVXVNNIINT8_SET): New. (OPTION_MASK_ISA2_AVXVNNIINT8_UNSET): Ditto. (ix86_handle_option): Handle -mavxvnniint8. * common/config/i386/i386-cpuinfo.h (enum processor_features): Add FEATURE_AVXVNNIINT8. * common/config/i386/i386-isas.h: Add ISA_NAME_TABLE_ENTRY for avxvnniint8. * config.gcc: Add avxvnniint8intrin.h. * config/i386/avxvnniint8intrin.h: New file. * config/i386/cpuid.h (bit_AVXVNNIINT8): New. * config/i386/i386-builtin.def: Add new builtins. * config/i386/i386-c.cc (ix86_target_macros_internal): Define __AVXVNNIINT8__. * config/i386/i386-options.cc (isa2_opts): Add -mavxvnniint8. (ix86_valid_target_attribute_inner_p): Handle avxvnniint8. * config/i386/i386-isa.def: Add DEF_PTA(AVXVNNIINT8) New.. * config/i386/i386.opt: Add option -mavxvnniint8. * config/i386/immintrin.h: Include avxvnniint8intrin.h. * config/i386/sse.md (UNSPEC_VPMADDUBSWACCD UNSPEC_VPMADDUBSWACCSSD,UNSPEC_VPMADDWDACCD, UNSPEC_VPMADDWDACCSSD): Rename according to new style. (vpdp_): New define_insn. * doc/extend.texi: Document avxvnniint8. * doc/invoke.texi: Document -mavxvnniint8. * doc/sourcebuild.texi: Document target avxvnniint8. gcc/testsuite/ChangeLog * g++.dg/other/i386-2.C: Add -mavxvnniint8. * g++.dg/other/i386-3.C: Ditto. * gcc.target/i386/avx-check.h: Add avxvnniint8 check. * gcc.target/i386/sse-12.c: Add -mavxvnniint8. * gcc.target/i386/sse-13.c: Ditto. * gcc.target/i386/sse-14.c: Ditto. * gcc.target/i386/sse-22.c: Ditto. * gcc.target/i386/sse-23.c: Ditto. * gcc.target/i386/funcspec-56.inc: Add new target attribute. * lib/target-supports.exp (check_effective_target_avxvnniint8): New. * gcc.target/i386/avxvnniint8-1.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbssd-2.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbssds-2.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbsud-2.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbsuds-2.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbuud-2.c: Ditto. * gcc.target/i386/avxvnniint8-vpdpbuuds-2.c: Ditto. Co-authored-by: Hongyu Wang Co-authored-by: Haochen Jiang --- gcc/common/config/i386/cpuinfo.h | 2 + gcc/common/config/i386/i386-common.cc | 22 ++- gcc/common/config/i386/i386-cpuinfo.h | 1 + gcc/common/config/i386/i386-isas.h | 2 + gcc/config.gcc | 2 +- gcc/config/i386/avxvnniint8intrin.h | 138 ++++++++++++++++++ gcc/config/i386/cpuid.h | 1 + gcc/config/i386/i386-builtin.def | 14 ++ gcc/config/i386/i386-c.cc | 2 + gcc/config/i386/i386-isa.def | 1 + gcc/config/i386/i386-options.cc | 4 +- gcc/config/i386/i386.opt | 5 + gcc/config/i386/immintrin.h | 2 + gcc/config/i386/sse.md | 71 ++++++--- gcc/doc/extend.texi | 5 + gcc/doc/invoke.texi | 9 +- gcc/doc/sourcebuild.texi | 3 + gcc/testsuite/g++.dg/other/i386-2.C | 2 +- gcc/testsuite/g++.dg/other/i386-3.C | 2 +- gcc/testsuite/gcc.target/i386/avx-check.h | 3 + gcc/testsuite/gcc.target/i386/avxvnniint8-1.c | 43 ++++++ .../gcc.target/i386/avxvnniint8-vpdpbssd-2.c | 72 +++++++++ .../gcc.target/i386/avxvnniint8-vpdpbssds-2.c | 72 +++++++++ .../gcc.target/i386/avxvnniint8-vpdpbsud-2.c | 72 +++++++++ .../gcc.target/i386/avxvnniint8-vpdpbsuds-2.c | 72 +++++++++ .../gcc.target/i386/avxvnniint8-vpdpbuud-2.c | 72 +++++++++ .../gcc.target/i386/avxvnniint8-vpdpbuuds-2.c | 72 +++++++++ gcc/testsuite/gcc.target/i386/funcspec-56.inc | 2 + gcc/testsuite/gcc.target/i386/sse-12.c | 2 +- gcc/testsuite/gcc.target/i386/sse-13.c | 2 +- gcc/testsuite/gcc.target/i386/sse-14.c | 2 +- gcc/testsuite/gcc.target/i386/sse-22.c | 4 +- gcc/testsuite/gcc.target/i386/sse-23.c | 2 +- gcc/testsuite/lib/target-supports.exp | 12 ++ 34 files changed, 758 insertions(+), 34 deletions(-) create mode 100644 gcc/config/i386/avxvnniint8intrin.h create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-1.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssd-2.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssds-2.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsud-2.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsuds-2.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuud-2.c create mode 100644 gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuuds-2.c diff --git a/gcc/common/config/i386/cpuinfo.h b/gcc/common/config/i386/cpuinfo.h index 9bb21c6cacc..bed88003f8e 100644 --- a/gcc/common/config/i386/cpuinfo.h +++ b/gcc/common/config/i386/cpuinfo.h @@ -795,6 +795,8 @@ get_available_features (struct __processor_model *cpu_model, set_feature (FEATURE_AVXVNNI); if (eax & bit_AVXIFMA) set_feature (FEATURE_AVXIFMA); + if (edx & bit_AVXVNNIINT8) + set_feature (FEATURE_AVXVNNIINT8); } if (avx512_usable) { diff --git a/gcc/common/config/i386/i386-common.cc b/gcc/common/config/i386/i386-common.cc index 4de7906b247..6a2a7e3d25a 100644 --- a/gcc/common/config/i386/i386-common.cc +++ b/gcc/common/config/i386/i386-common.cc @@ -108,6 +108,7 @@ along with GCC; see the file COPYING3. If not see #define OPTION_MASK_ISA2_AMX_TILE_SET OPTION_MASK_ISA2_AMX_TILE #define OPTION_MASK_ISA2_AMX_INT8_SET OPTION_MASK_ISA2_AMX_INT8 #define OPTION_MASK_ISA2_AMX_BF16_SET OPTION_MASK_ISA2_AMX_BF16 +#define OPTION_MASK_ISA2_AVXVNNIINT8_SET OPTION_MASK_ISA2_AVXVNNIINT8 /* SSE4 includes both SSE4.1 and SSE4.2. -msse4 should be the same as -msse4.2. */ @@ -214,7 +215,7 @@ along with GCC; see the file COPYING3. If not see (OPTION_MASK_ISA_AVX2 | OPTION_MASK_ISA_AVX512F_UNSET) #define OPTION_MASK_ISA2_AVX2_UNSET \ (OPTION_MASK_ISA2_AVXIFMA_UNSET | OPTION_MASK_ISA2_AVXVNNI_UNSET \ - | OPTION_MASK_ISA2_AVX512F_UNSET) + | OPTION_MASK_ISA2_AVXVNNIINT8_UNSET | OPTION_MASK_ISA2_AVX512F_UNSET) #define OPTION_MASK_ISA_AVX512F_UNSET \ (OPTION_MASK_ISA_AVX512F | OPTION_MASK_ISA_AVX512CD_UNSET \ | OPTION_MASK_ISA_AVX512PF_UNSET | OPTION_MASK_ISA_AVX512ER_UNSET \ @@ -278,6 +279,7 @@ along with GCC; see the file COPYING3. If not see #define OPTION_MASK_ISA2_KL_UNSET \ (OPTION_MASK_ISA2_KL | OPTION_MASK_ISA2_WIDEKL_UNSET) #define OPTION_MASK_ISA2_WIDEKL_UNSET OPTION_MASK_ISA2_WIDEKL +#define OPTION_MASK_ISA2_AVXVNNIINT8_UNSET OPTION_MASK_ISA2_AVXVNNIINT8 /* SSE4 includes both SSE4.1 and SSE4.2. -mno-sse4 should the same as -mno-sse4.1. */ @@ -1142,6 +1144,24 @@ ix86_handle_option (struct gcc_options *opts, } return true; + case OPT_mavxvnniint8: + if (value) + { + opts->x_ix86_isa_flags2 |= OPTION_MASK_ISA2_AVXVNNIINT8_SET; + opts->x_ix86_isa_flags2_explicit |= + OPTION_MASK_ISA2_AVXVNNIINT8_SET; + opts->x_ix86_isa_flags |= OPTION_MASK_ISA_AVX2_SET; + opts->x_ix86_isa_flags_explicit |= OPTION_MASK_ISA_AVX2_SET; + } + else + { + opts->x_ix86_isa_flags2 &= + ~OPTION_MASK_ISA2_AVXVNNIINT8_UNSET; + opts->x_ix86_isa_flags2_explicit |= + OPTION_MASK_ISA2_AVXVNNIINT8_UNSET; + } + return true; + case OPT_mfma: if (value) { diff --git a/gcc/common/config/i386/i386-cpuinfo.h b/gcc/common/config/i386/i386-cpuinfo.h index 968f9a56a6c..9a6b92fab79 100644 --- a/gcc/common/config/i386/i386-cpuinfo.h +++ b/gcc/common/config/i386/i386-cpuinfo.h @@ -241,6 +241,7 @@ enum processor_features FEATURE_X86_64_V3, FEATURE_X86_64_V4, FEATURE_AVXIFMA, + FEATURE_AVXVNNIINT8, CPU_FEATURE_MAX }; diff --git a/gcc/common/config/i386/i386-isas.h b/gcc/common/config/i386/i386-isas.h index b05b4bb8f0d..8c1f351056c 100644 --- a/gcc/common/config/i386/i386-isas.h +++ b/gcc/common/config/i386/i386-isas.h @@ -176,4 +176,6 @@ ISA_NAMES_TABLE_START ISA_NAMES_TABLE_ENTRY("x86-64-v3", FEATURE_X86_64_V3, P_X86_64_V3, NULL) ISA_NAMES_TABLE_ENTRY("x86-64-v4", FEATURE_X86_64_V4, P_X86_64_V4, NULL) ISA_NAMES_TABLE_ENTRY("avxifma", FEATURE_AVXIFMA, P_NONE, "-mavxifma") + ISA_NAMES_TABLE_ENTRY("avxvnniint8", FEATURE_AVXVNNIINT8, + P_NONE, "-mavxvnniint8") ISA_NAMES_TABLE_END diff --git a/gcc/config.gcc b/gcc/config.gcc index d086dbdf8fb..3f7804a54b7 100644 --- a/gcc/config.gcc +++ b/gcc/config.gcc @@ -422,7 +422,7 @@ i[34567]86-*-* | x86_64-*-*) amxbf16intrin.h x86gprintrin.h uintrintrin.h hresetintrin.h keylockerintrin.h avxvnniintrin.h mwaitintrin.h avx512fp16intrin.h avx512fp16vlintrin.h - avxifmaintrin.h" + avxifmaintrin.h avxvnniint8intrin.h" ;; ia64-*-*) extra_headers=ia64intrin.h diff --git a/gcc/config/i386/avxvnniint8intrin.h b/gcc/config/i386/avxvnniint8intrin.h new file mode 100644 index 00000000000..362e6f65c2a --- /dev/null +++ b/gcc/config/i386/avxvnniint8intrin.h @@ -0,0 +1,138 @@ +/* Copyright (C) 2020 Free Software Foundation, Inc. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + Under Section 7 of GPL version 3, you are granted additional + permissions described in the GCC Runtime Library Exception, version + 3.1, as published by the Free Software Foundation. + + You should have received a copy of the GNU General Public License and + a copy of the GCC Runtime Library Exception along with this program; + see the files COPYING3 and COPYING.RUNTIME respectively. If not, see + . */ + +#if !defined _IMMINTRIN_H_INCLUDED +#error "Never use directly; include instead." +#endif + +#ifndef _AVXVNNIINT8INTRIN_H_INCLUDED +#define _AVXVNNIINT8INTRIN_H_INCLUDED + +#if !defined(__AVXVNNIINT8__) +#pragma GCC push_options +#pragma GCC target("avxvnniint8") +#define __DISABLE_AVXVNNIINT8__ +#endif /* __AVXVNNIINT8__ */ + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbssd_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbssd128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbssds_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbssds128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbsud_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbsud128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbsuds_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbsuds128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbuud_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbuud128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m128i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm_dpbuuds_epi32 (__m128i __W, __m128i __A, __m128i __B) +{ + return (__m128i) + __builtin_ia32_vpdpbuuds128 ((__v4si) __W, (__v4si) __A, (__v4si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbssd_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbssd256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbssds_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbssds256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbsud_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbsud256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbsuds_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbsuds256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbuud_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbuud256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +extern __inline __m256i +__attribute__((__gnu_inline__, __always_inline__, __artificial__)) +_mm256_dpbuuds_epi32 (__m256i __W, __m256i __A, __m256i __B) +{ + return (__m256i) + __builtin_ia32_vpdpbuuds256 ((__v8si) __W, (__v8si) __A, (__v8si) __B); +} + +#ifdef __DISABLE_AVXVNNIINT8__ +#undef __DISABLE_AVXVNNIINT8__ +#pragma GCC pop_options +#endif /* __DISABLE_AVXVNNIINT8__ */ + +#endif /* __AVXVNNIINT8INTRIN_H_INCLUDED */ diff --git a/gcc/config/i386/cpuid.h b/gcc/config/i386/cpuid.h index 9885699efd5..f5fad22149a 100644 --- a/gcc/config/i386/cpuid.h +++ b/gcc/config/i386/cpuid.h @@ -49,6 +49,7 @@ #define bit_RDRND (1 << 30) /* %edx */ +#define bit_AVXVNNIINT8 (1 << 4) #define bit_CMPXCHG8B (1 << 8) #define bit_CMOV (1 << 15) #define bit_MMX (1 << 23) diff --git a/gcc/config/i386/i386-builtin.def b/gcc/config/i386/i386-builtin.def index 4a89099a00f..e6edae5728b 100644 --- a/gcc/config/i386/i386-builtin.def +++ b/gcc/config/i386/i386-builtin.def @@ -2696,6 +2696,20 @@ BDESC (OPTION_MASK_ISA_AVX512VNNI | OPTION_MASK_ISA_AVX512VL, OPTION_MASK_ISA2_A BDESC (OPTION_MASK_ISA_AVX512VNNI | OPTION_MASK_ISA_AVX512VL, 0, CODE_FOR_vpdpwssds_v4si_mask, "__builtin_ia32_vpdpwssds_v4si_mask", IX86_BUILTIN_VPDPWSSDSV4SI_MASK, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI_UQI) BDESC (OPTION_MASK_ISA_AVX512VNNI | OPTION_MASK_ISA_AVX512VL, 0, CODE_FOR_vpdpwssds_v4si_maskz, "__builtin_ia32_vpdpwssds_v4si_maskz", IX86_BUILTIN_VPDPWSSDSV4SI_MASKZ, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI_UQI) +/* AVXVNNIINT8 */ +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbssd_v8si, "__builtin_ia32_vpdpbssd256", IX86_BUILTIN_VPDPBSSDV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbssds_v8si, "__builtin_ia32_vpdpbssds256", IX86_BUILTIN_VPDPBSSDSV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbsud_v8si, "__builtin_ia32_vpdpbsud256", IX86_BUILTIN_VPDPBSUDV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbsuds_v8si, "__builtin_ia32_vpdpbsuds256", IX86_BUILTIN_VPDPBSUDSV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbuud_v8si, "__builtin_ia32_vpdpbuud256", IX86_BUILTIN_VPDPBUUDV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbuuds_v8si, "__builtin_ia32_vpdpbuuds256", IX86_BUILTIN_VPDPBUUDSV8SI, UNKNOWN, (int) V8SI_FTYPE_V8SI_V8SI_V8SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbssd_v4si, "__builtin_ia32_vpdpbssd128", IX86_BUILTIN_VPDPBSSDV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbssds_v4si, "__builtin_ia32_vpdpbssds128", IX86_BUILTIN_VPDPBSSDSV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbsud_v4si, "__builtin_ia32_vpdpbsud128", IX86_BUILTIN_VPDPBSUDV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbsuds_v4si, "__builtin_ia32_vpdpbsuds128", IX86_BUILTIN_VPDPBSUDSV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbuud_v4si, "__builtin_ia32_vpdpbuud128", IX86_BUILTIN_VPDPBUUDV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) +BDESC (0, OPTION_MASK_ISA2_AVXVNNIINT8, CODE_FOR_vpdpbuuds_v4si, "__builtin_ia32_vpdpbuuds128", IX86_BUILTIN_VPDPBUUDSV4SI, UNKNOWN, (int) V4SI_FTYPE_V4SI_V4SI_V4SI) + /* VPCLMULQDQ */ BDESC (OPTION_MASK_ISA_VPCLMULQDQ | OPTION_MASK_ISA_AVX512VL, 0, CODE_FOR_vpclmulqdq_v2di, "__builtin_ia32_vpclmulqdq_v2di", IX86_BUILTIN_VPCLMULQDQ2, UNKNOWN, (int) V2DI_FTYPE_V2DI_V2DI_INT) BDESC (OPTION_MASK_ISA_VPCLMULQDQ | OPTION_MASK_ISA_AVX, 0, CODE_FOR_vpclmulqdq_v4di, "__builtin_ia32_vpclmulqdq_v4di", IX86_BUILTIN_VPCLMULQDQ4, UNKNOWN, (int) V4DI_FTYPE_V4DI_V4DI_INT) diff --git a/gcc/config/i386/i386-c.cc b/gcc/config/i386/i386-c.cc index 3494ec035d5..a9a35c0a18a 100644 --- a/gcc/config/i386/i386-c.cc +++ b/gcc/config/i386/i386-c.cc @@ -635,6 +635,8 @@ ix86_target_macros_internal (HOST_WIDE_INT isa_flag, def_or_undef (parse_in, "__AVXVNNI__"); if (isa_flag2 & OPTION_MASK_ISA2_AVXIFMA) def_or_undef (parse_in, "__AVXIFMA__"); + if (isa_flag2 & OPTION_MASK_ISA2_AVXVNNIINT8) + def_or_undef (parse_in, "__AVXVNNIINT8__"); if (TARGET_IAMCU) { def_or_undef (parse_in, "__iamcu"); diff --git a/gcc/config/i386/i386-isa.def b/gcc/config/i386/i386-isa.def index 6e0254ce418..c95b917c6ce 100644 --- a/gcc/config/i386/i386-isa.def +++ b/gcc/config/i386/i386-isa.def @@ -110,3 +110,4 @@ DEF_PTA(WIDEKL) DEF_PTA(AVXVNNI) DEF_PTA(AVX512FP16) DEF_PTA(AVXIFMA) +DEF_PTA(AVXVNNIINT8) diff --git a/gcc/config/i386/i386-options.cc b/gcc/config/i386/i386-options.cc index 5facb64c2a8..3e6d04433a6 100644 --- a/gcc/config/i386/i386-options.cc +++ b/gcc/config/i386/i386-options.cc @@ -227,7 +227,8 @@ static struct ix86_target_opts isa2_opts[] = { "-mwidekl", OPTION_MASK_ISA2_WIDEKL }, { "-mavxvnni", OPTION_MASK_ISA2_AVXVNNI }, { "-mavx512fp16", OPTION_MASK_ISA2_AVX512FP16 }, - { "-mavxifma", OPTION_MASK_ISA2_AVXIFMA } + { "-mavxifma", OPTION_MASK_ISA2_AVXIFMA }, + { "-mavxvnniint8", OPTION_MASK_ISA2_AVXVNNIINT8 } }; static struct ix86_target_opts isa_opts[] = { @@ -1074,6 +1075,7 @@ ix86_valid_target_attribute_inner_p (tree fndecl, tree args, char *p_strings[], IX86_ATTR_ISA ("avxvnni", OPT_mavxvnni), IX86_ATTR_ISA ("avx512fp16", OPT_mavx512fp16), IX86_ATTR_ISA ("avxifma", OPT_mavxifma), + IX86_ATTR_ISA ("avxvnniint8", OPT_mavxvnniint8), /* enum options */ IX86_ATTR_ENUM ("fpmath=", OPT_mfpmath_), diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index 36e28b7063d..53d534f6392 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -1219,3 +1219,8 @@ mavxifma Target Mask(ISA2_AVXIFMA) Var(ix86_isa_flags2) Save Support MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, and AVXIFMA built-in functions and code generation. + +mavxvnniint8 +Target Mask(ISA2_AVXVNNIINT8) Var(ix86_isa_flags2) Save +Support MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2 and +AVXVNNIINT8 built-in functions and code generation. diff --git a/gcc/config/i386/immintrin.h b/gcc/config/i386/immintrin.h index e9d4e975243..ddea249d09b 100644 --- a/gcc/config/i386/immintrin.h +++ b/gcc/config/i386/immintrin.h @@ -46,6 +46,8 @@ #include +#include + #include #include diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index 331347569ea..6513d189b09 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -166,10 +166,10 @@ UNSPEC_VPSHLDV ;; For AVX512VNNI support - UNSPEC_VPMADDUBSWACCD - UNSPEC_VPMADDUBSWACCSSD - UNSPEC_VPMADDWDACCD - UNSPEC_VPMADDWDACCSSD + UNSPEC_VPDPBUSD + UNSPEC_VPDPBUSDS + UNSPEC_VPDPWSSD + UNSPEC_VPDPWSSDS ;; For VAES support UNSPEC_VAESDEC @@ -200,6 +200,13 @@ UNSPEC_COMPLEX_FCMUL UNSPEC_COMPLEX_MASK + ;; For AVX-VNNI-INT8 support + UNSPEC_VPDPBSSD + UNSPEC_VPDPBSSDS + UNSPEC_VPDPBSUD + UNSPEC_VPDPBSUDS + UNSPEC_VPDPBUUD + UNSPEC_VPDPBUUDS ]) (define_c_enum "unspecv" [ @@ -28541,7 +28548,7 @@ [(match_operand:V16SI 1 "register_operand" "0") (match_operand:V16SI 2 "register_operand" "v") (match_operand:V16SI 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDUBSWACCD))] + UNSPEC_VPDPBUSD))] "TARGET_AVX512VNNI" "vpdpbusd\t{%3, %2, %0|%0, %2, %3}" [(set_attr ("prefix") ("evex"))]) @@ -28552,7 +28559,7 @@ [(match_operand:VI4_AVX2 1 "register_operand" "0,0") (match_operand:VI4_AVX2 2 "register_operand" "x,v") (match_operand:VI4_AVX2 3 "nonimmediate_operand" "xm,vm")] - UNSPEC_VPMADDUBSWACCD))] + UNSPEC_VPDPBUSD))] "TARGET_AVXVNNI || (TARGET_AVX512VNNI && TARGET_AVX512VL)" "@ %{vex%} vpdpbusd\t{%3, %2, %0|%0, %2, %3} @@ -28567,7 +28574,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDUBSWACCD) + UNSPEC_VPDPBUSD) (match_dup 1) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28596,7 +28603,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm") - ] UNSPEC_VPMADDUBSWACCD) + ] UNSPEC_VPDPBUSD) (match_operand:VI4_AVX512VL 4 "const0_operand") (match_operand: 5 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28609,7 +28616,7 @@ [(match_operand:V16SI 1 "register_operand" "0") (match_operand:V16SI 2 "register_operand" "v") (match_operand:V16SI 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDUBSWACCSSD))] + UNSPEC_VPDPBUSDS))] "TARGET_AVX512VNNI" "vpdpbusds\t{%3, %2, %0|%0, %2, %3}" [(set_attr ("prefix") ("evex"))]) @@ -28620,7 +28627,7 @@ [(match_operand:VI4_AVX2 1 "register_operand" "0,0") (match_operand:VI4_AVX2 2 "register_operand" "x,v") (match_operand:VI4_AVX2 3 "nonimmediate_operand" "xm,vm")] - UNSPEC_VPMADDUBSWACCSSD))] + UNSPEC_VPDPBUSDS))] "TARGET_AVXVNNI || (TARGET_AVX512VNNI && TARGET_AVX512VL)" "@ %{vex%} vpdpbusds\t{%3, %2, %0|%0, %2, %3} @@ -28635,7 +28642,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDUBSWACCSSD) + UNSPEC_VPDPBUSDS) (match_dup 1) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28664,7 +28671,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDUBSWACCSSD) + UNSPEC_VPDPBUSDS) (match_operand:VI4_AVX512VL 4 "const0_operand") (match_operand: 5 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28677,7 +28684,7 @@ [(match_operand:V16SI 1 "register_operand" "0") (match_operand:V16SI 2 "register_operand" "v") (match_operand:V16SI 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCD))] + UNSPEC_VPDPWSSD))] "TARGET_AVX512VNNI" "vpdpwssd\t{%3, %2, %0|%0, %2, %3}" [(set_attr ("prefix") ("evex"))]) @@ -28688,7 +28695,7 @@ [(match_operand:VI4_AVX2 1 "register_operand" "0,0") (match_operand:VI4_AVX2 2 "register_operand" "x,v") (match_operand:VI4_AVX2 3 "nonimmediate_operand" "xm,vm")] - UNSPEC_VPMADDWDACCD))] + UNSPEC_VPDPWSSD))] "TARGET_AVXVNNI || (TARGET_AVX512VNNI && TARGET_AVX512VL)" "@ %{vex%} vpdpwssd\t{%3, %2, %0|%0, %2, %3} @@ -28703,7 +28710,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCD) + UNSPEC_VPDPWSSD) (match_dup 1) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28732,7 +28739,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCD) + UNSPEC_VPDPWSSD) (match_operand:VI4_AVX512VL 4 "const0_operand") (match_operand: 5 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28745,7 +28752,7 @@ [(match_operand:V16SI 1 "register_operand" "0") (match_operand:V16SI 2 "register_operand" "v") (match_operand:V16SI 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCSSD))] + UNSPEC_VPDPWSSDS))] "TARGET_AVX512VNNI" "vpdpwssds\t{%3, %2, %0|%0, %2, %3}" [(set_attr ("prefix") ("evex"))]) @@ -28756,7 +28763,7 @@ [(match_operand:VI4_AVX2 1 "register_operand" "0,0") (match_operand:VI4_AVX2 2 "register_operand" "x,v") (match_operand:VI4_AVX2 3 "nonimmediate_operand" "xm,vm")] - UNSPEC_VPMADDWDACCSSD))] + UNSPEC_VPDPWSSDS))] "TARGET_AVXVNNI || (TARGET_AVX512VNNI && TARGET_AVX512VL)" "@ %{vex%} vpdpwssds\t{%3, %2, %0|%0, %2, %3} @@ -28771,7 +28778,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCSSD) + UNSPEC_VPDPWSSDS) (match_dup 1) (match_operand: 4 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -28800,7 +28807,7 @@ [(match_operand:VI4_AVX512VL 1 "register_operand" "0") (match_operand:VI4_AVX512VL 2 "register_operand" "v") (match_operand:VI4_AVX512VL 3 "nonimmediate_operand" "vm")] - UNSPEC_VPMADDWDACCSSD) + UNSPEC_VPDPWSSDS) (match_operand:VI4_AVX512VL 4 "const0_operand") (match_operand: 5 "register_operand" "Yk")))] "TARGET_AVX512VNNI" @@ -29241,3 +29248,27 @@ gcc_unreachable (); DONE; }) + +(define_int_iterator VPDOTPROD + [UNSPEC_VPDPBSSD + UNSPEC_VPDPBSSDS + UNSPEC_VPDPBSUD + UNSPEC_VPDPBSUDS + UNSPEC_VPDPBUUD + UNSPEC_VPDPBUUDS]) + +(define_int_attr vpdotprodtype + [(UNSPEC_VPDPBSSD "bssd") (UNSPEC_VPDPBSSDS "bssds") + (UNSPEC_VPDPBSUD "bsud") (UNSPEC_VPDPBSUDS "bsuds") + (UNSPEC_VPDPBUUD "buud") (UNSPEC_VPDPBUUDS "buuds")]) + +(define_insn "vpdp_" + [(set (match_operand:VI4_AVX 0 "register_operand" "=x") + (unspec:VI4_AVX + [(match_operand:VI4_AVX 1 "register_operand" "0") + (match_operand:VI4_AVX 2 "register_operand" "x") + (match_operand:VI4_AVX 3 "nonimmediate_operand" "xm")] + VPDOTPROD))] + "TARGET_AVXVNNIINT8" + "vpdp\t{%3, %2, %0|%0, %2, %3}" + [(set_attr "prefix" "vex")]) diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi index edecf5c0070..9a8de9fc226 100644 --- a/gcc/doc/extend.texi +++ b/gcc/doc/extend.texi @@ -7065,6 +7065,11 @@ Enable/disable the generation of the AVXVNNI instructions. @cindex @code{target("avxifma")} function attribute, x86 Enable/disable the generation of the AVXIFMA instructions. +@item avxvnniint8 +@itemx no-avxvnniint8 +@cindex @code{target("avxvnniint8")} function attribute, x86 +Enable/disable the generation of the AVXVNNIINT8 instructions. + @item cld @itemx no-cld @cindex @code{target("cld")} function attribute, x86 diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 2cd617a9d44..1e8c49e391c 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -1436,7 +1436,7 @@ See RS/6000 and PowerPC Options. -mavx5124fmaps -mavx512vnni -mavx5124vnniw -mprfchw -mrdpid @gol -mrdseed -msgx -mavx512vp2intersect -mserialize -mtsxldtrk@gol -mamx-tile -mamx-int8 -mamx-bf16 -muintr -mhreset -mavxvnni@gol --mavx512fp16 -mavxifma @gol +-mavx512fp16 -mavxifma -mavxvnniint8 @gol -mcldemote -mms-bitfields -mno-align-stringops -minline-all-stringops @gol -minline-stringops-dynamically -mstringop-strategy=@var{alg} @gol -mkl -mwidekl @gol @@ -32896,6 +32896,9 @@ preferred alignment to @option{-mpreferred-stack-boundary=2}. @need 200 @itemx -mavxifma @opindex mavxifma +@need 200 +@itemx -mavxvnniint8 +@opindex mavxvnniint8 These switches enable the use of instructions in the MMX, SSE, SSE2, SSE3, SSSE3, SSE4, SSE4A, SSE4.1, SSE4.2, AVX, AVX2, AVX512F, AVX512PF, AVX512ER, AVX512CD, AVX512VL, AVX512BW, AVX512DQ, AVX512IFMA, AVX512VBMI, SHA, @@ -32906,8 +32909,8 @@ XSAVEOPT, XSAVEC, XSAVES, RTM, HLE, TBM, MWAITX, CLZERO, PKU, AVX512VBMI2, GFNI, VAES, WAITPKG, VPCLMULQDQ, AVX512BITALG, MOVDIRI, MOVDIR64B, AVX512BF16, ENQCMD, AVX512VPOPCNTDQ, AVX5124FMAPS, AVX512VNNI, AVX5124VNNIW, SERIALIZE, UINTR, HRESET, AMXTILE, AMXINT8, AMXBF16, KL, WIDEKL, AVXVNNI, AVX512FP16, -AVXIFMA or CLDEMOTE extended instruction sets. Each has a corresponding -@option{-mno-} option to disable use of these instructions. +AVXIFMA, AVXVNNIINT8 or CLDEMOTE extended instruction sets. Each has a +corresponding @option{-mno-} option to disable use of these instructions. These extensions are also available as built-in functions: see @ref{x86 Built-in Functions}, for details of the functions enabled and diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi index 0173acf4a65..e21a1d381e0 100644 --- a/gcc/doc/sourcebuild.texi +++ b/gcc/doc/sourcebuild.texi @@ -2493,6 +2493,9 @@ Target supports the execution of @code{avx512vp2intersect} instructions. @item avxifma Target supports the execution of @code{avxifma} instructions. +@item avxvnniint8 +Target supports the execution of @code{avxvnniint8} instructions. + @item amx_tile Target supports the execution of @code{amx-tile} instructions. diff --git a/gcc/testsuite/g++.dg/other/i386-2.C b/gcc/testsuite/g++.dg/other/i386-2.C index 5388606779b..ebd01fe47bc 100644 --- a/gcc/testsuite/g++.dg/other/i386-2.C +++ b/gcc/testsuite/g++.dg/other/i386-2.C @@ -1,5 +1,5 @@ /* { dg-do compile { target i?86-*-* x86_64-*-* } } */ -/* { dg-options "-O -pedantic-errors -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma" } */ +/* { dg-options "-O -pedantic-errors -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma -mavxvnniint8" } */ /* Test that {,x,e,p,t,s,w,a,b,i}mmintrin.h, mm3dnow.h, fma4intrin.h, xopintrin.h, abmintrin.h, bmiintrin.h, tbmintrin.h, lwpintrin.h, diff --git a/gcc/testsuite/g++.dg/other/i386-3.C b/gcc/testsuite/g++.dg/other/i386-3.C index 86cedd3d32f..b66498f1d4c 100644 --- a/gcc/testsuite/g++.dg/other/i386-3.C +++ b/gcc/testsuite/g++.dg/other/i386-3.C @@ -1,5 +1,5 @@ /* { dg-do compile { target i?86-*-* x86_64-*-* } } */ -/* { dg-options "-O -fkeep-inline-functions -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma" } */ +/* { dg-options "-O -fkeep-inline-functions -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma -mavxvnniint8" } */ /* Test that {,x,e,p,t,s,w,a,b,i}mmintrin.h, mm3dnow.h, fma4intrin.h, xopintrin.h, abmintrin.h, bmiintrin.h, tbmintrin.h, lwpintrin.h, diff --git a/gcc/testsuite/gcc.target/i386/avx-check.h b/gcc/testsuite/gcc.target/i386/avx-check.h index 24ee6ab4efd..77507ca2edc 100644 --- a/gcc/testsuite/gcc.target/i386/avx-check.h +++ b/gcc/testsuite/gcc.target/i386/avx-check.h @@ -25,6 +25,9 @@ main () && avx_os_support () #ifdef AVXIFMA && __builtin_cpu_supports ("avxifma") +#endif +#ifdef AVXVNNIINT8 + && __builtin_cpu_supports ("avxvnniint8") #endif ) { diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-1.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-1.c new file mode 100644 index 00000000000..d6942f34d6e --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-1.c @@ -0,0 +1,43 @@ +/* { dg-do compile } */ +/* { dg-options "-mavxvnniint8 -O2" } */ +/* { dg-final { scan-assembler-times "vpdpbssd\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbssd\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbssds\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbssds\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbsud\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbsud\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbsuds\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbsuds\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbuud\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbuud\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbuuds\[ \\t\]+\[^\{\n\]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+\[^\n\r]*%ymm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ +/* { dg-final { scan-assembler-times "vpdpbuuds\[ \\t\]+\[^\{\n\]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+\[^\n\r]*%xmm\[0-9\]+(?:\n|\[ \\t\]+#)" 1 } } */ + + +#include + +volatile __m256i x,y,z; +volatile __m128i x_,y_,z_; +volatile __mmask8 m; + +void extern +avxvnniint8_test (void) +{ + x = _mm256_dpbssd_epi32 (x, y, z); + x_ = _mm_dpbssd_epi32 (x_, y_, z_); + + x = _mm256_dpbssds_epi32 (x, y, z); + x_ = _mm_dpbssds_epi32 (x_, y_, z_); + + x = _mm256_dpbsud_epi32 (x, y, z); + x_ = _mm_dpbsud_epi32 (x_, y_, z_); + + x = _mm256_dpbsuds_epi32 (x, y, z); + x_ = _mm_dpbsuds_epi32 (x_, y_, z_); + + x = _mm256_dpbuud_epi32 (x, y, z); + x_ = _mm_dpbuud_epi32 (x_, y_, z_); + + x = _mm256_dpbuuds_epi32 (x, y, z); + x_ = _mm_dpbuuds_epi32 (x_, y_, z_); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssd-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssd-2.c new file mode 100644 index 00000000000..5016de39621 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssd-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (int *r, int *dst, char *s1, char *s2, int size) +{ + short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (short) s1[i] * (short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + long long test = (long long) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test; + } +} + +void +TEST (void) +{ + int i; + union256i_d res_256; + union256i_b src2_256; + union256i_b src1_256; + int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbssd_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_d (res_256, res_ref_256)) + abort (); + + union128i_d res_128; + union128i_b src2_128; + union128i_b src1_128; + int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbssd_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_d (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssds-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssds-2.c new file mode 100644 index 00000000000..6de5062e917 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbssds-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (int *r, int *dst, char *s1, char *s2, int size) +{ + short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (short) s1[i] * (short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + long long test = (long long) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test > 0x7FFFFFFF ? 0x7FFFFFFF : test; + } +} + +void +TEST (void) +{ + int i; + union256i_d res_256; + union256i_b src2_256; + union256i_b src1_256; + int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbssds_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_d (res_256, res_ref_256)) + abort (); + + union128i_d res_128; + union128i_b src2_128; + union128i_b src1_128; + int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbssds_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_d (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsud-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsud-2.c new file mode 100644 index 00000000000..6e4ffd1c7be --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsud-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (int *r, int *dst, char *s1, unsigned char *s2, int size) +{ + short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (short) s1[i] * (unsigned short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + long long test = (long long) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test; + } +} + +void +TEST (void) +{ + int i; + union256i_d res_256; + union256i_b src1_256; + union256i_ub src2_256; + int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbsud_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_d (res_256, res_ref_256)) + abort (); + + union128i_d res_128; + union128i_b src1_128; + union128i_ub src2_128; + int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbsud_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_d (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsuds-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsuds-2.c new file mode 100644 index 00000000000..ad4b6047ecd --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbsuds-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (int *r, int *dst, char *s1, unsigned char *s2, int size) +{ + short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (short) s1[i] * (unsigned short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + long long test = (long long) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test > 0x7FFFFFFF ? 0x7FFFFFFF : test; + } +} + +void +TEST (void) +{ + int i; + union256i_d res_256; + union256i_b src1_256; + union256i_ub src2_256; + int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbsuds_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_d (res_256, res_ref_256)) + abort (); + + union128i_d res_128; + union128i_b src1_128; + union128i_ub src2_128; + int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbsuds_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_d (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuud-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuud-2.c new file mode 100644 index 00000000000..6590915a459 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuud-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (unsigned int *r, unsigned int *dst, unsigned char *s1, unsigned char *s2, int size) +{ + unsigned short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (unsigned short) s1[i] * (unsigned short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + unsigned int test = (unsigned int) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test; + } +} + +void +TEST (void) +{ + int i; + union256i_ud res_256; + union256i_ub src2_256; + union256i_ub src1_256; + unsigned int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbuud_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_ud (res_256, res_ref_256)) + abort (); + + union128i_ud res_128; + union128i_ub src2_128; + union128i_ub src1_128; + unsigned int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbuud_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_ud (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuuds-2.c b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuuds-2.c new file mode 100644 index 00000000000..970e4a5d408 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avxvnniint8-vpdpbuuds-2.c @@ -0,0 +1,72 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavxvnniint8" } */ +/* { dg-require-effective-target avxvnniint8 } */ +#define AVXVNNIINT8 +#ifndef CHECK +#define CHECK "avx-check.h" +#endif + +#ifndef TEST +#define TEST avx_test +#endif + +#include CHECK + +static void +CALC (unsigned int *r, unsigned int *dst, unsigned char *s1, unsigned char *s2, int size) +{ + unsigned short tempres[32]; + for (int i = 0; i < size; i++) { + tempres[i] = (unsigned short) s1[i] * (unsigned short) s2[i]; + } + for (int i = 0; i < size / 4; i++) { + unsigned int test = (unsigned int) dst[i] + tempres[i * 4] + tempres[i * 4 + 1] + + tempres[i * 4 + 2] + tempres[i * 4 + 3]; + r[i] = test > 0xFFFFFFFF ? 0xFFFFFFFF : test; + } +} + +void +TEST (void) +{ + int i; + union256i_ud res_256; + union256i_ub src2_256; + union256i_ub src1_256; + unsigned int res_ref_256[8]; + + for (i = 0; i < 32; i++) + { + int sign = i % 2 ? 1 : -1; + src1_256.a[i] = 10 + 3 * i + sign; + src2_256.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 8; i++) + res_256.a[i] = 0x7fffffff; + + CALC (res_ref_256, res_256.a, src1_256.a, src2_256.a, 32); + res_256.x = _mm256_dpbuuds_epi32 (res_256.x, src1_256.x, src2_256.x); + if (check_union256i_ud (res_256, res_ref_256)) + abort (); + + union128i_ud res_128; + union128i_ub src2_128; + union128i_ub src1_128; + unsigned int res_ref_128[4]; + + for (i = 0; i < 16; i++) + { + int sign = i % 2 ? 1 : -1; + src1_128.a[i] = 10 + 3 * i * i + sign; + src2_128.a[i] = sign * 10 * i * i; + } + + for (i = 0; i < 4; i++) + res_128.a[i] = 0x7fffffff; + + CALC (res_ref_128, res_128.a, src1_128.a, src2_128.a, 16); + res_128.x = _mm_dpbuuds_epi32 (res_128.x, src1_128.x, src2_128.x); + if (check_union128i_ud (res_128, res_ref_128)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/funcspec-56.inc b/gcc/testsuite/gcc.target/i386/funcspec-56.inc index 466555c0d06..a681bffe3e7 100644 --- a/gcc/testsuite/gcc.target/i386/funcspec-56.inc +++ b/gcc/testsuite/gcc.target/i386/funcspec-56.inc @@ -81,6 +81,7 @@ extern void test_widekl (void) __attribute__((__target__("widekl"))); extern void test_avxvnni (void) __attribute__((__target__("avxvnni"))); extern void test_avx512fp16 (void) __attribute__((__target__("avx512fp16"))); extern void test_avxifma (void) __attribute__((__target__("avxifma"))); +extern void test_avxvnniint8 (void) __attribute__((__target__("avxvnniint8"))); extern void test_no_sgx (void) __attribute__((__target__("no-sgx"))); extern void test_no_avx5124fmaps(void) __attribute__((__target__("no-avx5124fmaps"))); @@ -163,6 +164,7 @@ extern void test_no_widekl (void) __attribute__((__target__("no-widekl"))); extern void test_no_avxvnni (void) __attribute__((__target__("no-avxvnni"))); extern void test_no_avx512fp16 (void) __attribute__((__target__("no-avx512fp16"))); extern void test_no_avxifma (void) __attribute__((__target__("no-avxifma"))); +extern void test_no_avxvnniint8 (void) __attribute__((__target__("no-avxvnniint8"))); extern void test_arch_nocona (void) __attribute__((__target__("arch=nocona"))); extern void test_arch_core2 (void) __attribute__((__target__("arch=core2"))); diff --git a/gcc/testsuite/gcc.target/i386/sse-12.c b/gcc/testsuite/gcc.target/i386/sse-12.c index fde56261d8f..ddde2df6657 100644 --- a/gcc/testsuite/gcc.target/i386/sse-12.c +++ b/gcc/testsuite/gcc.target/i386/sse-12.c @@ -3,7 +3,7 @@ popcntintrin.h gfniintrin.h and mm_malloc.h are usable with -O -std=c89 -pedantic-errors. */ /* { dg-do compile } */ -/* { dg-options "-O -std=c89 -pedantic-errors -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512bw -mavx512dq -mavx512vl -mavx512vbmi -mavx512vbmi2 -mavx512ifma -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavxifma" } */ +/* { dg-options "-O -std=c89 -pedantic-errors -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512bw -mavx512dq -mavx512vl -mavx512vbmi -mavx512vbmi2 -mavx512ifma -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavxifma -mavxvnniint8" } */ #include diff --git a/gcc/testsuite/gcc.target/i386/sse-13.c b/gcc/testsuite/gcc.target/i386/sse-13.c index bb29555babe..2b293216c6f 100644 --- a/gcc/testsuite/gcc.target/i386/sse-13.c +++ b/gcc/testsuite/gcc.target/i386/sse-13.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O2 -Werror-implicit-function-declaration -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vbmi2 -mavx512ifma -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mavx512vp2intersect -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma" } */ +/* { dg-options "-O2 -Werror-implicit-function-declaration -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vbmi2 -mavx512ifma -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mavx512vp2intersect -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mavx512bitalg -mpconfig -mwbnoinvd -mavx512bf16 -menqcmd -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma -mavxvnniint8" } */ /* { dg-add-options bind_pic_locally } */ #include diff --git a/gcc/testsuite/gcc.target/i386/sse-14.c b/gcc/testsuite/gcc.target/i386/sse-14.c index f2701ddaaf9..78b51048b90 100644 --- a/gcc/testsuite/gcc.target/i386/sse-14.c +++ b/gcc/testsuite/gcc.target/i386/sse-14.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O0 -Werror-implicit-function-declaration -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mpconfig -mwbnoinvd -mavx512vl -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma" } */ +/* { dg-options "-O0 -Werror-implicit-function-declaration -march=k8 -msse4a -m3dnow -mavx -mavx2 -mfma4 -mxop -maes -mpclmul -mpopcnt -mabm -mlzcnt -mbmi -mbmi2 -mtbm -mlwp -mfsgsbase -mrdrnd -mf16c -mfma -mrtm -mrdseed -mprfchw -madx -mfxsr -mxsaveopt -mavx512f -mavx512er -mavx512cd -mavx512pf -msha -mprefetchwt1 -mxsavec -mxsaves -mclflushopt -mavx512dq -mavx512bw -mavx512vl -mavx512ifma -mavx512vbmi -mavx512vbmi2 -mavx5124fmaps -mavx5124vnniw -mavx512vpopcntdq -mclwb -mmwaitx -mclzero -mpku -msgx -mrdpid -mgfni -mpconfig -mwbnoinvd -mavx512vl -mavx512bf16 -menqcmd -mavx512vp2intersect -mserialize -mtsxldtrk -mamx-tile -mamx-int8 -mamx-bf16 -mkl -mwidekl -mavxvnni -mavx512fp16 -mavxifma -mavxvnniint8" } */ /* { dg-add-options bind_pic_locally } */ #include diff --git a/gcc/testsuite/gcc.target/i386/sse-22.c b/gcc/testsuite/gcc.target/i386/sse-22.c index 3d196975b1e..cc1c8cfa4be 100644 --- a/gcc/testsuite/gcc.target/i386/sse-22.c +++ b/gcc/testsuite/gcc.target/i386/sse-22.c @@ -103,7 +103,7 @@ #ifndef DIFFERENT_PRAGMAS -#pragma GCC target ("sse4a,3dnow,avx,avx2,fma4,xop,aes,pclmul,popcnt,abm,lzcnt,bmi,bmi2,tbm,lwp,fsgsbase,rdrnd,f16c,rtm,rdseed,prfchw,adx,fxsr,xsaveopt,avx512f,avx512er,avx512cd,avx512pf,sha,prefetchwt1,avx512vl,avx512bw,avx512dq,avx512vbmi,avx512vbmi2,avx512ifma,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,gfni,avx512bitalg,avx512bf16,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma") +#pragma GCC target ("sse4a,3dnow,avx,avx2,fma4,xop,aes,pclmul,popcnt,abm,lzcnt,bmi,bmi2,tbm,lwp,fsgsbase,rdrnd,f16c,rtm,rdseed,prfchw,adx,fxsr,xsaveopt,avx512f,avx512er,avx512cd,avx512pf,sha,prefetchwt1,avx512vl,avx512bw,avx512dq,avx512vbmi,avx512vbmi2,avx512ifma,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,gfni,avx512bitalg,avx512bf16,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma,avxvnniint8") #endif /* Following intrinsics require immediate arguments. They @@ -220,7 +220,7 @@ test_4 (_mm_cmpestrz, int, __m128i, int, __m128i, int, 1) /* immintrin.h (AVX/AVX2/RDRND/FSGSBASE/F16C/RTM/AVX512F/SHA) */ #ifdef DIFFERENT_PRAGMAS -#pragma GCC target ("avx,avx2,rdrnd,fsgsbase,f16c,rtm,avx512f,avx512er,avx512cd,avx512pf,sha,avx512vl,avx512bw,avx512dq,avx512ifma,avx512vbmi,avx512vbmi2,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,gfni,avx512bitalg,avx512bf16,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma") +#pragma GCC target ("avx,avx2,rdrnd,fsgsbase,f16c,rtm,avx512f,avx512er,avx512cd,avx512pf,sha,avx512vl,avx512bw,avx512dq,avx512ifma,avx512vbmi,avx512vbmi2,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,gfni,avx512bitalg,avx512bf16,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma,avxvnniint8") #endif #include test_1 (_cvtss_sh, unsigned short, float, 1) diff --git a/gcc/testsuite/gcc.target/i386/sse-23.c b/gcc/testsuite/gcc.target/i386/sse-23.c index d3a233f90fc..270f4483491 100644 --- a/gcc/testsuite/gcc.target/i386/sse-23.c +++ b/gcc/testsuite/gcc.target/i386/sse-23.c @@ -843,6 +843,6 @@ #define __builtin_ia32_vpclmulqdq_v2di(A, B, C) __builtin_ia32_vpclmulqdq_v2di(A, B, 1) #define __builtin_ia32_vpclmulqdq_v8di(A, B, C) __builtin_ia32_vpclmulqdq_v8di(A, B, 1) -#pragma GCC target ("sse4a,3dnow,avx,avx2,fma4,xop,aes,pclmul,popcnt,abm,lzcnt,bmi,bmi2,tbm,lwp,fsgsbase,rdrnd,f16c,fma,rtm,rdseed,prfchw,adx,fxsr,xsaveopt,avx512f,avx512er,avx512cd,avx512pf,sha,prefetchwt1,xsavec,xsaves,clflushopt,avx512bw,avx512dq,avx512vl,avx512vbmi,avx512ifma,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,clwb,mwaitx,clzero,pku,sgx,rdpid,gfni,avx512vbmi2,vpclmulqdq,avx512bitalg,pconfig,wbnoinvd,avx512bf16,enqcmd,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma") +#pragma GCC target ("sse4a,3dnow,avx,avx2,fma4,xop,aes,pclmul,popcnt,abm,lzcnt,bmi,bmi2,tbm,lwp,fsgsbase,rdrnd,f16c,fma,rtm,rdseed,prfchw,adx,fxsr,xsaveopt,avx512f,avx512er,avx512cd,avx512pf,sha,prefetchwt1,xsavec,xsaves,clflushopt,avx512bw,avx512dq,avx512vl,avx512vbmi,avx512ifma,avx5124fmaps,avx5124vnniw,avx512vpopcntdq,clwb,mwaitx,clzero,pku,sgx,rdpid,gfni,avx512vbmi2,vpclmulqdq,avx512bitalg,pconfig,wbnoinvd,avx512bf16,enqcmd,avx512vp2intersect,serialize,tsxldtrk,amx-tile,amx-int8,amx-bf16,kl,widekl,avxvnni,avx512fp16,avxifma,avxvnniint8") #include diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp index 3d779081ff6..d60d8c699ac 100644 --- a/gcc/testsuite/lib/target-supports.exp +++ b/gcc/testsuite/lib/target-supports.exp @@ -9534,6 +9534,18 @@ proc check_effective_target_avxifma { } { } "-O0 -mavxifma" ] } +# Return 1 if avxvnniint8 instructions can be compiled. +proc check_effective_target_avxvnniint8 { } { + return [check_no_compiler_messages avxvnniint8 object { + typedef int __v8si __attribute__ ((__vector_size__ (32))); + __v8si + _mm256_dpbssd_epi32 (__v8si __A, __v8si __B, __v8si __C) + { + return __builtin_ia32_vpdpbssd256 (__A, __B, __C); + } + } "-O0 -mavxvnniint8" ] +} + # Return 1 if sse instructions can be compiled. proc check_effective_target_sse { } { return [check_no_compiler_messages sse object {