From patchwork Fri Jun 16 02:09:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: liuhongt X-Patchwork-Id: 108782 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1038697vqr; Thu, 15 Jun 2023 19:14:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4vNnTkMrxFwhucTClgR8OEU4c9r7Fut06kpfSiGuZxx0TbMyIQZfOfcrgaUP6b46h25nDl X-Received: by 2002:a17:907:3f18:b0:983:cb6c:8aa3 with SMTP id hq24-20020a1709073f1800b00983cb6c8aa3mr592357ejc.59.1686881642562; Thu, 15 Jun 2023 19:14:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686881642; cv=none; d=google.com; s=arc-20160816; b=CsLc7EIOwZY1usLOQ0VJ88rdT1n/m1ziyyK9Ij7SDMsw9v4ARDzp44SKOzMop0kCgX suIdCT5/O13FukvvVLSVSn/5nWfo9hKTGxNrUzjiL73e1UBen2FNTmMs+4ktsl34F9HP FD9L/LtBcVM7MOlhO5YFdA7qRwBWf+9nZBMxiLuizpRY+WQQF7V03ehEcnP3m2KyDa8r mKWC+E264U8csCFEy4BXakG8kgL9HIJAtsl/SzjbOBl3ZiS9o8+utYLwgA8QnQk+azUQ I+6YGUMfXIsISeNVT6FHHrAAfM3VuTaaEbwcBLwMHKcLBeHjzvTOe/HcTgVq3FDwsZQG 9Ocw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=llw6ipOTuT3Cvy6RofQYITL9Q42VfakEcpcOdpCVmO4=; b=l6p8JZectpmtJ7KrDJa+4vn56FHK7tGyllyyWOOAoHSFLmUH80DeOTTKyiLCMQttkB RU63I1sMaIw4MMK5U3nJE9ysgqxzfNNCH3p+71WSR53YFgWNn8PjedYr9BqcWAWcJkZ1 HKL8seOohlCAQFfo0dVKzOzMzNqsG/iUCMQqZoAyRNQiBT31oDB9/9KcsJX0Ga6Rsztq mVnHu0OeJ1gG4YH0HmdGLWtqA/2nW3ZhrGiGfuW2ocrnArw3lvvAUwx3jNnllHVB9yIp PbGcYLF6LGTCV5U9NvUExRqXle7QxJs0V5z4yJexwBoutYnz0cLJ3ryzbc63/HD+jEv2 WHrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=x1aSCRYR; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id n4-20020a17090625c400b00974e7637ea3si9394367ejb.702.2023.06.15.19.14.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jun 2023 19:14:02 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=x1aSCRYR; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5FA753855581 for ; Fri, 16 Jun 2023 02:14:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5FA753855581 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1686881641; bh=llw6ipOTuT3Cvy6RofQYITL9Q42VfakEcpcOdpCVmO4=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=x1aSCRYRO3JR5oUgJB+fsTCoDMOLPnsaSQcw7TZ1IhEzOquT1vK9kglVX7talHGwb mQ7iHFKHNHu6zVldjAcI/i49PwGfEcCn251rVlP0rho0RFfUb7XW1mAYIb2UABMNSF MH/34QnVA9nTmFgXIAlY3OpzlnfXKGbsASmTMT1g= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by sourceware.org (Postfix) with ESMTPS id 3F0953857342 for ; Fri, 16 Jun 2023 02:12:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3F0953857342 X-IronPort-AV: E=McAfee;i="6600,9927,10742"; a="359094494" X-IronPort-AV: E=Sophos;i="6.00,246,1681196400"; d="scan'208";a="359094494" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jun 2023 19:12:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10742"; a="825537975" X-IronPort-AV: E=Sophos;i="6.00,246,1681196400"; d="scan'208";a="825537975" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by fmsmga002.fm.intel.com with ESMTP; 15 Jun 2023 19:11:59 -0700 Received: from shliclel4217.sh.intel.com (shliclel4217.sh.intel.com [10.239.240.127]) by shvmail03.sh.intel.com (Postfix) with ESMTP id 6972A10079A5; Fri, 16 Jun 2023 10:11:58 +0800 (CST) To: gcc-patches@gcc.gnu.org Cc: ubizjak@gmail.com Subject: [PATCH 2/2] Refined 256/512-bit vpacksswb/vpackssdw patterns. Date: Fri, 16 Jun 2023 10:09:58 +0800 Message-Id: <20230616020958.1413585-2-hongtao.liu@intel.com> X-Mailer: git-send-email 2.39.1.388.g2fc9e9ca3c In-Reply-To: <20230616020958.1413585-1-hongtao.liu@intel.com> References: <20230616020958.1413585-1-hongtao.liu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: liuhongt via Gcc-patches From: liuhongt Reply-To: liuhongt Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768823605602945784?= X-GMAIL-MSGID: =?utf-8?q?1768823605602945784?= The packing in vpacksswb/vpackssdw is not a simple concat, it's an interweave from src1 and src2 for every 128 bit(or 64-bit for the ss_truncate result). .i.e. dst[192-255] = ss_truncate (src2[128-255]) dst[128-191] = ss_truncate (src1[128-255]) dst[64-127] = ss_truncate (src2[0-127]) dst[0-63] = ss_truncate (src1[0-127] The patch refined those patterns with an extra vec_select for the interweave. The patch will fix below testcase which failed after g:921b841350c4fc298d09f6c5674663e0f4208610 added constant-folding for SS_TRUNCATE FAIL: gcc.target/i386/avx2-vpackssdw-2.c execution test. Bootstrapped and regtested on x86_64-pc-linux-gnu. Ok for trunk? gcc/ChangeLog: PR target/110235 * config/i386/sse.md (_packsswb): Split to below 3 new define_insns. (sse2_packsswb): New define_insn. (avx2_packsswb): Ditto. (avx512bw_packsswb): Ditto. (_packssdw): Split to below 3 new define_insns. (sse2_packssdw): New define_insn. (avx2_packssdw): Ditto. (avx512bw_packssdw): Ditto. gcc/testsuite/ChangeLog: * gcc.target/i386/avx512bw-vpackssdw-3.c: New test. * gcc.target/i386/avx512bw-vpacksswb-3.c: New test. --- gcc/config/i386/sse.md | 165 ++++++++++++++++-- .../gcc.target/i386/avx512bw-vpackssdw-3.c | 55 ++++++ .../gcc.target/i386/avx512bw-vpacksswb-3.c | 50 ++++++ 3 files changed, 252 insertions(+), 18 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/avx512bw-vpackssdw-3.c create mode 100644 gcc/testsuite/gcc.target/i386/avx512bw-vpacksswb-3.c diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index 83e3f534fd2..cc4e4620257 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -17762,14 +17762,14 @@ (define_expand "vec_pack_sbool_trunc_qi" DONE; }) -(define_insn "_packsswb" - [(set (match_operand:VI1_AVX512 0 "register_operand" "=x,") - (vec_concat:VI1_AVX512 - (ss_truncate: - (match_operand: 1 "register_operand" "0,")) - (ss_truncate: - (match_operand: 2 "vector_operand" "xBm,m"))))] - "TARGET_SSE2 && && " +(define_insn "sse2_packsswb" + [(set (match_operand:V16QI 0 "register_operand" "=x,Yw") + (vec_concat:V16QI + (ss_truncate:V8QI + (match_operand:V8HI 1 "register_operand" "0,Yw")) + (ss_truncate:V8QI + (match_operand:V8HI 2 "vector_operand" "xBm,Ywm"))))] + "TARGET_SSE2 && && " "@ packsswb\t{%2, %0|%0, %2} vpacksswb\t{%2, %1, %0|%0, %1, %2}" @@ -17777,16 +17777,93 @@ (define_insn "_packsswb" (set_attr "type" "sselog") (set_attr "prefix_data16" "1,*") (set_attr "prefix" "orig,") - (set_attr "mode" "")]) + (set_attr "mode" "TI")]) -(define_insn "_packssdw" - [(set (match_operand:VI2_AVX2 0 "register_operand" "=x,") - (vec_concat:VI2_AVX2 - (ss_truncate: - (match_operand: 1 "register_operand" "0,")) - (ss_truncate: - (match_operand: 2 "vector_operand" "xBm,m"))))] - "TARGET_SSE2 && && " +(define_insn "avx2_packsswb" + [(set (match_operand:V32QI 0 "register_operand" "=Yw") + (vec_select:V32QI + (vec_concat:V32QI + (ss_truncate:V16QI + (match_operand:V16HI 1 "register_operand" "Yw")) + (ss_truncate:V16QI + (match_operand:V16HI 2 "vector_operand" "Ywm"))) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7) + (const_int 16) (const_int 17) + (const_int 18) (const_int 19) + (const_int 20) (const_int 21) + (const_int 22) (const_int 23) + (const_int 8) (const_int 9) + (const_int 10) (const_int 11) + (const_int 12) (const_int 13) + (const_int 14) (const_int 15) + (const_int 24) (const_int 25) + (const_int 26) (const_int 27) + (const_int 28) (const_int 29) + (const_int 30) (const_int 31)])))] + "TARGET_AVX2 && && " + "vpacksswb\t{%2, %1, %0|%0, %1, %2}" + [(set_attr "type" "sselog") + (set_attr "prefix" "") + (set_attr "mode" "OI")]) + +(define_insn "avx512bw_packsswb" + [(set (match_operand:V64QI 0 "register_operand" "=v") + (vec_select:V64QI + (vec_concat:V64QI + (ss_truncate:V32QI + (match_operand:V32HI 1 "register_operand" "v")) + (ss_truncate:V32QI + (match_operand:V32HI 2 "vector_operand" "vm"))) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7) + (const_int 32) (const_int 33) + (const_int 34) (const_int 35) + (const_int 36) (const_int 37) + (const_int 38) (const_int 39) + (const_int 8) (const_int 9) + (const_int 10) (const_int 11) + (const_int 12) (const_int 13) + (const_int 14) (const_int 15) + (const_int 40) (const_int 41) + (const_int 42) (const_int 43) + (const_int 44) (const_int 45) + (const_int 46) (const_int 47) + (const_int 16) (const_int 17) + (const_int 18) (const_int 19) + (const_int 20) (const_int 21) + (const_int 22) (const_int 23) + (const_int 48) (const_int 49) + (const_int 50) (const_int 51) + (const_int 52) (const_int 53) + (const_int 54) (const_int 55) + (const_int 24) (const_int 25) + (const_int 26) (const_int 27) + (const_int 28) (const_int 29) + (const_int 30) (const_int 31) + (const_int 56) (const_int 57) + (const_int 58) (const_int 59) + (const_int 60) (const_int 61) + (const_int 62) (const_int 63)])))] + + "TARGET_AVX512BW" + "vpacksswb\t{%2, %1, %0|%0, %1, %2}" + [(set_attr "type" "sselog") + (set_attr "prefix" "") + (set_attr "mode" "XI")]) + +(define_insn "sse2_packssdw" + [(set (match_operand:V8HI 0 "register_operand" "=x,Yw") + (vec_concat:V8HI + (ss_truncate:V4HI + (match_operand:V4SI 1 "register_operand" "0,Yw")) + (ss_truncate:V4HI + (match_operand:V4SI 2 "vector_operand" "xBm,Ywm"))))] + "TARGET_SSE2 && && " "@ packssdw\t{%2, %0|%0, %2} vpackssdw\t{%2, %1, %0|%0, %1, %2}" @@ -17794,7 +17871,59 @@ (define_insn "_packssdw" (set_attr "type" "sselog") (set_attr "prefix_data16" "1,*") (set_attr "prefix" "orig,") - (set_attr "mode" "")]) + (set_attr "mode" "TI")]) + +(define_insn "avx2_packssdw" + [(set (match_operand:V16HI 0 "register_operand" "=Yw") + (vec_select:V16HI + (vec_concat:V16HI + (ss_truncate:V8HI + (match_operand:V8SI 1 "register_operand" "Yw")) + (ss_truncate:V8HI + (match_operand:V8SI 2 "vector_operand" "Ywm"))) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 8) (const_int 9) + (const_int 10) (const_int 11) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7) + (const_int 12) (const_int 13) + (const_int 14) (const_int 15)])))] + "TARGET_AVX2 && && " + "vpackssdw\t{%2, %1, %0|%0, %1, %2}" + [(set_attr "type" "sselog") + (set_attr "prefix" "") + (set_attr "mode" "OI")]) + +(define_insn "avx512bw_packssdw" + [(set (match_operand:V32HI 0 "register_operand" "=v") + (vec_select:V32HI + (vec_concat:V32HI + (ss_truncate:V16HI + (match_operand:V16SI 1 "register_operand" "v")) + (ss_truncate:V16HI + (match_operand:V16SI 2 "vector_operand" "vm"))) + (parallel [(const_int 0) (const_int 1) + (const_int 2) (const_int 3) + (const_int 16) (const_int 17) + (const_int 18) (const_int 19) + (const_int 4) (const_int 5) + (const_int 6) (const_int 7) + (const_int 20) (const_int 21) + (const_int 22) (const_int 23) + (const_int 8) (const_int 9) + (const_int 10) (const_int 11) + (const_int 24) (const_int 25) + (const_int 26) (const_int 27) + (const_int 12) (const_int 13) + (const_int 14) (const_int 15) + (const_int 28) (const_int 29) + (const_int 30) (const_int 31)])))] + "TARGET_AVX512BW" + "vpackssdw\t{%2, %1, %0|%0, %1, %2}" + [(set_attr "type" "sselog") + (set_attr "prefix" "") + (set_attr "mode" "XI")]) ;; This is different from rtl unsigned saturation, the instruction does ;; unsigned saturation for signed value. diff --git a/gcc/testsuite/gcc.target/i386/avx512bw-vpackssdw-3.c b/gcc/testsuite/gcc.target/i386/avx512bw-vpackssdw-3.c new file mode 100644 index 00000000000..ae839e8d2c6 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avx512bw-vpackssdw-3.c @@ -0,0 +1,55 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavx512bw" } */ +/* { dg-require-effective-target avx512bw } */ + +#define AVX512BW +#include "avx512f-helper.h" + +#define DST_SIZE (AVX512F_LEN / 16) +#define SRC_SIZE (AVX512F_LEN / 32) + +#include "limits.h" + +#include "avx512f-mask-type.h" + +static short +int_to_short (int iVal) +{ + short sVal; + + if (iVal < -32768) + sVal = -32768; + else if (iVal > 32767) + sVal = 32767; + else + sVal = iVal; + + return sVal; +} + +void +TEST (void) +{ + union512i_d s1, s2; + union512i_w res1; + short dst_ref[32]; + int i; + + s1.x = _mm512_set_epi32 (1, 2, 3, 4, 65000, 20, 30, 90, 88, 44, 33, 22, 11, 98, 76, -65000); + s2.x = _mm512_set_epi32 (80, 40, 31, 21, 10, 99, 74, -65000, 2, 3, 4, 5, 65010, 21, 31, 91); + res1.x = _mm512_packs_epi32 (s1.x, s2.x); + for (int i = 0; i != 4; i++) + { + dst_ref[i] = int_to_short (s1.a[i]); + dst_ref[i + 4] = int_to_short (s2.a[i]); + dst_ref[i + 8] = int_to_short (s1.a[i + 4]); + dst_ref[i + 12] = int_to_short (s2.a[i + 4]); + dst_ref[i + 16] = int_to_short (s1.a[i + 8]); + dst_ref[i + 20] = int_to_short (s2.a[i + 8]); + dst_ref[i + 24] = int_to_short (s1.a[i + 12]); + dst_ref[i + 28] = int_to_short (s2.a[i + 12]); + } + + if (check_union512i_w (res1, dst_ref)) + abort (); +} diff --git a/gcc/testsuite/gcc.target/i386/avx512bw-vpacksswb-3.c b/gcc/testsuite/gcc.target/i386/avx512bw-vpacksswb-3.c new file mode 100644 index 00000000000..056c735ae0e --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/avx512bw-vpacksswb-3.c @@ -0,0 +1,50 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -mavx512bw" } */ +/* { dg-require-effective-target avx512bw } */ + +#define AVX512BW +#include "avx512f-helper.h" + +static char +short_to_byte (short iVal) +{ + short sVal; + + if (iVal < -128) + sVal = -128; + else if (iVal > 127) + sVal = 127; + else + sVal = iVal; + + return sVal; +} + +void +TEST (void) +{ + union512i_w s1, s2; + union512i_b res1; + char dst_ref[64]; + int i; + + s1.x = _mm512_set_epi16 (1, 2, 3, 4, 650, 20, 30, 90, 88, 44, 33, 22, 11, 98, 76, -650, + 128, 230, -112, -128, -3, -4, -7, 9, 10, 11, 12, 13, -223, 10, 8, 11); + s2.x = _mm512_set_epi16 (80, 40, 31, 21, 10, 99, 74, -650, 2, 3, 4, 5, 650, 21, 31, 91, + 280, -140, 310, 20, 9, 98, 73, -651, 3, 4, 5, 6, 651, 22, 32, 92); + res1.x = _mm512_packs_epi16 (s1.x, s2.x); + for (int i = 0; i != 8; i++) + { + dst_ref[i] = short_to_byte (s1.a[i]); + dst_ref[i + 8] = short_to_byte (s2.a[i]); + dst_ref[i + 16] = short_to_byte (s1.a[i + 8]); + dst_ref[i + 24] = short_to_byte (s2.a[i + 8]); + dst_ref[i + 32] = short_to_byte (s1.a[i + 16]); + dst_ref[i + 40] = short_to_byte (s2.a[i + 16]); + dst_ref[i + 48] = short_to_byte (s1.a[i + 24]); + dst_ref[i + 56] = short_to_byte (s2.a[i + 24]); + } + + if (check_union512i_b (res1, dst_ref)) + abort (); +}