From patchwork Tue Jun 13 09:30:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 107212 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp412497vqr; Tue, 13 Jun 2023 02:31:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6h5R1Nbbv3fr80yCbZMiifLym6dp2LedG0IjHu3clHTxxSlLFq9ZVA/BPa+LLPNuWC0Jyz X-Received: by 2002:a50:ef08:0:b0:514:8d9d:33a2 with SMTP id m8-20020a50ef08000000b005148d9d33a2mr6048652eds.13.1686648698206; Tue, 13 Jun 2023 02:31:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686648698; cv=none; d=google.com; s=arc-20160816; b=cGDruUaEf0KYFL9QTsd00rpRoMDgRRy071uPO/u/BM1HN+u5DIREetve5/pfI7EgN3 8U1F1mue20wpZCJsmTrOtTEDNyqGjYjcjo/SLTGwX9TI8yUPGQUhZ3vBQo7D0sw2E7Rn rjFDkEYQ0R/ioDKpzYKJCh7JCw20uh/O7yBxK6QxoRGfnP/nP3Fmsyo+u9dIGxJwqg1N bNTgQUBwuNjelvGuoxcIAbQGbIYNew0atydHMo12Tmr5PFpwpqzz4SWWow8LLISHwD7q IxZjEsgNRWYMTL+7ms61v4ZrK2h3Ns2v6l85aM8LNGyisc4BX7kw0SbSc24uXVxGNrQ3 Q4qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=hRYXN/qHAhQ3X5IGSlAS8hVt3WO/QOrZx8Nma5l2MyE=; b=fNb6f0jva3stXOLHFdmJW30s/Fxa0UPbO7g6uKt2k8Wizlx5cjleAlwQT8PymEfP8K xzR5T65n+RibDJTkMAf8NYF0hXLvT+bzsUj4CPKmqvlxWtGSp2oAZN50nfPv+bBGxTew IvnR4XfxZ7Q0cN3VpXXfBIOE0VSBx9iMy2ULE+AYZkUfPbdxlE5kECQJPjnlIqBJNdJ6 utPit3wQCuhVdvFBuihXTItODOSXtP51wQ6meYr9M5Zm+Y1OlN5BZjbI6StTyvt+dbI5 aYdbHZYPLFxHdpgSJgzcjTJ61FnmBywNzbqCA2KL4BAqz/xI1GnF9EYBY+6k1vYUvERg D89Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id k20-20020a05640212d400b0050bc27dd649si7311437edx.616.2023.06.13.02.31.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Jun 2023 02:31:38 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BB6B93858280 for ; Tue, 13 Jun 2023 09:31:29 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg1.qq.com (smtpbgsg1.qq.com [54.254.200.92]) by sourceware.org (Postfix) with ESMTPS id 55C8C3858D32 for ; Tue, 13 Jun 2023 09:31:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 55C8C3858D32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp71t1686648657ts9l73k9 Received: from server1.localdomain ( [58.60.1.22]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 13 Jun 2023 17:30:56 +0800 (CST) X-QQ-SSF: 01400000000000F0S000000A0000000 X-QQ-FEAT: 3M0okmaRx3h5bWqPZMrhKHr4EAYzIycL7IQtQ50aFxgwJAKqYrxs7EcZxiWpL qCU02ZDRs1fGudEwMFLhd9cxw2ZuS32zipm3EHkgiLJKApsJVNiPUbUNNrEblIeua5ierbH u9PBOls0BB2OSOCteSKOPl8StZX0rWg3NMxgwionRK5+1wWdIQqUQJPEZIwf2myoMtdbsCU nwLdkWHHGa2PKlA3sX80zMiteM+RpcFTWpBCljpq0xpDkwCgsRpzm2Me9UjGTZVW6Ha0GoH JnRKztc3fQlpgGkfvsffiRm5r/QFE3NIeQ81lv4UGnv6A+/LyIv6brNcQI5hZ+VTTG8BqRI FKOayMTMGLYC7PTcOgoKdxsGkrZMfO3NlsXiVOBQDvL9ru0Mnhvrtc5GQI4/7OsfBa5l5tb s4cpmqQHvuc= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 11134943305901473351 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, palmer@dabbelt.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Fix bug of VLA SLP auto-vectorization Date: Tue, 13 Jun 2023 17:30:55 +0800 Message-Id: <20230613093055.329955-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768579345175962796?= X-GMAIL-MSGID: =?utf-8?q?1768579345175962796?= From: Juzhe-Zhong Sorry for producing bugs in the previous VLA SLP patch. Consider this following permutation: _85 = VEC_PERM_EXPR <{ 99, 17, ... }, { 11, 80, ... }, { 0, POLY_INT_CST [4, 4], 1, POLY_INT_CST [5, 4], 2, POLY_INT_CST [6, 4], ... }>; The correct result should be: _85 = { 99, 11, 17, 80, ... } However, I did wrong in the previous patch. Code sequence before this patch: set mask = { 0, 1, 0, 1, ... } set v0 = { 99, 17, 99, 17, ... } set v1 = { 11, 80, 11, 80, ... } set index = viota (mask) = { 0, 0, 1, 1, 2, 2, ... } set result = vrgather_mu (v0, v1, index, mask) = { 99, 11, 99, 80 } The result is incorrect. After this patch: set mask = { 0, 1, 0, 1, ... } set index = viota (mask) = { 0, 0, 1, 1, 2, 2, ... } set v0 = vrgather ({ 99, 17, 99, 17, ... }, index) = { 99, 99, 17, 17, ... } set v1 = { 11, 80, 11, 80, ... } set result = vrgather_mu (v0, v1, index, mask) = { 99, 11, 17, 80 } The result is what we expected. This issue was discovered in the test I appended in this patch with --param=riscv-autovec-lmul=2. gcc/ChangeLog: * config/riscv/riscv-v.cc (emit_vlmax_decompress_insn): Fix bug. (shuffle_decompress_patterns): Ditto. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/partial/slp-12.c: New test. * gcc.target/riscv/rvv/autovec/partial/slp_run-12.c: New test. --- gcc/config/riscv/riscv-v.cc | 8 ++--- .../riscv/rvv/autovec/partial/slp-12.c | 33 +++++++++++++++++++ .../riscv/rvv/autovec/partial/slp_run-12.c | 30 +++++++++++++++++ 3 files changed, 67 insertions(+), 4 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp-12.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp_run-12.c diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 3ce2eb7f2ad..d797326d736 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -866,7 +866,7 @@ emit_vlmax_masked_gather_mu_insn (rtx target, rtx op, rtx sel, rtx mask) e q r d c b v a # v11 destination after vrgather using viota.m under mask */ static void -emit_vlmax_decompress_insn (rtx target, rtx op, rtx mask) +emit_vlmax_decompress_insn (rtx target, rtx op0, rtx op1, rtx mask) { machine_mode data_mode = GET_MODE (target); machine_mode sel_mode = related_int_vector_mode (data_mode).require (); @@ -876,7 +876,8 @@ emit_vlmax_decompress_insn (rtx target, rtx op, rtx mask) rtx sel = gen_reg_rtx (sel_mode); rtx iota_ops[] = {sel, mask}; emit_vlmax_insn (code_for_pred_iota (sel_mode), RVV_UNOP, iota_ops); - emit_vlmax_masked_gather_mu_insn (target, op, sel, mask); + emit_vlmax_gather_insn (target, op0, sel); + emit_vlmax_masked_gather_mu_insn (target, op1, sel, mask); } /* Emit merge instruction. */ @@ -2444,8 +2445,7 @@ shuffle_decompress_patterns (struct expand_vec_perm_d *d) rtx const_vec = gen_const_vector_dup (sel_mode, 1); rtx mask = gen_reg_rtx (mask_mode); expand_vec_cmp (mask, EQ, vid_repeat, const_vec); - emit_move_insn (d->target, op0); - emit_vlmax_decompress_insn (d->target, op1, mask); + emit_vlmax_decompress_insn (d->target, op0, op1, mask); return true; } diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp-12.c new file mode 100644 index 00000000000..4131fd71a74 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp-12.c @@ -0,0 +1,33 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define VEC_PERM(TYPE) \ + TYPE __attribute__ ((noinline, noclone)) \ + vec_slp_##TYPE (TYPE *restrict a, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + { \ + a[i * 8] += 99; \ + a[i * 8 + 1] += 11; \ + a[i * 8 + 2] += 17; \ + a[i * 8 + 3] += 80; \ + a[i * 8 + 4] += 63; \ + a[i * 8 + 5] += 37; \ + a[i * 8 + 6] += 24; \ + a[i * 8 + 7] += 81; \ + } \ + } + +#define TEST_ALL(T) \ + T (int8_t) \ + T (uint8_t) \ + T (int16_t) \ + T (uint16_t) \ + T (int32_t) \ + T (uint32_t) \ + T (int64_t) \ + T (uint64_t) + +TEST_ALL (VEC_PERM) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp_run-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp_run-12.c new file mode 100644 index 00000000000..42b06b687e9 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/partial/slp_run-12.c @@ -0,0 +1,30 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "slp-12.c" + +#define N (59 * 8) + +#define HARNESS(TYPE) \ + { \ + TYPE a[N], b[8] = { 99, 11, 17, 80, 63, 37, 24, 81 }; \ + for (unsigned int i = 0; i < N; ++i) \ + { \ + a[i] = i * 2 + i % 5; \ + asm volatile ("" ::: "memory"); \ + } \ + vec_slp_##TYPE (a, N / 8); \ + for (unsigned int i = 0; i < N; ++i) \ + { \ + TYPE orig = i * 2 + i % 5; \ + TYPE expected = orig + b[i % 8]; \ + if (a[i] != expected) \ + __builtin_abort (); \ + } \ + } + +int __attribute__ ((optimize (1))) +main (void) +{ + TEST_ALL (HARNESS) +}