From patchwork Mon May 15 02:26:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 93796 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6614436vqo; Sun, 14 May 2023 19:27:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4X80KMX0PGD4dYw3rWpjV96Y1jfx5Sd4nOWVhtGaYmojNm+7TNNkhI4d8Z4O5qtdB+b4bf X-Received: by 2002:a05:6402:12d5:b0:50c:9582:e968 with SMTP id k21-20020a05640212d500b0050c9582e968mr21805678edx.36.1684117667143; Sun, 14 May 2023 19:27:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684117667; cv=none; d=google.com; s=arc-20160816; b=YHO61W6HXFbSb9cw0ZAARhRw89rgA1sY4gjGWV6ANf7sO96EI09ESWTyWjn0jH9fb7 i2i5LNNVAvaTmLp5QndvLtJ292Up9YWsNKIc4p8QNC+NZMLHRWHafLNoEYxm6HaOj+Ik x0uRjSQApFd1jhI5gG/KZtjMYAKTHtJIOOREmeg2vGbHrITMcr/2T0XrHQqdKK2wZRaz rOhGi9eumsPlrFDffrhb14zH0ft6ANJya4Rjl/l8ChgRntkrUbGF/fUf6+FTe1EoED6R FyVHmPKZF9SpAJiG22UNqU9VpKFVcl8n1F7C/1lnrOBQOxzl9N+nm8OeMVl3/TTNbCMU 0gMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=5NBJAF/U3MGbrTFjaJBk07m3Zlv3OOlb+48dFslY59E=; b=mIcDNy1NEkE6U8x2Xj+Di6z5o0vcIJNgTabuo5MeyouJbKPKit0bA7HbDvgVzHtm/z NSrcToPC2099lVNUr9tkqujt+VsdNjwxdqWiQgIwPBocgPbelHg+M114zpxkQXK958w2 bPqccgKQ7M2UzGAJ6D9Sg8iMFk/luq1kJ1Te1smLqT6d1bPZ2pqCtMPsJqfdgEPBuRgE xYlPvBkZkrQlqexDzg+UkBuYVdbOjZEbiRIerw7B5+J6VfSk9bx59RHEMbKX/TbUWb/2 U3nPEPkEta9kgHq9PxIPQfT/h6Duh35w0sMQYcnU0YRUbKfFTRj1fpSE6uIfykT+WIo+ fzug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id u11-20020aa7db8b000000b0050bde7eb4ccsi10779431edt.682.2023.05.14.19.27.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 19:27:47 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 48E403857348 for ; Mon, 15 May 2023 02:27:42 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgeu2.qq.com (smtpbgeu2.qq.com [18.194.254.142]) by sourceware.org (Postfix) with ESMTPS id 482B53858D32 for ; Mon, 15 May 2023 02:27:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 482B53858D32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp66t1684117610t200ugge Received: from server1.localdomain ( [58.60.1.22]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 15 May 2023 10:26:48 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: ILHsT53NKPgsCL5XJ6bipglaK36YQh2ODRjPJkFeUYqkCOxhlkOnUArObAVZ5 uHkoDwAQiLsWTd1yIDCa9a4KUXNSTX7JUdBuAQ4BiMNF7tAa9jYgegFRM0jMut7EQuUwoSl IfRB3N4/7lxVNjvLCAu/bMLhSRt/z7X31Snp4vnr1ig8InSebvZ75aABqjCJ957tzGbIoO6 joL4lj262E/HQA4UyRT2KO6WVp3PfuKRYGjZIXz+wze/h5CvKFVVdPCEuWJ51RHdNPJmOtK CKaaW3TW0vfno5BB2TdHfnwDwDVln7w2IW4gZytyuvA13WqtxUbgtwFHKb0f47GkjsYZSxf oEugFBYtJw3zgsfQ1emfAlSOnoStEzKNXFwJ4ERnZdJ6a5wPvBdVkwjE0Qn5Ru6QLHtampy AQhv8uukmwo= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 15043565124055795335 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, palmer@dabbelt.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Add VECTOR_ALIGNMENT_REACHABLE && BUILTIN_VECTORIZATION_COST target hook to optimize RVV VLS auto-vectorization codegen Date: Mon, 15 May 2023 10:26:47 +0800 Message-Id: <20230515022647.182902-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, T_SPF_HELO_TEMPERROR autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765925366680675392?= X-GMAIL-MSGID: =?utf-8?q?1765925366680675392?= From: Juzhe-Zhong This patch is to optimize the codegen of RVV VLS auto-vectorizaiton codegen due to alignment. void __attribute__((noinline, noclone)) f3 (int * __restrict dst, int * __restrict op1, int * __restrict op2, int count) { for (int i = 0; i < count; ++i) dst[i] = op1[i] + op2[i]; } Before this patch: f3: ble a3,zero,.L1 srli a5,a1,2 negw a5,a5 andi a4,a5,3 sext.w a3,a3 beq a4,zero,.L3 lw a7,0(a1) lw a6,0(a2) andi a5,a5,2 addw a6,a6,a7 sw a6,0(a0) beq a5,zero,.L3 lw a7,4(a1) lw a5,4(a2) li a6,3 addw a5,a5,a7 sw a5,4(a0) bne a4,a6,.L3 lw a6,8(a2) lw a5,8(a1) addw a5,a5,a6 sw a5,8(a0) .L3: subw a3,a3,a4 slli a6,a4,2 slli a5,a3,32 srli a5,a5,32 add a1,a1,a6 add a2,a2,a6 add a0,a0,a6 li a3,4 .L6: mv a4,a5 bleu a5,a3,.L5 li a4,4 .L5: vsetvli zero,a4,e32,m1,ta,ma vle32.v v1,0(a1) vle32.v v2,0(a2) vsetivli zero,4,e32,m1,ta,ma sub a5,a5,a4 vadd.vv v1,v1,v2 vsetvli zero,a4,e32,m1,ta,ma vse32.v v1,0(a0) addi a1,a1,16 addi a2,a2,16 addi a0,a0,16 bne a5,zero,.L6 .L1: ret After this patch: f3: ble a3,zero,.L1 li a4,4 .L4: mv a5,a3 bleu a3,a4,.L3 li a5,4 .L3: vsetvli zero,a5,e32,m1,ta,ma vle32.v v2,0(a1) vle32.v v1,0(a2) vsetivli zero,4,e32,m1,ta,ma sub a3,a3,a5 vadd.vv v1,v1,v2 vsetvli zero,a5,e32,m1,ta,ma vse32.v v1,0(a0) addi a2,a2,16 addi a0,a0,16 addi a1,a1,16 bne a3,zero,.L4 .L1: ret The TARGET_VECTORIZE_VECTOR_ALIGNMENT_REACHABLE is directly coming from ARM SVE. The TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST is same as GCN port that vectorize all cases by default. We will need to support accurate vector cost model in the future. gcc/ChangeLog: * config/riscv/riscv.cc (riscv_simd_vector_alignment_reachable): New function. (riscv_vectorization_cost): New function. (TARGET_VECTORIZE_VECTOR_ALIGNMENT_REACHABLE): New target hook. (TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST): New target hook. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/align-2.c: New test. --- gcc/config/riscv/riscv.cc | 39 +++++++++++++++++++ .../gcc.target/riscv/rvv/autovec/align-2.c | 12 ++++++ 2 files changed, 51 insertions(+) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/align-2.c diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc index a5776a550b2..54306327cb3 100644 --- a/gcc/config/riscv/riscv.cc +++ b/gcc/config/riscv/riscv.cc @@ -7517,6 +7517,39 @@ riscv_vectorize_preferred_vector_alignment (const_tree type) return TYPE_ALIGN (type); } +/* Implement target hook TARGET_VECTORIZE_VECTOR_ALIGNMENT_REACHABLE. */ + +static bool +riscv_simd_vector_alignment_reachable (const_tree type, bool is_packed) +{ + if (is_packed) + return false; + + /* For fixed-length vectors, check that the vectorizer will aim for + full-vector alignment. This isn't true for generic GCC vectors + that are wider than the ABI maximum of 128 bits. */ + poly_uint64 preferred_alignment + = riscv_vectorize_preferred_vector_alignment (type); + if (TREE_CODE (TYPE_SIZE (type)) == INTEGER_CST + && maybe_ne (wi::to_widest (TYPE_SIZE (type)), preferred_alignment)) + return false; + + /* Vectors whose size is <= BIGGEST_ALIGNMENT are naturally aligned. */ + return true; +} + +/* Implement TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST. */ + +int +riscv_vectorization_cost (enum vect_cost_for_stmt ARG_UNUSED (type_of_cost), + tree ARG_UNUSED (vectype), int ARG_UNUSED (misalign)) +{ + /* TODO: Always vectorize. The vectorization COST model is not accurate, + we will need to support accurate vectorization COST model according + to '-mtune' in the future. */ + return 1; +} + /* Initialize the GCC target structure. */ #undef TARGET_ASM_ALIGNED_HI_OP #define TARGET_ASM_ALIGNED_HI_OP "\t.half\t" @@ -7792,6 +7825,12 @@ riscv_vectorize_preferred_vector_alignment (const_tree type) #undef TARGET_VECTORIZE_PREFERRED_VECTOR_ALIGNMENT #define TARGET_VECTORIZE_PREFERRED_VECTOR_ALIGNMENT \ riscv_vectorize_preferred_vector_alignment +#undef TARGET_VECTORIZE_VECTOR_ALIGNMENT_REACHABLE +#define TARGET_VECTORIZE_VECTOR_ALIGNMENT_REACHABLE \ + riscv_simd_vector_alignment_reachable +#undef TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST +#define TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST \ + riscv_vectorization_cost struct gcc_target targetm = TARGET_INITIALIZER; diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/align-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/align-2.c new file mode 100644 index 00000000000..812584e9d25 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/align-2.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv32gcv -mabi=ilp32d -O3 --param riscv-autovec-preference=fixed-vlmax" } */ + +void __attribute__((noinline, noclone)) +f (int * __restrict dst, int * __restrict op1, int * __restrict op2, int count) +{ + for (int i = 0; i < count; ++i) + dst[i] = op1[i] + op2[i]; +} + +/* { dg-final { scan-assembler-not "lw" } } */ +/* { dg-final { scan-assembler-not "sw" } } */