From patchwork Wed Jun 14 11:47:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 107912 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1181752vqr; Wed, 14 Jun 2023 04:49:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4qacvOenrTGsKManiEppkZb4/wkBWkZ+5+F2tAuDssV9LkwTVtOsu0XTVc5w7WVa2S/Uuk X-Received: by 2002:a17:907:971d:b0:96f:ea85:3ef6 with SMTP id jg29-20020a170907971d00b0096fea853ef6mr14945870ejc.62.1686743345853; Wed, 14 Jun 2023 04:49:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686743345; cv=none; d=google.com; s=arc-20160816; b=iZkZfmq2Wmzb24JI5R47NodlSdGI4vyR/1ZHheefct8nuacS9PzwHczVQEDRvKVPCb zsr1k9baSIAY87PiU3C9ttwtodjWGdfTAkCFJEGfDCubeuBcCHTeQtjVoGyeaHihc9pC RlSdCVF+eOBcC5HEpYPQ1AT8KycZYCSaWfQODIAe4M0tAJA5QX2jlfq+F4R6xV2AxZMv ULPvw0mhoWb89eX/S2E2PhyJT1iodaO9xBdnC1el/8Rp4ritn6iY21Z4FOGgROLUKWdR j8JUAWuTUsyBdJkAf6MqRZtdRUcVVg0Plo0yQmBXQzhau9zWhpO5fPgghnT2efb6eLws 7IYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:sender:errors-to:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mime-version:user-agent:subject:cc:to:date:dmarc-filter :delivered-to:dkim-signature:dkim-filter; bh=3VlIc111TgtYk2PJBOsfXn+o47DbKnyuQTX9E+ubCAQ=; b=WJVATfHmLgyrHypdWq5w6nK5cQKi1DVCpiTwmwbZZQDpuqjDYLH0k2DtKYS7IujDgh mde062o+KGe5ArHkpVZrZTROlOCOKcxhekvdW7oH/cnjCpA4BzLpgh7MorgeHkOGKlt7 YSsLvzQg9zyF1aaJKzt674rJ+nFNiG1088Gz1Ue4C9beHzlYOiFLb1cQHm7faaENXRNh v7QRUBsyaikmhVmCE0cznom5LX6BBdDooa6wTNFI/c5RcIrrJ6W8H3lYuGNTj9YPEHro LDV8MZcPROZUcj05GdYNkrhpc8JjOPz1Pe055GF91JH+R1HkccDjCNxXDyzAD3miJKtQ aRMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=mIVnDuWv; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id c23-20020a170906171700b009789497a6b9si6716625eje.292.2023.06.14.04.49.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jun 2023 04:49:05 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=mIVnDuWv; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 66BFB3858288 for ; Wed, 14 Jun 2023 11:48:43 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 66BFB3858288 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1686743323; bh=3VlIc111TgtYk2PJBOsfXn+o47DbKnyuQTX9E+ubCAQ=; h=Date:To:cc:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=mIVnDuWvPnw6tVadqiRREd8b82N+7FC4RDaapssF9qCGzmS71It1NH3e9wdPlvQHQ nwBzGpvZbn5zbwVXOMXYl3BBXOqCCt497r5i8e5DaXI+JfgYc3PaPyiK46QnYzVkML UHYMdLVGOfJGGwJ5NAXJyMjMyjImxfXxI0fuu0tY= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id 443CF3857701 for ; Wed, 14 Jun 2023 11:47:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 443CF3857701 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 5BAC82185C; Wed, 14 Jun 2023 11:47:21 +0000 (UTC) Received: from wotan.suse.de (wotan.suse.de [10.160.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 4E00B2C142; Wed, 14 Jun 2023 11:47:21 +0000 (UTC) Date: Wed, 14 Jun 2023 11:47:21 +0000 (UTC) To: gcc-patches@gcc.gnu.org cc: richard.sandiford@arm.com Subject: [PATCH 2/3] Add loop_vinfo argument to vect_get_loop_mask User-Agent: Alpine 2.22 (LSU 394 2020-01-19) MIME-Version: 1.0 X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Biener via Gcc-patches From: Richard Biener Reply-To: Richard Biener Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" Message-Id: <20230614114843.66BFB3858288@sourceware.org> X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768678590805106794?= X-GMAIL-MSGID: =?utf-8?q?1768678590805106794?= This adds a loop_vinfo argument for future use, making the next patch smaller. * tree-vectorizer.h (vect_get_loop_mask): Add loop_vec_info argument. * tree-vect-loop.cc (vect_get_loop_mask): Likewise. (vectorize_fold_left_reduction): Adjust. (vect_transform_reduction): Likewise. (vectorizable_live_operation): Likewise. * tree-vect-stmts.cc (vectorizable_call): Likewise. (vectorizable_operation): Likewise. (vectorizable_store): Likewise. (vectorizable_load): Likewise. (vectorizable_condition): Likewise. --- gcc/tree-vect-loop.cc | 16 +++++++++------- gcc/tree-vect-stmts.cc | 36 +++++++++++++++++++----------------- gcc/tree-vectorizer.h | 3 ++- 3 files changed, 30 insertions(+), 25 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index a9695e5b25d..1897e720389 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -6637,7 +6637,7 @@ vectorize_fold_left_reduction (loop_vec_info loop_vinfo, gimple *new_stmt; tree mask = NULL_TREE; if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo)) - mask = vect_get_loop_mask (gsi, masks, vec_num, vectype_in, i); + mask = vect_get_loop_mask (loop_vinfo, gsi, masks, vec_num, vectype_in, i); /* Handle MINUS by adding the negative. */ if (reduc_fn != IFN_LAST && code == MINUS_EXPR) @@ -7950,8 +7950,8 @@ vect_transform_reduction (loop_vec_info loop_vinfo, gcc_assert (commutative_binary_op_p (code, op.type)); std::swap (vop[0], vop[1]); } - tree mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, - vectype_in, i); + tree mask = vect_get_loop_mask (loop_vinfo, gsi, masks, + vec_num * ncopies, vectype_in, i); gcall *call = gimple_build_call_internal (cond_fn, 4, mask, vop[0], vop[1], vop[0]); new_temp = make_ssa_name (vec_dest, call); @@ -7967,8 +7967,8 @@ vect_transform_reduction (loop_vec_info loop_vinfo, if (masked_loop_p && mask_by_cond_expr) { - tree mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, - vectype_in, i); + tree mask = vect_get_loop_mask (loop_vinfo, gsi, masks, + vec_num * ncopies, vectype_in, i); build_vect_cond_expr (code, vop, mask, gsi); } @@ -10075,7 +10075,8 @@ vectorizable_live_operation (vec_info *vinfo, the loop mask for the final iteration. */ gcc_assert (ncopies == 1 && !slp_node); tree scalar_type = TREE_TYPE (STMT_VINFO_VECTYPE (stmt_info)); - tree mask = vect_get_loop_mask (gsi, &LOOP_VINFO_MASKS (loop_vinfo), + tree mask = vect_get_loop_mask (loop_vinfo, gsi, + &LOOP_VINFO_MASKS (loop_vinfo), 1, vectype, 0); tree scalar_res = gimple_build (&stmts, CFN_EXTRACT_LAST, scalar_type, mask, vec_lhs_phi); @@ -10359,7 +10360,8 @@ vect_record_loop_mask (loop_vec_info loop_vinfo, vec_loop_masks *masks, arrangement. */ tree -vect_get_loop_mask (gimple_stmt_iterator *gsi, vec_loop_masks *masks, +vect_get_loop_mask (loop_vec_info, + gimple_stmt_iterator *gsi, vec_loop_masks *masks, unsigned int nvectors, tree vectype, unsigned int index) { rgroup_controls *rgm = &(*masks)[nvectors - 1]; diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index a7acc032d47..47baf35227f 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -3692,7 +3692,8 @@ vectorizable_call (vec_info *vinfo, unsigned int vec_num = vec_oprnds0.length (); /* Always true for SLP. */ gcc_assert (ncopies == 1); - vargs[varg++] = vect_get_loop_mask (gsi, masks, vec_num, + vargs[varg++] = vect_get_loop_mask (loop_vinfo, + gsi, masks, vec_num, vectype_out, i); } size_t k; @@ -3733,7 +3734,8 @@ vectorizable_call (vec_info *vinfo, unsigned int vec_num = vec_oprnds0.length (); /* Always true for SLP. */ gcc_assert (ncopies == 1); - tree mask = vect_get_loop_mask (gsi, masks, vec_num, + tree mask = vect_get_loop_mask (loop_vinfo, + gsi, masks, vec_num, vectype_out, i); vargs[mask_opno] = prepare_vec_mask (loop_vinfo, TREE_TYPE (mask), mask, @@ -3758,7 +3760,7 @@ vectorizable_call (vec_info *vinfo, int varg = 0; if (masked_loop_p && reduc_idx >= 0) - vargs[varg++] = vect_get_loop_mask (gsi, masks, ncopies, + vargs[varg++] = vect_get_loop_mask (loop_vinfo, gsi, masks, ncopies, vectype_out, j); for (i = 0; i < nargs; i++) { @@ -3777,7 +3779,7 @@ vectorizable_call (vec_info *vinfo, if (mask_opno >= 0 && masked_loop_p) { - tree mask = vect_get_loop_mask (gsi, masks, ncopies, + tree mask = vect_get_loop_mask (loop_vinfo, gsi, masks, ncopies, vectype_out, j); vargs[mask_opno] = prepare_vec_mask (loop_vinfo, TREE_TYPE (mask), mask, @@ -6823,8 +6825,8 @@ vectorizable_operation (vec_info *vinfo, } else if (masked_loop_p && mask_out_inactive) { - tree mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, - vectype, i); + tree mask = vect_get_loop_mask (loop_vinfo, gsi, masks, + vec_num * ncopies, vectype, i); auto_vec vops (5); vops.quick_push (mask); vops.quick_push (vop0); @@ -6865,8 +6867,8 @@ vectorizable_operation (vec_info *vinfo, if (loop_vinfo->scalar_cond_masked_set.contains ({ op0, ncopies})) { - mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, - vectype, i); + mask = vect_get_loop_mask (loop_vinfo, gsi, masks, + vec_num * ncopies, vectype, i); vop0 = prepare_vec_mask (loop_vinfo, TREE_TYPE (mask), mask, vop0, gsi); @@ -6875,8 +6877,8 @@ vectorizable_operation (vec_info *vinfo, if (loop_vinfo->scalar_cond_masked_set.contains ({ op1, ncopies })) { - mask = vect_get_loop_mask (gsi, masks, vec_num * ncopies, - vectype, i); + mask = vect_get_loop_mask (loop_vinfo, gsi, masks, + vec_num * ncopies, vectype, i); vop1 = prepare_vec_mask (loop_vinfo, TREE_TYPE (mask), mask, vop1, gsi); @@ -8760,8 +8762,8 @@ vectorizable_store (vec_info *vinfo, tree final_mask = NULL; if (loop_masks) - final_mask = vect_get_loop_mask (gsi, loop_masks, ncopies, - vectype, j); + final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, + ncopies, vectype, j); if (vec_mask) final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, final_mask, vec_mask, gsi); @@ -8814,7 +8816,7 @@ vectorizable_store (vec_info *vinfo, tree final_mask = NULL_TREE; if (loop_masks) - final_mask = vect_get_loop_mask (gsi, loop_masks, + final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, vec_num * ncopies, vectype, vec_num * j + i); if (vec_mask) @@ -10133,8 +10135,8 @@ vectorizable_load (vec_info *vinfo, tree final_mask = NULL_TREE; if (loop_masks) - final_mask = vect_get_loop_mask (gsi, loop_masks, ncopies, - vectype, j); + final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, + ncopies, vectype, j); if (vec_mask) final_mask = prepare_vec_mask (loop_vinfo, mask_vectype, final_mask, vec_mask, gsi); @@ -10184,7 +10186,7 @@ vectorizable_load (vec_info *vinfo, tree final_mask = NULL_TREE; if (loop_masks && memory_access_type != VMAT_INVARIANT) - final_mask = vect_get_loop_mask (gsi, loop_masks, + final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, vec_num * ncopies, vectype, vec_num * j + i); if (vec_mask) @@ -11208,7 +11210,7 @@ vectorizable_condition (vec_info *vinfo, if (masks) { tree loop_mask - = vect_get_loop_mask (gsi, masks, vec_num * ncopies, + = vect_get_loop_mask (loop_vinfo, gsi, masks, vec_num * ncopies, vectype, i); tree tmp2 = make_ssa_name (vec_cmp_type); gassign *g diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index af25d20bd7e..767a0774d45 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2302,7 +2302,8 @@ extern tree vect_halve_mask_nunits (tree, machine_mode); extern tree vect_double_mask_nunits (tree, machine_mode); extern void vect_record_loop_mask (loop_vec_info, vec_loop_masks *, unsigned int, tree, tree); -extern tree vect_get_loop_mask (gimple_stmt_iterator *, vec_loop_masks *, +extern tree vect_get_loop_mask (loop_vec_info, gimple_stmt_iterator *, + vec_loop_masks *, unsigned int, tree, unsigned int); extern void vect_record_loop_len (loop_vec_info, vec_loop_lens *, unsigned int, tree, unsigned int);