From patchwork Fri Jun 23 13:48:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 112141 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp5785782vqr; Fri, 23 Jun 2023 06:49:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7ZIdwd2xB2L6/f7TH69itAhBSUHgxJ1UI80u5+pQTi5Lo8mICrdFrD8TXISq/Mj9Ks7p3J X-Received: by 2002:a17:907:9811:b0:988:ce71:7a84 with SMTP id ji17-20020a170907981100b00988ce717a84mr11945078ejc.27.1687528149584; Fri, 23 Jun 2023 06:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687528149; cv=none; d=google.com; s=arc-20160816; b=nOz7J5YG9xNIP0DMg1JVQDqozXbo1OIi+Uh7SXRCgOm2/eRW8xi2mUlSAfEoL6RfsB bchUyGWnTh55CVG3hmbdsJnRlpY8XdJI9sKQJR1SBETq03jk95oxfmicgIPDk5IlzY6u 1CZEmGCBjU61ZybAEE3blMV3McsnW9WWXLxSgX6m8laSyS+U2l1pyDlxTP7+duYgz8vh fKCiEiP4299I6lkHpw9jUW2VP2+jil/m04fKi1T2tJXtnLFQIvs6AYnn9AOBaoirgO3S 0PSSjRWXjdv1UWm9/al4zrROTq2OJcOeqVx2kg/xGTZYVhtfCqkDEc3E0sZecGzkPLvw /vMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=qU2ISKl19cX58uJYe0TMKPwSZVD3t1Z4E086HpApaR0=; b=IHNP9gU2T64PbFVy7+ttluzWFQkTHB8pr3eCtarBriG0P2R+vHk5LNEOyz05NeC3tD FK7RZoREc5eHxUSdS33nCvX3vnfRJ4/GQLXRmUAiQO6Zrrud/xinqT818up7ZCrvVq7s E7nIaQH+WsQMxAVALIWcrCz799s9/lq7QLOcpCpeJ7JKuOJ/bGLrpzsLdKfk09yDysmg V/1YWkwbyurQvFC6hSgjiSP187aAefIZa25SyDTe7oV2SzDLiOmqj/MWCWNueNf1WowU tNvO0L6UX9kCn6YgOP4RzN6dmrYhX+Y8h7uTV32Yx88Iw3qrok4JiYIWiLyAsj2BWqtZ blYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id pg1-20020a170907204100b009829a18f65asi5200173ejb.329.2023.06.23.06.49.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jun 2023 06:49:09 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0FE823858025 for ; Fri, 23 Jun 2023 13:49:04 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgbr1.qq.com (smtpbgbr1.qq.com [54.207.19.206]) by sourceware.org (Postfix) with ESMTPS id 71CC13858CD1 for ; Fri, 23 Jun 2023 13:48:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 71CC13858CD1 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp72t1687528109t7qbwcmj Received: from rios-cad5.localdomain ( [58.60.1.11]) by bizesmtp.qq.com (ESMTP) with id ; Fri, 23 Jun 2023 21:48:28 +0800 (CST) X-QQ-SSF: 01400000000000G0S000000A0000000 X-QQ-FEAT: yFznwLkTCH4AwSexdW17oqcHFXMkHyU4omcwkSwyvLhMkarngNFSiJ2DD7hqn 10uaMiVo7aW9ug5TZF0rmeC8ozrF5VEDzI8rBcLL7sI9ONv+4Dte9Aifmb7GfVtwfDmjarA ru+krKhzX8CUaKagF3dHhmEta/Xc5OW/Rd5GcUc7zJc05hWM/lLN4iVsLhWMs1Mme9rBAKM l9Wcu9+r5p+vTjyqXasSHOAY9phZeppQG7MLJolzNS4qDze3xDlDKxHFbayZx/hoJlkHzq0 ShL3SKYZMfiopBQQw3dExX5JsQUNagsIRY1Ms66cQC9C0J8LWtvOSjM7P6xofeCBWwhzn1s fE5yVIgb8QQEZ/ETELLYzP4PAiMzgKscjHhjtaETQQQA4dj+/2gdssZ6byyQJD1aCjIQktd X-QQ-GoodBg: 2 X-BIZMAIL-ID: 2955109731661241390 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: rguenther@suse.de, richard.sandiford@arm.com, Ju-Zhe Zhong Subject: [PATCH] GIMPLE_FOLD: Apply LEN_MASK_{LOAD,STORE} into GIMPLE_FOLD Date: Fri, 23 Jun 2023 21:48:27 +0800 Message-Id: <20230623134827.4093245-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769501516995284371?= X-GMAIL-MSGID: =?utf-8?q?1769501516995284371?= From: Ju-Zhe Zhong Hi, since we are going to have LEN_MASK_{LOAD,STORE} into loopVectorizer. Currenly, 1. we can fold MASK_{LOAD,STORE} into MEM when mask is all ones. 2. we can fold LEN_{LOAD,STORE} into MEM when (len - bias) is VF. Now, I think it makes sense that we can support fold LEN_MASK_{LOAD,STORE} into MEM when both mask = all ones and (len - bias) is VF. gcc/ChangeLog: * gimple-fold.cc (arith_overflowed_p): Apply LEN_MASK_{LOAD,STORE}. (gimple_fold_partial_load_store_mem_ref): Ditto. (gimple_fold_partial_store): Ditto. (gimple_fold_call): Ditto. --- gcc/gimple-fold.cc | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/gcc/gimple-fold.cc b/gcc/gimple-fold.cc index 55e80567708..3d46b76edeb 100644 --- a/gcc/gimple-fold.cc +++ b/gcc/gimple-fold.cc @@ -5370,10 +5370,10 @@ arith_overflowed_p (enum tree_code code, const_tree type, return wi::min_precision (wres, sign) > TYPE_PRECISION (type); } -/* If IFN_{MASK,LEN}_LOAD/STORE call CALL is unconditional, return a MEM_REF - for the memory it references, otherwise return null. VECTYPE is the - type of the memory vector. MASK_P indicates it's for MASK if true, - otherwise it's for LEN. */ +/* If IFN_{MASK,LEN,LEN_MASK}_LOAD/STORE call CALL is unconditional, + return a MEM_REF for the memory it references, otherwise return null. + VECTYPE is the type of the memory vector. MASK_P indicates it's for + MASK if true, otherwise it's for LEN. */ static tree gimple_fold_partial_load_store_mem_ref (gcall *call, tree vectype, bool mask_p) @@ -5400,6 +5400,16 @@ gimple_fold_partial_load_store_mem_ref (gcall *call, tree vectype, bool mask_p) if (maybe_ne (wi::to_poly_widest (basic_len) - wi::to_widest (bias), GET_MODE_SIZE (TYPE_MODE (vectype)))) return NULL_TREE; + + /* For LEN_MASK_{LOAD,STORE}, we should also check whether + the mask is all ones mask. */ + internal_fn ifn = gimple_call_internal_fn (call); + if (ifn == IFN_LEN_MASK_LOAD || ifn == IFN_LEN_MASK_STORE) + { + tree mask = gimple_call_arg (call, internal_fn_mask_index (ifn)); + if (!integer_all_onesp (mask)) + return NULL_TREE; + } } unsigned HOST_WIDE_INT align = tree_to_uhwi (alias_align); @@ -5438,7 +5448,8 @@ static bool gimple_fold_partial_store (gimple_stmt_iterator *gsi, gcall *call, bool mask_p) { - tree rhs = gimple_call_arg (call, 3); + internal_fn ifn = gimple_call_internal_fn (call); + tree rhs = gimple_call_arg (call, internal_fn_stored_value_index (ifn)); if (tree lhs = gimple_fold_partial_load_store_mem_ref (call, TREE_TYPE (rhs), mask_p)) { @@ -5676,9 +5687,11 @@ gimple_fold_call (gimple_stmt_iterator *gsi, bool inplace) changed |= gimple_fold_partial_store (gsi, stmt, true); break; case IFN_LEN_LOAD: + case IFN_LEN_MASK_LOAD: changed |= gimple_fold_partial_load (gsi, stmt, false); break; case IFN_LEN_STORE: + case IFN_LEN_MASK_STORE: changed |= gimple_fold_partial_store (gsi, stmt, false); break; default: