From patchwork Tue Jun 13 02:03:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kewen.Lin" X-Patchwork-Id: 107019 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp251417vqr; Mon, 12 Jun 2023 19:06:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68+7hj1hmqvGyYn7HVACLALZoyyco08yAK3Z5RcxqmN05xCByhX+VISZh4yA2Tw2qpMHq7 X-Received: by 2002:a17:906:730b:b0:973:bcf6:1d4 with SMTP id di11-20020a170906730b00b00973bcf601d4mr13215176ejc.76.1686621993753; Mon, 12 Jun 2023 19:06:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686621993; cv=none; d=google.com; s=arc-20160816; b=dk8cRmYdVkuwjJt7gT5yMnm2bGq5+qozifwJHtFyNQ0/vu6Zjmm4ph9A1BO/7XU6CO fxzi539OO9ohtRSAAEDwjdKewr0yfvf5gTBdN5PCPXjeEcJLOdu/jA/3qt8sjIQVIMnv Z6elRK9vZbJxcJ6ZTtGHeaOVhzuutJHgaL6O5EZ8c6GTirimtI1W9cXYjhRdGVK9pZXL 4hTSCjjBgayUVqYg9Cs/FvkNvHnB5fWYxAtc9xauSh3FqMjB6LHzjS7vO34gtRazxeuY bjJfL0TCJcUQWbAy150OrjHkX84U2b2c9QYZ6S74MZGNO4MbXC4PQ0qOeY9DNB3ut5se uKBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=iSFZQDIFqKy7C8+BrTU+LC3kCipl5K4VhKTUUgxNPoY=; b=mt/gNo9uy9M0L+lyaSh1O3JKPVa5z04GRUPdNmasEUH4l+DSuJxQwBmf8dqEozowS2 2w0h/Iv9iiFlQbRweYkyF4yWXIhiOzK8HsnAW6yeO5PFKgO9v/PjoepvOnfMtv9I5iec vMmsVQEttlXULtNMhLFO218AKq9I58vAyF+XyxmXB+cLz9hFHStPsOSnd9CeoXLfgAky kkDWTnjKtGyR3XI0XEWMlurx65XXYHuZpQ1a3H5/VwTvu1UKuhWlmGthoDY5t55zzCk4 FiUqi8gjYJ2gUcLGwZhCspYTnIbB4BFD7LZJBvyfsNrzIdE0Ia4GJFftABLCCqkJuIpb USDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IKlutqAP; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id am10-20020a170906568a00b00974e764c89bsi5997700ejc.57.2023.06.12.19.06.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 19:06:33 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=IKlutqAP; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2079438558A6 for ; Tue, 13 Jun 2023 02:05:45 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2079438558A6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1686621945; bh=iSFZQDIFqKy7C8+BrTU+LC3kCipl5K4VhKTUUgxNPoY=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=IKlutqAPyw7vPJCLopsgKJOrz6Ui9HdY+lFwVGFKeM88p00rjfB2LSW70nBNS1sew wduqM6Tu78Or3lF7I53+r4p0NreEZ4W3ZCB+UQF2pdVsNNI9Wv/w4xZRK9lBjy1bLF ohyeiQqxyBSgpW7R207h/jldMXSPZ9BKqRLs4sAs= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id CF3DA3858D33 for ; Tue, 13 Jun 2023 02:03:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CF3DA3858D33 Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35D217Z6020252; Tue, 13 Jun 2023 02:03:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r6fa2829w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 02:03:49 +0000 Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35D22xQ6025525; Tue, 13 Jun 2023 02:03:49 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3r6fa2828s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 02:03:49 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35D1lqvZ025458; Tue, 13 Jun 2023 02:03:46 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3r4gee1ue3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 02:03:46 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35D23iPv18023006 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Jun 2023 02:03:44 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6FF8F20043; Tue, 13 Jun 2023 02:03:44 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 81B9620040; Tue, 13 Jun 2023 02:03:43 +0000 (GMT) Received: from trout.aus.stglabs.ibm.com (unknown [9.40.194.100]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 13 Jun 2023 02:03:43 +0000 (GMT) To: gcc-patches@gcc.gnu.org Cc: richard.guenther@gmail.com, richard.sandiford@arm.com, segher@kernel.crashing.org, bergner@linux.ibm.com Subject: [PATCH 2/9] vect: Adjust vectorizable_load costing on VMAT_GATHER_SCATTER && gs_info.decl Date: Mon, 12 Jun 2023 21:03:23 -0500 Message-Id: <9bad792a4bcef35fbd9906245bf3493672b340fe.1686573640.git.linkw@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: Z-tFxoFxGVOAJ2XOlvMjuOK3EcjvOhDb X-Proofpoint-GUID: c-yv9Djaz3OGVHKcHdarypMAH1gdqogk X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-12_18,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 impostorscore=0 suspectscore=0 spamscore=0 adultscore=0 lowpriorityscore=0 mlxscore=0 priorityscore=1501 malwarescore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130016 X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kewen Lin via Gcc-patches From: "Kewen.Lin" Reply-To: Kewen Lin Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768551343692288197?= X-GMAIL-MSGID: =?utf-8?q?1768551343692288197?= This patch adds one extra argument cost_vec to function vect_build_gather_load_calls, so that we can do costing next to the tranform in vect_build_gather_load_calls. For now, the implementation just follows the handlings in vect_model_load_cost, it isn't so good, so placing one FIXME for any further improvement. This patch should not cause any functional changes. gcc/ChangeLog: * tree-vect-stmts.cc (vect_build_gather_load_calls): Add the handlings on costing with one extra argument cost_vec. (vectorizable_load): Adjust the call to vect_build_gather_load_calls. (vect_model_load_cost): Assert it won't get VMAT_GATHER_SCATTER with gs_info.decl set any more. --- gcc/tree-vect-stmts.cc | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 44514658be3..744cdf40e26 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -1135,6 +1135,8 @@ vect_model_load_cost (vec_info *vinfo, slp_tree slp_node, stmt_vector_for_cost *cost_vec) { + gcc_assert (memory_access_type != VMAT_GATHER_SCATTER || !gs_info->decl); + unsigned int inside_cost = 0, prologue_cost = 0; bool grouped_access_p = STMT_VINFO_GROUPED_ACCESS (stmt_info); @@ -2819,7 +2821,8 @@ vect_build_gather_load_calls (vec_info *vinfo, stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, gimple **vec_stmt, gather_scatter_info *gs_info, - tree mask) + tree mask, + stmt_vector_for_cost *cost_vec) { loop_vec_info loop_vinfo = dyn_cast (vinfo); class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); @@ -2831,6 +2834,23 @@ vect_build_gather_load_calls (vec_info *vinfo, stmt_vec_info stmt_info, poly_uint64 gather_off_nunits = TYPE_VECTOR_SUBPARTS (gs_info->offset_vectype); + /* FIXME: Keep the previous costing way in vect_model_load_cost by costing + N scalar loads, but it should be tweaked to use target specific costs + on related gather load calls. */ + if (!vec_stmt) + { + unsigned int assumed_nunits = vect_nunits_for_cost (vectype); + unsigned int inside_cost; + inside_cost = record_stmt_cost (cost_vec, ncopies * assumed_nunits, + scalar_load, stmt_info, 0, vect_body); + if (dump_enabled_p ()) + dump_printf_loc (MSG_NOTE, vect_location, + "vect_model_load_cost: inside_cost = %d, " + "prologue_cost = 0 .\n", + inside_cost); + return; + } + tree arglist = TYPE_ARG_TYPES (TREE_TYPE (gs_info->decl)); tree rettype = TREE_TYPE (TREE_TYPE (gs_info->decl)); tree srctype = TREE_VALUE (arglist); arglist = TREE_CHAIN (arglist); @@ -9483,13 +9503,8 @@ vectorizable_load (vec_info *vinfo, if (memory_access_type == VMAT_GATHER_SCATTER && gs_info.decl) { - if (costing_p) - vect_model_load_cost (vinfo, stmt_info, ncopies, vf, memory_access_type, - alignment_support_scheme, misalignment, &gs_info, - slp_node, cost_vec); - else - vect_build_gather_load_calls (vinfo, stmt_info, gsi, vec_stmt, &gs_info, - mask); + vect_build_gather_load_calls (vinfo, stmt_info, gsi, vec_stmt, &gs_info, + mask, cost_vec); return true; }