From patchwork Wed Dec 13 12:30:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177942 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7739456dys; Wed, 13 Dec 2023 04:32:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IFwi8qCbXmsDGPXLBxyUrzpuQVSoh+EtBBo8ScNF2TvWX1qtDSkGCZoRc2FPIo1IKWuluK9 X-Received: by 2002:a05:6122:180a:b0:4ac:7cf9:9ef4 with SMTP id ay10-20020a056122180a00b004ac7cf99ef4mr6997660vkb.2.1702470752391; Wed, 13 Dec 2023 04:32:32 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470752; cv=pass; d=google.com; s=arc-20160816; b=UacpQGWXoO3RJWEFtPRBk+HFI2Zy76KrGT20pZGo9fhXw0uun5nPzMAMnx1XOwEEeF aLGoNegQPr6C6Cfcbl1nQyBpcbDbZmop4zVeeEsbR27bcq+RzsC8/qDLeJM+oS2guK+B 6pHr3oEaSguf7lnSTPk0FvIag76hzWbLFWac5/hkKr870VWsNEXDS3jXY3ow+WHQ1eN0 SaEK9uVuV/IOAgIaYKspkmbZaRuqXYHZ990aAylaEg6qxlNf69JGtq2KX96gtdd0/Kwd 6jnb1weTnYBN4ij7as3U8/v/gP088En7Vdut/373hyNfcJmEV4BCnFzhPHCfrhVnXaww gliQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=9n4M7ojf975L20RESRcujrAHYf0j3fGmQk2any59nC4=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=YaTvHifpgMZQ8UXMpsOXZch/Zq+hFtJItRHut6ICW3MZVMppBTr4uGPc+LS/z3m4+L ZTERZCYmIgCq6cOK9SmUkfp5uhJclfovjFrjO54AHhJ8VZ9EFN95krFJ9h4ahNljDVR6 4WpZWfI5BuyANnjzNbV/RnG7L/nvY8fK9eNNIX+nWXHEknxnyqJXRWZzU/vsldtIl3ot nLxi85qh37XwusnBYlhHFDwr9owkaoTJyS6/sTNJ4A+6vbcQkQjf+NYLDTGOE1TTII2o pu0+ZGiXPSvnAw96Wd6lOcCU8HMrZ7bZjiToGqrCcPp1XTR4VCz9PehYhX0m8dLiKcxR tbRg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=m7vg46Gp; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=u1zj+jIR; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=m7vg46Gp; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id s186-20020a1fdbc3000000b004b2dbc6566dsi2376118vkg.109.2023.12.13.04.32.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:32:32 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=m7vg46Gp; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=u1zj+jIR; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=m7vg46Gp; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0020C385332C for ; Wed, 13 Dec 2023 12:32:23 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by sourceware.org (Postfix) with ESMTPS id D39A63858C78 for ; Wed, 13 Dec 2023 12:31:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D39A63858C78 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D39A63858C78 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470718; cv=none; b=SasaEdlpzWM4uT37SVCWkLLfHtiZN/9fO/x+zaejdl6RtcoIHKpFIt8Jev4VyD4/cWFQ/9wmzO83nxVmbGYNXNxKZtwQ0WVx8hubK23ou7ovo0go/d0TIfYMXfs74jWxUxwhrNsOk5bndsr9z/NGj9t8r6ABIRcOFqeFqGGmzsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470718; c=relaxed/simple; bh=dnmWxz4adjCr+94DbArPnY0bhNrXUw1fLCD6pyZZD3Y=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=v+G1AJ92N9XxMxbWpzXPxPU15eocJ512IgX45WMGc3o83sPqmlLUJHIH4rgKcARmi1y7ROz7kzE2lMUUJOvUyrDZki1sO/8+GDvgPP6vlYSWKx+o1L5M7JTS6kGxlMvCukdugYLiS/XOEmXG1KwveH1FbUe2tCZIp2uDElCxNT0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BB408221C9 for ; Wed, 13 Dec 2023 12:31:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470715; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=9n4M7ojf975L20RESRcujrAHYf0j3fGmQk2any59nC4=; b=m7vg46Gpj3t9z4z1ZFRDMK/A8ADUuwRzTxnhA0PKupUn0oHpmyYulgf+VeNFLQtFk9AWz0 dFA+oW8FQorrbD2Du75CQGRyrqpK0SwRCrReeSxcP1esE/nDQU0mVtA7d91447wWVnL14l B7wFcifALnFypUHRZB2WsUrroLtpfmU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470715; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=9n4M7ojf975L20RESRcujrAHYf0j3fGmQk2any59nC4=; b=u1zj+jIRo/x/zU/jHOHYJOyqjVYT6nFPIzL9tz/+77uFBO1GVVBPe7Ff501+KS0bduJsuu +IKD/KLlN27ry/CA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470715; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=9n4M7ojf975L20RESRcujrAHYf0j3fGmQk2any59nC4=; b=m7vg46Gpj3t9z4z1ZFRDMK/A8ADUuwRzTxnhA0PKupUn0oHpmyYulgf+VeNFLQtFk9AWz0 dFA+oW8FQorrbD2Du75CQGRyrqpK0SwRCrReeSxcP1esE/nDQU0mVtA7d91447wWVnL14l B7wFcifALnFypUHRZB2WsUrroLtpfmU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470715; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=9n4M7ojf975L20RESRcujrAHYf0j3fGmQk2any59nC4=; b=u1zj+jIRo/x/zU/jHOHYJOyqjVYT6nFPIzL9tz/+77uFBO1GVVBPe7Ff501+KS0bduJsuu +IKD/KLlN27ry/CA== Date: Wed, 13 Dec 2023 13:30:52 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/6] Reduce the number of get_vectype_for_scalar_type calls MIME-Version: 1.0 X-Spam-Score: -0.60 Authentication-Results: smtp-out1.suse.de; none X-Spam-Score: 2.31 X-Spamd-Result: default: False [2.31 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(2.91)[0.972]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Flag: NO X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123224.0020C385332C@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785169971560288027 X-GMAIL-MSGID: 1785169971560288027 The following removes get_vectype_for_scalar_type calls when we already have the vector type computed. It also avoids some premature and possibly redundant or unnecessary check during data-ref analysis for gathers. * tree-vect-data-refs.cc (vect_analyze_data_refs): Do not check for a vector type for gather/scatter offset. vect_check_gather_scatter does that already. * tree-vect-loop.cc (get_initial_def_for_reduction): Use the computed vector type. * tree-vect-stmts.cc (vectorizable_operation): Use the known vector type for constants/externs. --- gcc/tree-vect-data-refs.cc | 4 +--- gcc/tree-vect-loop.cc | 2 +- gcc/tree-vect-stmts.cc | 3 ++- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index d5c9c4a11c2..107dffe0a64 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -4721,9 +4721,7 @@ vect_analyze_data_refs (vec_info *vinfo, poly_uint64 *min_vf, bool *fatal) gather_scatter_info gs_info; if (!vect_check_gather_scatter (stmt_info, as_a (vinfo), - &gs_info) - || !get_vectype_for_scalar_type (vinfo, - TREE_TYPE (gs_info.offset))) + &gs_info)) { if (fatal) *fatal = false; diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 6261cd1be1d..3a0731f3bea 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -5466,7 +5466,7 @@ get_initial_def_for_reduction (loop_vec_info loop_vinfo, { class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); tree scalar_type = TREE_TYPE (init_val); - tree vectype = get_vectype_for_scalar_type (loop_vinfo, scalar_type); + tree vectype = STMT_VINFO_VECTYPE (reduc_info); tree init_def; gimple_seq stmts = NULL; diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 390c8472fd6..5ad306e2b08 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -6914,7 +6914,8 @@ vectorizable_operation (vec_info *vinfo, S2: z = x + 1 - VS2_0 */ vect_get_vec_defs (vinfo, stmt_info, slp_node, ncopies, - op0, &vec_oprnds0, op1, &vec_oprnds1, op2, &vec_oprnds2); + op0, vectype, &vec_oprnds0, op1, vectype, &vec_oprnds1, + op2, vectype, &vec_oprnds2); /* Arguments are ready. Create the new vector stmt. */ FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) { From patchwork Wed Dec 13 12:30:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177944 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7740112dys; Wed, 13 Dec 2023 04:33:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IFH6RXxjML26bnOhrptZ+1+judY6BfmzygZqc7fWTS6K8Xd2P0domjH0/xibTN5aUe1viMB X-Received: by 2002:a05:620a:27cc:b0:77e:c67:32e4 with SMTP id i12-20020a05620a27cc00b0077e0c6732e4mr7952770qkp.23.1702470816028; Wed, 13 Dec 2023 04:33:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470816; cv=pass; d=google.com; s=arc-20160816; b=ty0IjZKckuZ7EuaX0J6FKAU5Y0/R07YoLjZ5gbW/YV/TYxFQxQXBsVOlVkjxwOhAnR 538PLe5Ig7PA16oBHbb8FVc1uONOW1r1o7J8GJjmr5/OsK2+j/tN4j7QEHkKtzwS6mJV w5b3xf9QTNW3oRqWnYMjdO2ydQqTViD+DQ3wZ8XujunuDcM0UBviBJPIAgRSRgr9q0L0 f9dL8gJhcARd+aiH0B6foHyywQjm7+eBDMcWkrMsTtjhZaKwIBbY6UlPlgR7G+LY7xTB RV+yL62mLCxoGiFx1ZAntT8dUas4zCLKzSffI56GHZEH1d0ROUuM3VJ9equKXeXGsMdf ZM8A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=oPkb6Fzlxk+KirHk/3o/0G4CEY93oQIRclOhTQwpzRE=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=VjYa4cK4SlbT++xUg1k2RM7aicuFdq4Mb7Db06P7T2cqEF7xDxyb2ckG/kuCDlx2mB eh2I9DPJHtj/Y1qMlJUyCX4bdp6FykZ11I3TI/egf5nqWJcc/F7pvKiwnOiHNjsy7Xp0 6OA9DIEnIceEcWRnHeL4wNolYjUV3T6POweU4AZWFekl2yj4Xwqj3z2rvE2H4zfYmXwc HNB0ghzGlQicDOgzcugt6pMdkoy9jn7wV2FFmcZ7ObCs3wy05KwXys7MUKStozU5TDW6 9yf3AfOEPUxWX6unyYFyq2qZuI4D88aFAP1T5fnJbqkB6Cxsr0lo825bw4DbBJQYNMPp O9zg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=HXg6UDCm; dkim=neutral (no key) header.i=@suse.de; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=HXg6UDCm; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id q21-20020a05620a0d9500b0077f7cf586d2si5760375qkl.38.2023.12.13.04.33.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:33:36 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=HXg6UDCm; dkim=neutral (no key) header.i=@suse.de; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=HXg6UDCm; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BA2D6384DEF7 for ; Wed, 13 Dec 2023 12:32:57 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2a07:de40:b251:101:10:150:64:2]) by sourceware.org (Postfix) with ESMTPS id 0B8AF385C419 for ; Wed, 13 Dec 2023 12:32:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0B8AF385C419 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0B8AF385C419 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a07:de40:b251:101:10:150:64:2 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470734; cv=none; b=kkC4r5nd3ztjmtP2Ld5ESYPYy+1lkteP3rsx3kFce/pDG9Lx9DojuwPKFplH7kc93JMNU+Pf7gmlnNCq78elN+DwKIdl95g+5aEbUNBC4+QmiUtYa72dzWb+Vw9G5Fc/gC74UEn3m1rRIkM8L/sV6/zFawbm135fzE44FOYQye4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470734; c=relaxed/simple; bh=6NF+dbvFAhpKWz0CHGKc9MGUBosC8UXfFMfHSD/bhYc=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=vgm0vnuIEDqpKPAHcJkVx4QA3F5+I+4lErXLDQSTXFA53sgNTbNiEjLzBpEVSolF2HoAScaFZiSYJ3TlVXRbHVn9xi+xTHBE0/NxzwHjF2n1ZHGkJCuRB5cxySqCvZaVhfp6NyDzuhY2uw8gmwQt9BDCyhJoqmdtqodCs3I8vI8= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id F2BA61F392 for ; Wed, 13 Dec 2023 12:32:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470723; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=oPkb6Fzlxk+KirHk/3o/0G4CEY93oQIRclOhTQwpzRE=; b=HXg6UDCmIvPl00jeAfji+C/9OmkQbIGNPtxfESWzfiVJlefWPA8RP4PKtg+uFcrkL7maHO HpHLUiJQv6U/cpEbYKMgHIKRQ342oS8GoOCd3ffEQ8EgIAsPlg0r7ds7ChJMCVP9FBfM30 1IvhvfDo0qIR+bRk1HBdT7rcH8ZW6+E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470723; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=oPkb6Fzlxk+KirHk/3o/0G4CEY93oQIRclOhTQwpzRE=; b=q3rUINkpM6r75bsEmMms6L8crUqcTbqp6EseOU2jPtPTJ09Ou0beRE/6UAAwAnC6c5zSIB G8BfFeMuMvwVdQAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470723; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=oPkb6Fzlxk+KirHk/3o/0G4CEY93oQIRclOhTQwpzRE=; b=HXg6UDCmIvPl00jeAfji+C/9OmkQbIGNPtxfESWzfiVJlefWPA8RP4PKtg+uFcrkL7maHO HpHLUiJQv6U/cpEbYKMgHIKRQ342oS8GoOCd3ffEQ8EgIAsPlg0r7ds7ChJMCVP9FBfM30 1IvhvfDo0qIR+bRk1HBdT7rcH8ZW6+E= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470723; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=oPkb6Fzlxk+KirHk/3o/0G4CEY93oQIRclOhTQwpzRE=; b=q3rUINkpM6r75bsEmMms6L8crUqcTbqp6EseOU2jPtPTJ09Ou0beRE/6UAAwAnC6c5zSIB G8BfFeMuMvwVdQAQ== Date: Wed, 13 Dec 2023 13:30:59 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 2/6] Set LOOP_VINFO_VECT_FACTOR only when it is final MIME-Version: 1.0 X-Spam-Score: -0.60 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: 2.31 X-Spamd-Result: default: False [2.31 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(2.91)[0.970]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Flag: NO X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123257.BA2D6384DEF7@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785170038256694221 X-GMAIL-MSGID: 1785170038256694221 The following makes sure to keep LOOP_VINFO_VECT_FACTOR at the indetermined value zero until it is final, making LOOP_VINFO_VECT_FACTOR an rvalue and changing some direct references to use the macro. * tree-vectorizer.h (LOOP_VINFO_VECT_FACTOR): Make an rvalue. * tree-vect-loop.cc (vect_determine_vectorization_factor): Do not set LOOP_VINFO_VECT_FACTOR, return value via reference. (vect_update_vf_for_slp): Likewise. (vect_analyze_loop_2): Set LOOP_VINFO_VECT_FACTOR only ever to its final value. Perform SLP optimization after setting the vectorization factor. * tree-vect-slp.cc (vect_slp_analyze_node_operations_1): Use LOOP_VINFO_VECT_FACTOR. (vect_slp_analyze_node_operations): Likewise. * tree-vectorizer.cc (vect_transform_loops): Likewise. --- gcc/tree-vect-loop.cc | 43 +++++++++++++++++++++++------------------- gcc/tree-vect-slp.cc | 4 ++-- gcc/tree-vectorizer.cc | 2 +- gcc/tree-vectorizer.h | 2 +- 4 files changed, 28 insertions(+), 23 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 3a0731f3bea..3af4160426b 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -282,12 +282,12 @@ vect_determine_vf_for_stmt (vec_info *vinfo, */ static opt_result -vect_determine_vectorization_factor (loop_vec_info loop_vinfo) +vect_determine_vectorization_factor (loop_vec_info loop_vinfo, + poly_uint64 &vectorization_factor) { class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); basic_block *bbs = LOOP_VINFO_BBS (loop_vinfo); unsigned nbbs = loop->num_nodes; - poly_uint64 vectorization_factor = 1; tree scalar_type = NULL_TREE; gphi *phi; tree vectype; @@ -296,6 +296,8 @@ vect_determine_vectorization_factor (loop_vec_info loop_vinfo) DUMP_VECT_SCOPE ("vect_determine_vectorization_factor"); + vectorization_factor = 1; + for (i = 0; i < nbbs; i++) { basic_block bb = bbs[i]; @@ -370,7 +372,6 @@ vect_determine_vectorization_factor (loop_vec_info loop_vinfo) if (known_le (vectorization_factor, 1U)) return opt_result::failure_at (vect_location, "not vectorized: unsupported data-type\n"); - LOOP_VINFO_VECT_FACTOR (loop_vinfo) = vectorization_factor; return opt_result::success (); } @@ -1937,17 +1938,16 @@ vect_create_loop_vinfo (class loop *loop, vec_info_shared *shared, statements update the vectorization factor. */ static void -vect_update_vf_for_slp (loop_vec_info loop_vinfo) +vect_update_vf_for_slp (loop_vec_info loop_vinfo, + poly_uint64 &vectorization_factor) { class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); basic_block *bbs = LOOP_VINFO_BBS (loop_vinfo); int nbbs = loop->num_nodes; - poly_uint64 vectorization_factor; int i; DUMP_VECT_SCOPE ("vect_update_vf_for_slp"); - vectorization_factor = LOOP_VINFO_VECT_FACTOR (loop_vinfo); gcc_assert (known_ne (vectorization_factor, 0U)); /* If all the stmts in the loop can be SLPed, we perform only SLP, and @@ -2006,7 +2006,6 @@ vect_update_vf_for_slp (loop_vec_info loop_vinfo) LOOP_VINFO_SLP_UNROLLING_FACTOR (loop_vinfo)); } - LOOP_VINFO_VECT_FACTOR (loop_vinfo) = vectorization_factor; if (dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, @@ -2809,7 +2808,8 @@ vect_analyze_loop_2 (loop_vec_info loop_vinfo, bool &fatal, return opt_result::failure_at (vect_location, "bad data dependence.\n"); LOOP_VINFO_MAX_VECT_FACTOR (loop_vinfo) = max_vf; - ok = vect_determine_vectorization_factor (loop_vinfo); + poly_uint64 vectorization_factor; + ok = vect_determine_vectorization_factor (loop_vinfo, vectorization_factor); if (!ok) { if (dump_enabled_p ()) @@ -2821,7 +2821,7 @@ vect_analyze_loop_2 (loop_vec_info loop_vinfo, bool &fatal, /* Compute the scalar iteration cost. */ vect_compute_single_scalar_iteration_cost (loop_vinfo); - poly_uint64 saved_vectorization_factor = LOOP_VINFO_VECT_FACTOR (loop_vinfo); + poly_uint64 saved_vectorization_factor = vectorization_factor; if (slp) { @@ -2839,13 +2839,7 @@ vect_analyze_loop_2 (loop_vec_info loop_vinfo, bool &fatal, vect_detect_hybrid_slp (loop_vinfo); /* Update the vectorization factor based on the SLP decision. */ - vect_update_vf_for_slp (loop_vinfo); - - /* Optimize the SLP graph with the vectorization factor fixed. */ - vect_optimize_slp (loop_vinfo); - - /* Gather the loads reachable from the SLP graph entries. */ - vect_gather_slp_loads (loop_vinfo); + vect_update_vf_for_slp (loop_vinfo, vectorization_factor); } } @@ -2863,11 +2857,12 @@ start_over: during finish_cost the first time we ran the analyzis for this vector mode. */ if (applying_suggested_uf) - LOOP_VINFO_VECT_FACTOR (loop_vinfo) *= loop_vinfo->suggested_unroll_factor; + vectorization_factor *= loop_vinfo->suggested_unroll_factor; /* Now the vectorization factor is final. */ - poly_uint64 vectorization_factor = LOOP_VINFO_VECT_FACTOR (loop_vinfo); gcc_assert (known_ne (vectorization_factor, 0U)); + gcc_assert (known_eq (LOOP_VINFO_VECT_FACTOR (loop_vinfo), 0U)); + loop_vinfo->vectorization_factor = vectorization_factor; if (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) && dump_enabled_p ()) { @@ -2882,6 +2877,15 @@ start_over: && maybe_lt (max_vf, LOOP_VINFO_VECT_FACTOR (loop_vinfo))) return opt_result::failure_at (vect_location, "bad data dependence.\n"); + if (slp) + { + /* Optimize the SLP graph with the vectorization factor fixed. */ + vect_optimize_slp (loop_vinfo); + + /* Gather the loads reachable from the SLP graph entries. */ + vect_gather_slp_loads (loop_vinfo); + } + loop_vinfo->vector_costs = init_cost (loop_vinfo, false); /* Analyze the alignment of the data-refs in the loop. @@ -3303,7 +3307,8 @@ again: /* Roll back state appropriately. No SLP this time. */ slp = false; /* Restore vectorization factor as it were without SLP. */ - LOOP_VINFO_VECT_FACTOR (loop_vinfo) = saved_vectorization_factor; + vectorization_factor = saved_vectorization_factor; + loop_vinfo->vectorization_factor = 0; /* Free the SLP instances. */ FOR_EACH_VEC_ELT (LOOP_VINFO_SLP_INSTANCES (loop_vinfo), j, instance) vect_free_slp_instance (instance); diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 6799b9375ae..efda358a7f6 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -6126,7 +6126,7 @@ vect_slp_analyze_node_operations_1 (vec_info *vinfo, slp_tree node, { poly_uint64 vf; if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) - vf = loop_vinfo->vectorization_factor; + vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); else vf = 1; unsigned int group_size = SLP_TREE_LANES (node); @@ -6399,7 +6399,7 @@ vect_slp_analyze_node_operations (vec_info *vinfo, slp_tree node, unsigned group_size = SLP_TREE_LANES (child); poly_uint64 vf = 1; if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) - vf = loop_vinfo->vectorization_factor; + vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); SLP_TREE_NUMBER_OF_VEC_STMTS (child) = vect_get_num_vectors (vf * group_size, vector_type); /* And cost them. */ diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc index d97e2b54c25..08ff932fb53 100644 --- a/gcc/tree-vectorizer.cc +++ b/gcc/tree-vectorizer.cc @@ -1015,7 +1015,7 @@ vect_transform_loops (hash_table *&simduid_to_vf_htab, if (!simduid_to_vf_htab) simduid_to_vf_htab = new hash_table (15); simduid_to_vf_data->simduid = DECL_UID (loop->simduid); - simduid_to_vf_data->vf = loop_vinfo->vectorization_factor; + simduid_to_vf_data->vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo); *simduid_to_vf_htab->find_slot (simduid_to_vf_data, INSERT) = simduid_to_vf_data; } diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 1810833a324..a2bab8676af 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -967,7 +967,7 @@ public: #define LOOP_VINFO_EPIL_USING_PARTIAL_VECTORS_P(L) \ (L)->epil_using_partial_vectors_p #define LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS(L) (L)->partial_load_store_bias -#define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor +#define LOOP_VINFO_VECT_FACTOR(L) ((L)->vectorization_factor + 0) #define LOOP_VINFO_MAX_VECT_FACTOR(L) (L)->max_vectorization_factor #define LOOP_VINFO_MASKS(L) (L)->masks #define LOOP_VINFO_LENS(L) (L)->lens From patchwork Wed Dec 13 12:31:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177946 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7740246dys; Wed, 13 Dec 2023 04:33:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IEhXjAhZJbQCKFqHa1VWrkpjeqtUn5Ber/lGPv35r/E/qJWWm50Yqwo+3GAYS0JTvji+MWG X-Received: by 2002:a05:620a:2401:b0:77f:684:5b1 with SMTP id d1-20020a05620a240100b0077f068405b1mr12712533qkn.47.1702470829002; Wed, 13 Dec 2023 04:33:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470828; cv=pass; d=google.com; s=arc-20160816; b=dLyIjHvBOKWDvlcxZUefVryU7xWZY7LPZgzLExPazw0DZ7Qkpn0xQuDWU5VsW+oEG0 SgOIkG+rtS0+9ESrEohWva9k9oQ8f4I1yxK4rknD8dyJbM6kchTPuQ2iu5xaNv9G7KAS Ew1VSPcE9CgsJA9qVV02JOuL0icb825b0e47DpHbk12AF4BSOm/WiO5fbwyqgwb1vSaK HH38BMOkpNNoMAprOJjibq7qQpijZRJxgN/PQ18K7OrfiEZ2GAFu5fkmEZ9DgdyxsvJa 6I/N8QX+oLCBTT6EEMr64+flEcI4bjIc/Rr61FUScwwZfMmr5uuQa2Awt6P4Pj5KKOkZ M15w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=mfXlIng+8FgI0E2tngmvDv1pQx+ENORQ55TmFs5hFP0=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=ISoK6Wygz3hC8548dGA3bW9khUVCG5eL2Jv23dFEZx6jXr3WpbwsnQtF9mHXFmBO+D h98DimHjI3iTLNFEJje3PPCdWYoQD4zfUfmGxRqPhp4HprcBSOutMQEF4CgnUxtV0TVk QojJn5ddzgHS85EClbciI0YLEAUiC9lGs9Wfldvnxm7DM7u/LXffKbAjMKGyW7V1gP5H CNjnteLzL9P8ISVKeBDJRXtp0x97EoDTfZ0TEZ8HdDMmXQ56OArtXyfCZzpb66tL/yVi AZQtP1GaQTzv6XBEcV/wRpirdKGoYABKN1O/0+FhLB5QQSmuQn4x29dGh0uHoIuibtUt QE4A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=vr4LiQp4; dkim=neutral (no key) header.i=@suse.de header.b=FcsFUZpA; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=vr4LiQp4; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id m16-20020a05620a291000b0077892f61ac1si2414328qkp.616.2023.12.13.04.33.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:33:48 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=vr4LiQp4; dkim=neutral (no key) header.i=@suse.de header.b=FcsFUZpA; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=vr4LiQp4; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2BC39388300A for ; Wed, 13 Dec 2023 12:33:06 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by sourceware.org (Postfix) with ESMTPS id 86031385DC0C for ; Wed, 13 Dec 2023 12:32:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 86031385DC0C Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 86031385DC0C Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470736; cv=none; b=eU19Xprv6qE7MTgvuAHszhokIP0jeoIrTGZVQcfAHp0h07xnD7HuIpBKdlOx/qHODfJ3IPlX7peG7KItuFKk9Cs02dR50ge4ktvlK971IIUikhxyNptq0wr+rMmCSrzw4nHeP4iUyct1UPCZ642qmKKw3QC09AvBgLDtQHEUVSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470736; c=relaxed/simple; bh=9AoLUVakKT9MoJY2SO6oIUmTGVWy+UdS9k6hQaoUBTs=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=SAF0syDqLpcHP6cAQsAuVFc5CQ2hWQtWDokHyDoOfkYIRvY8KxOIoIUnao7KqYOaP1P3JOOCUJqQLSlN0QRbBkBRy/3dUHzI3dhDK9PIeLtcg6EhQA1e/FFjNQePp3ggIskRTstK1/KEUCCXovKKEOuIzjuhUjBdqomP/pM3//4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7D59E1F394 for ; Wed, 13 Dec 2023 12:32:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470729; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=mfXlIng+8FgI0E2tngmvDv1pQx+ENORQ55TmFs5hFP0=; b=vr4LiQp4M6BjeEXALl5rc3FxbEHFBkzOcXzZeKFNoPF4ZJJ00M9N0eEtNjixnNH4hKGCLK PNaFePrv3ZjC+95+QSCpsEY5SlkfumB4UIIici+D0vfds01KCNxeGDqgtP2vlBM9pkRPcw xm0RBT3BP4mo6wMQc1ZCBcXxZ+aYbRo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470729; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=mfXlIng+8FgI0E2tngmvDv1pQx+ENORQ55TmFs5hFP0=; b=FcsFUZpA79giVhPs/7BSc2QLXYPfpvt1QoYvAP/5gKEfMOdlE+3/JgZ7nXuW818yhXSU4L +UYdDCnKFE30oSBQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470729; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=mfXlIng+8FgI0E2tngmvDv1pQx+ENORQ55TmFs5hFP0=; b=vr4LiQp4M6BjeEXALl5rc3FxbEHFBkzOcXzZeKFNoPF4ZJJ00M9N0eEtNjixnNH4hKGCLK PNaFePrv3ZjC+95+QSCpsEY5SlkfumB4UIIici+D0vfds01KCNxeGDqgtP2vlBM9pkRPcw xm0RBT3BP4mo6wMQc1ZCBcXxZ+aYbRo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470729; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=mfXlIng+8FgI0E2tngmvDv1pQx+ENORQ55TmFs5hFP0=; b=FcsFUZpA79giVhPs/7BSc2QLXYPfpvt1QoYvAP/5gKEfMOdlE+3/JgZ7nXuW818yhXSU4L +UYdDCnKFE30oSBQ== Date: Wed, 13 Dec 2023 13:31:05 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 3/6] Query an appropriate offset vector type in vect_gather_scatter_fn_p MIME-Version: 1.0 X-Spam-Score: 1.88 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: 4.17 X-Spamd-Result: default: False [4.17 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(2.88)[0.961]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-1.11)[88.32%] X-Spam-Flag: NO X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123306.2BC39388300A@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785170051672612474 X-GMAIL-MSGID: 1785170051672612474 The gather_load optab and friends require the offset vector mode to have the same number of lanes as the data vector mode. Restrict the vector type query to that when searching for a proper offset type. * tree-vect-data-refs.cc (vect_gather_scatter_fn_p): Use get_related_vectype_for_scalar_type to get at the offset vector type. --- gcc/tree-vect-data-refs.cc | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index 107dffe0a64..59e296e7976 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -3913,7 +3913,7 @@ vect_prune_runtime_alias_test_list (loop_vec_info loop_vinfo) *IFN_OUT and the vector type for the offset in *OFFSET_VECTYPE_OUT. */ bool -vect_gather_scatter_fn_p (vec_info *vinfo, bool read_p, bool masked_p, +vect_gather_scatter_fn_p (vec_info *, bool read_p, bool masked_p, tree vectype, tree memory_type, tree offset_type, int scale, internal_fn *ifn_out, tree *offset_vectype_out) @@ -3948,13 +3948,18 @@ vect_gather_scatter_fn_p (vec_info *vinfo, bool read_p, bool masked_p, for (;;) { - tree offset_vectype = get_vectype_for_scalar_type (vinfo, offset_type); - if (!offset_vectype) - return false; + /* The optabs require the same number of elements in the offset + vector as in the data vector. */ + tree offset_vectype + = get_related_vectype_for_scalar_type (TYPE_MODE (vectype), offset_type, + TYPE_VECTOR_SUBPARTS (vectype)); /* Test whether the target supports this combination. */ - if (internal_gather_scatter_fn_supported_p (ifn, vectype, memory_type, - offset_vectype, scale)) + if (!offset_vectype) + ; + else if (internal_gather_scatter_fn_supported_p (ifn, vectype, + memory_type, + offset_vectype, scale)) { *ifn_out = ifn; *offset_vectype_out = offset_vectype; From patchwork Wed Dec 13 12:31:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177943 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7739890dys; Wed, 13 Dec 2023 04:33:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IGMVwWdNr0ssmuDSEv6Dgd2bS5IdoVwJ8WxxZr/1XhnodgeLOp6AVrdaQBX25mxzMh4OLqI X-Received: by 2002:a05:620a:2fb:b0:77f:16ec:d713 with SMTP id a27-20020a05620a02fb00b0077f16ecd713mr9657713qko.90.1702470793348; Wed, 13 Dec 2023 04:33:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470793; cv=pass; d=google.com; s=arc-20160816; b=cQ52hJMlWJBuT5PMGHA5MeMtYmpAzL6WmBDbhfiZS0l3ybK8Pa3IEC9cYaxSCgPxIQ 804+zCkbaO2boGL9+rB5gJMvBtshKUAnqoR73yDRdzyW1QyBepAtt1tX2rQY9gknG5c7 Codoft7GNEJXIlwS/i9Kac1ZtZ3wHMdU+Gh6CoQsnONMGORgBq1LGsHRUSYEzXRTSoiW znkqH/wTNUqA4goUG8AWIgch/h3UrYnqirVKcK6mk9S+HBNOZ4Yv8dE+lPjg5qvoCXoB OFwQ1liD8tHa+2XqtGV5FxZhyGOVWdRI06r1fF+CXhKjgHwHfkTMd0WgTW+G/s1DyJlG 11eQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=rBVGXBK8wR+OMVjG3F9Ma8pvhxT1DalWuytRq4vCJMk=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=wkC7ImMp9rSWYStt5h7nofwAG/rSFv1I3p+I941HpJrQMFiCp4L7FcMClS/QcWDo3r uf3KQxFrGq8x06vHo/a2f/OdkVIR8RuD9pePsnyM5N8mXa9nT+rPT53d5vqoFel8s0sY D8TTaQJ8Jepk4/QjaNAz0kQnjVtIYWE3zp6HIzXqezRmyP/s8jIBNt7X0UkkyemHP+BB xg+bYWiX4RTO0mfh3F5aFZjvAh0gYCCXPPBx//ZSpdwMMAr6X57Q04nKcQ3rcVHUlabx Fxbb2qlGX0xPt/gpzvLyke1n5GL6yDI4lDuCjmv8MkzM6PuKGHfnZSxoClX3kEG2ZB+g TC4w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=B4cVxTiZ; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=tLpiZBgk; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=B4cVxTiZ; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=tLpiZBgk; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id bn34-20020a05620a2ae200b0077f4e80e131si12024741qkb.692.2023.12.13.04.33.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:33:13 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=B4cVxTiZ; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=tLpiZBgk; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=B4cVxTiZ; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=tLpiZBgk; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 16BE53845159 for ; Wed, 13 Dec 2023 12:32:46 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by sourceware.org (Postfix) with ESMTPS id 7C67D385417D for ; Wed, 13 Dec 2023 12:32:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7C67D385417D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 7C67D385417D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470739; cv=none; b=awBvL/vML4s4YoPbP6Cd4J76UJ2UZgPhVqIjHEWL5oQpHf5CfSAgbzUdtMz3cJYOS1xMtuMXMpDRtrW6mNsfUYiMPd6b/5ri0WplxmORzWSDVBZ0oQ396X3vhCkmxBtOI84fprZeIoXxI0+CnzQWbffWH9BAzvh2d4cnSRVVqcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470739; c=relaxed/simple; bh=WJufVMc/Ov1THtao/zb21RB3U5Ae9rt7F7lPyCqlLD0=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=twqsECaXw7oPDUD5c862W24nsueZAU1INEU8ClUxB4Ttb9zpj65gz9AjMd+HxjbTpAtlIItatgylzmiIyh6g1jPfY19G8YpUD+eptoBVw5R8c3976pZDtS0W0iglmuk3cYHrnjo7YZ+B8yVcAKGwPtq4/4ktG43eSGzCTaULxcw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6DBAB221C9 for ; Wed, 13 Dec 2023 12:32:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470736; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=rBVGXBK8wR+OMVjG3F9Ma8pvhxT1DalWuytRq4vCJMk=; b=B4cVxTiZFNQGEpAJt35s9pmsrCcEndL4o7sJPY1UCN1eAKtMb5kIc28z1yfuST9P+qmyn/ J1vwJhfpY5lskbps6d9Zfb1+JeJwRfpDd6OEJEk5rmDR/jZvr3wASox/s5kTpvgsXODsC3 ecA0VfXm4zIFqNW+DMbFk2FpqBStE+s= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470736; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=rBVGXBK8wR+OMVjG3F9Ma8pvhxT1DalWuytRq4vCJMk=; b=tLpiZBgk3D94DrH613RiEWsoRDZi3pV4se66QwXXQq/lZNUycVmsWEhC9RKrIeGiyFmsip FMYEkOuFYZ4CYlBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470736; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=rBVGXBK8wR+OMVjG3F9Ma8pvhxT1DalWuytRq4vCJMk=; b=B4cVxTiZFNQGEpAJt35s9pmsrCcEndL4o7sJPY1UCN1eAKtMb5kIc28z1yfuST9P+qmyn/ J1vwJhfpY5lskbps6d9Zfb1+JeJwRfpDd6OEJEk5rmDR/jZvr3wASox/s5kTpvgsXODsC3 ecA0VfXm4zIFqNW+DMbFk2FpqBStE+s= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470736; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=rBVGXBK8wR+OMVjG3F9Ma8pvhxT1DalWuytRq4vCJMk=; b=tLpiZBgk3D94DrH613RiEWsoRDZi3pV4se66QwXXQq/lZNUycVmsWEhC9RKrIeGiyFmsip FMYEkOuFYZ4CYlBA== Date: Wed, 13 Dec 2023 13:31:12 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 4/6] More explicit vector types MIME-Version: 1.0 X-Spam-Level: ** X-Spam-Score: 2.31 X-Spam-Level: X-Spam-Score: -0.60 X-Spamd-Result: default: False [-0.60 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Flag: NO Authentication-Results: smtp-out1.suse.de; none X-Spam-Status: No, score=-11.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123246.16BE53845159@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785170014653156774 X-GMAIL-MSGID: 1785170014653156774 This reduces more calls to get_vectype_for_scalar_type. * tree-vect-loop.cc (vect_transform_cycle_phi): Specify the vector type for invariant/external defs. * tree-vect-stmts.cc (vectorizable_shift): For invariant or external shifted operands use the result vector type. Specify the vector type for invariant/external defs. (vectorizable_store): Likewise. --- gcc/tree-vect-loop.cc | 2 +- gcc/tree-vect-stmts.cc | 20 ++++++-------------- 2 files changed, 7 insertions(+), 15 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 3af4160426b..9e531921e29 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -8749,7 +8749,7 @@ vect_transform_cycle_phi (loop_vec_info loop_vinfo, correctly if ncopies is not one. */ vect_get_vec_defs_for_operand (loop_vinfo, reduc_stmt_info, ncopies, initial_def, - &vec_initial_defs); + &vec_initial_defs, vectype_out); } else if (STMT_VINFO_REDUC_TYPE (reduc_info) == CONST_COND_REDUCTION || STMT_VINFO_REDUC_TYPE (reduc_info) == COND_REDUCTION) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 5ad306e2b08..88401a2a00b 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -6126,19 +6126,11 @@ vectorizable_shift (vec_info *vinfo, "use not simple.\n"); return false; } - /* If op0 is an external or constant def, infer the vector type - from the scalar type. */ + /* If op0 is an external or constant def, use the output vector type. */ if (!vectype) - vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (op0), slp_node); + vectype = vectype_out; if (vec_stmt) gcc_assert (vectype); - if (!vectype) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "no vectype for scalar type\n"); - return false; - } nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); nunits_in = TYPE_VECTOR_SUBPARTS (vectype); @@ -6426,8 +6418,8 @@ vectorizable_shift (vec_info *vinfo, (a special case for certain kind of vector shifts); otherwise, operand 1 should be of a vector type (the usual case). */ vect_get_vec_defs (vinfo, stmt_info, slp_node, ncopies, - op0, &vec_oprnds0, - vec_oprnd1 ? NULL_TREE : op1, &vec_oprnds1); + op0, vectype, &vec_oprnds0, + vec_oprnd1 ? NULL_TREE : op1, op1_vectype, &vec_oprnds1); /* Arguments are ready. Create the new vector stmt. */ FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0) @@ -8537,7 +8529,7 @@ vectorizable_store (vec_info *vinfo, op = vect_get_store_rhs (next_stmt_info); if (!costing_p) vect_get_vec_defs (vinfo, next_stmt_info, slp_node, ncopies, op, - &vec_oprnds); + vectype, &vec_oprnds); else update_prologue_cost (&prologue_cost, op); unsigned int group_el = 0; @@ -9303,7 +9295,7 @@ vectorizable_store (vec_info *vinfo, { vect_get_vec_defs_for_operand (vinfo, next_stmt_info, ncopies, op, - gvec_oprnds[i]); + gvec_oprnds[i], vectype); vec_oprnd = (*gvec_oprnds[i])[0]; dr_chain.quick_push (vec_oprnd); } From patchwork Wed Dec 13 12:31:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177945 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7740173dys; Wed, 13 Dec 2023 04:33:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IFSl/Eyum1nxKUJHdWvl0FTfJwLkCwjm04yCxebz4dRGFF7khd+dsCQOnofZdoGiWJh+l7D X-Received: by 2002:ac8:7c4c:0:b0:425:9f26:6c2a with SMTP id o12-20020ac87c4c000000b004259f266c2amr11075604qtv.66.1702470821740; Wed, 13 Dec 2023 04:33:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470821; cv=pass; d=google.com; s=arc-20160816; b=og2QGTva666iOhmv+ee5qoOWbL314nLWmhzCCBikylU5sBTDb9mSKqkkRFZDr7XVbN sOpv5ecZS8X9u+WLUe5H8xnaQGlwinbfueMuoFaYeeCiUNmrXu9OSQ7j7L3Becd/CQqi KJ7IfRpACfQL0PfPCDKTRXCGwyn09/J30b56gfGWaBOD2G7Kv+lb4Do8a0s4NYyMhUJv n0Uzwzejg7OquAYTzkgYHwlhfybftcqBOPmOZhqIVpydr82Hzqci7YbWtsZ3v+39qnua Is9osoAyuumEuI661yCxxBlH8knK0N2Wn1UZ7qUD5iZ+s3s/t/YnDAeY49gXzYABFKZq 52AA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=RxPFXp5HVv6XD1I3U89s9QK/tPG74VE2yelGOv63peA=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=IFq5FYlWwN3NSdtHMBe2oJ3BwI7xb4xiaJfrq9vVhX5dzL6GJv/ilPag5APyeu7C7I 6j66bQwxxX3emE49ta609pG0q0wG1Eiv4V9pFcGTKSrVBpgMR5yC5wCzp31E3hFyEn7A UWu2o/NHBN+ibfc2Pgq7j7uY3RXrDjSU+5nBVEOcxTs9nKIAGzYa1/1hCRnIYK1BoM3d M+GO/5RCa8Je6Z69uJ1o+VD8msHrdJGzIihyorEthMQ4XXkmL3eh512CNFfwd8/8GtJo eQGs4zXQzwszWFATzu+1c5h44UFhzx6Q0j3dpvk9hRXh+EHZIfRA4qu+S4fdRLp+eENN bYEg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="rJHFR/jP"; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b="/vcB+iyJ"; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="rJHFR/jP"; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b="/vcB+iyJ"; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id 10-20020ac8574a000000b0042402e963a9si14085624qtx.444.2023.12.13.04.33.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:33:41 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="rJHFR/jP"; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b="/vcB+iyJ"; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b="rJHFR/jP"; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b="/vcB+iyJ"; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DFB4D387543C for ; Wed, 13 Dec 2023 12:33:00 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2a07:de40:b251:101:10:150:64:1]) by sourceware.org (Postfix) with ESMTPS id 9C8AE3861012 for ; Wed, 13 Dec 2023 12:32:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9C8AE3861012 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 9C8AE3861012 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a07:de40:b251:101:10:150:64:1 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470755; cv=none; b=G8skjlFseK+uZzoYiQaGvLdcW02b8aY8cjGUzqnY6IKQvLmt7wYADN7VDizOeINcINNV3Hmbusb7VpFDkbQZRQKtPBE74soVK8pdGByCqjySlEpGMTImxrv2Bx6myvqSny01x67inAjeNXQAlGGs8j69IDiQHku9a+Bk9jkyBW8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470755; c=relaxed/simple; bh=ulv++fLJGin07VrQ2HnZhKSn51GRkqEmBXu78m2F+xA=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=G2UwZFSyzhTiNgGLYgoimD5uxDMSMhd83DKJHAYMX6vlVSJoZvvsOXi2DRq2G6gMEsZJuKbBvJA98YRrKlN1kYQhxES3uSz+3K5o7wuF2JVRubuKL+O/3Etrctm2hjhtwdg/n85A6dqYKgj3eolBwNFyz0QVrFTh7fqG5OIpgWI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9E84B221C9 for ; Wed, 13 Dec 2023 12:32:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470743; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=RxPFXp5HVv6XD1I3U89s9QK/tPG74VE2yelGOv63peA=; b=rJHFR/jPSnh2SGW/Kin8Fy1gO1lLOmVsxpI4XaVAKz9wIqtXKtTsSADb46AtnxkoJXAmef 5DCBzwIZ7BfJ6fakorMhlo927UTQCGLtNyApyjvFoaxcJP3u0VOEq0qf79wUalHYP/KODi HRPvwY7jkSTDitOVq7bZsa+DE/4LcKE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470743; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=RxPFXp5HVv6XD1I3U89s9QK/tPG74VE2yelGOv63peA=; b=/vcB+iyJIfNoDzZAeDPEBzZAq0BOWEqmPYZx8Bci2KQWnCq2WKQTdKiyclAqtmQ7eFbXaT 9+fDXhDtQRWIvyCA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470743; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=RxPFXp5HVv6XD1I3U89s9QK/tPG74VE2yelGOv63peA=; b=rJHFR/jPSnh2SGW/Kin8Fy1gO1lLOmVsxpI4XaVAKz9wIqtXKtTsSADb46AtnxkoJXAmef 5DCBzwIZ7BfJ6fakorMhlo927UTQCGLtNyApyjvFoaxcJP3u0VOEq0qf79wUalHYP/KODi HRPvwY7jkSTDitOVq7bZsa+DE/4LcKE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470743; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=RxPFXp5HVv6XD1I3U89s9QK/tPG74VE2yelGOv63peA=; b=/vcB+iyJIfNoDzZAeDPEBzZAq0BOWEqmPYZx8Bci2KQWnCq2WKQTdKiyclAqtmQ7eFbXaT 9+fDXhDtQRWIvyCA== Date: Wed, 13 Dec 2023 13:31:20 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 5/6] Allow poly_uint64 for group_size args to vector type query routines MIME-Version: 1.0 X-Spam-Level: ** X-Spam-Score: 2.32 X-Spam-Level: X-Spam-Score: -0.60 X-Spamd-Result: default: False [-0.60 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Flag: NO Authentication-Results: smtp-out1.suse.de; none X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123300.DFB4D387543C@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785170044204879209 X-GMAIL-MSGID: 1785170044204879209 The following changes the unsigned group_size argument to a poly_uint64 one to avoid too much special-casing in callers for VLA vectors when passing down the effective maximum desirable vector size to vector type query routines. The intent is to be able to pass down the vectorization factor (times the SLP group size) eventually. * tree-vectorizer.h (get_vectype_for_scalar_type, get_mask_type_for_scalar_type, vect_get_vector_types_for_stmt): Change group_size argument to poly_uint64 type. (vect_get_mask_type_for_stmt): Remove prototype for no longer existing function. * tree-vect-stmts.cc (get_vectype_for_scalar_type): Change group_size argument to poly_uint64. (get_mask_type_for_scalar_type): Likewise. (vect_get_vector_types_for_stmt): Likewise. --- gcc/tree-vect-stmts.cc | 25 ++++++++++++++----------- gcc/tree-vectorizer.h | 7 +++---- 2 files changed, 17 insertions(+), 15 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 88401a2a00b..a5e26b746fb 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -13297,14 +13297,14 @@ get_related_vectype_for_scalar_type (machine_mode prevailing_mode, tree get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, - unsigned int group_size) + poly_uint64 group_size) { /* For BB vectorization, we should always have a group size once we've constructed the SLP tree; the only valid uses of zero GROUP_SIZEs are tentative requests during things like early data reference analysis and pattern recognition. */ if (is_a (vinfo)) - gcc_assert (vinfo->slp_instances.is_empty () || group_size != 0); + gcc_assert (vinfo->slp_instances.is_empty () || known_ne (group_size, 0)); else group_size = 0; @@ -13320,9 +13320,11 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, /* If the natural choice of vector type doesn't satisfy GROUP_SIZE, try again with an explicit number of elements. */ + uint64_t cst_group_size; if (vectype - && group_size - && maybe_ge (TYPE_VECTOR_SUBPARTS (vectype), group_size)) + && group_size.is_constant (&cst_group_size) + && cst_group_size != 0 + && maybe_ge (TYPE_VECTOR_SUBPARTS (vectype), cst_group_size)) { /* Start with the biggest number of units that fits within GROUP_SIZE and halve it until we find a valid vector type. @@ -13336,7 +13338,7 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, even though the group is not a multiple of that vector size. The BB vectorizer will then try to carve up the group into smaller pieces. */ - unsigned int nunits = 1 << floor_log2 (group_size); + unsigned int nunits = 1 << floor_log2 (cst_group_size); do { vectype = get_related_vectype_for_scalar_type (vinfo->vector_mode, @@ -13372,7 +13374,7 @@ get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type, slp_tree node) tree get_mask_type_for_scalar_type (vec_info *vinfo, tree scalar_type, - unsigned int group_size) + poly_uint64 group_size) { tree vectype = get_vectype_for_scalar_type (vinfo, scalar_type, group_size); @@ -14243,7 +14245,7 @@ opt_result vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, tree *stmt_vectype_out, tree *nunits_vectype_out, - unsigned int group_size) + poly_uint64 group_size) { gimple *stmt = stmt_info->stmt; @@ -14252,7 +14254,7 @@ vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, are tentative requests during things like early data reference analysis and pattern recognition. */ if (is_a (vinfo)) - gcc_assert (vinfo->slp_instances.is_empty () || group_size != 0); + gcc_assert (vinfo->slp_instances.is_empty () || known_ne (group_size, 0)); else group_size = 0; @@ -14281,7 +14283,7 @@ vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, tree vectype; tree scalar_type = NULL_TREE; - if (group_size == 0 && STMT_VINFO_VECTYPE (stmt_info)) + if (known_eq (group_size, 0U) && STMT_VINFO_VECTYPE (stmt_info)) { vectype = STMT_VINFO_VECTYPE (stmt_info); if (dump_enabled_p ()) @@ -14310,10 +14312,11 @@ vect_get_vector_types_for_stmt (vec_info *vinfo, stmt_vec_info stmt_info, if (dump_enabled_p ()) { - if (group_size) + if (known_ne (group_size, 0U)) dump_printf_loc (MSG_NOTE, vect_location, "get vectype for scalar type (group size %d):" - " %T\n", group_size, scalar_type); + " %T\n", (int)constant_lower_bound (group_size), + scalar_type); else dump_printf_loc (MSG_NOTE, vect_location, "get vectype for scalar type: %T\n", scalar_type); diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index a2bab8676af..95240504d18 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -2204,9 +2204,9 @@ extern edge vec_init_loop_exit_info (class loop *); /* In tree-vect-stmts.cc. */ extern tree get_related_vectype_for_scalar_type (machine_mode, tree, poly_uint64 = 0); -extern tree get_vectype_for_scalar_type (vec_info *, tree, unsigned int = 0); +extern tree get_vectype_for_scalar_type (vec_info *, tree, poly_uint64 = 0); extern tree get_vectype_for_scalar_type (vec_info *, tree, slp_tree); -extern tree get_mask_type_for_scalar_type (vec_info *, tree, unsigned int = 0); +extern tree get_mask_type_for_scalar_type (vec_info *, tree, poly_uint64 = 0); extern tree get_mask_type_for_scalar_type (vec_info *, tree, slp_tree); extern tree get_same_sized_vectype (tree, tree); extern bool vect_chooses_same_modes_p (vec_info *, machine_mode); @@ -2295,8 +2295,7 @@ extern tree vect_gen_while (gimple_seq *, tree, tree, tree, extern tree vect_gen_while_not (gimple_seq *, tree, tree, tree); extern opt_result vect_get_vector_types_for_stmt (vec_info *, stmt_vec_info, tree *, - tree *, unsigned int = 0); -extern opt_tree vect_get_mask_type_for_stmt (stmt_vec_info, unsigned int = 0); + tree *, poly_uint64 = 0); /* In tree-vect-data-refs.cc. */ extern bool vect_can_force_dr_alignment_p (const_tree, poly_uint64); From patchwork Wed Dec 13 12:31:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 177947 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp7740414dys; Wed, 13 Dec 2023 04:34:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IFkqEGQjk0auWuDKmr0WNzFS3arnXzTMNB0E9DoVo6bdDtlsYfq6VDV3fJSjVrQFbZvjgu2 X-Received: by 2002:a05:622a:308:b0:425:8cbf:c8b3 with SMTP id q8-20020a05622a030800b004258cbfc8b3mr11948875qtw.99.1702470850150; Wed, 13 Dec 2023 04:34:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702470850; cv=pass; d=google.com; s=arc-20160816; b=yFyg4SkTXoi/k3o8eHmELFOIB7of2WXWC8vq8SeFwpPZYEm7cc4km8/Ly26Rn6AV4M +flpW5O+pc9C7I4F3TCrBGypo8v97vFUjnrpkMWsNhwEbYcgcHSRQoId+fIWQeluh9u0 FOA1V0nHVFCUGLe3XEYcjGf1ii6d/ApiEGUKZABQC0nSAcONvzV538PcZuvedkeAs0Nb 1knpF/iXE37n2LkhKALskUQ3CriS4HpeHt5QshbFzNO3PMEE6d1PzXJzwnInWZmS04+2 xJtMBQaxHQkJvguH+Yc7L4wF8Yv0kcqDBdsmADTTBJRaxac0TSjhbn7kp2Ajj8YGP3pO 3xBA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:to:from:date:dkim-signature:dkim-signature:dkim-signature :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=tigVD7Cv8uKq8yX9Ni0e6GOKDE0vHrIMimMk4SlCDvY=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=I967wTsTxfWWnF2fDEk+F/0c0yOg3hEPcPJvhhtSQTa2x8rVxah/nMCxlEQvDi8BgY CkwGYRI7sTuLgfFNyw/IpzF2uJBSHQTN/h+Gd2SLhbFtdznld2UA5l9OjUvHrLma5CCQ Jgld9M4jGz59NUDXpEh6lHu6cVJEFQTlvIgErEtH8lPQvR84kr16JGUcTcOcYTDWbxAS WL8pdYatJQw2OTjxzrJY+hiDXf+9mBQA+Mtl40xFjkgp4/y1RuT7Zijy82pxN17srxsv h7UVm9HNWHioCKEmwze3iaEXBfE4DSZ/FEBWP5Ed3L28l6PUErARWYzU9Y4OlIcZLi+H S/Ww== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=DGn9nqU0; dkim=neutral (no key) header.i=@suse.de; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=DGn9nqU0; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id jv12-20020a05622aa08c00b00423852de519si11809339qtb.758.2023.12.13.04.34.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 04:34:10 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=DGn9nqU0; dkim=neutral (no key) header.i=@suse.de; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=DGn9nqU0; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4544D3858D39 for ; Wed, 13 Dec 2023 12:33:36 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by sourceware.org (Postfix) with ESMTPS id DF380385AC3E for ; Wed, 13 Dec 2023 12:32:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DF380385AC3E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org DF380385AC3E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470754; cv=none; b=ayDD7paIKxrOpuvG9CjxPdLiRQeNjMsW2IM66pGod7H6EZwRCt6GoJan43T4znTooiu/Mt6Du2iIRqKxKjPcb7fZchcs0Gh6O0nPbb7yTIhq6EsQr8DJG6RYoyq8B0/WBSGiBO6k7ENhsFA/9NfLmoZHIlg2golO2PGZkOXqc9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702470754; c=relaxed/simple; bh=nZgoYd/u5i3oZdiXU7Yjk/oTMwvcfIeP1xG2LDzVGpA=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=wkai8131mhOOiRXdHJN1ZF5KOyq/rwvu+RPYt3bNdp6neaW5n8sy4XgGfDhDgIRV6Xi5iIWMgr+UgsCvKBf9xpxFd1Us6QbtUuS6s5yGonV6Hj9ZRmNwZK4har9gOKI36leoHP/Yu4sEjIjCfmsItDRynC3CYWv27675N+owKG0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C41D21F392 for ; Wed, 13 Dec 2023 12:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470750; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=tigVD7Cv8uKq8yX9Ni0e6GOKDE0vHrIMimMk4SlCDvY=; b=DGn9nqU0nA2houb76ikEmkCSITGkRt/wwrHrrWDhng1LjoKKz4l5nJXabgpvrNyq153K9m ybv8EMIdawWiaLLtr7LuNcDSWUlncPElVTH9jvAFKI4VvDHeOJ94+TOEz2/uvfAL04atGm QMLa9MhQHkq1MKqPCCTIVSkY0Cr4sJc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470750; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=tigVD7Cv8uKq8yX9Ni0e6GOKDE0vHrIMimMk4SlCDvY=; b=JvW8sAojXtwJJMnWSWN+hyr70YluzM3xaq64I84y9XzOzATYTWmVy6dVOHP2rxQrBOFdK9 VJp4xeJdPaW0LjAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1702470750; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=tigVD7Cv8uKq8yX9Ni0e6GOKDE0vHrIMimMk4SlCDvY=; b=DGn9nqU0nA2houb76ikEmkCSITGkRt/wwrHrrWDhng1LjoKKz4l5nJXabgpvrNyq153K9m ybv8EMIdawWiaLLtr7LuNcDSWUlncPElVTH9jvAFKI4VvDHeOJ94+TOEz2/uvfAL04atGm QMLa9MhQHkq1MKqPCCTIVSkY0Cr4sJc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1702470750; h=from:from:reply-to:date:date:to:to:cc:mime-version:mime-version: content-type:content-type; bh=tigVD7Cv8uKq8yX9Ni0e6GOKDE0vHrIMimMk4SlCDvY=; b=JvW8sAojXtwJJMnWSWN+hyr70YluzM3xaq64I84y9XzOzATYTWmVy6dVOHP2rxQrBOFdK9 VJp4xeJdPaW0LjAA== Date: Wed, 13 Dec 2023 13:31:27 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 6/6] Defer assigning vector types until after VF is determined MIME-Version: 1.0 X-Spam-Score: -0.60 Authentication-Results: smtp-out2.suse.de; none X-Spam-Score: 2.08 X-Spamd-Result: default: False [2.08 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(2.68)[0.894]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; DBL_BLOCKED_OPENRESOLVER(0.00)[gcc.target:url]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Flag: NO X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20231213123336.4544D3858D39@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785170074421451927 X-GMAIL-MSGID: 1785170074421451927 The following defers, for non-gather/scatter and non-pattern stmts, setting of STMT_VINFO_VECTYPE until after we computed the desired vectorization factor. This allows us to use larger vector types when the vectorization factor and the preferred vector mode allow, reducing the number of vector stmt copies and enabling vectorization in the first place if ncopies restrictions requires the use of different size vector types like for PR65947. vectorizable_operation handles some of the required vector type inference. * tree-vect-data-refs.cc (vect_analyze_data_refs): Do not set STMT_VINFO_VECTYPE unless this is a gather/scatter. * tree-vect-loop.cc (vect_determine_vf_for_stmt_1): Do not set STMT_VINFO_VECTYPE, only determine the VF. (vect_determine_vectorization_factor): Likewise. (vect_analyze_loop_2): Set STMT_VINFO_VECTYPE where missing and non-mask. Choose larger vectors to reduce the number of stmt copies. * tree-vect-stmts.cc (vect_analyze_stmt): Allow not specified vector type for mask producers. (vectorizable_operation): Refactor to handle STMT_VINFO_VECTYPE inference from operands. * gcc.dg/vect/pr65947-7.c: Adjust. * gcc.target/i386/vect-multi-size-1.c: New testcase. --- gcc/testsuite/gcc.dg/vect/pr65947-7.c | 2 +- .../gcc.target/i386/vect-multi-size-1.c | 17 ++ gcc/tree-vect-data-refs.cc | 11 +- gcc/tree-vect-loop.cc | 148 +++++++++++++++--- gcc/tree-vect-stmts.cc | 121 +++++++------- 5 files changed, 202 insertions(+), 97 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/vect-multi-size-1.c diff --git a/gcc/testsuite/gcc.dg/vect/pr65947-7.c b/gcc/testsuite/gcc.dg/vect/pr65947-7.c index 58c46df5c54..8f8adce3d91 100644 --- a/gcc/testsuite/gcc.dg/vect/pr65947-7.c +++ b/gcc/testsuite/gcc.dg/vect/pr65947-7.c @@ -53,4 +53,4 @@ main (void) } /* { dg-final { scan-tree-dump "optimizing condition reduction with FOLD_EXTRACT_LAST" "vect" { target vect_fold_extract_last } } } */ -/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" { target aarch64*-*-* } } } */ +/* { dg-final { scan-tree-dump "LOOP VECTORIZED" "vect" { target { aarch64*-*-* } || { vect_multiple_sizes } } } } */ diff --git a/gcc/testsuite/gcc.target/i386/vect-multi-size-1.c b/gcc/testsuite/gcc.target/i386/vect-multi-size-1.c new file mode 100644 index 00000000000..a0dd3cf9801 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/vect-multi-size-1.c @@ -0,0 +1,17 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -march=znver4 -fdump-tree-vect" } */ + +double x[1024]; +char y[1024]; +void foo () +{ + for (int i = 0 ; i < 16; ++i) + { + x[i] = i; + y[i] = i; + } +} + +/* We expect to see AVX512 vectors for x[] and a SSE vector for y[]. */ +/* { dg-final { scan-tree-dump-times "MEM " 2 "vect" } } */ +/* { dg-final { scan-tree-dump-times "MEM " 1 "vect" } } */ diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index 59e296e7976..80057474af9 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -4716,18 +4716,19 @@ vect_analyze_data_refs (vec_info *vinfo, poly_uint64 *min_vf, bool *fatal) vf = TYPE_VECTOR_SUBPARTS (vectype); *min_vf = upper_bound (*min_vf, vf); - /* Leave the BB vectorizer to pick the vector type later, based on - the final dataref group size and SLP node size. */ - if (is_a (vinfo)) - STMT_VINFO_VECTYPE (stmt_info) = vectype; - if (gatherscatter != SG_NONE) { + /* ??? We should perform a coarser check here, or none at all. + We're checking this again later, in particular during + relevancy analysis where we hook on the discovered offset + operand. */ + STMT_VINFO_VECTYPE (stmt_info) = vectype; gather_scatter_info gs_info; if (!vect_check_gather_scatter (stmt_info, as_a (vinfo), &gs_info)) { + STMT_VINFO_VECTYPE (stmt_info) = NULL_TREE; if (fatal) *fatal = false; return opt_result::failure_at diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 9e531921e29..f226135cb1d 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -189,22 +189,19 @@ vect_determine_vf_for_stmt_1 (vec_info *vinfo, stmt_vec_info stmt_info, if (!res) return res; - if (stmt_vectype) + if (nunits_vectype) { - if (STMT_VINFO_VECTYPE (stmt_info)) - /* The only case when a vectype had been already set is for stmts - that contain a data ref, or for "pattern-stmts" (stmts generated - by the vectorizer to represent/replace a certain idiom). */ - gcc_assert ((STMT_VINFO_DATA_REF (stmt_info) - || vectype_maybe_set_p) - && STMT_VINFO_VECTYPE (stmt_info) == stmt_vectype); - else - STMT_VINFO_VECTYPE (stmt_info) = stmt_vectype; + poly_uint64 saved_vf = *vf; + vect_update_max_nunits (vf, nunits_vectype); + if (maybe_ne (*vf, saved_vf) && dump_enabled_p ()) + { + dump_printf_loc (MSG_NOTE, vect_location, "updated " + "vectorization factor to "); + dump_dec (MSG_NOTE, *vf); + dump_printf (MSG_NOTE, "\n"); + } } - if (nunits_vectype) - vect_update_max_nunits (vf, nunits_vectype); - return opt_result::success (); } @@ -330,20 +327,17 @@ vect_determine_vectorization_factor (loop_vec_info loop_vinfo, "not vectorized: unsupported " "data-type %T\n", scalar_type); - STMT_VINFO_VECTYPE (stmt_info) = vectype; - - if (dump_enabled_p ()) - dump_printf_loc (MSG_NOTE, vect_location, "vectype: %T\n", - vectype); - if (dump_enabled_p ()) + poly_uint64 saved_vectorization_factor = vectorization_factor; + vect_update_max_nunits (&vectorization_factor, vectype); + if (maybe_ne (vectorization_factor, saved_vectorization_factor) + && dump_enabled_p ()) { - dump_printf_loc (MSG_NOTE, vect_location, "nunits = "); - dump_dec (MSG_NOTE, TYPE_VECTOR_SUBPARTS (vectype)); + dump_printf_loc (MSG_NOTE, vect_location, "updated " + "vectorization factor to "); + dump_dec (MSG_NOTE, vectorization_factor); dump_printf (MSG_NOTE, "\n"); } - - vect_update_max_nunits (&vectorization_factor, vectype); } } @@ -2864,6 +2858,114 @@ start_over: gcc_assert (known_eq (LOOP_VINFO_VECT_FACTOR (loop_vinfo), 0U)); loop_vinfo->vectorization_factor = vectorization_factor; + /* At this point we have the vectorization factor that should determine + the vector types to use decided. The unrolling factor should not + influence that since otherwise we'd eventually use larger vectors + rather than doing actual effective unrolling. + + Note that with re-starting without SLP we actually will have the + original loop VF so we're off here - but then non-SLP should go + away ... */ + /* Check that nothing set STMT_VINFO_VECTYPE so nothing could have + relied on it. ??? Same for SLP. ??? That also catches pattern + stmts which might be more difficult to "fix". */ + for (stmt_vec_info stmt_info : loop_vinfo->stmt_vec_infos) + { + if (!stmt_info + || gimple_clobber_p (stmt_info->stmt)) + continue; + + if (!STMT_VINFO_RELEVANT_P (stmt_info) + && !STMT_VINFO_LIVE_P (stmt_info)) + continue; + + if (STMT_VINFO_VECTYPE (stmt_info)) + { + /* Pattern stmts and gather/scatter may have a precomputed + vector type. */ + gcc_assert (STMT_VINFO_RELATED_STMT (stmt_info) + || STMT_VINFO_GATHER_SCATTER_P (stmt_info)); + continue; + } + + /* ??? This is still a coarse vector type decision. Multiple + up/down passes over use-def chains should be used to set + vector types from within vectorizable_* itself, in a new + special mode. Possibly identifying the responsible worker early. + Not worth spending much time on this in the non-SLP path. */ + tree stmt_vectype, nunits_vectype; + opt_result res + = vect_get_vector_types_for_stmt (loop_vinfo, stmt_info, &stmt_vectype, + &nunits_vectype); + gcc_assert (res); + if (!stmt_vectype) + /* OMP SIMD calls without LHS. */ + continue; + + tree scalar_type = NULL_TREE; + if (vect_use_mask_type_p (stmt_info)) + { + if (is_a (stmt_info->stmt)) + { + /* Only with BB vectorization or as PHI in a nested cycle. */ + gcc_assert (flow_bb_inside_loop_p (LOOP_VINFO_LOOP (loop_vinfo), + gimple_bb (stmt_info->stmt))); + /* ??? vectorizable_* should set the vector type. */ + continue; + } + else + { + tree_code code = gimple_assign_rhs_code (stmt_info->stmt); + if (is_gimple_assign (stmt_info->stmt) + && TREE_CODE_CLASS (code) == tcc_comparison) + scalar_type = TREE_TYPE (gimple_assign_rhs1 (stmt_info->stmt)); + else + /* ??? vectorizable_* should set the vector type. */ + continue; + } + } + else + scalar_type = TREE_TYPE (stmt_vectype); + + /* Try to use a larger vector type when the above one has less lanes + than the chosen VF, up to the one recommended by the perferred vector + mode hook. This keeps ncopies down, generating more efficient code + and in some cases enables vectorizing in the first place. */ + tree preferred_vectype = get_related_vectype_for_scalar_type (VOIDmode, + scalar_type, + 0); + if (known_lt (TYPE_VECTOR_SUBPARTS (stmt_vectype), + LOOP_VINFO_VECT_FACTOR (loop_vinfo)) + && known_lt (TYPE_VECTOR_SUBPARTS (stmt_vectype), + TYPE_VECTOR_SUBPARTS (preferred_vectype)) + && ordered_p (TYPE_VECTOR_SUBPARTS (preferred_vectype), + LOOP_VINFO_VECT_FACTOR (loop_vinfo))) + { + /* ??? Could try all nunits between stmt_vectype and MIN. */ + poly_uint64 nunits + = ordered_min (LOOP_VINFO_VECT_FACTOR (loop_vinfo), + TYPE_VECTOR_SUBPARTS (preferred_vectype)); + tree cand = get_related_vectype_for_scalar_type + (TYPE_MODE (preferred_vectype), scalar_type, nunits); + if (cand) + { + if (VECTOR_BOOLEAN_TYPE_P (stmt_vectype)) + cand = truth_type_for (cand); + stmt_vectype = cand; + } + } + + if (dump_enabled_p ()) + { + dump_printf_loc (MSG_NOTE, vect_location, + "==> examining statement: %G", stmt_info->stmt); + dump_printf_loc (MSG_NOTE, vect_location, "vectype: %T\n", + stmt_vectype); + } + + STMT_VINFO_VECTYPE (stmt_info) = stmt_vectype; + } + if (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) && dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index a5e26b746fb..da27404aadb 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -6490,7 +6490,6 @@ vectorizable_operation (vec_info *vinfo, int ndts = 3; poly_uint64 nunits_in; poly_uint64 nunits_out; - tree vectype_out; int ncopies, vec_num; int i; vec vec_oprnds0 = vNULL; @@ -6550,25 +6549,6 @@ vectorizable_operation (vec_info *vinfo, return false; } - scalar_dest = gimple_assign_lhs (stmt); - vectype_out = STMT_VINFO_VECTYPE (stmt_info); - - /* Most operations cannot handle bit-precision types without extra - truncations. */ - bool mask_op_p = VECTOR_BOOLEAN_TYPE_P (vectype_out); - if (!mask_op_p - && !type_has_mode_precision_p (TREE_TYPE (scalar_dest)) - /* Exception are bitwise binary operations. */ - && code != BIT_IOR_EXPR - && code != BIT_XOR_EXPR - && code != BIT_AND_EXPR) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "bit-precision arithmetic not supported.\n"); - return false; - } - slp_tree slp_op0; if (!vect_is_simple_use (vinfo, stmt_info, slp_node, 0, &op0, &slp_op0, &dt[0], &vectype)) @@ -6580,47 +6560,6 @@ vectorizable_operation (vec_info *vinfo, } bool is_invariant = (dt[0] == vect_external_def || dt[0] == vect_constant_def); - /* If op0 is an external or constant def, infer the vector type - from the scalar type. */ - if (!vectype) - { - /* For boolean type we cannot determine vectype by - invariant value (don't know whether it is a vector - of booleans or vector of integers). We use output - vectype because operations on boolean don't change - type. */ - if (VECT_SCALAR_BOOLEAN_TYPE_P (TREE_TYPE (op0))) - { - if (!VECT_SCALAR_BOOLEAN_TYPE_P (TREE_TYPE (scalar_dest))) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "not supported operation on bool value.\n"); - return false; - } - vectype = vectype_out; - } - else - vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (op0), - slp_node); - } - if (vec_stmt) - gcc_assert (vectype); - if (!vectype) - { - if (dump_enabled_p ()) - dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, - "no vectype for scalar type %T\n", - TREE_TYPE (op0)); - - return false; - } - - nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); - nunits_in = TYPE_VECTOR_SUBPARTS (vectype); - if (maybe_ne (nunits_out, nunits_in)) - return false; - tree vectype2 = NULL_TREE, vectype3 = NULL_TREE; slp_tree slp_op1 = NULL, slp_op2 = NULL; if (op_type == binary_op || op_type == ternary_op) @@ -6635,9 +6574,8 @@ vectorizable_operation (vec_info *vinfo, } is_invariant &= (dt[1] == vect_external_def || dt[1] == vect_constant_def); - if (vectype2 - && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype2))) - return false; + if (!vectype) + vectype = vectype2; } if (op_type == ternary_op) { @@ -6651,9 +6589,52 @@ vectorizable_operation (vec_info *vinfo, } is_invariant &= (dt[2] == vect_external_def || dt[2] == vect_constant_def); - if (vectype3 - && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype3))) - return false; + if (!vectype) + vectype = vectype3; + } + + if (!vectype) + vectype = STMT_VINFO_VECTYPE (stmt_info); + if (!vectype) + { + /* We want to pre-assign sth here. */ + gcc_assert (!vec_stmt + && is_invariant + && !vect_use_mask_type_p (stmt_info)); + vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (op0), slp_node); + } + + tree vectype_out = vectype; + nunits_out = TYPE_VECTOR_SUBPARTS (vectype_out); + nunits_in = TYPE_VECTOR_SUBPARTS (vectype); + if (maybe_ne (nunits_out, nunits_in)) + return false; + /* ??? Isn't the constraint the types are the same apart from + signednes (ABSU_EXPR for example)? The rest suggests this as + we are using 'vectype' for constants/invariants. */ + if (vectype2 + && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype2))) + return false; + if (vectype3 + && maybe_ne (nunits_out, TYPE_VECTOR_SUBPARTS (vectype3))) + return false; + + scalar_dest = gimple_assign_lhs (stmt); + + /* Most operations cannot handle bit-precision types without extra + truncations. */ + bool mask_op_p = VECTOR_BOOLEAN_TYPE_P (vectype_out); + if (!mask_op_p + && !type_has_mode_precision_p (TREE_TYPE (scalar_dest)) + /* Exception are bitwise binary operations. */ + && code != BIT_IOR_EXPR + && code != BIT_XOR_EXPR + && code != BIT_AND_EXPR) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "bit-precision arithmetic not supported.\n"); + return false; } /* Multiple types in SLP are handled by creating the appropriate number of @@ -6788,6 +6769,8 @@ vectorizable_operation (vec_info *vinfo, return false; } + if (!STMT_VINFO_VECTYPE (stmt_info)) + STMT_VINFO_VECTYPE (stmt_info) = vectype; STMT_VINFO_TYPE (stmt_info) = op_vec_info_type; DUMP_VECT_SCOPE ("vectorizable_operation"); vect_model_simple_cost (vinfo, stmt_info, @@ -12890,7 +12873,9 @@ vect_analyze_stmt (vec_info *vinfo, { gcall *call = dyn_cast (stmt_info->stmt); gcc_assert (STMT_VINFO_VECTYPE (stmt_info) - || (call && gimple_call_lhs (call) == NULL_TREE)); + || (call && gimple_call_lhs (call) == NULL_TREE) + /* ??? Inconsistently so. */ + || vect_use_mask_type_p (stmt_info)); *need_to_vectorize = true; }