From patchwork Wed Nov 8 15:02:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 163078 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:aa0b:0:b0:403:3b70:6f57 with SMTP id k11csp971491vqo; Wed, 8 Nov 2023 07:03:30 -0800 (PST) X-Google-Smtp-Source: AGHT+IE+NbIa362NKctte2RBQ+M288ZkYOCNYzLgrqE+jrfWl9z+l/hMi/7Wxjbtcz6TXyst5dRn X-Received: by 2002:a05:6870:2381:b0:1ef:ad05:f49b with SMTP id e1-20020a056870238100b001efad05f49bmr2182570oap.37.1699455810587; Wed, 08 Nov 2023 07:03:30 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699455810; cv=pass; d=google.com; s=arc-20160816; b=eyao5XXuQyvE7SPYsE70tIc7aI3Xg3KMG6jCAFVKBvJCNNzjQFyqIkiw/4XxpDUCPV 373D4bT09k6NiXVFfmkDdSJo7K4/tqTMrFhfuJa1Svp/l0Xa0141XzbOoZSWGVOPH6gx Vl/jSm6dAMbk7SWsCoHrvDlY4JZ0hMd93JhI8VfN4jLfklAM2IUP4DpbBEfyFXLFf2m0 rgFRcSXYEOHMtFtWRdge7YYQuFNKMnA5qDOWP7hQ71qins0RY7TslTUYmHnNUDYZv8kW tyjaQIWRNbrg/aMtt0Kk0QY3r5jkrh5Y/Maf599WJKtlM3cNmBbnuNOwVASzfoLREht5 pdaw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:message-id:mime-version:subject :to:from:date:dkim-signature:dkim-signature:arc-filter:dmarc-filter :delivered-to; bh=o8JPehAFx6piKK9u4IkX6AIr9TOHOq6s6TWfgUMe5j8=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=qmeWBuGnWIKO2j8vmc1TYrjxiECNJ354dpppKZ2sbNUV1bM75dYFz/MboEXlvJnY2D NjzKMjHxJgYjzr57S8y7tzzmoWvmVOHsHyvYJzff2zMpUHdDjQ0io2w0+NLX3R8BiyR8 k+9qx0cCGSmuwN1N/2UdbmS65W75h8ADcowUcmNHYnQohK/na3bnqhT5ZZouPu8Ni7BS zlYMNGde1Y1RPdVRYn+o5QmjZit8x/9oNudp4Ie6NEXXffEUNzgDxR2VDi57R2TTbsVq 8fBnzxCbTSm2Mess6wh64/imToZkf8kbKsq42bD8jFNKYjHS7lT1WpAit8SCZfr1uQH0 J7dg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=v7oDQtka; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=fgtCdoqv; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id n15-20020ab0748f000000b007a874d659dbsi1130023uap.178.2023.11.08.07.03.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 07:03:30 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=v7oDQtka; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519 header.b=fgtCdoqv; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EE3653854143 for ; Wed, 8 Nov 2023 15:03:28 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id A41883858D1E for ; Wed, 8 Nov 2023 15:02:55 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A41883858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A41883858D1E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.220.29 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699455785; cv=none; b=ZiAPVMpgsNCKVM82QiBxXq7r/KUwNuCAxqrePdS/xQcWqev2k0yrnlN7UcrfRrPxiosvP0dIQ7VCl6C2Sbl+XuWtroKv9T/8lzfoO2GL0nKaukO/fiRkDgymhj4WUpPWtrMvRLBkVAFHQ73T8BDEUQNi3kYhTgDaFBG5wUOgQuw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699455785; c=relaxed/simple; bh=EAV01csWsr/9+ICOG5qwjDXEVAYJI9rZ4HIrXlNTUlU=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:MIME-Version: Message-Id; b=vIjKGF1nwZmoxKY7g2jeQ6njb5tCq8ljv7psvMNbLSyL3pigQZRdPVbuYrZMcPC66gTmueb7xOzzPzK3tBENzNuLLM1Da1gXIiSnUeKWf7QtsR5xBLuF9SmJmDbZQkerG3dNCx8BqkNwasNJwOh/HJ6x53T0dsN9IX2UwvlE2LA= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id A3CC11F45A for ; Wed, 8 Nov 2023 15:02:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699455774; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=o8JPehAFx6piKK9u4IkX6AIr9TOHOq6s6TWfgUMe5j8=; b=v7oDQtkaJh+o9qhiUIqkbxGWl6YJyvmq/sLQTk1Ik0CW5zjtVAUjwIBtdhSHux20pz++Ep KdsMYRbtby3Qwm/cmpmiAskiSED4LPtqbJwDZEAwmqEvcHtt/tJer7loA+qBKNZ1AVQE/O rwTwNmqe86rWGtHpj9TB0g7rJgpgzz4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699455774; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=o8JPehAFx6piKK9u4IkX6AIr9TOHOq6s6TWfgUMe5j8=; b=fgtCdoqvKeMstKZOY65u6tU5B7ibDHs2sx5NjyeqYAsVfhwW498DNPLejKK819cQVgwAGz u0qpGlJphbb+fWBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8F9F9133F5 for ; Wed, 8 Nov 2023 15:02:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id bCLJIR6jS2XkdwAAMHmgww (envelope-from ) for ; Wed, 08 Nov 2023 15:02:54 +0000 Date: Wed, 8 Nov 2023 16:02:54 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH 1/4] Fix SLP of masked loads MIME-Version: 1.0 Message-Id: <20231108150254.8F9F9133F5@imap2.suse-dmz.suse.de> X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782008575698183559 X-GMAIL-MSGID: 1782008575698183559 The following adjusts things to use the correct mask operand for the SLP of masked loads and gathers. Test coverage is from runtime fails of i386 specific AVX512 tests when enabling single-lane SLP. Bootstrap and regtest running on x86_64-unknown-linux-gnu. * tree-vect-stmts.cc (vectorizable_load): Use the correct vectorized mask operand. --- gcc/tree-vect-stmts.cc | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index 65883e04ad7..096a857f2dd 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -10920,9 +10920,6 @@ vectorizable_load (vec_info *vinfo, gsi, stmt_info, bump); } - if (mask && !costing_p) - vec_mask = vec_masks[j]; - gimple *new_stmt = NULL; for (i = 0; i < vec_num; i++) { @@ -10931,6 +10928,8 @@ vectorizable_load (vec_info *vinfo, tree bias = NULL_TREE; if (!costing_p) { + if (mask) + vec_mask = vec_masks[vec_num * j + i]; if (loop_masks) final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, @@ -11285,8 +11284,6 @@ vectorizable_load (vec_info *vinfo, at_loop, offset, &dummy, gsi, &ptr_incr, simd_lane_access_p, bump); - if (mask) - vec_mask = vec_masks[0]; } else if (!costing_p) { @@ -11297,8 +11294,6 @@ vectorizable_load (vec_info *vinfo, else dataref_ptr = bump_vector_ptr (vinfo, dataref_ptr, ptr_incr, gsi, stmt_info, bump); - if (mask) - vec_mask = vec_masks[j]; } if (grouped_load || slp_perm) @@ -11312,6 +11307,8 @@ vectorizable_load (vec_info *vinfo, tree bias = NULL_TREE; if (!costing_p) { + if (mask) + vec_mask = vec_masks[vec_num * j + i]; if (loop_masks) final_mask = vect_get_loop_mask (loop_vinfo, gsi, loop_masks, vec_num * ncopies, vectype,