From patchwork Thu Aug 3 08:10:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 130380 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp988590vqx; Thu, 3 Aug 2023 01:12:37 -0700 (PDT) X-Google-Smtp-Source: APBJJlH2rUw2cLVQ3YAcpQX5uywiohpUMa1sssaPzuMu/rREKA97A9UrkKuFbMQ2SEh/6wCKGfEJ X-Received: by 2002:a17:906:7397:b0:99b:ef8a:d64a with SMTP id f23-20020a170906739700b0099bef8ad64amr7796043ejl.0.1691050357363; Thu, 03 Aug 2023 01:12:37 -0700 (PDT) Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id y12-20020a170906558c00b0099bd7b268easi11644310ejp.121.2023.08.03.01.12.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Aug 2023 01:12:37 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=baHjUiEZ; arc=fail (signature failed); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C1B98385AF92 for ; Thu, 3 Aug 2023 08:12:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C1B98385AF92 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1691050342; bh=AMzD3GJnpe/nNGWZUobyEHwqzi1A7TLBY3i8cu5eZNM=; h=Date:Subject:To:Cc:References:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=baHjUiEZygsa/+mBM6IWa7pVuzonV0nM3JXRaf8fdpMD6Sh5GUNVtrF75+MmY/FZQ ji06Xfj7K4BSIqE6FuHydU7Pdkk78gT2YQ+dV628tKDIMXHMrL0Eekh6yq8msWFEC2 A/Lf0FJqOFw6QEAvnkYQkIi5PoQTSBaiEwR8UVUE= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2047.outbound.protection.outlook.com [40.107.22.47]) by sourceware.org (Postfix) with ESMTPS id B42823858280 for ; Thu, 3 Aug 2023 08:10:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B42823858280 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bQWGWAROUUYYSnev3tGh6Gnux3iyayYtrjEMbU0jorh9DrMas00a9ZRFEdrWfUhEwJOL9T0YkYcYl07IdpaszsOf9JuWufkauQNVYcq3gdH/nwzOAIbSu26uXpBMzgwzo+2S+d7fZFdJMMd7wFmpCZtuLW8lJHrm86hzaSyT09miGguEsvwRs/5czVmvMIhu2UgpjNsDrDv0UeMTZvRmI92Le17T54CdXgxTlVQe6CsLgprjBbdDYc9NhNqLijOJZe61U4r1k5osUBUcLJYsBPTYMy1kJ4l8j2jmX17SzcwrtoF5zfgccAmSzk/nsSa11BS1LcpTpyz58VYt7tn9EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AMzD3GJnpe/nNGWZUobyEHwqzi1A7TLBY3i8cu5eZNM=; b=BplkVjGxNFMXzoR0RDAoepACZm/6XQW97JswMFdnZC3XaYbH7HgnWf7Rr9DVjEnn9gXLehgrdj5v7FVj9N9p3zrYXkpfNNUiWEGsMK/PT5ZoaI4JTi6aWtbYR3gOkOw7TdH5BmJAx23Yuz8ildYSHBPs8V6LBcvKVMHDiQPkhOSFZPMBevojWT5HKnQfIKKH0E4+NR2f/NclN2UgM9YutS/k7MXCS+McfUcN5KlOGAD89bjY+8A5L4kdfZYixQY1SHeQwZc3w8HiuenwFCAu4h+PWDV2N9CO7WCamS+GZie+yCjCdHdan4FIEaMLDfh+vUYaZh2WgWBmHFS3OkAWMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Received: from DU2PR04MB8790.eurprd04.prod.outlook.com (2603:10a6:10:2e1::23) by AS4PR04MB9265.eurprd04.prod.outlook.com (2603:10a6:20b:4e0::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 08:10:48 +0000 Received: from DU2PR04MB8790.eurprd04.prod.outlook.com ([fe80::e5cf:5743:ab60:b14e]) by DU2PR04MB8790.eurprd04.prod.outlook.com ([fe80::e5cf:5743:ab60:b14e%5]) with mapi id 15.20.6631.046; Thu, 3 Aug 2023 08:10:48 +0000 Message-ID: Date: Thu, 3 Aug 2023 10:10:46 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.13.1 Subject: [PATCH 03/10] x86: "ssemuladd" adjustments Content-Language: en-US To: "gcc-patches@gcc.gnu.org" Cc: Uros Bizjak , Hongtao Liu , Jan Hubicka , Kirill Yukhin References: <9bef986f-7441-c4a9-5bfd-7bbed0a3d950@suse.com> In-Reply-To: <9bef986f-7441-c4a9-5bfd-7bbed0a3d950@suse.com> X-ClientProxiedBy: FR0P281CA0203.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:ad::9) To DU2PR04MB8790.eurprd04.prod.outlook.com (2603:10a6:10:2e1::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DU2PR04MB8790:EE_|AS4PR04MB9265:EE_ X-MS-Office365-Filtering-Correlation-Id: a71b057a-c38d-4870-f671-08db93f92541 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4+vpJ16N9iryyrj3JWGr7iIoO1cHTKkKg5dRNbYgK3osAhDtyyxpD/VZVOkSZP9jsuf0/KCcb3TaZlwIHH9IISRwmb5mjtKpnNARWpaK4+gqLOsjbLMPgN2xJwpwo50NoSUy6SHKFqrvZkyzrjVx5EHn3iveXkOp4GgCZ4w4wwtc/MkO2bm6ScrjkVTzITxyOC+b5HwRmt/0BvSFV/QFVZ6G1N/TtzYKFSaI8c/H+UYdUa+96D4ro6HSpBN1BQoofwiCZ7eha9cqzsQP1XQaB6Z04tVN/jWPfIrbBo0dar7QqJIVxqPcjF0t2Re9WEkioxBUTp/2bRg04XvrH0MhTbthzHtxprRxqDhR5GYwsRjHEo4vKkDGGVbE5SvBGZ6/bTCXovM7YAyXqGnKaFi4hZq9uSdLSLpJpWu8QNGnoYVahNpJ+BCzPMl1yCxLjLu+V9i8ISf5lYGyAYQRlCwc/+ZCaeZ91Aekfx3noWWqWHljkbgH3dfFFtDluTp06s7wap6/PEsSywPT4yCU2/I9g35ip78rYl44dk4QBmIGF35RlFRgViB433JF8cRItuNOMTILNkLLpIazOYetLfx45Yba0BWfqhzY54pXH7a0cEMA69mApwrwr5kC15pOyW4XxcabnbV1UcXKWE566UQYeQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DU2PR04MB8790.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(136003)(396003)(346002)(39860400002)(366004)(376002)(451199021)(2616005)(6506007)(26005)(186003)(316002)(2906002)(66946007)(4326008)(66476007)(66556008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(6486002)(6512007)(54906003)(478600001)(38100700002)(31696002)(86362001)(36756003)(31686004)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?v7BpMHT21oyWlnwbH3p3ponjC5yj?= =?utf-8?q?wB755IY2jQJabvcZcLiboOwT4VvZGfJhkoSxOrBuUva9wjtR0u6RQwaGSm10yh9eV?= =?utf-8?q?gRbUUzs+EA/VRq0ePXqqBslrW8bQcAykTWljJeHyj4GBN1y4UDqInelSLk5pYUi3R?= =?utf-8?q?ojz6n7LAH8KcdCwSCv9qDPbl25xcZicEMPpxkAE5asKWJwCUGVtVn0+S6XUrkTWON?= =?utf-8?q?kyi6+jWaP+RaJYcFNTPkAQxREdjIev5qa1f3QjJvYn58TiEZreOMVqUb6wuki46vh?= =?utf-8?q?jab2xVgXftJkr5/+Zeu2VexbjfbzFf+Z+z1OB+/+0EBNW2wUk2j2Vns5WwPCFQH5k?= =?utf-8?q?2RcnUYxDwrn9GA69OM25y9Q6Gg+BKhPnHgbYICqi+t23Oh5Quf7yL3sHsjFH847Zj?= =?utf-8?q?Acfmn+WGKw8lcts+eCNbe6R3o1l+WIyFDXYaN20N8OxM8zXRIo7SR4BR9aZZgXMO/?= =?utf-8?q?HzxiJrtwlHLt6ynqPSefkjZKFzmg/+XgSErv4tONjYfnlPojeXlm01gtRN3ZrhElW?= =?utf-8?q?kUyRZydgVFQ/DAYCMHzzwMh6cf475hCnPmHP28Q1zBqzFg49tfUUWatQZzn1qWyvu?= =?utf-8?q?mRAVokdKXwiMTTXw5ZDy5pLy4GwwRp29hC0jiELHHZsawyG4IlS4Q+5x2sZa/MZTJ?= =?utf-8?q?8XOn8G97ImzS70l3IljtNXtOyKKhQQHFbksaoNZoJcf9kAYpEMJnZaZkBBdB2YMf6?= =?utf-8?q?jVzEe4CGvMcG2hCw94pqoPM/Fg/2eAHXdiv/FKt+qt/3jAJXHQRhty/bed4Dn3AUP?= =?utf-8?q?SsPDoJ7CLkMmu8H6rU3P8d2WD3tccCmXfNyD7MMnVj4iPvKJWjMEHL/o9bJtiQXq9?= =?utf-8?q?An3BDSOWmK/GxqBwaFU/fHzluFZYcfIkUzc53INJLNKp41zQnqCMRGP6vy3yTOMrc?= =?utf-8?q?55OdHeAL5ZYXPVoF783W4v7fWSjt7VFPHe9kzl6K4/4OOZk0k4VakUUZtmHzxuKCC?= =?utf-8?q?+4LWaCwG1t2FUTvy3mDYvfEhFFuPWgzR4/RWUvr9kPdJnmiPCVD2uSOcLYsoUIUp+?= =?utf-8?q?RPuJpcRfHrJDHJxurLey36FdGIWtoeZrwEjdBdSYsjgGL3DBlwWxIDd+M3jd85aEA?= =?utf-8?q?GPSyT53im7UP/v57uBGEDFzSa3BnpM6pXxL0f1b6U8ngi2o/9xhZr9MCgBtxuDs7s?= =?utf-8?q?M8AQiX6AGBi9m7zsYBleTJH7AafI+37h7U9YzldyAB5kHeXqtJsTEAJimQAOabf1P?= =?utf-8?q?TZe+1Uo2Y8avR96iz/8xVFrGRk8MrKGlvbk3bZwNUPVPlbOH/rlfQN9urc/aMG/UI?= =?utf-8?q?3CcI4JCMqkI7C5/iDyv4f9q0TUQWnHH+HYsMjBf/qTWCEXnsG05K8+mritItewMfl?= =?utf-8?q?pNHfya/TiiAnUzHHsYfIJ0l5Bwx4hNV25X5uWu3RaL6lKtngM56Ptv68Yt90Es37b?= =?utf-8?q?tRQ8dOLGpesrZYAmVEThsb1kspMjIzMwNobzxnmrdWmCdsOZjbSHrcOA240O7zguK?= =?utf-8?q?rPAc9vRqBFHky3el4iHVEWh2sJDB5N1Qbg6HGDNoNDfgHkIZ/FSCoX2mLJp8/5VQN?= =?utf-8?q?cx4mMa0Q4lrV?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: a71b057a-c38d-4870-f671-08db93f92541 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8790.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 08:10:48.5739 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5wgeqaW+HoqPX9bbgjn4sD/p1U0O20pcQWpQv2wA3YZ7wNVUgK8ArZpZIf5Sn0EmjhzVDDNJg9PJDuruQXTFSw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS4PR04MB9265 X-Spam-Status: No, score=-3027.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jan Beulich via Gcc-patches From: Jan Beulich Reply-To: Jan Beulich Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773194819792266641 X-GMAIL-MSGID: 1773194819792266641 They're all VEX3- (also covering XOP) or EVEX-encoded. Express that in the default calculation of "prefix". FMA4 insns also all have a 1-byte immediate operand. Where the default calculation is not sufficient / applicable, add explicit "prefix" attributes. While there also add a "mode" attribute to fma___pair. gcc/ * config/i386/i386.md (isa): Move up. (length_immediate): Handle "fma4". (prefix): Handle "ssemuladd". * config/i386/sse.md (*fma_fmadd_): Add "prefix" attribute. (fma_fmadd_): Likewise. (_fmadd__mask): Likewise. (_fmadd__mask3): Likewise. (fma_fmsub_): Likewise. (_fmsub__mask): Likewise. (_fmsub__mask3): Likewise. (*fma_fnmadd_): Likewise. (fma_fnmadd_): Likewise. (_fnmadd__mask): Likewise. (_fnmadd__mask3): Likewise. (fma_fnmsub_): Likewise. (_fnmsub__mask): Likewise. (_fnmsub__mask3): Likewise. (fma_fmaddsub_): Likewise. (_fmaddsub__mask): Likewise. (_fmaddsub__mask3): Likewise. (fma_fmsubadd_): Likewise. (_fmsubadd__mask): Likewise. (_fmsubadd__mask3): Likewise. (*fmai_fmadd_): Likewise. (*fmai_fmsub_): Likewise. (*fmai_fnmadd_): Likewise. (*fmai_fnmsub_): Likewise. (avx512f_vmfmadd__mask): Likewise. (avx512f_vmfmadd__mask3): Likewise. (avx512f_vmfmadd__maskz_1): Likewise. (*avx512f_vmfmsub__mask): Likewise. (avx512f_vmfmsub__mask3): Likewise. (*avx512f_vmfmsub__maskz_1): Likewise. (avx512f_vmfnmadd__mask): Likewise. (avx512f_vmfnmadd__mask3): Likewise. (avx512f_vmfnmadd__maskz_1): Likewise. (*avx512f_vmfnmsub__mask): Likewise. (*avx512f_vmfnmsub__mask3): Likewise. (*avx512f_vmfnmsub__maskz_1): Likewise. (*fma4i_vmfmadd_): Likewise. (*fma4i_vmfmsub_): Likewise. (*fma4i_vmfnmadd_): Likewise. (*fma4i_vmfnmsub_): Likewise. (fma__): Likewise. (___mask): Likewise. (avx512fp16_fma_sh_v8hf): Likewise. (avx512fp16_sh_v8hf_mask): Likewise. (xop_p): Likewise. (xop_pdql): Likewise. (xop_pdqh): Likewise. (xop_pwd): Likewise. (xop_pwd): Likewise. (fma___pair): Likewise. Add "mode" attribute. --- a/gcc/config/i386/i386.md +++ b/gcc/config/i386/i386.md @@ -531,12 +531,23 @@ (const_string "unknown")] (const_string "integer"))) +;; Used to control the "enabled" attribute on a per-instruction basis. +(define_attr "isa" "base,x64,nox64,x64_sse2,x64_sse4,x64_sse4_noavx, + x64_avx,x64_avx512bw,x64_avx512dq,aes, + sse_noavx,sse2,sse2_noavx,sse3,sse3_noavx,sse4,sse4_noavx, + avx,noavx,avx2,noavx2,bmi,bmi2,fma4,fma,avx512f,noavx512f, + avx512bw,noavx512bw,avx512dq,noavx512dq,fma_or_avx512vl, + avx512vl,noavx512vl,avxvnni,avx512vnnivl,avx512fp16,avxifma, + avx512ifmavl,avxneconvert,avx512bf16vl,vpclmulqdqvl" + (const_string "base")) + ;; The (bounding maximum) length of an instruction immediate. (define_attr "length_immediate" "" (cond [(eq_attr "type" "incdec,setcc,icmov,str,lea,other,multi,idiv,leave, bitmanip,imulx,msklog,mskmov") (const_int 0) - (eq_attr "type" "sse4arg") + (ior (eq_attr "type" "sse4arg") + (eq_attr "isa" "fma4")) (const_int 1) (eq_attr "unit" "i387,sse,mmx") (const_int 0) @@ -637,6 +648,10 @@ (const_string "vex") (eq_attr "mode" "XI,V16SF,V8DF") (const_string "evex") + (eq_attr "type" "ssemuladd") + (if_then_else (eq_attr "isa" "fma4") + (const_string "vex") + (const_string "maybe_evex")) (eq_attr "type" "sse4arg") (const_string "vex") ] @@ -842,16 +857,6 @@ ;; Define attribute to indicate unaligned ssemov insns (define_attr "movu" "0,1" (const_string "0")) -;; Used to control the "enabled" attribute on a per-instruction basis. -(define_attr "isa" "base,x64,nox64,x64_sse2,x64_sse4,x64_sse4_noavx, - x64_avx,x64_avx512bw,x64_avx512dq,aes, - sse_noavx,sse2,sse2_noavx,sse3,sse3_noavx,sse4,sse4_noavx, - avx,noavx,avx2,noavx2,bmi,bmi2,fma4,fma,avx512f,noavx512f, - avx512bw,noavx512bw,avx512dq,noavx512dq,fma_or_avx512vl, - avx512vl,noavx512vl,avxvnni,avx512vnnivl,avx512fp16,avxifma, - avx512ifmavl,avxneconvert,avx512bf16vl,vpclmulqdqvl" - (const_string "base")) - ;; Define instruction set of MMX instructions (define_attr "mmx_isa" "base,native,sse,sse_noavx,avx" (const_string "base")) --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -5422,6 +5422,7 @@ vfmadd213\t{%3, %2, %0|%0, %2, %3} vfmadd231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "cond_fma" @@ -5461,6 +5462,7 @@ vfmadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmadd__mask3" @@ -5475,6 +5477,7 @@ "TARGET_AVX512F" "vfmadd231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*fma_fmsub_" @@ -5522,6 +5525,7 @@ vfmsub213\t{%3, %2, %0|%0, %2, %3} vfmsub231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "cond_fms" @@ -5563,6 +5567,7 @@ vfmsub132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmsub213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmsub__mask3" @@ -5578,6 +5583,7 @@ "TARGET_AVX512F && " "vfmsub231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*fma_fnmadd_" @@ -5625,6 +5631,7 @@ vfnmadd213\t{%3, %2, %0|%0, %2, %3} vfnmadd231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "cond_fnma" @@ -5666,6 +5673,7 @@ vfnmadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfnmadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fnmadd__mask3" @@ -5681,6 +5689,7 @@ "TARGET_AVX512F && " "vfnmadd231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*fma_fnmsub_" @@ -5730,6 +5739,7 @@ vfnmsub213\t{%3, %2, %0|%0, %2, %3} vfnmsub231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "cond_fnms" @@ -5773,6 +5783,7 @@ vfnmsub132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfnmsub213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fnmsub__mask3" @@ -5789,6 +5800,7 @@ "TARGET_AVX512F" "vfnmsub231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) ;; FMA parallel floating point multiply addsub and subadd operations. @@ -5889,6 +5901,7 @@ vfmaddsub213\t{%3, %2, %0|%0, %2, %3} vfmaddsub231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmaddsub__mask" @@ -5906,6 +5919,7 @@ vfmaddsub132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmaddsub213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmaddsub__mask3" @@ -5921,6 +5935,7 @@ "TARGET_AVX512F" "vfmaddsub231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*fma_fmsubadd_" @@ -5956,6 +5971,7 @@ vfmsubadd213\t{%3, %2, %0|%0, %2, %3} vfmsubadd231\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmsubadd__mask" @@ -5974,6 +5990,7 @@ vfmsubadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmsubadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "_fmsubadd__mask3" @@ -5990,6 +6007,7 @@ "TARGET_AVX512F" "vfmsubadd231\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) ;; FMA3 floating point scalar intrinsics. These merge result with @@ -6057,6 +6075,7 @@ vfmadd132\t{%2, %3, %0|%0, %3, %2} vfmadd213\t{%3, %2, %0|%0, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "maybe_evex") (set_attr "mode" "")]) (define_insn "*fmai_fmsub_" @@ -6074,6 +6093,7 @@ vfmsub132\t{%2, %3, %0|%0, %3, %2} vfmsub213\t{%3, %2, %0|%0, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "maybe_evex") (set_attr "mode" "")]) (define_insn "*fmai_fnmadd_" @@ -6091,6 +6111,7 @@ vfnmadd132\t{%2, %3, %0|%0, %3, %2} vfnmadd213\t{%3, %2, %0|%0, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "maybe_evex") (set_attr "mode" "")]) (define_insn "*fmai_fnmsub_" @@ -6109,6 +6130,7 @@ vfnmsub132\t{%2, %3, %0|%0, %3, %2} vfnmsub213\t{%3, %2, %0|%0, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "maybe_evex") (set_attr "mode" "")]) (define_insn "avx512f_vmfmadd__mask" @@ -6128,6 +6150,7 @@ vfmadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "avx512f_vmfmadd__mask3" @@ -6145,6 +6168,7 @@ "TARGET_AVX512F" "vfmadd231\t{%2, %1, %0%{%4%}|%0%{%4%}, %3, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "avx512f_vmfmadd__maskz" @@ -6178,6 +6202,7 @@ vfmadd132\t{%2, %3, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %3, %2} vfmadd213\t{%3, %2, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*avx512f_vmfmsub__mask" @@ -6198,6 +6223,7 @@ vfmsub132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfmsub213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "avx512f_vmfmsub__mask3" @@ -6216,6 +6242,7 @@ "TARGET_AVX512F" "vfmsub231\t{%2, %1, %0%{%4%}|%0%{%4%}, %3, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*avx512f_vmfmsub__maskz_1" @@ -6236,6 +6263,7 @@ vfmsub132\t{%2, %3, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %3, %2} vfmsub213\t{%3, %2, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "avx512f_vmfnmadd__mask" @@ -6256,6 +6284,7 @@ vfnmadd132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfnmadd213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "avx512f_vmfnmadd__mask3" @@ -6274,6 +6303,7 @@ "TARGET_AVX512F" "vfnmadd231\t{%2, %1, %0%{%4%}|%0%{%4%}, %3, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "avx512f_vmfnmadd__maskz" @@ -6308,6 +6338,7 @@ vfnmadd132\t{%2, %3, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %3, %2} vfnmadd213\t{%3, %2, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__mask" @@ -6329,6 +6360,7 @@ vfnmsub132\t{%2, %3, %0%{%4%}|%0%{%4%}, %3, %2} vfnmsub213\t{%3, %2, %0%{%4%}|%0%{%4%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__mask3" @@ -6348,6 +6380,7 @@ "TARGET_AVX512F" "vfnmsub231\t{%2, %1, %0%{%4%}|%0%{%4%}, %3, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn "*avx512f_vmfnmsub__maskz_1" @@ -6369,6 +6402,7 @@ vfnmsub132\t{%2, %3, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %3, %2} vfnmsub213\t{%3, %2, %0%{%5%}%{z%}|%0%{%5%}%{z%}, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) ;; FMA4 floating point scalar intrinsics. These write the @@ -6398,6 +6432,7 @@ "TARGET_FMA4" "vfmadd\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "")]) (define_insn "*fma4i_vmfmsub_" @@ -6413,6 +6448,7 @@ "TARGET_FMA4" "vfmsub\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "")]) (define_insn "*fma4i_vmfnmadd_" @@ -6428,6 +6464,7 @@ "TARGET_FMA4" "vfnmadd\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "")]) (define_insn "*fma4i_vmfnmsub_" @@ -6444,6 +6481,7 @@ "TARGET_FMA4" "vfnmsub\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "")]) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; @@ -6591,6 +6629,7 @@ "TARGET_AVX512FP16 && && " "v\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_insn_and_split "fma__fadd_fmul" @@ -6654,7 +6693,9 @@ UNSPEC_COMPLEX_F_C_MA_PAIR))] "TARGET_AVX512FP16" "vph\t{%2, %1, %0|%0, %1, %2}" - [(set_attr "type" "ssemuladd")]) + [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") + (set_attr "mode" "")]) (define_insn_and_split "fma__fmaddc_bcst" [(set (match_operand:VF_AVX512FP16VL 0 "register_operand") @@ -6726,6 +6767,7 @@ "TARGET_AVX512FP16 && " "v\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "")]) (define_expand "cmul3" @@ -6913,6 +6955,7 @@ "TARGET_AVX512FP16" "vsh\t{%2, %1, %0|%0, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "V8HF")]) (define_insn "avx512fp16_sh_v8hf_mask" @@ -6932,6 +6975,7 @@ "TARGET_AVX512FP16" "vsh\t{%2, %1, %0%{%4%}|%0%{%4%}, %1, %2}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "evex") (set_attr "mode" "V8HF")]) (define_insn "avx512fp16_sh_v8hf" @@ -24721,6 +24765,7 @@ "TARGET_XOP" "vp\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "TI")]) (define_insn "xop_pdql" @@ -24739,6 +24784,7 @@ "TARGET_XOP" "vpdql\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "TI")]) (define_insn "xop_pdqh" @@ -24757,6 +24803,7 @@ "TARGET_XOP" "vpdqh\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "TI")]) ;; XOP parallel integer multiply/add instructions for the intrinisics @@ -24778,6 +24825,7 @@ "TARGET_XOP" "vpwd\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "TI")]) (define_insn "xop_pwd" @@ -24810,6 +24858,7 @@ "TARGET_XOP" "vpwd\t{%3, %2, %1, %0|%0, %1, %2, %3}" [(set_attr "type" "ssemuladd") + (set_attr "prefix" "vex") (set_attr "mode" "TI")]) ;; XOP parallel XMM conditional moves