From patchwork Thu Sep 7 14:06:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 137646 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp2981652vqo; Thu, 7 Sep 2023 07:08:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHfgq5OR6r55lpI8enzK0rV0mpLR1lDUaHv10pWMxsKsPDkgStH9+bKO6daxo97Tf4yAFhT X-Received: by 2002:a17:906:30c7:b0:9a5:9f3c:961e with SMTP id b7-20020a17090630c700b009a59f3c961emr3671047ejb.18.1694095693798; Thu, 07 Sep 2023 07:08:13 -0700 (PDT) Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id hb8-20020a170906b88800b0099bdfd4cc50si10665416ejb.830.2023.09.07.07.08.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Sep 2023 07:08:13 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Yt8ypfYH; arc=fail (signature failed); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5AB5C385559F for ; Thu, 7 Sep 2023 14:07:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5AB5C385559F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1694095661; bh=7hy32rVejwU35hI5ypXPasMUgKN1EIT3roJ0/t8xwnM=; h=To:CC:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=Yt8ypfYH4EkH8u7QPdL3sq+D8xELNnXu8jMlsquuZQxRQ9oijRouCRczl6KR/ee87 pRC+UVqRKk5HsoCkyClDLxswX3oP9dmYxvvWCW/2sR6fFCa4pAhqKFqiICv7LFs7SX r65u3x4roILcp2tzw7Bd8hBVevLEIwcv5VjJjwS4= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2083.outbound.protection.outlook.com [40.107.22.83]) by sourceware.org (Postfix) with ESMTPS id 947553857017 for ; Thu, 7 Sep 2023 14:06:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 947553857017 Received: from AM6PR10CA0010.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:89::23) by AS8PR08MB5877.eurprd08.prod.outlook.com (2603:10a6:20b:291::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.34; Thu, 7 Sep 2023 14:06:48 +0000 Received: from AM7EUR03FT003.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:89:cafe::df) by AM6PR10CA0010.outlook.office365.com (2603:10a6:209:89::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.30 via Frontend Transport; Thu, 7 Sep 2023 14:06:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT003.mail.protection.outlook.com (100.127.140.227) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.28 via Frontend Transport; Thu, 7 Sep 2023 14:06:48 +0000 Received: ("Tessian outbound 5c548696a0e7:v175"); Thu, 07 Sep 2023 14:06:48 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 24a3415e6fb5e5db X-CR-MTA-TID: 64aa7808 Received: from b4a6fb749828.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0EB5F828-36C1-468A-949C-3DB7EB239B86.1; Thu, 07 Sep 2023 14:06:37 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b4a6fb749828.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 07 Sep 2023 14:06:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iPnKFjm8ObpdXGgXgQEInPo353Cv9XBvmUO6kIAcgu7ozaSDjgQXqGf+gEScX3oPiaflbCG6Et5TMMX9OnfkubTDqHOJywrp+MM3tavYn/s/iCsXZfr9Q3GrPqUVy9vLP9fCHJFfipQDlKpy0L49k8sUQ+2DcCuVwoVCpO3rgDs1TZjqKn1p1GgJGtbXHUDEvRTFS1YBbZ0nCJw3w63BOECisVj5b0c7Ove67QfUMwGcrzh5d8Sisd2Yj3pMCFXwDGomDkbbO5X1ElTjkCoQXDXcCrDy/XESGFaz5j8nGgdDRBh0/oHUQcdZVgubO4J6FICCFFV5Z65Y05UpVzuQpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7hy32rVejwU35hI5ypXPasMUgKN1EIT3roJ0/t8xwnM=; b=nLcrABTnUsDKdS7m6MUCuE+oSuxIOx2iNZ9bg7OcWDSPl3OLXg4xYoMtU6l0t3qZqDB7SksLhAjrvUTJIZPAfVjK+ooqJHCch6lwL+DVUd08i6HcJPBvP15IwwF28C1OAuSvv2cE35xt/+tZSzOhUCa2kBMRIJzPsG2pyIxRAwTiFnQtuvPaEOz+SVMRu6EkQPmCHBrf/IlVQobPkUnpgAa7weSsEE8ZLFavOU/xGM3GJvSL7tc0cEdXM1+p4I4xZRTp+OVvlbd185vkmy8RmbknlrvQk2XE+8Yd96BwzIDKaq2CzEaiPvlT+QGl2QC+kOFXlQeQyZum90FZoYFhww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Received: from PAWPR08MB8982.eurprd08.prod.outlook.com (2603:10a6:102:33f::20) by DU0PR08MB7567.eurprd08.prod.outlook.com (2603:10a6:10:317::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.34; Thu, 7 Sep 2023 14:06:34 +0000 Received: from PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::ff3d:6e95:9971:a7e]) by PAWPR08MB8982.eurprd08.prod.outlook.com ([fe80::ff3d:6e95:9971:a7e%5]) with mapi id 15.20.6745.034; Thu, 7 Sep 2023 14:06:34 +0000 To: Richard Earnshaw , Kyrylo Tkachov CC: GCC Patches Subject: [PATCH] ARM: Block predication on atomics [PR111235] Thread-Topic: [PATCH] ARM: Block predication on atomics [PR111235] Thread-Index: AQHZ4ZOgjHPVvmJO2Ey0nSX1JQY5yw== Date: Thu, 7 Sep 2023 14:06:33 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: PAWPR08MB8982:EE_|DU0PR08MB7567:EE_|AM7EUR03FT003:EE_|AS8PR08MB5877:EE_ X-MS-Office365-Filtering-Correlation-Id: 17bee9c2-0d89-4146-3006-08dbafabad92 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3QYdwLQH2c7djE6x6X1goOX2WUAQKWTTrIS5LlxOP/K8NymrpHi3k94fTBQvh++qCp9N8IuIN/AYLwen6JIy3JklpOxFGN+imrehAD5Bci5f3On2HQxpmD0F5svbyt00P5XfI17A0rydCseCTkWHpbJVJ2R8W2bwX0phzdJ/00hYc9yx/jY/YcrYVeEcb1ENevNjOlGyGpO1kFW+OIf3My8PfU3VQbvDeJcLgqi78cB0r7Ut0qa7ZPTwZuB4RASb//jPE2B+CfZPxUEe8Bz4MXRNSc0P7MDBX/p6nIohdyhyYW31mGtQVUbxCcHAKP/Cfpua3VIC8GcpafgoOqQgSYRtghb/QRg60taaXUs8gjQZTW7i5F8ekNiKTsEjMQp/1pK0WSyH2fUfL8BzpgMKgHxIoZVtNNmYRtIYO2y1/98FE9abojhaO9Mt/G50wqJMI9M/woOLCKz6XwPTalMEZnPHu11NjR8qF/WjuWX1q50DFVIxTvJrTjLlSAclGetajLuTNfDJ8J2b4hIYc4rI5iG/wdZsx6cZNc0se0gf+NUxmGeRdoL8XRY+TidUzvfFONbBxkdajPkr2ArK0ZNGuWEPdj2BIl2ot9I8a/iDz4Y= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PAWPR08MB8982.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(346002)(136003)(396003)(366004)(376002)(39860400002)(186009)(1800799009)(451199024)(38070700005)(38100700002)(122000001)(26005)(6506007)(7696005)(9686003)(83380400001)(55016003)(86362001)(84970400001)(5660300002)(30864003)(41300700001)(66556008)(52536014)(66446008)(316002)(66946007)(66476007)(110136005)(91956017)(33656002)(8676002)(4326008)(8936002)(6636002)(64756008)(76116006)(2906002)(71200400001)(478600001); DIR:OUT; SFP:1101; MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7567 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT003.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: a82937b7-e6ad-4f00-8451-08dbafaba4bb X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eTtHm+ADIhwR1xt4m7ZZWK4EPPXe9MfLV7V2jJuqpcoxPFOpEwjub+hEm1ziuLwJDC6AlmgkVu2V3yfw9FX/kZ+NFeqITXwQqGbOl+tW5K8baSEcuLOtx4vMnauaNn+sIGIimVmbw77d4VhE/Qah9eJzp23PfdOj9VddIP8xP39UftOCwDeYzD0X9wU0V/mvGmWN3oyICs4OMVaXpIgKHwDciGXkDfGsJRHMQAvG5KNf47vWKTfojgxpV8RIYEjfGOoBMBVELm2w684kCLXr36kF485cJgna4J0HMRe6nlKlmDTXqHvoFFL/l+OAeVBI0apPOalXT9yhAjW1GemIkq0iQy9NUbMcaty16vVmHzx5alR3PsZQBYaxGaIcPNFvJnsOkEJjWpIahioi+e2zvfCgRzVV+oknD7JUEUR3bSmRaI1RXw9SEcSUd4OEH3aCEligyU83W9l9D40Yf48xdmT3y4tvUfRjrApXag8du7OI/dPKKMqzzf1SdWyAlNzkAGxi5LcQuRLdjnxhTCte8knTtwJpvenei314Er5rnN31ddFmQUvyUK1/YPWXEsJPADR79C8K+exBefF/DO+/UHtlKh5yWuWEVKAoSoPD5CO0+IMkXXr0olJI7/RJ/Eba7tQfbnmJooQ0OUl7s4kH5QjS1i8dZFGhijVEl4kAOhgvU3ZzeEEv8/zYgGTaOSYF1cwRECe/NRmVgteWswjleOMQp8EeBO5VC4M+sC0lITg= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(396003)(136003)(39860400002)(376002)(186009)(451199024)(1800799009)(82310400011)(40470700004)(36840700001)(46966006)(356005)(33656002)(86362001)(84970400001)(55016003)(40480700001)(26005)(478600001)(70206006)(52536014)(316002)(110136005)(5660300002)(6506007)(7696005)(81166007)(6636002)(70586007)(41300700001)(9686003)(8936002)(4326008)(8676002)(2906002)(47076005)(40460700003)(82740400003)(36860700001)(83380400001)(30864003)(336012); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2023 14:06:48.8583 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 17bee9c2-0d89-4146-3006-08dbafabad92 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT003.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB5877 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Wilco Dijkstra via Gcc-patches From: Wilco Dijkstra Reply-To: Wilco Dijkstra Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776388085721074883 X-GMAIL-MSGID: 1776388085721074883 The v7 memory ordering model allows reordering of conditional atomic instructions. To avoid this, make all atomic patterns unconditional. Expand atomic loads and stores for all architectures so the memory access can be wrapped into an UNSPEC. Passes regress/bootstrap, OK for commit? gcc/ChangeLog/ PR target/111235 * config/arm/constraints.md: Remove Pf constraint. * onfig/arm/sync.md (arm_atomic_load): Add new pattern. (arm_atomic_load_acquire): Likewise. (arm_atomic_store): Likewise. (arm_atomic_store_release): Likewise. (atomic_load): Always expand atomic loads explicitly. (atomic_store): Always expand atomic stores explicitly. (arm_atomic_loaddi2_ldrd): Remove predication. (arm_load_exclusive): Likewise. (arm_load_acquire_exclusive): Likewise. (arm_load_exclusivesi): Likewise. (arm_load_acquire_exclusivesi: Likewise. (arm_load_exclusivedi): Likewise. (arm_load_acquire_exclusivedi): Likewise. (arm_store_exclusive): Likewise. (arm_store_release_exclusivedi): Likewise. (arm_store_release_exclusive): Likewise. * gcc/config/arm/unspecs.md: Add VUNSPEC_LDR and VUNSPEC_STR. gcc/testsuite/ChangeLog/ PR target/111235 * gcc.target/arm/pr111235.c: Add new test. Reviewed-by: Ramana Radhakrishnan diff --git a/gcc/config/arm/constraints.md b/gcc/config/arm/constraints.md index 05a4ebbdd67601d7b92aa44a619d17634cc69f17..d7c4a1b0cd785f276862048005e6cfa57cdcb20d 100644 --- a/gcc/config/arm/constraints.md +++ b/gcc/config/arm/constraints.md @@ -36,7 +36,7 @@ ;; in Thumb-1 state: Pa, Pb, Pc, Pd, Pe ;; in Thumb-2 state: Ha, Pj, PJ, Ps, Pt, Pu, Pv, Pw, Px, Py, Pz, Rd, Rf, Rb, Ra, ;; Rg, Ri -;; in all states: Pf, Pg +;; in all states: Pg ;; The following memory constraints have been used: ;; in ARM/Thumb-2 state: Uh, Ut, Uv, Uy, Un, Um, Us, Up, Uf, Ux, Ul @@ -239,13 +239,6 @@ (define_constraint "Pe" (and (match_code "const_int") (match_test "TARGET_THUMB1 && ival >= 256 && ival <= 510"))) -(define_constraint "Pf" - "Memory models except relaxed, consume or release ones." - (and (match_code "const_int") - (match_test "!is_mm_relaxed (memmodel_from_int (ival)) - && !is_mm_consume (memmodel_from_int (ival)) - && !is_mm_release (memmodel_from_int (ival))"))) - (define_constraint "Pg" "@internal In Thumb-2 state a constant in range 1 to 32" (and (match_code "const_int") diff --git a/gcc/config/arm/sync.md b/gcc/config/arm/sync.md index 7626bf3c443285dc63b4c4367b11a879a99c93c6..2210810f67f37ce043b8fdc73b4f21b54c5b1912 100644 --- a/gcc/config/arm/sync.md +++ b/gcc/config/arm/sync.md @@ -62,68 +62,110 @@ (define_insn "*memory_barrier" (set_attr "conds" "unconditional") (set_attr "predicable" "no")]) -(define_insn "atomic_load" - [(set (match_operand:QHSI 0 "register_operand" "=r,r,l") +(define_insn "arm_atomic_load" + [(set (match_operand:QHSI 0 "register_operand" "=r,l") (unspec_volatile:QHSI - [(match_operand:QHSI 1 "arm_sync_memory_operand" "Q,Q,Q") - (match_operand:SI 2 "const_int_operand" "n,Pf,n")] ;; model + [(match_operand:QHSI 1 "memory_operand" "m,m")] + VUNSPEC_LDR))] + "" + "ldr\t%0, %1" + [(set_attr "arch" "32,any")]) + +(define_insn "arm_atomic_load_acquire" + [(set (match_operand:QHSI 0 "register_operand" "=r") + (unspec_volatile:QHSI + [(match_operand:QHSI 1 "arm_sync_memory_operand" "Q")] VUNSPEC_LDA))] "TARGET_HAVE_LDACQ" - { - if (aarch_mm_needs_acquire (operands[2])) - { - if (TARGET_THUMB1) - return "lda\t%0, %1"; - else - return "lda%?\t%0, %1"; - } - else - { - if (TARGET_THUMB1) - return "ldr\t%0, %1"; - else - return "ldr%?\t%0, %1"; - } - } - [(set_attr "arch" "32,v8mb,any") - (set_attr "predicable" "yes")]) + "lda\t%0, %C1" +) -(define_insn "atomic_store" - [(set (match_operand:QHSI 0 "memory_operand" "=Q,Q,Q") +(define_insn "arm_atomic_store" + [(set (match_operand:QHSI 0 "memory_operand" "=m,m") + (unspec_volatile:QHSI + [(match_operand:QHSI 1 "register_operand" "r,l")] + VUNSPEC_STR))] + "" + "str\t%1, %0"; + [(set_attr "arch" "32,any")]) + +(define_insn "arm_atomic_store_release" + [(set (match_operand:QHSI 0 "arm_sync_memory_operand" "=Q") (unspec_volatile:QHSI - [(match_operand:QHSI 1 "general_operand" "r,r,l") - (match_operand:SI 2 "const_int_operand" "n,Pf,n")] ;; model + [(match_operand:QHSI 1 "register_operand" "r")] VUNSPEC_STL))] "TARGET_HAVE_LDACQ" - { - if (aarch_mm_needs_release (operands[2])) - { - if (TARGET_THUMB1) - return "stl\t%1, %0"; - else - return "stl%?\t%1, %0"; - } - else - { - if (TARGET_THUMB1) - return "str\t%1, %0"; - else - return "str%?\t%1, %0"; - } - } - [(set_attr "arch" "32,v8mb,any") - (set_attr "predicable" "yes")]) + "stl\t%1, %C0") + + +(define_expand "atomic_load" + [(match_operand:QHSI 0 "register_operand") ;; val out + (match_operand:QHSI 1 "arm_sync_memory_operand") ;; memory + (match_operand:SI 2 "const_int_operand")] ;; model + "" +{ + memmodel model = memmodel_from_int (INTVAL (operands[2])); + + if (TARGET_HAVE_LDACQ && !is_mm_relaxed (model)) + { + emit_insn (gen_arm_atomic_load_acquire (operands[0], operands[1])); + DONE; + } + + /* The seq_cst model needs a barrier before the load to block reordering with + earlier accesses. */ + if (is_mm_seq_cst (model)) + expand_mem_thread_fence (model); + + emit_insn (gen_arm_atomic_load (operands[0], operands[1])); + + /* All non-relaxed models need a barrier after the load when load-acquire + instructions are not available. */ + if (!is_mm_relaxed (model)) + expand_mem_thread_fence (model); + + DONE; +}) + +(define_expand "atomic_store" + [(match_operand:QHSI 0 "arm_sync_memory_operand") ;; memory + (match_operand:QHSI 1 "register_operand") ;; store value + (match_operand:SI 2 "const_int_operand")] ;; model + "" +{ + memmodel model = memmodel_from_int (INTVAL (operands[2])); + + if (TARGET_HAVE_LDACQ && !is_mm_relaxed (model)) + { + emit_insn (gen_arm_atomic_store_release (operands[0], operands[1])); + DONE; + } + + /* All non-relaxed models need a barrier after the load when load-acquire + instructions are not available. */ + if (!is_mm_relaxed (model)) + expand_mem_thread_fence (model); + + emit_insn (gen_arm_atomic_store (operands[0], operands[1])); + + /* The seq_cst model needs a barrier after the store to block reordering with + later accesses. */ + if (is_mm_seq_cst (model)) + expand_mem_thread_fence (model); + + DONE; +}) ;; An LDRD instruction usable by the atomic_loaddi expander on LPAE targets (define_insn "arm_atomic_loaddi2_ldrd" [(set (match_operand:DI 0 "register_operand" "=r") (unspec_volatile:DI - [(match_operand:DI 1 "arm_sync_memory_operand" "Q")] + [(match_operand:DI 1 "memory_operand" "m")] VUNSPEC_LDRD_ATOMIC))] "ARM_DOUBLEWORD_ALIGN && TARGET_HAVE_LPAE" - "ldrd%?\t%0, %H0, %C1" - [(set_attr "predicable" "yes")]) + "ldrd\t%0, %H0, %1" +) ;; There are three ways to expand this depending on the architecture ;; features available. As for the barriers, a load needs a barrier @@ -152,6 +194,11 @@ (define_expand "atomic_loaddi" DONE; } + /* The seq_cst model needs a barrier before the load to block reordering with + earlier accesses. */ + if (is_mm_seq_cst (model)) + expand_mem_thread_fence (model); + /* On LPAE targets LDRD and STRD accesses to 64-bit aligned locations are 64-bit single-copy atomic. We still need barriers in the appropriate places to implement the ordering constraints. */ @@ -160,7 +207,6 @@ (define_expand "atomic_loaddi" else emit_insn (gen_arm_load_exclusivedi (operands[0], operands[1])); - /* All non-relaxed models need a barrier after the load when load-acquire instructions are not available. */ if (!is_mm_relaxed (model)) @@ -446,54 +492,42 @@ (define_insn_and_split "atomic_nand_fetch" [(set_attr "arch" "32,v8mb")]) (define_insn "arm_load_exclusive" - [(set (match_operand:SI 0 "s_register_operand" "=r,r") + [(set (match_operand:SI 0 "s_register_operand" "=r") (zero_extend:SI (unspec_volatile:NARROW - [(match_operand:NARROW 1 "mem_noofs_operand" "Ua,Ua")] + [(match_operand:NARROW 1 "mem_noofs_operand" "Ua")] VUNSPEC_LL)))] "TARGET_HAVE_LDREXBH" - "@ - ldrex%?\t%0, %C1 - ldrex\t%0, %C1" - [(set_attr "arch" "32,v8mb") - (set_attr "predicable" "yes")]) + "ldrex\t%0, %C1" +) (define_insn "arm_load_acquire_exclusive" - [(set (match_operand:SI 0 "s_register_operand" "=r,r") + [(set (match_operand:SI 0 "s_register_operand" "=r") (zero_extend:SI (unspec_volatile:NARROW - [(match_operand:NARROW 1 "mem_noofs_operand" "Ua,Ua")] + [(match_operand:NARROW 1 "mem_noofs_operand" "Ua")] VUNSPEC_LAX)))] "TARGET_HAVE_LDACQ" - "@ - ldaex%?\\t%0, %C1 - ldaex\\t%0, %C1" - [(set_attr "arch" "32,v8mb") - (set_attr "predicable" "yes")]) + "ldaex\\t%0, %C1" +) (define_insn "arm_load_exclusivesi" - [(set (match_operand:SI 0 "s_register_operand" "=r,r") + [(set (match_operand:SI 0 "s_register_operand" "=r") (unspec_volatile:SI - [(match_operand:SI 1 "mem_noofs_operand" "Ua,Ua")] + [(match_operand:SI 1 "mem_noofs_operand" "Ua")] VUNSPEC_LL))] "TARGET_HAVE_LDREX" - "@ - ldrex%?\t%0, %C1 - ldrex\t%0, %C1" - [(set_attr "arch" "32,v8mb") - (set_attr "predicable" "yes")]) + "ldrex\t%0, %C1" +) (define_insn "arm_load_acquire_exclusivesi" - [(set (match_operand:SI 0 "s_register_operand" "=r,r") + [(set (match_operand:SI 0 "s_register_operand" "=r") (unspec_volatile:SI - [(match_operand:SI 1 "mem_noofs_operand" "Ua,Ua")] + [(match_operand:SI 1 "mem_noofs_operand" "Ua")] VUNSPEC_LAX))] "TARGET_HAVE_LDACQ" - "@ - ldaex%?\t%0, %C1 - ldaex\t%0, %C1" - [(set_attr "arch" "32,v8mb") - (set_attr "predicable" "yes")]) + "ldaex\t%0, %C1" +) (define_insn "arm_load_exclusivedi" [(set (match_operand:DI 0 "s_register_operand" "=r") @@ -501,8 +535,8 @@ (define_insn "arm_load_exclusivedi" [(match_operand:DI 1 "mem_noofs_operand" "Ua")] VUNSPEC_LL))] "TARGET_HAVE_LDREXD" - "ldrexd%?\t%0, %H0, %C1" - [(set_attr "predicable" "yes")]) + "ldrexd\t%0, %H0, %C1" +) (define_insn "arm_load_acquire_exclusivedi" [(set (match_operand:DI 0 "s_register_operand" "=r") @@ -510,8 +544,8 @@ (define_insn "arm_load_acquire_exclusivedi" [(match_operand:DI 1 "mem_noofs_operand" "Ua")] VUNSPEC_LAX))] "TARGET_HAVE_LDACQEXD && ARM_DOUBLEWORD_ALIGN" - "ldaexd%?\t%0, %H0, %C1" - [(set_attr "predicable" "yes")]) + "ldaexd\t%0, %H0, %C1" +) (define_insn "arm_store_exclusive" [(set (match_operand:SI 0 "s_register_operand" "=&r") @@ -530,14 +564,11 @@ (define_insn "arm_store_exclusive" Note that the 1st register always gets the lowest word in memory. */ gcc_assert ((REGNO (operands[2]) & 1) == 0 || TARGET_THUMB2); - return "strexd%?\t%0, %2, %H2, %C1"; + return "strexd\t%0, %2, %H2, %C1"; } - if (TARGET_THUMB1) - return "strex\t%0, %2, %C1"; - else - return "strex%?\t%0, %2, %C1"; + return "strex\t%0, %2, %C1"; } - [(set_attr "predicable" "yes")]) +) (define_insn "arm_store_release_exclusivedi" [(set (match_operand:SI 0 "s_register_operand" "=&r") @@ -550,20 +581,16 @@ (define_insn "arm_store_release_exclusivedi" { /* See comment in arm_store_exclusive above. */ gcc_assert ((REGNO (operands[2]) & 1) == 0 || TARGET_THUMB2); - return "stlexd%?\t%0, %2, %H2, %C1"; + return "stlexd\t%0, %2, %H2, %C1"; } - [(set_attr "predicable" "yes")]) +) (define_insn "arm_store_release_exclusive" - [(set (match_operand:SI 0 "s_register_operand" "=&r,&r") + [(set (match_operand:SI 0 "s_register_operand" "=&r") (unspec_volatile:SI [(const_int 0)] VUNSPEC_SLX)) - (set (match_operand:QHSI 1 "mem_noofs_operand" "=Ua,Ua") + (set (match_operand:QHSI 1 "mem_noofs_operand" "=Ua") (unspec_volatile:QHSI - [(match_operand:QHSI 2 "s_register_operand" "r,r")] + [(match_operand:QHSI 2 "s_register_operand" "r")] VUNSPEC_SLX))] "TARGET_HAVE_LDACQ" - "@ - stlex%?\t%0, %2, %C1 - stlex\t%0, %2, %C1" - [(set_attr "arch" "32,v8mb") - (set_attr "predicable" "yes")]) + "stlex\t%0, %2, %C1") diff --git a/gcc/config/arm/unspecs.md b/gcc/config/arm/unspecs.md index dccda283573208bd5db4f04c10ae9dbbfdda49dd..ba86ce0992f2b14a5e65ac09784b3fb3539de035 100644 --- a/gcc/config/arm/unspecs.md +++ b/gcc/config/arm/unspecs.md @@ -221,7 +221,9 @@ (define_c_enum "unspecv" [ VUNSPEC_SC ; Represent a store-register-exclusive. VUNSPEC_LAX ; Represent a load-register-acquire-exclusive. VUNSPEC_SLX ; Represent a store-register-release-exclusive. - VUNSPEC_LDA ; Represent a store-register-acquire. + VUNSPEC_LDR ; Represent a load-register-relaxed. + VUNSPEC_LDA ; Represent a load-register-acquire. + VUNSPEC_STR ; Represent a store-register-relaxed. VUNSPEC_STL ; Represent a store-register-release. VUNSPEC_GET_FPSCR ; Represent fetch of FPSCR content. VUNSPEC_SET_FPSCR ; Represent assign of FPSCR content. diff --git a/gcc/testsuite/gcc.target/arm/pr111235.c b/gcc/testsuite/gcc.target/arm/pr111235.c new file mode 100644 index 0000000000000000000000000000000000000000..923b231afa888d326bcdc0ecabbcf8ba223d365a --- /dev/null +++ b/gcc/testsuite/gcc.target/arm/pr111235.c @@ -0,0 +1,35 @@ +/* { dg-do compile } */ +/* { dg-options "-std=c11 -O" } */ +/* { dg-require-effective-target arm_arch_v7a_ok } */ +/* { dg-add-options arm_arch_v7a } */ + +int t0 (int *p, int x) +{ + if (x > 100) + x = atomic_load_explicit (p, memory_order_relaxed); + return x + 1; +} + +long long t1 (long long *p, int x) +{ + if (x > 100) + x = atomic_load_explicit (p, memory_order_relaxed); + return x + 1; +} + +void t2 (int *p, int x) +{ + if (x > 100) + atomic_store_explicit (p, x, memory_order_relaxed); +} + +void t3 (long long *p, long long x) +{ + if (x > 100) + atomic_store_explicit (p, x, memory_order_relaxed); +} + + +/* { dg-final { scan-assembler-times "ldrexd\tr\[0-9\]+, r\[0-9\]+, \\\[r\[0-9\]+\\\]" 1 } } */ +/* { dg-final { scan-assembler-times "dmb\tish" 1 } } */ +