From patchwork Thu Aug 3 10:43:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Jelinek X-Patchwork-Id: 130459 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp1055338vqx; Thu, 3 Aug 2023 03:44:20 -0700 (PDT) X-Google-Smtp-Source: APBJJlHEproeeGBAINofHzPN6TdvEPBUTSayhmVyDkOHuLuI1CEZxlysjtlvxLvFNM77KA8bmSv9 X-Received: by 2002:aa7:c595:0:b0:522:29ec:3e0a with SMTP id g21-20020aa7c595000000b0052229ec3e0amr7330091edq.28.1691059460384; Thu, 03 Aug 2023 03:44:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691059460; cv=none; d=google.com; s=arc-20160816; b=ZI8aofDfTCUsW15/rs0QN2EdObCfoftmyF7UsjWHCAnLiiPBdoYSgCYKca5H4UbJhg +g3hYvSYrGoKiy+j7K/57lWHHDxd18wZnPEdhoo9PXbX+5ErTmvQIEjT8/8Hx5T3amHs 8LgjWuN6hKmu+A/+tCGuGYpLsgRoQ9t+GvGHd7WJjTt27XiqHJiHkcDVwG3+haJDCfvp BBaQYAyd7bmzmD2oysEY7/vWO4UdKwLzSvTuDh/oqNcE82zAM1jx8zM9132d/D0axJTS +xb1ct9jHFLM4UzBE8P8zEhpFW/cFFBEltbcOJg39RDEb0lB37Q+mI4gR/hhB1ceqBDH nxeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-disposition:mime-version:message-id:subject:cc:to:date :dmarc-filter:delivered-to:dkim-signature:dkim-filter; bh=SWLrmxTvPGsmO1nq3Su3NWRhFsVzWqbcqhx62WibYbY=; fh=aeZHQn/lA9+KGzRlDhL1qO9SE/pAMSdkbwH21Dg0FsA=; b=jeQGhEVEFfSU/Vhf32sKqelx8qORWL5HbnWbwqaaXSw6jLMctCGdDgx0DioMck6tcc 1L28SV/hW7KeFPyj3JHhroW20q575G0W8R+OKwiJhN1Wuj+O5HCv/RGoYtM2McBkTw9O cYSAx/axdrm/3LrC2CC54Fm7nf310aVWlqLIDjOBremcSIILlops99HbXlxHjXDnkOPx 0eApoL0Hg8R9qy53W70NJ4q2CAojxd4vSm4xWkiT+7TXjETuLgN9mAIbrTHwtiZV2j2p ZAoPoKkcAtSV3w/ZPQP6rk7xi+XXKzSEhdgKmarWRTy70Eyx0lNpzFcit/7CqAdT0CRh v5tA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=P4D01Qa5; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id u15-20020aa7d88f000000b005221dcc6470si3307157edq.591.2023.08.03.03.44.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Aug 2023 03:44:20 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=P4D01Qa5; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EB86B385842D for ; Thu, 3 Aug 2023 10:44:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EB86B385842D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1691059459; bh=SWLrmxTvPGsmO1nq3Su3NWRhFsVzWqbcqhx62WibYbY=; h=Date:To:Cc:Subject:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=P4D01Qa5KMC/u7h/DrFWa2kD7eRjyfulPQPiTztQ6LtdcwgxSfWPlGQzjpAuZa2uO /4+bw5s+3pj6Q9J9T3TnqFffPbYfodNeTK1Z09ZZkhDLxeaNpmPODKpxg4MAEjnHY8 mOPyfQGxaBdiqbMPKhL34h57PT9Xjz3GTDwJ72Lo= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 70DAB3858D1E for ; Thu, 3 Aug 2023 10:43:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 70DAB3858D1E Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-494-BiYLw9FkMvOhIBVFTumFJQ-1; Thu, 03 Aug 2023 06:43:32 -0400 X-MC-Unique: BiYLw9FkMvOhIBVFTumFJQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 610D9801E80; Thu, 3 Aug 2023 10:43:32 +0000 (UTC) Received: from tucnak.zalov.cz (unknown [10.45.224.18]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0ABD11454143; Thu, 3 Aug 2023 10:43:31 +0000 (UTC) Received: from tucnak.zalov.cz (localhost [127.0.0.1]) by tucnak.zalov.cz (8.17.1/8.17.1) with ESMTPS id 373AhTYd1169910 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Thu, 3 Aug 2023 12:43:29 +0200 Received: (from jakub@localhost) by tucnak.zalov.cz (8.17.1/8.17.1/Submit) id 373AhSkD1169909; Thu, 3 Aug 2023 12:43:28 +0200 Date: Thu, 3 Aug 2023 12:43:28 +0200 To: "Joseph S. Myers" Cc: gcc-patches@gcc.gnu.org Subject: [PATCH] c-family: Add _BitInt support for __atomic_*fetch* [PR102989] Message-ID: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jakub Jelinek via Gcc-patches From: Jakub Jelinek Reply-To: Jakub Jelinek Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773204364730651975 X-GMAIL-MSGID: 1773204364730651975 Hi! The following patch implements the lowering of __atomic_*fetch* functions where first argument is a pointer to (optionally _Atomic) _BitInt which either doesn't have size 1, 2, 4, 8 or 16 bytes or has 16 byte size but target doesn't support TImode. Patch on top of the _BitInt patch series. Tested on x86_64-linux. 2023-08-03 Jakub Jelinek PR c/102989 gcc/c-family/ * c-common.cc (sync_resolve_size): Add ORIG_FORMAT argument. If FETCH && !ORIG_FORMAT, type is BITINT_TYPE, return -1 if size isn't one of 1, 2, 4, 8 or 16 or if it is 16 but TImode is not supported. (atomic_bitint_fetch_using_cas_loop): New function. (resolve_overloaded_builtin): Adjust sync_resolve_size caller. If -1 is returned, use atomic_bitint_fetch_using_cas_loop to lower it. Formatting fix. gcc/testsuite/ * gcc.dg/bitint-18.c: New test. Jakub --- gcc/c-family/c-common.cc.jj 2023-07-11 15:28:55.119673958 +0200 +++ gcc/c-family/c-common.cc 2023-08-03 12:10:50.852085519 +0200 @@ -7190,12 +7190,16 @@ speculation_safe_value_resolve_return (t /* A helper function for resolve_overloaded_builtin in resolving the overloaded __sync_ builtins. Returns a positive power of 2 if the first operand of PARAMS is a pointer to a supported data type. - Returns 0 if an error is encountered. + Returns 0 if an error is encountered. Return -1 for _BitInt + __atomic*fetch* with unsupported type which should be handled by + a cas loop. FETCH is true when FUNCTION is one of the _FETCH_OP_ or _OP_FETCH_ + built-ins. ORIG_FORMAT is for __sync_* rather than __atomic_* built-ins. */ static int -sync_resolve_size (tree function, vec *params, bool fetch) +sync_resolve_size (tree function, vec *params, bool fetch, + bool orig_format) { /* Type of the argument. */ tree argtype; @@ -7230,9 +7234,19 @@ sync_resolve_size (tree function, vec *orig_params) +{ + enum tree_code code = ERROR_MARK; + bool return_old_p = false; + switch (orig_code) + { + case BUILT_IN_ATOMIC_ADD_FETCH_N: + code = PLUS_EXPR; + break; + case BUILT_IN_ATOMIC_SUB_FETCH_N: + code = MINUS_EXPR; + break; + case BUILT_IN_ATOMIC_AND_FETCH_N: + code = BIT_AND_EXPR; + break; + case BUILT_IN_ATOMIC_NAND_FETCH_N: + break; + case BUILT_IN_ATOMIC_XOR_FETCH_N: + code = BIT_XOR_EXPR; + break; + case BUILT_IN_ATOMIC_OR_FETCH_N: + code = BIT_IOR_EXPR; + break; + case BUILT_IN_ATOMIC_FETCH_ADD_N: + code = PLUS_EXPR; + return_old_p = true; + break; + case BUILT_IN_ATOMIC_FETCH_SUB_N: + code = MINUS_EXPR; + return_old_p = true; + break; + case BUILT_IN_ATOMIC_FETCH_AND_N: + code = BIT_AND_EXPR; + return_old_p = true; + break; + case BUILT_IN_ATOMIC_FETCH_NAND_N: + return_old_p = true; + break; + case BUILT_IN_ATOMIC_FETCH_XOR_N: + code = BIT_XOR_EXPR; + return_old_p = true; + break; + case BUILT_IN_ATOMIC_FETCH_OR_N: + code = BIT_IOR_EXPR; + return_old_p = true; + break; + default: + gcc_unreachable (); + } + + if (orig_params->length () != 3) + { + if (orig_params->length () < 3) + error_at (loc, "too few arguments to function %qE", orig_function); + else + error_at (loc, "too many arguments to function %qE", orig_function); + return error_mark_node; + } + + tree stmts = push_stmt_list (); + + tree nonatomic_lhs_type = TREE_TYPE (TREE_TYPE ((*orig_params)[0])); + nonatomic_lhs_type = TYPE_MAIN_VARIANT (nonatomic_lhs_type); + gcc_assert (TREE_CODE (nonatomic_lhs_type) == BITINT_TYPE); + + tree lhs_addr = (*orig_params)[0]; + tree val = convert (nonatomic_lhs_type, (*orig_params)[1]); + tree model = convert (integer_type_node, (*orig_params)[2]); + if (TREE_SIDE_EFFECTS (lhs_addr)) + { + tree var = create_tmp_var_raw (TREE_TYPE (lhs_addr)); + lhs_addr = build4 (TARGET_EXPR, TREE_TYPE (lhs_addr), var, lhs_addr, + NULL_TREE, NULL_TREE); + add_stmt (lhs_addr); + } + if (TREE_SIDE_EFFECTS (val)) + { + tree var = create_tmp_var_raw (nonatomic_lhs_type); + val = build4 (TARGET_EXPR, nonatomic_lhs_type, var, val, NULL_TREE, + NULL_TREE); + add_stmt (val); + } + if (TREE_SIDE_EFFECTS (model)) + { + tree var = create_tmp_var_raw (integer_type_node); + model = build4 (TARGET_EXPR, integer_type_node, var, model, NULL_TREE, + NULL_TREE); + add_stmt (model); + } + + tree old = create_tmp_var_raw (nonatomic_lhs_type); + tree old_addr = build_unary_op (loc, ADDR_EXPR, old, false); + TREE_ADDRESSABLE (old) = 1; + suppress_warning (old); + + tree newval = create_tmp_var_raw (nonatomic_lhs_type); + tree newval_addr = build_unary_op (loc, ADDR_EXPR, newval, false); + TREE_ADDRESSABLE (newval) = 1; + suppress_warning (newval); + + tree loop_decl = create_artificial_label (loc); + tree loop_label = build1 (LABEL_EXPR, void_type_node, loop_decl); + + tree done_decl = create_artificial_label (loc); + tree done_label = build1 (LABEL_EXPR, void_type_node, done_decl); + + vec *params; + vec_alloc (params, 6); + + /* __atomic_load (addr, &old, SEQ_CST). */ + tree fndecl = builtin_decl_explicit (BUILT_IN_ATOMIC_LOAD); + params->quick_push (lhs_addr); + params->quick_push (old_addr); + params->quick_push (build_int_cst (integer_type_node, MEMMODEL_RELAXED)); + tree func_call = resolve_overloaded_builtin (loc, fndecl, params); + if (func_call == NULL_TREE) + func_call = build_function_call_vec (loc, vNULL, fndecl, params, NULL); + old = build4 (TARGET_EXPR, nonatomic_lhs_type, old, func_call, NULL_TREE, + NULL_TREE); + add_stmt (old); + params->truncate (0); + + /* loop: */ + add_stmt (loop_label); + + /* newval = old + val; */ + tree rhs; + switch (code) + { + case PLUS_EXPR: + case MINUS_EXPR: + if (!TYPE_OVERFLOW_WRAPS (nonatomic_lhs_type)) + { + tree utype + = build_bitint_type (TYPE_PRECISION (nonatomic_lhs_type), 1); + rhs = convert (nonatomic_lhs_type, + build2_loc (loc, code, utype, + convert (utype, old), + convert (utype, val))); + } + else + rhs = build2_loc (loc, code, nonatomic_lhs_type, old, val); + break; + case BIT_AND_EXPR: + case BIT_IOR_EXPR: + case BIT_XOR_EXPR: + rhs = build2_loc (loc, code, nonatomic_lhs_type, old, val); + break; + case ERROR_MARK: + rhs = build2_loc (loc, BIT_AND_EXPR, nonatomic_lhs_type, + build1_loc (loc, BIT_NOT_EXPR, + nonatomic_lhs_type, old), val); + break; + default: + gcc_unreachable (); + } + rhs = build4 (TARGET_EXPR, nonatomic_lhs_type, newval, rhs, NULL_TREE, + NULL_TREE); + SET_EXPR_LOCATION (rhs, loc); + add_stmt (rhs); + + /* if (__atomic_compare_exchange (addr, &old, &new, false, model, model)) + goto done; */ + fndecl = builtin_decl_explicit (BUILT_IN_ATOMIC_COMPARE_EXCHANGE); + params->quick_push (lhs_addr); + params->quick_push (old_addr); + params->quick_push (newval_addr); + params->quick_push (integer_zero_node); + params->quick_push (model); + if (tree_fits_uhwi_p (model) + && (tree_to_uhwi (model) == MEMMODEL_RELEASE + || tree_to_uhwi (model) == MEMMODEL_ACQ_REL)) + params->quick_push (build_int_cst (integer_type_node, MEMMODEL_RELAXED)); + else + params->quick_push (model); + func_call = resolve_overloaded_builtin (loc, fndecl, params); + if (func_call == NULL_TREE) + func_call = build_function_call_vec (loc, vNULL, fndecl, params, NULL); + + tree goto_stmt = build1 (GOTO_EXPR, void_type_node, done_decl); + SET_EXPR_LOCATION (goto_stmt, loc); + + tree stmt + = build3 (COND_EXPR, void_type_node, func_call, goto_stmt, NULL_TREE); + SET_EXPR_LOCATION (stmt, loc); + add_stmt (stmt); + + /* goto loop; */ + goto_stmt = build1 (GOTO_EXPR, void_type_node, loop_decl); + SET_EXPR_LOCATION (goto_stmt, loc); + add_stmt (goto_stmt); + + /* done: */ + add_stmt (done_label); + + tree ret = create_tmp_var_raw (nonatomic_lhs_type); + stmt = build2_loc (loc, MODIFY_EXPR, void_type_node, ret, + return_old_p ? old : newval); + add_stmt (stmt); + + /* Finish the compound statement. */ + stmts = pop_stmt_list (stmts); + + return build4 (TARGET_EXPR, nonatomic_lhs_type, ret, stmts, NULL_TREE, + NULL_TREE); +} + + /* Some builtin functions are placeholders for other expressions. This function should be called immediately after parsing the call expression before surrounding code has committed to the type of the expression. @@ -8030,19 +8261,22 @@ resolve_overloaded_builtin (location_t l /* The following are not _FETCH_OPs and must be accepted with pointers to _Bool (or C++ bool). */ if (fetch_op) - fetch_op = - (orig_code != BUILT_IN_SYNC_BOOL_COMPARE_AND_SWAP_N - && orig_code != BUILT_IN_SYNC_VAL_COMPARE_AND_SWAP_N - && orig_code != BUILT_IN_SYNC_LOCK_TEST_AND_SET_N - && orig_code != BUILT_IN_SYNC_LOCK_RELEASE_N); + fetch_op = (orig_code != BUILT_IN_SYNC_BOOL_COMPARE_AND_SWAP_N + && orig_code != BUILT_IN_SYNC_VAL_COMPARE_AND_SWAP_N + && orig_code != BUILT_IN_SYNC_LOCK_TEST_AND_SET_N + && orig_code != BUILT_IN_SYNC_LOCK_RELEASE_N); - int n = sync_resolve_size (function, params, fetch_op); + int n = sync_resolve_size (function, params, fetch_op, orig_format); tree new_function, first_param, result; enum built_in_function fncode; if (n == 0) return error_mark_node; + if (n == -1) + return atomic_bitint_fetch_using_cas_loop (loc, orig_code, + function, params); + fncode = (enum built_in_function)((int)orig_code + exact_log2 (n) + 1); new_function = builtin_decl_explicit (fncode); if (!sync_resolve_params (loc, function, new_function, params, --- gcc/testsuite/gcc.dg/bitint-18.c.jj 2023-08-03 12:26:35.510922996 +0200 +++ gcc/testsuite/gcc.dg/bitint-18.c 2023-08-03 12:26:42.114831050 +0200 @@ -0,0 +1,44 @@ +/* PR c/102989 */ +/* { dg-do compile { target bitint } } */ +/* { dg-options "-std=c2x -pedantic-errors" } */ + +_Atomic _BitInt(15) a; +_Atomic(_BitInt(15)) b; +_Atomic _BitInt(115) c; +_Atomic _BitInt(192) d; +_Atomic _BitInt(575) e; +unsigned _BitInt(575) f; + +__attribute__((noipa)) _BitInt(575) +foo (_BitInt(575) x) +{ + return x; +} + +__attribute__((noipa)) int +bar (int x) +{ + return x; +} + +__attribute__((noipa)) _Atomic _BitInt(575) * +baz (_Atomic _BitInt(575) *x) +{ + return x; +} + +int +main () +{ + a += 1wb; + b -= 2wb; + c *= 3wb; + d /= 4wb; + e -= 5wb; + f = __atomic_fetch_add (&e, 54342985743985743985743895743834298574985734895743895734895wb, __ATOMIC_SEQ_CST); + f += __atomic_sub_fetch (&e, 13110356772307144130089534440127211568864891923061809853784155727841516341877716905506658630804426134644404380556711020290072702485839594283061059349912463486203837251238365wb, __ATOMIC_SEQ_CST); + f += __atomic_fetch_and (&e, -33740418462630594385361724744395454079240140931656245750192534103967695265126850678980088699287669565365078793986191778469857714756111026776864987769580622009237241167211461wb, __ATOMIC_RELAXED); + f += __atomic_xor_fetch (&e, 30799001892772360282132495459823194445423296347702377756575214695893559890977912003055702776548378201752339680602420936304294728688029412276600086349055079523071860836114234wb, __ATOMIC_SEQ_CST); + f += __atomic_fetch_or (baz (&e), foo (-6581969867283727911005990155704642154324773504588160884865628865547696324844988049982401783508268917375066790729408659617189350524019843499435572226770089390885472550659255wb), bar (__ATOMIC_RELAXED)); + f += __atomic_nand_fetch (&e, 55047840194947228224723671648125013926111290688378416557548660662319034233151051252215595447712248992759177463741832904590457754423713378627482465906620631734790561114905369wb, __ATOMIC_ACQ_REL); +}