From patchwork Fri Dec 15 00:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 178988 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp8967878dys; Thu, 14 Dec 2023 17:18:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IFIFQ3SWsy2ZEX3kVMyEoU++8ZzcSU1kVyzcWl3paSF8DR0xMN5P+mNa4JsCV055zMRVYqX X-Received: by 2002:a05:6808:e8a:b0:3b8:b063:6670 with SMTP id k10-20020a0568080e8a00b003b8b0636670mr10507614oil.103.1702603138191; Thu, 14 Dec 2023 17:18:58 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1702603138; cv=pass; d=google.com; s=arc-20160816; b=BLBtIEYmmPm85NQj5EpaSWZCnILAiIUK5r/LuoGjDAn1clq23M/C6EM22drzzsyduQ 8/bx9msJzZ28LWg+RC2ym+o2xBXgijE2nZJDkSJkjCYArVSalHUaiEk5JjnYCx8nzK/C 6vI9I8pzbNRQptpMJysv+XPvkpoSvKJ3bwlpgLdoQqWqY2p7pZoo0wPTn3E5TYhtFrqw RIbesI+46CvXcj7iVJihX95tjuGSJOsiHC3EAynnqWN0bE/onDq4mmYm9w0AYwKUqiNg g348StifMl+3M2GGMqcQD5rQ1dznlEoHLvpp/n8fIQ8syzTbth9Hr4U+XfGTl/Rl8fa/ 9tfg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:to:from:dkim-signature :arc-filter:dmarc-filter:delivered-to; bh=CcFbbzXBbpcPNjCECCIuKObB9+2PCZv3XLpSepg17xc=; fh=sJ+2/4g29YdyXkoRrFZSpsL2zxijepB7X/1rB0LDDh8=; b=K0N0aHZNDruigubV7/bQsDTVds/2uBYHcn3A5/Ybf6JcnH8Rwy7GdCd55xaVXssM3y 8M4XqicSqgEEvg/S+DcpetJ/0GEyKR151bd/Q/58hegwvkX7nX1N4S57kV2qd53v7BNp bDqSjjE827pGjW78hJSxbAjj7mff6qyBH8j/2fnjWvkzC6931u4uzrNCMoZKTlaQKq3W J8pN6EX1phiPUOSYL6YOGxqeJlJ1EoTjo1JoaxwBCzfZjH+Ta9Vll2M6iB9uO5Xvp9SW 5yWDbD61LwX7h/wDJ/uzSWkEfjk+sNqa99/HZ6obaxuO7AtqdKavN4LFfI6ptHntLbmb GCcg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=L3eqnO0A; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id qs21-20020a05620a395500b0077dd2def9b5si17414857qkn.330.2023.12.14.17.18.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Dec 2023 17:18:58 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=L3eqnO0A; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 311083881D34 for ; Fri, 15 Dec 2023 00:03:02 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 8D0F43865C25 for ; Fri, 15 Dec 2023 00:02:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8D0F43865C25 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 8D0F43865C25 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702598550; cv=none; b=kTP+VORV6s38/KnEE6DVUw0WPZkTzNKtyDTn32jkqWvYUFQ7ortK1pVLnWgLo7YCMsw4aEtNJ50CdVsg6elYJmhFrvN4rhYbbCjWB1gxz+YamIbu4s1Bw944f5tcQ0x/cHVdnu5uxQKv5AX7COLuQXcQ4wZUBQiEenHs7PsCyA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702598550; c=relaxed/simple; bh=Q8D34QKu09STdylSgHv7VsEJFpNwEPYIslJB4Efu0QU=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Mh3moQ0JQRXzf5UcdQW0DaYFxugDIlTyXHB3SdF10LhOsZX7eioJw9/GJek4zYuLzd6+tfvrYR3Cuhq+eQ77Wconn/bgdE04Ag4HqQRw0HLK5lZmgBMVsK/BPJU56+hB8NA8k5c7LD/1dNimJCmfrbwohgHSn+e/G2CyAGbp3aU= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1702598549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CcFbbzXBbpcPNjCECCIuKObB9+2PCZv3XLpSepg17xc=; b=L3eqnO0AsxAyMMFu1v3mS8/t8MB8VKtjAZLVKSrCeaEHvpLdfA3EKtxgvjrIX+Oj3VUm6n w/61N55CZmQoCi/esAlWXcF0YGaqb+qFG+DWDd4HzTjkWwogSqlkIriV3/5qV6kiQG4pJu 4OCThW6LxjAHU/3vl3lbjccy9Nl8MmI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-50-iKkXgYI3NFmKIRW7vn05Dw-1; Thu, 14 Dec 2023 19:02:26 -0500 X-MC-Unique: iKkXgYI3NFmKIRW7vn05Dw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B485929AA3B5; Fri, 15 Dec 2023 00:02:25 +0000 (UTC) Received: from localhost (unknown [10.42.28.145]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FC062166B31; Fri, 15 Dec 2023 00:02:25 +0000 (UTC) From: Jonathan Wakely To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [committed] libstdc++: Tweaks for std::format fast path Date: Fri, 15 Dec 2023 00:02:17 +0000 Message-ID: <20231215000224.2069098-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785308788507286646 X-GMAIL-MSGID: 1785308788507286646 Tested x86_64-linux. Pushed to trunk. -- >8 -- Fix an incorrect call to _Sink::_M_reserve() which should have passed the __n parameter. This was not actually a problem because it was in an discarded statement, because only the _Seq_sink> specialization was used. Also add some branch prediction hints, explanatory comments, and debug mode assertions to _Seq_sink. libstdc++-v3/ChangeLog: * include/std/format (_Seq_sink): Fix missing argument in discarded statement. Add comments, likely/unlikely attributes and debug assertions as sanity checks. --- libstdc++-v3/include/std/format | 40 +++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format index 1f8cd5c06be..6204fd0e3c1 100644 --- a/libstdc++-v3/include/std/format +++ b/libstdc++-v3/include/std/format @@ -2609,15 +2609,13 @@ namespace __format virtual _Reservation _M_reserve(size_t __n) { - auto __avail = _M_unused(); - if (__n <= __avail.size()) + if (__n <= _M_unused().size()) return { this }; if (__n <= _M_span.size()) // Cannot meet the request. { _M_overflow(); // Make more space available. - __avail = _M_unused(); - if (__n <= __avail.size()) + if (__n <= _M_unused().size()) return { this }; } return { nullptr }; @@ -2669,24 +2667,38 @@ namespace __format _M_overflow() override { auto __s = this->_M_used(); - if (__s.empty()) - return; + if (__s.empty()) [[unlikely]] + return; // Nothing in the buffer to transfer to _M_seq. + + // If _M_reserve was called then _M_bump must have been called too. + _GLIBCXX_DEBUG_ASSERT(__s.data() != _M_seq.data()); + if constexpr (__is_specialization_of<_Seq, basic_string>) _M_seq.append(__s.data(), __s.size()); else _M_seq.insert(_M_seq.end(), __s.begin(), __s.end()); + + // Make the whole of _M_buf available for the next write: this->_M_rewind(); } typename _Sink<_CharT>::_Reservation _M_reserve(size_t __n) override { + // We might already have n characters available in this->_M_unused(), + // but the whole point of this function is to be an optimization for + // the std::format("{}", x) case. We want to avoid writing to _M_buf + // and then copying that into a basic_string if possible, so this + // function prefers to create space directly in _M_seq rather than + // using _M_buf. + if constexpr (__is_specialization_of<_Seq, basic_string> || __is_specialization_of<_Seq, vector>) { - // Flush the buffer to _M_seq first: - if (this->_M_used().size()) - _M_overflow(); + // Flush the buffer to _M_seq first (should not be needed). + if (this->_M_used().size()) [[unlikely]] + _Seq_sink::_M_overflow(); + // Expand _M_seq to make __n new characters available: const auto __sz = _M_seq.size(); if constexpr (is_same_v || is_same_v) @@ -2696,12 +2708,14 @@ namespace __format }); else _M_seq.resize(__sz + __n); - // Set _M_used() to be a span over the original part of _M_seq: + + // Set _M_used() to be a span over the original part of _M_seq + // and _M_unused() to be the extra capacity we just created: this->_M_reset(_M_seq, __sz); return { this }; } else // Try to use the base class' buffer. - return _Sink<_CharT>::_M_reserve(); + return _Sink<_CharT>::_M_reserve(__n); } void @@ -2710,8 +2724,10 @@ namespace __format if constexpr (__is_specialization_of<_Seq, basic_string> || __is_specialization_of<_Seq, vector>) { + auto __s = this->_M_used(); + _GLIBCXX_DEBUG_ASSERT(__s.data() == _M_seq.data()); // Truncate the sequence to the part that was actually written to: - _M_seq.resize(this->_M_used().size() + __n); + _M_seq.resize(__s.size() + __n); // Switch back to using buffer: this->_M_reset(this->_M_buf); }