From patchwork Sat Nov 18 21:44:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 166656 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1398234vqn; Sat, 18 Nov 2023 14:48:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IGS6szOKmdF+t+J2mg3dICSq/N6ecYjiuZkQEZPm2kmtxlBVzkj9V98nDKNh2DWaCezR27O X-Received: by 2002:a05:622a:1004:b0:41c:c045:2c86 with SMTP id d4-20020a05622a100400b0041cc0452c86mr13044129qte.31.1700347694492; Sat, 18 Nov 2023 14:48:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1700347694; cv=pass; d=google.com; s=arc-20160816; b=of2NQmvn0Gn22arK6cy9vBNql+Vyb2UmgXfhaQtZJS7sO2s7eVpfJG+kwFEidKOe3Z va1pxsY+q2KzuN1X18fRNGYqLfzvHgExcoRJgCgcoA0k5o6wT2/oj4EeFn8IcXFrLgfD yrzI2kvN6x3Cu3cKZuq3CyeCGnM/qjNM+hFCnjEQW29xT5/RFVl3GHCjoa21QpBsMpG0 j35qjkX5F7CQ7CqcFAlhmaus1o3M62GHzAx8L0Emci78qph5L51vsdKOj9NcUDRP633e p+vfs3tyiXQsgu5kIZoh4O8w+YAUsK9d/1TG7SoHTg5gOKhAn8BOe4TDyI0ZfAvhYFci 9FTQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=QJ7w/tcQa5BNNWI3yT9ADrGlhNWu7Sns7eNxiqp7+BE=; fh=sJ+2/4g29YdyXkoRrFZSpsL2zxijepB7X/1rB0LDDh8=; b=f20UlyJ9esF2YmuHfFiKNksgGAthkeJ3g2f3bOfnzySzE8khMLWfd3eh/wPgrGIYGa Jiujlbwmops9IU6PUSRWsylzbdYcSDTUv90DdvfohI4ZnfEdLJ414g2LOGMQhnXy/zpU FtesYU4GqIu+MFTQ+vcB/UaGta7aBK6d7A+4THCdHUF9LPzfhxgvz1xqbuHaPtYc9zEk 9dKJDuVTDusiz/LMcXvBgHk9rxnJZAqM5GlYH00QxbHFBTAv+LXZrwXlh0IAaKq4kzub 9uWNd4JV8QaS14iNF02lBFj1w0RN4URJMi4NxUD30B+xLL5/g2oxSpmvMYcEhDkcpPyP 8vJw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=LvJwiLRA; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id v16-20020a05622a015000b0041e3dfe1bb9si4364169qtw.30.2023.11.18.14.48.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Nov 2023 14:48:14 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=LvJwiLRA; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 42F64385801C for ; Sat, 18 Nov 2023 21:45:01 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id E7CC13858D33 for ; Sat, 18 Nov 2023 21:44:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E7CC13858D33 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E7CC13858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700343877; cv=none; b=AXZzP4CgAjuGjTH9w+QZZe2tZHWxMm4rABm++yHG2vopcm/LMACLOlA4cX5MeI+vs90KDlid9u1FnGBK/gZYpsQQlWtbiW8pqD20OFwalX03rEZWNbwP1i4C3SICIe7N4ZRYve6SnBBwFeIUu8ECVZHmNin9I1xDc0Q9xQJq4JM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700343877; c=relaxed/simple; bh=OYlF2oxlzMDl4LUIF9ImBVMyUdsapOnoG778Oqjnce8=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=nVxJDu+zmCNs3RgI8Fc3sYZdtl4GHK45HFZ2mFJI5c9XXQR0l3i6PRY9Eqmk69RZpubHid2Yy0n4cs8/iVNVK7VwMTzJF2IPAqaOOchB9h4pq4T2V5qGwD7P/SZUv7JLLavznos8toOAjMVXxbKMGsGD9z1uPwVE2eRv0IsX6j8= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700343875; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QJ7w/tcQa5BNNWI3yT9ADrGlhNWu7Sns7eNxiqp7+BE=; b=LvJwiLRAbbFAz1cHZPhGpF1xjOz3Aq3hMVkhKKW7mN7bNqs6QW/1GIWZFwq22Fpnko0uR9 kRs3ZWS/7avr/h3YzM+gJlPsGxbSGyOPqzJtQ9V53i+ht1FZ2O5vkxZY88/rZ94u8rOVVz v327+ha6dshJ7k6PBQFKgytMpMHngHM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-253-pJnqBDzlPH2fQ3uDHtDhpA-1; Sat, 18 Nov 2023 16:44:33 -0500 X-MC-Unique: pJnqBDzlPH2fQ3uDHtDhpA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4C426811E7B; Sat, 18 Nov 2023 21:44:33 +0000 (UTC) Received: from localhost (unknown [10.42.28.9]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3D2D492BE0; Sat, 18 Nov 2023 21:44:32 +0000 (UTC) From: Jonathan Wakely To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [committed v2] libstdc++: Add fast path for std::format("{}", x) [PR110801] Date: Sat, 18 Nov 2023 21:44:02 +0000 Message-ID: <20231118214432.1636488-1-jwakely@redhat.com> In-Reply-To: <20231117155420.1513704-1-jwakely@redhat.com> References: <20231117155420.1513704-1-jwakely@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782827193629352683 X-GMAIL-MSGID: 1782943783958546249 Here's an improved version of this patch, which I've pushed to trunk. Testeed x86_64-linux. -- >8 -- This optimizes the simple case of formatting a single string, integer or bool, with no format-specifier (so no padding, alignment, alternate form etc.) libstdc++-v3/ChangeLog: PR libstdc++/110801 * include/std/format (_Sink_iter::_M_reserve): New member function. (_Sink::_Reservation): New nested class. (_Sink::_M_reserve, _Sink::_M_bump): New virtual functions. (_Seq_sink::_M_reserve, _Seq_sink::_M_bump): New virtual overrides. (_Iter_sink::_M_reserve): Likewise. (__do_vformat_to): Use new functions to optimize "{}" case. --- libstdc++-v3/include/std/format | 164 +++++++++++++++++++++++++++++++- 1 file changed, 163 insertions(+), 1 deletion(-) diff --git a/libstdc++-v3/include/std/format b/libstdc++-v3/include/std/format index 8ec1c8a0b9a..7c52cce5dbb 100644 --- a/libstdc++-v3/include/std/format +++ b/libstdc++-v3/include/std/format @@ -2442,6 +2442,10 @@ namespace __format iter_difference_t<_Out> size; }; +_GLIBCXX_BEGIN_NAMESPACE_CONTAINER +template class vector; +_GLIBCXX_END_NAMESPACE_CONTAINER + /// @cond undocumented namespace __format { @@ -2492,6 +2496,10 @@ namespace __format [[__gnu__::__always_inline__]] constexpr _Sink_iter operator++(int) { return *this; } + + auto + _M_reserve(size_t __n) const + { return _M_sink->_M_reserve(__n); } }; // Abstract base class for type-erased character sinks. @@ -2508,6 +2516,7 @@ namespace __format // Called when the span is full, to make more space available. // Precondition: _M_next != _M_span.begin() // Postcondition: _M_next != _M_span.end() + // TODO: remove the precondition? could make overflow handle it. virtual void _M_overflow() = 0; protected: @@ -2572,6 +2581,46 @@ namespace __format } } + // A successful _Reservation can be used to directly write + // up to N characters to the sink to avoid unwanted buffering. + struct _Reservation + { + // True if the reservation was successful, false otherwise. + explicit operator bool() const noexcept { return _M_sink; } + // A pointer to write directly to the sink. + _CharT* get() const noexcept { return _M_sink->_M_next.operator->(); } + // Add n to the _M_next iterator for the sink. + void _M_bump(size_t __n) { _M_sink->_M_bump(__n); } + _Sink* _M_sink; + }; + + // Attempt to reserve space to write n characters to the sink. + // If anything is written to the reservation then there must be a call + // to _M_bump(N2) before any call to another member function of *this, + // where N2 is the number of characters written. + virtual _Reservation + _M_reserve(size_t __n) + { + auto __avail = _M_unused(); + if (__n <= __avail.size()) + return { this }; + + if (__n <= _M_span.size()) // Cannot meet the request. + { + _M_overflow(); // Make more space available. + __avail = _M_unused(); + if (__n <= __avail.size()) + return { this }; + } + return { nullptr }; + } + + // Update the next output position after writing directly to the sink. + // pre: no calls to _M_write or _M_overflow since _M_reserve. + virtual void + _M_bump(size_t __n) + { _M_next += __n; } + public: _Sink(const _Sink&) = delete; _Sink& operator=(const _Sink&) = delete; @@ -2596,6 +2645,8 @@ namespace __format { } }; + using _GLIBCXX_STD_C::vector; + // A sink that fills a sequence (e.g. std::string, std::vector, std::deque). // Writes to a buffer then appends that to the sequence when it fills up. template @@ -2619,6 +2670,45 @@ namespace __format this->_M_rewind(); } + typename _Sink<_CharT>::_Reservation + _M_reserve(size_t __n) override + { + if constexpr (__is_specialization_of<_Seq, basic_string> + || __is_specialization_of<_Seq, vector>) + { + // Flush the buffer to _M_seq first: + if (this->_M_used().size()) + _M_overflow(); + // Expand _M_seq to make __n new characters available: + const auto __sz = _M_seq.size(); + if constexpr (is_same_v || is_same_v) + _M_seq.__resize_and_overwrite(__sz + __n, + [](auto, auto __n2) { + return __n2; + }); + else + _M_seq.resize(__sz + __n); + // Set _M_used() to be a span over the original part of _M_seq: + this->_M_reset(_M_seq, __sz); + return { this }; + } + else // Try to use the base class' buffer. + return _Sink<_CharT>::_M_reserve(); + } + + void + _M_bump(size_t __n) override + { + if constexpr (__is_specialization_of<_Seq, basic_string> + || __is_specialization_of<_Seq, vector>) + { + // Truncate the sequence to the part that was actually written to: + _M_seq.resize(this->_M_used().size() + __n); + // Switch back to using buffer: + this->_M_reset(this->_M_buf); + } + } + public: // TODO: for SSO string, use SSO buffer as initial span, then switch // to _M_buf if it overflows? Or even do that for all unused capacity? @@ -2744,6 +2834,21 @@ namespace __format } } + typename _Sink<_CharT>::_Reservation + _M_reserve(size_t __n) final + { + auto __avail = this->_M_unused(); + if (__n > __avail.size()) + { + if (_M_max >= 0) + return {}; // cannot grow + + auto __s = this->_M_used(); + this->_M_reset({__s.data(), __s.size() + __n}, __s.size()); + } + return { this }; + } + private: static span<_CharT> _S_make_span(_CharT* __ptr, iter_difference_t<_OutIter> __n, @@ -2773,7 +2878,7 @@ namespace __format uint64_t __off = reinterpret_cast(__ptr) % 1024; __n = (1024 - __off) / sizeof(_CharT); if (__n > 0) [[likely]] - return {__ptr, static_cast(__n)}; + return {__ptr, static_cast(__n)}; else // Misaligned/packed buffer of wchar_t? return {__ptr, 1}; } @@ -3835,6 +3940,63 @@ namespace __format else __sink_out = __sink.out(); + if constexpr (is_same_v<_CharT, char>) + if (__fmt.size() == 2 && __fmt[0] == '{' && __fmt[1] == '}') + { + bool __done = false; + std::visit_format_arg([&](auto& __arg) { + using _Tp = remove_cvref_t; + if constexpr (is_same_v<_Tp, bool>) + { + size_t __len = 4 + !__arg; + const char* __chars[] = { "false", "true" }; + if (auto __res = __sink_out._M_reserve(__len)) + { + __builtin_memcpy(__res.get(), __chars[__arg], __len); + __res._M_bump(__len); + __done = true; + } + } + else if constexpr (is_integral_v<_Tp>) + { + make_unsigned_t<_Tp> __uval; + const bool __neg = __arg < 0; + if (__neg) + __uval = make_unsigned_t<_Tp>(~__arg) + 1u; + else + __uval = __arg; + const auto __n = __detail::__to_chars_len(__uval) + __neg; + if (auto __res = __sink_out._M_reserve(__n)) + { + auto __ptr = __res.get(); + *__ptr = '-'; + __detail::__to_chars_10_impl(__ptr + (int)__neg, __n, + __uval); + __res._M_bump(__n); + __done = true; + } + } + else if constexpr (is_convertible_v<_Tp, string_view>) + { + string_view __sv = __arg; + if (auto __res = __sink_out._M_reserve(__sv.size())) + { + __builtin_memcpy(__res.get(), __sv.data(), __sv.size()); + __res._M_bump(__sv.size()); + __done = true; + } + } + }, __args.get(0)); + + if (__done) + { + if constexpr (is_same_v<_Out, _Sink_iter<_CharT>>) + return __sink_out; + else + return std::move(__sink)._M_finish().out; + } + } + auto __ctx = __loc == nullptr ? _Context(__args, __sink_out) : _Context(__args, __sink_out, *__loc);