From patchwork Wed Mar 15 19:32:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jannik_Gl=C3=BCckert?= X-Patchwork-Id: 70399 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp81673wrt; Wed, 15 Mar 2023 12:33:08 -0700 (PDT) X-Google-Smtp-Source: AK7set8SBngsl4SW/dAWli16OBcRxwfHrNaaQ1vasuoIDmch318XQOd1xh8uICQGWzM54D+S/CMD X-Received: by 2002:a17:906:cf87:b0:925:2bda:fb07 with SMTP id um7-20020a170906cf8700b009252bdafb07mr6907795ejb.65.1678908787703; Wed, 15 Mar 2023 12:33:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678908787; cv=none; d=google.com; s=arc-20160816; b=qG9XrxCnzXL562flLmwct3+ONi1b4Rqvo22Q4H5PkJO9inHYDLJ87xV98BpEQIJpSO HnJLIW5uRfPlIN2jAImt0254utDlG1Cvw3jFdtb26eSH7Nf7/YGxg9g2QQagYCpMtUpK sz6HsYSHk45b04W4s/oO5X6rnuxU24ZM9n6+/9qnLmIB03hD0Nete0jzORN7emfmUe76 5nqvsbzuu2d/Ro89XirtCMnwLZLH/+TVmcwCJLGxtIkQ9zOGuXZl3O2P51do8A/5MG31 4bO7QaWzj+8+MVk4WnXs60iHXMI1gwMwbTT3U/aLi3bAFYzB8IF/CZpsQwAB+HR3oBMH G7eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:cc:to:subject :message-id:date:mime-version:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=ns6ttBXOONOwqSGEd4rpD6R6+3J7CZOVzBIjkjHc8gQ=; b=buCR1WhzppwQG1MD1x2Jzlg6ouzA6G+ukYK03YR48k/9F40Cd3e5ZXmAfU8asTCqTF Oc651Jmvh/2S0omBr6T5rGx4PWgCNE1lO81aImqSOpOa4OjPW4RR+6cMrCtIuF7E2MYZ Dmnp5kGmigpabOskH2sXVTRSYpe7ZwSNnD5VSleruCEOWgVj+DEUjVnHkH5iY+bWi5ES wpz3mw4PeDnWLiPUJqJF/IRLrDGSgnq5ULw7LFvUBT2lrtj922Kv0VV9qB5S92InVfO8 sKd17Vat2YXDQujT4+E4sv9MmVVJiTHiksmPRT1HVg6guoRELvjnEr8IZg9cjc7syUGB YAoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=i4q5UbbJ; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id m21-20020a170906161500b0092beae94d7dsi6488904ejd.67.2023.03.15.12.33.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 12:33:07 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=i4q5UbbJ; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0CB68385B536 for ; Wed, 15 Mar 2023 19:32:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0CB68385B536 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1678908777; bh=ns6ttBXOONOwqSGEd4rpD6R6+3J7CZOVzBIjkjHc8gQ=; h=Date:Subject:To:Cc:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=i4q5UbbJhoFleOD0xeTnW/7sQ9sVMTTUSgwnikXx6MFZw81VBQK2Hq+74+LQ+AGMp 3lqgYnw1q/TSf12jVUSZO+WUDBxbAjeKL47xGGWgqxlGh9giybtSWz/xDEpDwywvb+ ou6NQCcBX7vi3IrRCRQppkMYHPwhXnB4dFrhM4R0= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by sourceware.org (Postfix) with ESMTPS id 2EA983858C83; Wed, 15 Mar 2023 19:32:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2EA983858C83 Received: by mail-qk1-x735.google.com with SMTP id bm1so2300770qkb.13; Wed, 15 Mar 2023 12:32:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678908732; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ns6ttBXOONOwqSGEd4rpD6R6+3J7CZOVzBIjkjHc8gQ=; b=s7kUHm9LEwBgELJuWCV8YbzuGGqLDnqNDQXusyYb9f3JVKkYb2pHYEvKHFa5NwbQ3k F1EepBJW7lElfsIqmjJNk2e8mKXzmCBoYsOEGkAJnIGUbYWkv3xGgcu7f2iX6r7CwqT0 5j/NiL6fZ/8rTdNsNyhRhq1tMmgQoohqL/5QaxJ60YRorHNbnWMC7nOuPtEoOMxF+RvS /lOYzpi6v+YyCV6XRqeqC9FLAsb6BGpAhvEBFexs6oJ+L6KHSa//XavOBofeqG5f+bVR tdxv8aYlTOibVW70APiIrgWKK17b+XdX3ydRH9SYvAMXil9rfrf5xv8EweoVTF4NHyCQ OOuQ== X-Gm-Message-State: AO0yUKW/4gu7Pl2FpOgQWSI26LJLcogYKu5qRLfl3Lpan6D0IY9azFTV 6yQx3r3HKc2mxccosy3/Ko10bP+gup0cY2OKur7NKpxI X-Received: by 2002:a05:620a:146b:b0:742:839d:72e8 with SMTP id j11-20020a05620a146b00b00742839d72e8mr5640899qkl.5.1678908732207; Wed, 15 Mar 2023 12:32:12 -0700 (PDT) MIME-Version: 1.0 Date: Wed, 15 Mar 2023 20:32:01 +0100 Message-ID: Subject: [PATCH v2 2/2] libstdc++: use copy_file_range, improve sendfile in filesystem::copy_file To: libstdc++@gcc.gnu.org Cc: gcc-patches@gcc.gnu.org X-Spam-Status: No, score=-10.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?q?Jannik_Gl=C3=BCckert_via_Gcc-patches?= From: =?utf-8?q?Jannik_Gl=C3=BCckert?= Reply-To: =?utf-8?q?Jannik_Gl=C3=BCckert?= Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760463460462931565?= X-GMAIL-MSGID: =?utf-8?q?1760463460462931565?= From c028a0072c7573cfac90289b3606ba19cb8272a6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jannik=20Gl=C3=BCckert?= Date: Wed, 8 Mar 2023 19:37:43 +0100 Subject: [PATCH 2/2] libstdc++: use copy_file_range MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit copy_file_range is a recent-ish syscall for copying files. It is similar to sendfile but allows filesystem-specific optimizations. Common are: Reflinks: BTRFS, XFS, ZFS (does not implement the syscall yet) Server-side copy: NFS, SMB, Ceph If copy_file_range is not available for the given files, fall back to sendfile / userspace copy. libstdc++-v3/ChangeLog: * acinclude.m4 (_GLIBCXX_USE_COPY_FILE_RANGE): define * config.h.in: Regenerate. * configure: Regenerate. * src/filesystem/ops-common.h: use copy_file_range in std::filesystem::copy_file Signed-off-by: Jannik Glückert --- libstdc++-v3/acinclude.m4 | 20 ++++++++ libstdc++-v3/config.h.in | 3 ++ libstdc++-v3/configure | 62 ++++++++++++++++++++++++ libstdc++-v3/src/filesystem/ops-common.h | 50 +++++++++++++++++++ 4 files changed, 135 insertions(+) diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4 index 85a09a5a869..4cf02dc6e4e 100644 --- a/libstdc++-v3/acinclude.m4 +++ b/libstdc++-v3/acinclude.m4 @@ -4581,6 +4581,7 @@ dnl _GLIBCXX_USE_UTIMENSAT dnl _GLIBCXX_USE_ST_MTIM dnl _GLIBCXX_USE_FCHMOD dnl _GLIBCXX_USE_FCHMODAT +dnl _GLIBCXX_USE_COPY_FILE_RANGE dnl _GLIBCXX_USE_SENDFILE dnl HAVE_LINK dnl HAVE_LSEEK @@ -4779,6 +4780,25 @@ dnl if test $glibcxx_cv_truncate = yes; then AC_DEFINE(HAVE_TRUNCATE, 1, [Define if truncate is available in .]) fi +dnl + AC_CACHE_CHECK([for copy_file_range that can copy files], + glibcxx_cv_copy_file_range, [dnl + case "${target_os}" in + linux*) + GCC_TRY_COMPILE_OR_LINK( + [#include ], + [copy_file_range(1, nullptr, 2, nullptr, 1, 0);], + [glibcxx_cv_copy_file_range=yes], + [glibcxx_cv_copy_file_range=no]) + ;; + *) + glibcxx_cv_copy_file_range=no + ;; + esac + ]) + if test $glibcxx_cv_copy_file_range = yes; then + AC_DEFINE(_GLIBCXX_USE_COPY_FILE_RANGE, 1, [Define if copy_file_range is available in .]) + fi dnl AC_CACHE_CHECK([for sendfile that can copy files], glibcxx_cv_sendfile, [dnl diff --git a/libstdc++-v3/src/filesystem/ops-common.h b/libstdc++-v3/src/filesystem/ops-common.h index 9e1b1d41dc5..202728baef2 100644 --- a/libstdc++-v3/src/filesystem/ops-common.h +++ b/libstdc++-v3/src/filesystem/ops-common.h @@ -49,6 +49,9 @@ #ifdef NEED_DO_COPY_FILE # include # include +# ifdef _GLIBCXX_USE_COPY_FILE_RANGE +# include // copy_file_range +# endif # ifdef _GLIBCXX_USE_SENDFILE # include // sendfile # include // lseek @@ -359,6 +362,31 @@ _GLIBCXX_BEGIN_NAMESPACE_FILESYSTEM } #ifdef NEED_DO_COPY_FILE +#ifdef _GLIBCXX_USE_COPY_FILE_RANGE + bool + copy_file_copy_file_range(int fd_in, int fd_out, size_t length) noexcept + { + // a zero-length file is either empty, or not copyable by this syscall + // return early to avoid the syscall cost + if (length == 0) + { + errno = EINVAL; + return false; + } + size_t bytes_left = length; + off64_t off_in = 0, off_out = 0; + ssize_t bytes_copied; + do { + bytes_copied = ::copy_file_range(fd_in, &off_in, fd_out, &off_out, bytes_left, 0); + bytes_left -= bytes_copied; + } while (bytes_left > 0 && bytes_copied > 0); + if (bytes_copied < 0) + { + return false; + } + return true; + } +#endif #if defined _GLIBCXX_USE_SENDFILE && ! defined _GLIBCXX_FILESYSTEM_IS_WINDOWS bool copy_file_sendfile(int fd_in, int fd_out, size_t length) noexcept @@ -527,6 +555,28 @@ _GLIBCXX_BEGIN_NAMESPACE_FILESYSTEM bool has_copied = false; +#ifdef _GLIBCXX_USE_COPY_FILE_RANGE + if (!has_copied) + has_copied = copy_file_copy_file_range(in.fd, out.fd, from_st->st_size); + if (!has_copied) + { + // EINVAL: src and dst are the same file (this is not cheaply detectable from userspace) + // EINVAL: copy_file_range is unsupported for this file type by the underlying filesystem + // ENOTSUP: undocumented, can arise with old kernels and NFS + // EOPNOTSUPP: filesystem does not implement copy_file_range + // ETXTBSY: src or dst is an active swapfile (nonsensical, but allowed with normal copying) + // EXDEV: src and dst are on different filesystems that do not support cross-fs copy_file_range + // ENOENT: undocumented, can arise with CIFS + // ENOSYS: unsupported by kernel or blocked by seccomp + if (errno != EINVAL && errno != ENOTSUP && errno != EOPNOTSUPP + && errno != ETXTBSY && errno != EXDEV && errno != ENOENT && errno != ENOSYS) + { + ec.assign(errno, std::generic_category()); + return false; + } + } +#endif + #if defined _GLIBCXX_USE_SENDFILE && ! defined _GLIBCXX_FILESYSTEM_IS_WINDOWS if (!has_copied) has_copied = copy_file_sendfile(in.fd, out.fd, from_st->st_size); -- 2.39.2