From patchwork Thu Oct 27 23:16:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Boeckel X-Patchwork-Id: 12004 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp507748wru; Thu, 27 Oct 2022 16:23:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6sJNl1PUDp/WvWHBFMxC29mQftP8PxEMvaQatWSKU93yy2vaBLKCgN2FKlTc/eEndd0TjT X-Received: by 2002:a05:6402:1f84:b0:455:27b8:27aa with SMTP id c4-20020a0564021f8400b0045527b827aamr47126185edc.243.1666912994599; Thu, 27 Oct 2022 16:23:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666912994; cv=none; d=google.com; s=arc-20160816; b=KlvxmakIfENQGYgUjJ8uCmoiw7C5lrf5ze0X5n1DJ9bExCKEfbzVfRGdhEWEOqkHAd UA2wZ24vhDBLIDbOMhSqZhj12CAoChL38Ccpxy4hRV+RlCxa2dsTsG3USEXskIf2Oi1h nY6eYjThjz2X/jXHgLZPAx/+NAqkuU0UQ92wGyMlJT4HOcO/TrbP22386AvhQufSU6+W PN33YJA8UEeLZ9K7FEPMJbUFGLzi61av8mnHCmkWlbqbsRAKYk8QoiWTwpxiFH1+Ku2p SE2/k+W34P4jAccMM7p2PhG7f1DSz2Ikjp2eM2er0ItnnfhBfEuflZDKBpZlX29IQgrE 4G9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:dmarc-filter:delivered-to:dkim-signature :dkim-filter; bh=WD7Nl3PzdQINQhUMU6SJY7Al+WrwPD5f/oQH2DRtcQI=; b=hSTVOCWofHVpaIUInLoHVSOFLEozl4fXZz7PmFvh7ImhQMEZHWOAl2ljS7fgO5qbTl KPCpwJCawtEHzTagcH6VpdLIvNuf9Kl8XQQ2KT3qPtPW+PTB2LFv2jZ2/RmknFfsKucL tgckJSS0wZkJeM0bMkaquMClsa954v1az4fCwrpp5IrpIetgaXMJTx04JEEx0qwvko0C QgVszywAup0RDn7HoC82TROOHQ4+XwcgNn3YRppZTuQRqOPk2KSb3ruQpGQpOjrFUvZo /i+rc9knT5tXgQE7X3YejhI/7Ix4mMBIcXD5378XaSMEm750+Oy2mw+DX2ciLyab3IZh lxKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=n7YSWCzv; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id sg9-20020a170907a40900b0078dd12d0a9bsi67900ejc.875.2022.10.27.16.23.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:23:14 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=n7YSWCzv; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 547823817765 for ; Thu, 27 Oct 2022 23:19:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 547823817765 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666912792; bh=WD7Nl3PzdQINQhUMU6SJY7Al+WrwPD5f/oQH2DRtcQI=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=n7YSWCzvsZpL6pGiC4k9t8eRBHo5+GwFHHE187Pkrre7uO6+bdXMmP0RLzDoc8kW0 WNoWeTtap0QTcg4gCoVfoyX2M4c7YVOWdRqiBfppGeJE27ydE/04NbIbpZP6QDNzUZ lY4hU1RGgfmFHclbfyPq8xII4+3Q3GCkSR9dmLuQ= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by sourceware.org (Postfix) with ESMTPS id B952B3851147 for ; Thu, 27 Oct 2022 23:17:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B952B3851147 Received: by mail-qk1-x72e.google.com with SMTP id z17so2368683qkj.8 for ; Thu, 27 Oct 2022 16:17:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WD7Nl3PzdQINQhUMU6SJY7Al+WrwPD5f/oQH2DRtcQI=; b=u+zX4JwqpHns6Vv2ClqGJKYgIlCsK8JMykRZGn2Gna8FjjxvYkLDonKcIDdvMiMkiH 85pLfFW/TKJzXZ3VAqE4jdsNJK4ja5Zin965dk4F1X+i4HT1z7QEgjTJCByjYeYtJJHT VZOv65PAvMPwF8Gxdsus0XPqaZ4aKL5I4hF9mDDF4KlgJVSzMGjb3BvoEWSaNDgB7fa6 g5pIiofxoS+tF4ClwNgAcA/ZTiPkqjSRpzU89LwGgtbW9Ed8nH+6IWjedLpgO02hL1L8 mGyoAxc5fKPhmM4N5PgiMBv7ss1iAbkBofBaABKjYVUba5DYSHmuo9KyDnV5+osk1bxF L9Cg== X-Gm-Message-State: ACrzQf3z92wFa2J0sRj64y7kgTbIgQwJC3hzvGCuBDxrbys+KdHpskQ0 ASuAlmyhv5PZS9zZ0tcQMgS2jCtZNPyCXw== X-Received: by 2002:ae9:e8c2:0:b0:6f9:2661:b674 with SMTP id a185-20020ae9e8c2000000b006f92661b674mr7735819qkg.392.1666912627197; Thu, 27 Oct 2022 16:17:07 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id d17-20020a05622a15d100b003434d3b5938sm1620908qty.2.2022.10.27.16.17.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:17:06 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH v2 1/3] libcpp: reject codepoints above 0x10FFFF Date: Thu, 27 Oct 2022 19:16:42 -0400 Message-Id: <20221027231645.67623-2-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221027231645.67623-1-ben.boeckel@kitware.com> References: <20221027231645.67623-1-ben.boeckel@kitware.com> MIME-Version: 1.0 X-Spam-Status: No, score=-10.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Boeckel via Gcc-patches From: Ben Boeckel Reply-To: Ben Boeckel Cc: gcc@gcc.gnu.org, brad.king@kitware.com, fortran@gcc.gnu.org, anlauf@gmx.de, Ben Boeckel , nathan@acm.org Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747884960059819116?= X-GMAIL-MSGID: =?utf-8?q?1747884960059819116?= Unicode does not support such values because they are unrepresentable in UTF-16. Signed-off-by: Ben Boeckel --- libcpp/ChangeLog | 6 ++++++ libcpp/charset.cc | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog index 18d5bcceaf0..4d707277531 100644 --- a/libcpp/ChangeLog +++ b/libcpp/ChangeLog @@ -1,3 +1,9 @@ +2022-10-27 Ben Boeckel + + * include/charset.cc: Reject encodings of codepoints above 0x10FFFF. + UTF-16 does not support such codepoints and therefore all Unicode + rejects such values. + 2022-10-19 Lewis Hyatt * include/cpplib.h (struct cpp_string): Use new "string_length" GTY. diff --git a/libcpp/charset.cc b/libcpp/charset.cc index 12a398e7527..e9da6674b5f 100644 --- a/libcpp/charset.cc +++ b/libcpp/charset.cc @@ -216,7 +216,7 @@ one_utf8_to_cppchar (const uchar **inbufp, size_t *inbytesleftp, if (c <= 0x3FFFFFF && nbytes > 5) return EILSEQ; /* Make sure the character is valid. */ - if (c > 0x7FFFFFFF || (c >= 0xD800 && c <= 0xDFFF)) return EILSEQ; + if (c > 0x10FFFF || (c >= 0xD800 && c <= 0xDFFF)) return EILSEQ; *cp = c; *inbufp = inbuf; @@ -320,7 +320,7 @@ one_utf32_to_utf8 (iconv_t bigend, const uchar **inbufp, size_t *inbytesleftp, s += inbuf[bigend ? 2 : 1] << 8; s += inbuf[bigend ? 3 : 0]; - if (s >= 0x7FFFFFFF || (s >= 0xD800 && s <= 0xDFFF)) + if (s > 0x10FFFF || (s >= 0xD800 && s <= 0xDFFF)) return EILSEQ; rval = one_cppchar_to_utf8 (s, outbufp, outbytesleftp); From patchwork Thu Oct 27 23:16:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Boeckel X-Patchwork-Id: 12002 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp506162wru; Thu, 27 Oct 2022 16:18:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ML92M9e0lvSLAARhTLT0v3k4RmRkMSiMeCuvLTWIeM9tzCvsAF1VaCJTH8xYj796XA22N X-Received: by 2002:a05:6402:5248:b0:461:f0fa:864e with SMTP id t8-20020a056402524800b00461f0fa864emr17988220edd.81.1666912726054; Thu, 27 Oct 2022 16:18:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666912726; cv=none; d=google.com; s=arc-20160816; b=bcz/XUwr1uVbzI0phfyHo4f3imGTTVlJYvPbAaGYPRaECPCuPPM80dIucX6JvUr/QQ 7gLKvBhkFi/YtNmISQsuCVHlBhsJ5e2PG1k054Yh0C2V8kOJ/+TY9EQKiKqbJuesKUZB l3KpKN7zaXa9eoGSi4qx3naQ4pZoojgS5274ow/wdrxs/zZA65J8l+oCD9Srd1Hth6eG DQYUmsOCw1ioYdgE31gOsBhAqMea7U6LkS9k4qkLrtbP7uLF7ftQ6WLKGmt5xAZsXkV0 g4coywv5kS+7+gt2n9gSiTWC1UCkOfMv58lbRjDNGxfOSB9+R+cGASZDYo3HxS1bGKg2 tLEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:dmarc-filter:delivered-to:dkim-signature :dkim-filter; bh=k1ckgw/Zitbryg1fYk9NUQK1HUEQUI6/L91coAjePPM=; b=WmAnMkslZgWWVn30Cg1cyoucDjcGfZDUHv3YQkjqV2RUxyoDrRHRPRz4HKoezjVahu 0kwTyUacxtnsymyboiVxaTlTd1lhW/ho3pWHX+JY9UXf/KUulbLFteWczXfYfhHER1hw Xtb7jtxXBQhXxYkH2A2CaObu+WXSlTGEJRJu7o+k9dMtdqPWPudeqyZqrO/ADvbFtd7R mjo3BCbwz+NhtlZt83vyzqRdKZKOEiRGv6pLx0ARDYJ+cs39HMfeWiU6U840K5mCgBp5 boTgNDS/xMTO5k1ficrbVRP1tM/yz+fmeZtHMue5xKs+n7ghMBEcnLx9uoPu7Ibx+kh9 HvqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b="rF/lhgMP"; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id qb6-20020a1709077e8600b0073155abc1b8si3157209ejc.154.2022.10.27.16.18.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:18:46 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b="rF/lhgMP"; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0F23D381D445 for ; Thu, 27 Oct 2022 23:17:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0F23D381D445 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666912679; bh=k1ckgw/Zitbryg1fYk9NUQK1HUEQUI6/L91coAjePPM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rF/lhgMPVukHtFElGOBxUSLxjB7k0/jSRMC6vL8OFoIWNxRRhehTFM/3tYLW6u7dD HiLzr0Xcf99uBTmjrHjmRCKaNMjqdT9w1trJeSULDQ4ucc/sBf9re+QLRJVBucRHQi /4PzYmCqR9sgqxPh4DPxlzh5inL64CgZnPJPpOaI= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com [IPv6:2607:f8b0:4864:20::f2c]) by sourceware.org (Postfix) with ESMTPS id D2715385AC26 for ; Thu, 27 Oct 2022 23:17:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D2715385AC26 Received: by mail-qv1-xf2c.google.com with SMTP id e15so2919152qvo.4 for ; Thu, 27 Oct 2022 16:17:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k1ckgw/Zitbryg1fYk9NUQK1HUEQUI6/L91coAjePPM=; b=5JPVaZYDmO6ve2QT36UQqqLp9fiRxd98iX0UAcN5AWBMPQgBEJ3Z9N1GWA1cg+Demr 9mIZ6ErssmtcN9GqoZHACf9eUQXNcp5+7dmPsztapzUJ4zTNVvd8mW0CNvtbbgD8tByv nyQuOhGMOK5VOjHrC77AEd+rfkacvDp/2J0K/c/gYov5tPqOVelP8+AukWJFIbQrZ1RG wZP+E5buTVboNa2g/rUmNqNQi5kOWQQ2UblL4Zj8U/5/2R31AAVoQ/6lJdexLAlYyLKZ BcTOEUoP76/qY25eEw3GyRcbJqM9/mII74aPZp1yEMxdCMRczzzk9zfyJOPamgdnMhMv f+yA== X-Gm-Message-State: ACrzQf1sgGstLVc98dZMkAwrQzhw3Zz+b7avrM37Au91FyyJ0Qy2edJ0 DXNWnIdPcCROsICGaUsi/eFmi5OwvrfokA== X-Received: by 2002:a05:6214:d46:b0:4b4:625b:7f0c with SMTP id 6-20020a0562140d4600b004b4625b7f0cmr44238248qvr.86.1666912629198; Thu, 27 Oct 2022 16:17:09 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id i16-20020a05620a405000b006eeb25369e9sm1881155qko.25.2022.10.27.16.17.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:17:08 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH v2 2/3] libcpp: add a function to determine UTF-8 validity of a C string Date: Thu, 27 Oct 2022 19:16:43 -0400 Message-Id: <20221027231645.67623-3-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221027231645.67623-1-ben.boeckel@kitware.com> References: <20221027231645.67623-1-ben.boeckel@kitware.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Boeckel via Gcc-patches From: Ben Boeckel Reply-To: Ben Boeckel Cc: gcc@gcc.gnu.org, brad.king@kitware.com, fortran@gcc.gnu.org, anlauf@gmx.de, Ben Boeckel , nathan@acm.org Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747884679069333862?= X-GMAIL-MSGID: =?utf-8?q?1747884679069333862?= This simplifies the interface for other UTF-8 validity detections when a simple "yes" or "no" answer is sufficient. Signed-off-by: Ben Boeckel --- libcpp/ChangeLog | 6 ++++++ libcpp/charset.cc | 18 ++++++++++++++++++ libcpp/internal.h | 2 ++ 3 files changed, 26 insertions(+) diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog index 4d707277531..4e2c7900ae2 100644 --- a/libcpp/ChangeLog +++ b/libcpp/ChangeLog @@ -1,3 +1,9 @@ +2022-10-27 Ben Boeckel + + * include/charset.cc: Add `_cpp_valid_utf8_str` which determines + whether a C string is valid UTF-8 or not. + * include/internal.h: Add prototype for `_cpp_valid_utf8_str`. + 2022-10-27 Ben Boeckel * include/charset.cc: Reject encodings of codepoints above 0x10FFFF. diff --git a/libcpp/charset.cc b/libcpp/charset.cc index e9da6674b5f..0524ab6beba 100644 --- a/libcpp/charset.cc +++ b/libcpp/charset.cc @@ -1864,6 +1864,24 @@ _cpp_valid_utf8 (cpp_reader *pfile, return true; } +extern bool +_cpp_valid_utf8_str (const char *name) +{ + const uchar* in = (const uchar*)name; + size_t len = strlen(name); + cppchar_t cp; + + while (*in) + { + if (one_utf8_to_cppchar(&in, &len, &cp)) + { + return false; + } + } + + return true; +} + /* Subroutine of convert_hex and convert_oct. N is the representation in the execution character set of a numeric escape; write it into the string buffer TBUF and update the end-of-string pointer therein. WIDE diff --git a/libcpp/internal.h b/libcpp/internal.h index badfd1b40da..4f2dd4a2f5c 100644 --- a/libcpp/internal.h +++ b/libcpp/internal.h @@ -834,6 +834,8 @@ extern bool _cpp_valid_utf8 (cpp_reader *pfile, struct normalize_state *nst, cppchar_t *cp); +extern bool _cpp_valid_utf8_str (const char *str); + extern void _cpp_destroy_iconv (cpp_reader *); extern unsigned char *_cpp_convert_input (cpp_reader *, const char *, unsigned char *, size_t, size_t, From patchwork Thu Oct 27 23:16:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Boeckel X-Patchwork-Id: 12003 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp506296wru; Thu, 27 Oct 2022 16:19:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6YFq34E1eZSCgglwCPsNihgpPwqVJgjdGvctP+Hj4K6p1shHEu3+pPUa+owu5UjyygB/5J X-Received: by 2002:a17:907:31c4:b0:78d:9b2f:4e1a with SMTP id xf4-20020a17090731c400b0078d9b2f4e1amr44277510ejb.306.1666912752986; Thu, 27 Oct 2022 16:19:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666912752; cv=none; d=google.com; s=arc-20160816; b=E+gqaoi1GTqQAjaHDapEwtjXnscqksHJs9bV2oSlbgPEhRhTQw83ZeqqfEY9kpEGAt T8Ub56zOglzL9AP2XMp7gZv7WMjR562YwIxfNauxnKOY5UBWhtlgUXfsdW06+J2T+8op MrCTa1DFOgBko2srFXttxccfaQ+N+68zr4yJe6uOg9gZqEHHo/pZnBbntcUQ4k8ZIAMO ghslhiYOQz8hR8CM54xQg1VcOPbKs0vPTDMPUMic1b71JgPRyAzxiSZ94r+5mb2dtMqO jUw0Ev+WiadE7BepMBY7qHXeJ6l1zZQ0JZxQQaLE+8+zBwVxSSCbdG/4tg4y2OUrCNdK R5HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:dmarc-filter:delivered-to:dkim-signature :dkim-filter; bh=Bp+SjHDJw2KQkOAmFKA296t4lHAtzAhYxhNlUoZF1jk=; b=d4wFzqxKsTu8U7F0b+5LH+jgoh15mbnpceCe532P6VyrClD+MlQlAoHxSr6UbtlgeD K2yNpVbxCENNzmcDIgMKz94kSr6RL4M98q8SxoerBmKbeWzgM0V5GZlr+RtFLofXvgxm 5KrbZz3Ic4XyM0HFktJMjq5gQfDMA1zo8r6nh7YrUIcIBdHBLC4F+nmabBelRBGSKYv2 O5mHyMBDTpgPEKM48IMrcDHELl7E3VRXfi4BljVTAGYHtYJfzYrbuWglsYbeuQSAX1Lp rwoaZcSNLB/1H7YxzecJiejf8vrlFJGsf+muO24q19tGu5gevbxaiuWVVYC0IUNoUyAu TDQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b="Nd/nEsd4"; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id 28-20020a508e5c000000b004485081f004si3175187edx.598.2022.10.27.16.19.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:19:12 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b="Nd/nEsd4"; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DD7AB3898398 for ; Thu, 27 Oct 2022 23:18:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DD7AB3898398 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666912689; bh=Bp+SjHDJw2KQkOAmFKA296t4lHAtzAhYxhNlUoZF1jk=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Nd/nEsd4s4Lq8HKiGqz6bOK0JrZWWRsMvhxUpB+qOoK4rsRiHppoC+22oIQTTOGmU aGtgU4raWnIUBhCcuwOhbsSwfHhKoTgk4r7d7x2f/0gylYGArQox0DPciqNrPFSTE9 U/ORXTG1Vau8gIg3qLul1d9y8m9824QBcQ62IaZk= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by sourceware.org (Postfix) with ESMTPS id 03DC2384A804 for ; Thu, 27 Oct 2022 23:17:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 03DC2384A804 Received: by mail-qk1-x72d.google.com with SMTP id j21so2364801qkk.9 for ; Thu, 27 Oct 2022 16:17:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bp+SjHDJw2KQkOAmFKA296t4lHAtzAhYxhNlUoZF1jk=; b=WIITmSCudy00XmyeDwChjWi4cutzr5XWogf7zILiSIkSxbY+EC0mzDkNIt6sxdTivp 8NP7TTzO5rrq+hfXYTE2C4CTfxW/PpLSw/ak2+oRWOUirCjjsX7/1xFCogwTu/U8NWgG 9lAFgPkK8neCYCtDOAkO+gkeW/ZmdDI4V4+73P8eVJm3NdfDij8e+PVaExbIjwpC9UoS 07AQ8xpO5LySTh20BvReiVnNAyWIjpYWpgvCK+xIfyfK1pphX9vcKZGfAGOUx9hjtPgv f+vF7PAaQl4Jgt2Idy0DIz1uzu8f0jEXZszhATGGX2hPOBIuRDnmNS0py0Re1Pboxt00 h/AA== X-Gm-Message-State: ACrzQf3Dp8f9S/af1VylC8qlfP14NTostxKQGKVNebt1QfhXKp64udR1 WD2cJOZhYSFx5y83PkyhjvB3twv+EjsWQw== X-Received: by 2002:a05:620a:4724:b0:6ee:b2c1:6864 with SMTP id bs36-20020a05620a472400b006eeb2c16864mr37108466qkb.418.1666912631192; Thu, 27 Oct 2022 16:17:11 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id n3-20020a05620a294300b006ed138e89f2sm1840905qkp.123.2022.10.27.16.17.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Oct 2022 16:17:10 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH v2 3/3] p1689r5: initial support Date: Thu, 27 Oct 2022 19:16:44 -0400 Message-Id: <20221027231645.67623-4-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221027231645.67623-1-ben.boeckel@kitware.com> References: <20221027231645.67623-1-ben.boeckel@kitware.com> MIME-Version: 1.0 X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Boeckel via Gcc-patches From: Ben Boeckel Reply-To: Ben Boeckel Cc: gcc@gcc.gnu.org, brad.king@kitware.com, fortran@gcc.gnu.org, anlauf@gmx.de, Ben Boeckel , nathan@acm.org Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747884706646687238?= X-GMAIL-MSGID: =?utf-8?q?1747884706646687238?= This patch implements support for [P1689R5][] to communicate to a build system the C++20 module dependencies to build systems so that they may build `.gcm` files in the proper order. Support is communicated through the following three new flags: - `-fdeps-format=` specifies the format for the output. Currently named `p1689r5`. - `-fdeps-file=` specifies the path to the file to write the format to. - `-fdep-output=` specifies the `.o` that will be written for the TU that is scanned. This is required so that the build system can correlate the dependency output with the actual compilation that will occur. CMake supports this format as of 17 Jun 2022 (to be part of 3.25.0) using an experimental feature selection (to allow for future usage evolution without committing to how it works today). While it remains experimental, docs may be found in CMake's documentation for experimental features. Future work may include using this format for Fortran module dependencies as well, however this is still pending work. [P1689R5]: https://isocpp.org/files/papers/P1689R5.html [cmake-experimental]: https://gitlab.kitware.com/cmake/cmake/-/blob/master/Help/dev/experimental.rst TODO: - header-unit information fields Header units (including the standard library headers) are 100% unsupported right now because the `-E` mechanism wants to import their BMIs. A new mode (i.e., something more workable than existing `-E` behavior) that mocks up header units as if they were imported purely from their path and content would be required. - non-utf8 paths The current standard says that paths that are not unambiguously represented using UTF-8 are not supported (because these cases are rare and the extra complication is not worth it at this time). Future versions of the format might have ways of encoding non-UTF-8 paths. For now, this patch just doesn't support non-UTF-8 paths (ignoring the "unambiguously represetable in UTF-8" case). - figure out why junk gets placed at the end of the file Sometimes it seems like the file gets a lot of `NUL` bytes appended to it. It happens rarely and seems to be the result of some `ftruncate`-style call which results in extra padding in the contents. Noting it here as an observation at least. Signed-off-by: Ben Boeckel --- gcc/ChangeLog | 5 + gcc/c-family/ChangeLog | 6 + gcc/c-family/c-opts.cc | 40 +++- gcc/c-family/c.opt | 12 + gcc/cp/ChangeLog | 5 + gcc/cp/module.cc | 3 +- gcc/doc/invoke.texi | 15 ++ gcc/testsuite/ChangeLog | 7 + gcc/testsuite/g++.dg/modules/depflags-f-MD.C | 2 + gcc/testsuite/g++.dg/modules/depflags-f.C | 1 + gcc/testsuite/g++.dg/modules/depflags-fi.C | 3 + gcc/testsuite/g++.dg/modules/depflags-fj-MD.C | 3 + gcc/testsuite/g++.dg/modules/depflags-fj.C | 4 + .../g++.dg/modules/depflags-fjo-MD.C | 4 + gcc/testsuite/g++.dg/modules/depflags-fjo.C | 5 + gcc/testsuite/g++.dg/modules/depflags-fo-MD.C | 3 + gcc/testsuite/g++.dg/modules/depflags-fo.C | 4 + gcc/testsuite/g++.dg/modules/depflags-j-MD.C | 2 + gcc/testsuite/g++.dg/modules/depflags-j.C | 3 + gcc/testsuite/g++.dg/modules/depflags-jo-MD.C | 3 + gcc/testsuite/g++.dg/modules/depflags-jo.C | 4 + gcc/testsuite/g++.dg/modules/depflags-o-MD.C | 2 + gcc/testsuite/g++.dg/modules/depflags-o.C | 3 + gcc/testsuite/g++.dg/modules/modules.exp | 11 + gcc/testsuite/g++.dg/modules/p1689-1.C | 18 ++ gcc/testsuite/g++.dg/modules/p1689-1.exp.json | 27 +++ gcc/testsuite/g++.dg/modules/p1689-2.C | 16 ++ gcc/testsuite/g++.dg/modules/p1689-2.exp.json | 16 ++ gcc/testsuite/g++.dg/modules/p1689-3.C | 14 ++ gcc/testsuite/g++.dg/modules/p1689-3.exp.json | 16 ++ gcc/testsuite/g++.dg/modules/p1689-4.C | 14 ++ gcc/testsuite/g++.dg/modules/p1689-4.exp.json | 14 ++ gcc/testsuite/g++.dg/modules/p1689-5.C | 14 ++ gcc/testsuite/g++.dg/modules/p1689-5.exp.json | 14 ++ gcc/testsuite/g++.dg/modules/test-p1689.py | 222 ++++++++++++++++++ gcc/testsuite/lib/modules.exp | 71 ++++++ libcpp/ChangeLog | 11 + libcpp/include/cpplib.h | 12 +- libcpp/include/mkdeps.h | 17 +- libcpp/init.cc | 13 +- libcpp/mkdeps.cc | 149 +++++++++++- 41 files changed, 789 insertions(+), 19 deletions(-) create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fi.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o-MD.C create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.exp.json create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.exp.json create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.exp.json create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.exp.json create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.C create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.exp.json create mode 100644 gcc/testsuite/g++.dg/modules/test-p1689.py create mode 100644 gcc/testsuite/lib/modules.exp diff --git a/gcc/ChangeLog b/gcc/ChangeLog index f9052da2f97..4bdf70701f1 100644 --- a/gcc/ChangeLog +++ b/gcc/ChangeLog @@ -1,3 +1,8 @@ +2022-10-27 Ben Boeckel + + * doc/invoke.texi: Document -fdeps-format=, -fdep-file=, and + -fdep-output= flags. + 2022-10-26 David Faust * config/bpf/bpf.cc: Support __builtin_preserve_field_info. diff --git a/gcc/c-family/ChangeLog b/gcc/c-family/ChangeLog index ee7b51179e1..819015fdaad 100644 --- a/gcc/c-family/ChangeLog +++ b/gcc/c-family/ChangeLog @@ -1,3 +1,9 @@ +2022-10-27 Ben Boeckel + + * c-opts.cc (c_common_handle_option): Add fdeps_file variable and + -fdeps-format=, -fdep-file=, and -fdep-output= parsing. + * c.opt: Add -fdeps-format=, -fdep-file=, and -fdep-output= flags. + 2022-10-26 Marek Polacek PR c++/106393 diff --git a/gcc/c-family/c-opts.cc b/gcc/c-family/c-opts.cc index 32b929e3ece..093432bddeb 100644 --- a/gcc/c-family/c-opts.cc +++ b/gcc/c-family/c-opts.cc @@ -77,6 +77,9 @@ static bool verbose; /* Dependency output file. */ static const char *deps_file; +/* Enhanced dependency output file. */ +static const char *fdeps_file; + /* The prefix given by -iprefix, if any. */ static const char *iprefix; @@ -360,6 +363,23 @@ c_common_handle_option (size_t scode, const char *arg, HOST_WIDE_INT value, deps_file = arg; break; + case OPT_fdep_format_: + if (!strcmp (arg, "p1689r5")) + cpp_opts->deps.format = DEPS_FMT_P1689R5; + else + error ("%<-fdep-format=%> unknown format %<%s%>", arg); + break; + + case OPT_fdep_file_: + deps_seen = true; + fdeps_file = arg; + break; + + case OPT_fdep_output_: + deps_seen = true; + defer_opt (code, arg); + break; + case OPT_MF: deps_seen = true; deps_file = arg; @@ -1271,6 +1291,7 @@ void c_common_finish (void) { FILE *deps_stream = NULL; + FILE *fdeps_stream = NULL; /* Note that we write the dependencies even if there are errors. This is useful for handling outdated generated headers that now trigger errors @@ -1299,9 +1320,24 @@ c_common_finish (void) locations with input_location, which would be incorrect now. */ override_libcpp_locations = false; + if (cpp_opts->deps.format != DEPS_FMT_NONE) + { + if (!fdeps_file) + fdeps_stream = out_stream; + else if (fdeps_file[0] == '-' && fdeps_file[1] == '\0') + fdeps_stream = stdout; + else + { + fdeps_stream = fopen (fdeps_file, "w"); + if (!fdeps_stream) + fatal_error (input_location, "opening dependency file %s: %m", + fdeps_file); + } + } + /* For performance, avoid tearing down cpplib's internal structures with cpp_destroy (). */ - cpp_finish (parse_in, deps_stream); + cpp_finish (parse_in, deps_stream, fdeps_stream); if (deps_stream && deps_stream != out_stream && deps_stream != stdout && (ferror (deps_stream) || fclose (deps_stream))) @@ -1373,6 +1409,8 @@ handle_deferred_opts (void) if (opt->code == OPT_MT || opt->code == OPT_MQ) deps_add_target (deps, opt->arg, opt->code == OPT_MQ); + else if (opt->code == OPT_fdep_output_) + deps_add_output (deps, opt->arg, true); } } diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt index 070f85c81d2..dcee27c6533 100644 --- a/gcc/c-family/c.opt +++ b/gcc/c-family/c.opt @@ -256,6 +256,18 @@ MT C ObjC C++ ObjC++ Joined Separate MissingArgError(missing makefile target after %qs) -MT Add a target that does not require quoting. +fdep-format= +C ObjC C++ ObjC++ NoDriverArg Joined MissingArgError(missing format after %qs) +Format for output dependency information. Supported (\"p1689r5\"). + +fdep-file= +C ObjC C++ ObjC++ NoDriverArg Joined MissingArgError(missing output path after %qs) +File for output dependency information. + +fdep-output= +C ObjC C++ ObjC++ NoDriverArg Joined MissingArgError(missing path after %qs) +-fdep-output=obj.o Output file for the compile step. + P C ObjC C++ ObjC++ Do not generate #line directives. diff --git a/gcc/cp/ChangeLog b/gcc/cp/ChangeLog index 4a490753778..d7fb7c8f3e6 100644 --- a/gcc/cp/ChangeLog +++ b/gcc/cp/ChangeLog @@ -1,3 +1,8 @@ +2022-10-27 Ben Boeckel + + * module.cc (preprocessed_module): Pass whether the module is + exported to dependency tracking. + 2022-10-26 Marek Polacek PR c++/106393 diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc index 9957df510e6..d9e760876b7 100644 --- a/gcc/cp/module.cc +++ b/gcc/cp/module.cc @@ -19794,7 +19794,8 @@ preprocessed_module (cpp_reader *reader) && (module->is_interface () || module->is_partition ())) deps_add_module_target (deps, module->get_flatname (), maybe_add_cmi_prefix (module->filename), - module->is_header()); + module->is_header (), + module->is_exported ()); else deps_add_module_dep (deps, module->get_flatname ()); } diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 9f0e5460861..3bd4024f9c5 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -2786,6 +2786,21 @@ is @option{-fpermitted-flt-eval-methods=c11}. The default when in a GNU dialect (@option{-std=gnu11} or similar) is @option{-fpermitted-flt-eval-methods=ts-18661-3}. +@item -fdep-file=@var{file} +@opindex fdep-file +Where to write structured dependency information. + +@item -fdep-format=@var{format} +@opindex fdep-format +The format to use for structured dependency information. @samp{p1689r5} is the +only supported format right now. Note that when this argument is specified, the +output of @samp{-MF} is stripped of some information (namely C++ modules) so +that it does not use extended makefile syntax not understood by most tools. + +@item -fdep-output=@var{file} +@opindex fdep-output +Analogous to @option{-MT} but for structured dependency information. + @item -fplan9-extensions @opindex fplan9-extensions Accept some non-standard constructs used in Plan 9 code. diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog index 56ccf843482..248f57e8035 100644 --- a/gcc/testsuite/ChangeLog +++ b/gcc/testsuite/ChangeLog @@ -1,3 +1,10 @@ +2022-10-27 Ben Boeckel + + * g++.dg/modules/depflags*: New tests. + * g++.dg/modules/p1689*: New tests. + * g++.dg/modules/test-p1689.py: New tool for validating P1689 output. + * lib/modules.exp: Support for validating P1689 outputs. + 2022-10-26 David Malcolm * gcc.dg/analyzer/fd-3.c (test_5): Expect "opened here" message diff --git a/gcc/testsuite/g++.dg/modules/depflags-f-MD.C b/gcc/testsuite/g++.dg/modules/depflags-f-MD.C new file mode 100644 index 00000000000..90e1c9983bd --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-f-MD.C @@ -0,0 +1,2 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-format=p1689r5 } diff --git a/gcc/testsuite/g++.dg/modules/depflags-f.C b/gcc/testsuite/g++.dg/modules/depflags-f.C new file mode 100644 index 00000000000..6192300879d --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-f.C @@ -0,0 +1 @@ +// { dg-additional-options -fdep-format=p1689r5 } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fi.C b/gcc/testsuite/g++.dg/modules/depflags-fi.C new file mode 100644 index 00000000000..4f649a11bdd --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fi.C @@ -0,0 +1,3 @@ +// { dg-additional-options -fdep-format=invalid } + +// { dg-prune-output "error: '-fdep-format=' unknown format 'invalid'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fj-MD.C b/gcc/testsuite/g++.dg/modules/depflags-fj-MD.C new file mode 100644 index 00000000000..a361d81f37f --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fj-MD.C @@ -0,0 +1,3 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-format=p1689r5 } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fj.C b/gcc/testsuite/g++.dg/modules/depflags-fj.C new file mode 100644 index 00000000000..4a140ec1f13 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fj.C @@ -0,0 +1,4 @@ +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-format=p1689r5 } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C b/gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C new file mode 100644 index 00000000000..18d765211b4 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C @@ -0,0 +1,4 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-output=depflags-1.C } +// { dg-additional-options -fdep-format=p1689r5 } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fjo.C b/gcc/testsuite/g++.dg/modules/depflags-fjo.C new file mode 100644 index 00000000000..6d239f63017 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fjo.C @@ -0,0 +1,5 @@ +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-output=depflags-1.C } +// { dg-additional-options -fdep-format=p1689r5 } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fo-MD.C b/gcc/testsuite/g++.dg/modules/depflags-fo-MD.C new file mode 100644 index 00000000000..a3a775b606a --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fo-MD.C @@ -0,0 +1,3 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=depflags-1.C } diff --git a/gcc/testsuite/g++.dg/modules/depflags-fo.C b/gcc/testsuite/g++.dg/modules/depflags-fo.C new file mode 100644 index 00000000000..29839978e59 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-fo.C @@ -0,0 +1,4 @@ +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=depflags-1.C } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-j-MD.C b/gcc/testsuite/g++.dg/modules/depflags-j-MD.C new file mode 100644 index 00000000000..d95c8e6c2e6 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-j-MD.C @@ -0,0 +1,2 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-file=depflags-3.json } diff --git a/gcc/testsuite/g++.dg/modules/depflags-j.C b/gcc/testsuite/g++.dg/modules/depflags-j.C new file mode 100644 index 00000000000..5f100b0f6e5 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-j.C @@ -0,0 +1,3 @@ +// { dg-additional-options -fdep-file=depflags-3.json } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-jo-MD.C b/gcc/testsuite/g++.dg/modules/depflags-jo-MD.C new file mode 100644 index 00000000000..44330794abc --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-jo-MD.C @@ -0,0 +1,3 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-output=depflags-1.C } diff --git a/gcc/testsuite/g++.dg/modules/depflags-jo.C b/gcc/testsuite/g++.dg/modules/depflags-jo.C new file mode 100644 index 00000000000..8eec6bba1d1 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-jo.C @@ -0,0 +1,4 @@ +// { dg-additional-options -fdep-file=depflags-3.json } +// { dg-additional-options -fdep-output=depflags-1.C } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/depflags-o-MD.C b/gcc/testsuite/g++.dg/modules/depflags-o-MD.C new file mode 100644 index 00000000000..429f1f85684 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-o-MD.C @@ -0,0 +1,2 @@ +// { dg-additional-options -MD } +// { dg-additional-options -fdep-output=depflags-1.C } diff --git a/gcc/testsuite/g++.dg/modules/depflags-o.C b/gcc/testsuite/g++.dg/modules/depflags-o.C new file mode 100644 index 00000000000..9a7326cc812 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depflags-o.C @@ -0,0 +1,3 @@ +// { dg-additional-options -fdep-output=depflags-1.C } + +// { dg-prune-output "error: to generate dependencies you must specify either '-M' or '-MM'" } diff --git a/gcc/testsuite/g++.dg/modules/modules.exp b/gcc/testsuite/g++.dg/modules/modules.exp index afb323d0efd..7fe8825144f 100644 --- a/gcc/testsuite/g++.dg/modules/modules.exp +++ b/gcc/testsuite/g++.dg/modules/modules.exp @@ -28,6 +28,7 @@ # { dg-module-do [link|run] [xfail] [options] } # link [and run] load_lib g++-dg.exp +load_lib modules.exp # If a testcase doesn't have special options, use these. global DEFAULT_CXXFLAGS @@ -237,6 +238,13 @@ proc cleanup_module_files { files } { } } +# delete the specified set of dep files +proc cleanup_dep_files { files } { + foreach file $files { + file_on_host delete $file + } +} + global testdir set testdir $srcdir/$subdir proc srcdir {} { @@ -310,6 +318,7 @@ foreach src [lsort [find $srcdir/$subdir {*_a.[CHX}]] { set std_list [module-init $src] foreach std $std_list { set mod_files {} + set dep_files {} global module_do set module_do {"compile" "P"} set asm_list {} @@ -346,6 +355,8 @@ foreach src [lsort [find $srcdir/$subdir {*_a.[CHX}]] { set mod_files [find $DEFAULT_REPO *.gcm] } cleanup_module_files $mod_files + + cleanup_dep_files $dep_files } } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-1.C b/gcc/testsuite/g++.dg/modules/p1689-1.C new file mode 100644 index 00000000000..245e30d09ce --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-1.C @@ -0,0 +1,18 @@ +// { dg-additional-options -E } +// { dg-additional-options -MT } +// { dg-additional-options p1689-1.json } +// { dg-additional-options -MD } +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=p1689-1.o } +// { dg-additional-options -fdep-file=p1689-1.json } + +// Export a module that uses modules, re-exports modules, and partitions. + +export module foo; +export import foo:part1; +import foo:part2; + +export import bar; + +// { dg-final { run-check-p1689-valid p1689-1.json p1689-1.exp.json } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-1.exp.json b/gcc/testsuite/g++.dg/modules/p1689-1.exp.json new file mode 100644 index 00000000000..c5648ac7ae5 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-1.exp.json @@ -0,0 +1,27 @@ +{ + "rules": [ + { + "primary-output": "p1689-1.o", + "provides": [ + { + "logical-name": "foo", + "is-interface": true + } + ], + "requires": [ + "__P1689_unordered__", + { + "logical-name": "bar" + }, + { + "logical-name": "foo:part1" + }, + { + "logical-name": "foo:part2" + } + ] + } + ], + "version": 0, + "revision": 0 +} diff --git a/gcc/testsuite/g++.dg/modules/p1689-2.C b/gcc/testsuite/g++.dg/modules/p1689-2.C new file mode 100644 index 00000000000..add07f59a0e --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-2.C @@ -0,0 +1,16 @@ +// { dg-additional-options -E } +// { dg-additional-options -MT } +// { dg-additional-options p1689-2.json } +// { dg-additional-options -MD } +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=p1689-2.o } +// { dg-additional-options -fdep-file=p1689-2.json } + +// Export a module partition that uses modules. + +export module foo:part1; + +#include + +// { dg-final { run-check-p1689-valid p1689-2.json p1689-2.exp.json } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-2.exp.json b/gcc/testsuite/g++.dg/modules/p1689-2.exp.json new file mode 100644 index 00000000000..6901172f277 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-2.exp.json @@ -0,0 +1,16 @@ +{ + "rules": [ + { + "primary-output": "p1689-2.o", + "provides": [ + { + "logical-name": "foo:part1", + "is-interface": true + } + ], + "requires": [] + } + ], + "version": 0, + "revision": 0 +} diff --git a/gcc/testsuite/g++.dg/modules/p1689-3.C b/gcc/testsuite/g++.dg/modules/p1689-3.C new file mode 100644 index 00000000000..3482c2f4903 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-3.C @@ -0,0 +1,14 @@ +// { dg-additional-options -E } +// { dg-additional-options -MT } +// { dg-additional-options p1689-3.json } +// { dg-additional-options -MD } +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=p1689-3.o } +// { dg-additional-options -fdep-file=p1689-3.json } + +// Provide a module partition. + +module foo:part2; + +// { dg-final { run-check-p1689-valid p1689-3.json p1689-3.exp.json } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-3.exp.json b/gcc/testsuite/g++.dg/modules/p1689-3.exp.json new file mode 100644 index 00000000000..5a40beacd22 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-3.exp.json @@ -0,0 +1,16 @@ +{ + "rules": [ + { + "primary-output": "p1689-3.o", + "provides": [ + { + "logical-name": "foo:part2", + "is-interface": false + } + ], + "requires": [] + } + ], + "version": 0, + "revision": 0 +} diff --git a/gcc/testsuite/g++.dg/modules/p1689-4.C b/gcc/testsuite/g++.dg/modules/p1689-4.C new file mode 100644 index 00000000000..88bac77a8f8 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-4.C @@ -0,0 +1,14 @@ +// { dg-additional-options -E } +// { dg-additional-options -MT } +// { dg-additional-options p1689-4.json } +// { dg-additional-options -MD } +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=p1689-4.o } +// { dg-additional-options -fdep-file=p1689-4.json } + +// Module implementation unit. + +module foo; + +// { dg-final { run-check-p1689-valid p1689-4.json p1689-4.exp.json } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-4.exp.json b/gcc/testsuite/g++.dg/modules/p1689-4.exp.json new file mode 100644 index 00000000000..b119f5654b1 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-4.exp.json @@ -0,0 +1,14 @@ +{ + "rules": [ + { + "primary-output": "p1689-4.o", + "requires": [] + { + "logical-name": "foo" + } + ] + } + ], + "version": 0, + "revision": 0 +} diff --git a/gcc/testsuite/g++.dg/modules/p1689-5.C b/gcc/testsuite/g++.dg/modules/p1689-5.C new file mode 100644 index 00000000000..e985277368b --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-5.C @@ -0,0 +1,14 @@ +// { dg-additional-options -E } +// { dg-additional-options -MT } +// { dg-additional-options p1689-5.json } +// { dg-additional-options -MD } +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -fdep-format=p1689r5 } +// { dg-additional-options -fdep-output=p1689-5.o } +// { dg-additional-options -fdep-file=p1689-5.json } + +// Use modules, don't provide anything. + +import bar; + +// { dg-final { run-check-p1689-valid p1689-5.json p1689-5.exp.json } } diff --git a/gcc/testsuite/g++.dg/modules/p1689-5.exp.json b/gcc/testsuite/g++.dg/modules/p1689-5.exp.json new file mode 100644 index 00000000000..18704ac8820 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/p1689-5.exp.json @@ -0,0 +1,14 @@ +{ + "rules": [ + { + "primary-output": "p1689-5.o", + "requires": [ + { + "logical-name": "bar" + } + ] + } + ], + "version": 0, + "revision": 0 +} diff --git a/gcc/testsuite/g++.dg/modules/test-p1689.py b/gcc/testsuite/g++.dg/modules/test-p1689.py new file mode 100644 index 00000000000..2f07cc361aa --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/test-p1689.py @@ -0,0 +1,222 @@ +import json + + +# Parameters. +ALL_ERRORS = False +REPLACEMENTS = {} + + +def _print_path(path): + '''Format a JSON path for output.''' + return '/'.join(path) + + +def _report_error(msg): + '''Report an error.''' + full_msg = 'ERROR: ' + msg + if ALL_ERRORS: + print(full_msg) + else: + raise RuntimeError(full_msg) + + +def _error_type_mismatch(path, actual, expect): + '''Report that there is a type mismatch.''' + _report_error('type mismatch at %s: actual: "%s" expect: "%s"' % (_print_path(path), actual, expect)) + + +def _error_unknown_type(path, typ): + '''Report that there is an unknown type in the JSON object.''' + _report_error('unknown type at %s: "%s"' % (_print_path(path), typ)) + + +def _error_length_mismatch(path, actual, expect): + '''Report a length mismatch in an object or array.''' + _report_error('length mismatch at %s: actual: "%s" expect: "%s"' % (_print_path(path), actual, expect)) + + +def _error_unexpect_value(path, actual, expect): + '''Report a value mismatch.''' + _report_error('value mismatch at %s: actual: "%s" expect: "%s"' % (_print_path(path), actual, expect)) + + +def _error_extra_key(path, key): + '''Report on a key that is unexpected.''' + _report_error('extra key at %s: "%s"' % (_print_path(path), key)) + + +def _error_missing_key(path, key): + '''Report on a key that is missing.''' + _report_error('extra key at %s: %s' % (_print_path(path), key)) + + +def _compare_object(path, actual, expect): + '''Compare a JSON object.''' + is_ok = True + + if not len(actual) == len(expect): + _error_length_mismatch(path, len(actual), len(expect)) + is_ok = False + + for key in actual: + if key not in expect: + _error_extra_key(path, key) + is_ok = False + else: + sub_error = compare_json(path + [key], actual[key], expect[key]) + if sub_error: + is_ok = False + + for key in expect: + if key not in actual: + _error_missing_key(path, key) + is_ok = False + + return is_ok + + +def _compare_array(path, actual, expect): + '''Compare a JSON array.''' + is_ok = True + + if not len(actual) == len(expect): + _error_length_mismatch(path, len(actual), len(expect)) + is_ok = False + + for (idx, (a, e)) in enumerate(zip(actual, expect)): + sub_error = compare_json(path + [str(idx)], a, e) + if sub_error: + is_ok = False + + return is_ok + + +def _make_replacements(value): + for (old, new) in REPLACEMENTS.values(): + value = value.replace(old, new) + return value + + +def _compare_string(path, actual, expect): + '''Compare a JSON string supporting replacements in the expected output.''' + expect = _make_replacements(expect) + + if not actual == expect: + _error_unexpect_value(path, actual, expect) + return False + else: + print('%s is ok: %s' % (_print_path(path), actual)) + return True + + +def _compare_number(path, actual, expect): + '''Compare a JSON integer.''' + if not actual == expect: + _error_unexpect_value(path, actual, expect) + return False + else: + print('%s is ok: %s' % (_print_path(path), actual)) + return True + + +def _inspect_ordering(arr): + req_ordering = True + + if not arr: + return arr, req_ordering + + if arr[0] == '__P1689_unordered__': + arr.pop(0) + req_ordering = False + + return arr, req_ordering + + +def compare_json(path, actual, expect): + actual_type = type(actual) + expect_type = type(expect) + + is_ok = True + + if not actual_type == expect_type: + _error_type_mismatch(path, actual_type, expect_type) + is_ok = False + elif actual_type == dict: + is_ok = _compare_object(path, actual, expect) + elif actual_type == list: + expect, req_ordering = _inspect_ordering(expect) + if not req_ordering: + actual = set(actual) + expect = set(expect) + is_ok = _compare_array(path, actual, expect) + elif actual_type == str: + is_ok = _compare_string(path, actual, expect) + elif actual_type == float: + is_ok = _compare_number(path, actual, expect) + elif actual_type == int: + is_ok = _compare_number(path, actual, expect) + elif actual_type == bool: + is_ok = _compare_number(path, actual, expect) + elif actual_type == type(None): + pass + else: + _error_unknown_type(path, actual_type) + is_ok = False + + return is_ok + + +def validate_p1689(actual, expect): + '''Validate a P1689 file against an expected output file. + + Returns `False` if it fails, `True` if they are the same. + ''' + with open(actual, 'r') as fin: + actual_content = fin.read() + with open(expect, 'r') as fin: + expect_content = fin.read() + + actual_json = json.loads(actual_content) + expect_json = json.loads(expect_content) + + return compare_json([], actual_json, expect_json) + + +if __name__ == '__main__': + import sys + + actual = None + expect = None + + # Parse arguments. + args = sys.argv[1:] + while args: + # Take an argument. + arg = args.pop(0) + + # Parse out replacement expressions. + if arg == '-r' or arg == '--replace': + replacement = args.pop(0) + (key, value) = replacement.split('=', maxsplit=1) + REPLACEMENTS[key] = value + # Flag to change how errors are reported. + elif arg == '-A' or arg == '--all': + ALL_ERRORS = True + # Required arguments. + elif arg == '-a' or arg == '--actual': + actual = args.pop(0) + elif arg == '-e' or arg == '--expect': + expect = args.pop(0) + + # Validate that we have the required arguments. + if actual is None: + raise RuntimeError('missing "actual" file') + if expect is None: + raise RuntimeError('missing "expect" file') + + # Do the actual work. + is_ok = validate_p1689(actual, expect) + + # Fail if errors are found. + if not is_ok: + sys.exit(1) diff --git a/gcc/testsuite/lib/modules.exp b/gcc/testsuite/lib/modules.exp new file mode 100644 index 00000000000..c7cfda6aae4 --- /dev/null +++ b/gcc/testsuite/lib/modules.exp @@ -0,0 +1,71 @@ +# Copyright (C) 1997-2022 Free Software Foundation, Inc. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# . + +# Verify various kinds of gcov output: line counts, branch percentages, +# and call return percentages. None of this is language-specific. + +load_lib "target-supports.exp" + +# +# clean-p1689-file -- delete a working file the compiler creates for p1689 +# +# TESTCASE is the name of the test. +# SUFFIX is file suffix + +proc clean-p1689-file { testcase suffix } { + set basename [file tail $testcase] + set base [file rootname $basename] + remote_file host delete $base.$suffix +} + +# +# clean-p1689 -- delete the working files the compiler creates for p1689 +# +# TESTCASE is the name of the test. +# +proc clean-p1689 { testcase } { + clean-p1689-file $testcase "d" + clean-p1689-file $testcase "json" +} + +# Call by dg-final to check a P1689 dependency file + +proc run-check-p1689-valid { depfile template } { + global srcdir subdir + # Extract the test file name from the arguments. + set testcase [file rootname [file tail $depfile]] + + verbose "Running P1689 validation for $testcase in $srcdir/$subdir" 2 + set testcase [remote_download host $testcase] + + set pytest_script "test-p1689.py" + if { ![check_effective_target_recent_python3] } { + unsupported "$pytest_script python3 is missing" + return + } + + verbose "running script" 1 + spawn -noecho python3 $srcdir/$subdir/$pytest_script --all --actual $depfile --expect $srcdir/$subdir/$template + + expect { + -re "ERROR: (\[^\r\n\]*)" { + fail $expect_out(0,string) + exp_continue + } + } + + clean-p1689 $testcase +} diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog index 4e2c7900ae2..34117d79fee 100644 --- a/libcpp/ChangeLog +++ b/libcpp/ChangeLog @@ -1,3 +1,14 @@ +2022-10-27 Ben Boeckel + + * include/cpplib.h: Add cpp_deps_format enum. + * include/cpplib.h (cpp_options): Add format field. + * include/cpplib.h (cpp_finish): Add dependency stream parameter. + * include/mkdeps.h (deps_add_module_target): Add new preprocessor + parameter used for C++ module tracking. + * init.cc (cpp_finish): Add new preprocessor parameter used for C++ + module tracking. + * mkdeps.cc (mkdeps): Implement P1689R5 output. + 2022-10-27 Ben Boeckel * include/charset.cc: Add `_cpp_valid_utf8_str` which determines diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h index 1d34c00669f..3b2e4f23204 100644 --- a/libcpp/include/cpplib.h +++ b/libcpp/include/cpplib.h @@ -302,6 +302,9 @@ typedef CPPCHAR_SIGNED_T cppchar_signed_t; /* Style of header dependencies to generate. */ enum cpp_deps_style { DEPS_NONE = 0, DEPS_USER, DEPS_SYSTEM }; +/* Format of header dependencies to generate. */ +enum cpp_deps_format { DEPS_FMT_NONE = 0, DEPS_FMT_P1689R5 }; + /* The possible normalization levels, from most restrictive to least. */ enum cpp_normalize_level { /* In NFKC. */ @@ -589,6 +592,9 @@ struct cpp_options /* Style of header dependencies to generate. */ enum cpp_deps_style style; + /* Format of header dependencies to generate. */ + enum cpp_deps_format format; + /* Assume missing files are generated files. */ bool missing_files; @@ -1112,9 +1118,9 @@ extern void cpp_post_options (cpp_reader *); extern void cpp_init_iconv (cpp_reader *); /* Call this to finish preprocessing. If you requested dependency - generation, pass an open stream to write the information to, - otherwise NULL. It is your responsibility to close the stream. */ -extern void cpp_finish (cpp_reader *, FILE *deps_stream); + generation, pass open stream(s) to write the information to, + otherwise NULL. It is your responsibility to close the stream(s). */ +extern void cpp_finish (cpp_reader *, FILE *deps_stream, FILE *fdeps_stream = NULL); /* Call this to release the handle at the end of preprocessing. Any use of the handle after this function returns is invalid. */ diff --git a/libcpp/include/mkdeps.h b/libcpp/include/mkdeps.h index 96d64641b1a..0bd284f903e 100644 --- a/libcpp/include/mkdeps.h +++ b/libcpp/include/mkdeps.h @@ -53,20 +53,29 @@ extern void deps_add_default_target (class mkdeps *, const char *); /* Adds a module target. The module name and cmi name are copied. */ extern void deps_add_module_target (struct mkdeps *, const char *module, - const char *cmi, bool is_header); + const char *cmi, bool is_header, + bool is_exported); /* Adds a module dependency. The module name is copied. */ extern void deps_add_module_dep (struct mkdeps *, const char *module); +/* Add an output. */ +extern void deps_add_output (struct mkdeps *, const char *, bool); + /* Add a dependency (appears on the right side of the colon) to the deps list. Dependencies will be printed in the order that they were entered with this function. By convention, the first dependency entered should be the primary source file. */ extern void deps_add_dep (class mkdeps *, const char *); -/* Write out a deps buffer to a specified file. The last argument - is the number of columns to word-wrap at (0 means don't wrap). */ -extern void deps_write (const cpp_reader *, FILE *, unsigned int); +/* Write out a deps buffer to a specified file. The third argument + is the number of columns to word-wrap at (0 means don't wrap). + The last argument indicates whether to output extra information + (namely modules). */ +extern void deps_write (const struct cpp_reader *, FILE *, unsigned int); + +/* Write out a deps buffer to a specified file in P1689R5 format. */ +extern void deps_write_p1689r5 (const struct mkdeps *, FILE *); /* Write out a deps buffer to a file, in a form that can be read back with deps_restore. Returns nonzero on error, in which case the diff --git a/libcpp/init.cc b/libcpp/init.cc index 5f34e3515d2..ca1e57b1669 100644 --- a/libcpp/init.cc +++ b/libcpp/init.cc @@ -855,7 +855,7 @@ read_original_directory (cpp_reader *pfile) Maybe it should also reset state, such that you could call cpp_start_read with a new filename to restart processing. */ void -cpp_finish (cpp_reader *pfile, FILE *deps_stream) +cpp_finish (struct cpp_reader *pfile, FILE *deps_stream, FILE *fdeps_stream) { /* Warn about unused macros before popping the final buffer. */ if (CPP_OPTION (pfile, warn_unused_macros)) @@ -869,8 +869,15 @@ cpp_finish (cpp_reader *pfile, FILE *deps_stream) while (pfile->buffer) _cpp_pop_buffer (pfile); - if (deps_stream) - deps_write (pfile, deps_stream, 72); + cpp_deps_format deps_format = CPP_OPTION (pfile, deps.format); + if (deps_format == DEPS_FMT_P1689R5 && fdeps_stream) + deps_write_p1689r5 (pfile->deps, fdeps_stream); + + if (CPP_OPTION (pfile, deps.style) != DEPS_NONE + && deps_stream) + { + deps_write (pfile, deps_stream, 72); + } /* Report on headers that could use multiple include guards. */ if (CPP_OPTION (pfile, print_include_names)) diff --git a/libcpp/mkdeps.cc b/libcpp/mkdeps.cc index 30e87d8b4d7..8e0171d3f00 100644 --- a/libcpp/mkdeps.cc +++ b/libcpp/mkdeps.cc @@ -81,7 +81,8 @@ public: }; mkdeps () - : module_name (NULL), cmi_name (NULL), is_header_unit (false), quote_lwm (0) + : primary_output (NULL), module_name (NULL), cmi_name (NULL) + , is_header_unit (false), is_exported (false), quote_lwm (0) { } ~mkdeps () @@ -90,6 +91,9 @@ public: for (i = targets.size (); i--;) free (const_cast (targets[i])); + free (const_cast (primary_output)); + for (i = outputs.size (); i--;) + free (const_cast (outputs[i])); for (i = deps.size (); i--;) free (const_cast (deps[i])); for (i = vpath.size (); i--;) @@ -103,6 +107,8 @@ public: public: vec targets; vec deps; + const char * primary_output; + vec outputs; vec vpath; vec modules; @@ -110,6 +116,7 @@ public: const char *module_name; const char *cmi_name; bool is_header_unit; + bool is_exported; unsigned short quote_lwm; }; @@ -288,6 +295,21 @@ deps_add_default_target (class mkdeps *d, const char *tgt) } } +/* Adds an output O. We make a copy, so it need not be a permanent + string. */ +void +deps_add_output (struct mkdeps *d, const char *o, bool is_primary) +{ + o = apply_vpath (d, o); + if (is_primary) + { + if (d->primary_output) + d->outputs.push (d->primary_output); + d->primary_output = xstrdup (o); + } else + d->outputs.push (xstrdup (o)); +} + void deps_add_dep (class mkdeps *d, const char *t) { @@ -325,12 +347,13 @@ deps_add_vpath (class mkdeps *d, const char *vpath) void deps_add_module_target (struct mkdeps *d, const char *m, - const char *cmi, bool is_header_unit) + const char *cmi, bool is_header_unit, bool is_exported) { gcc_assert (!d->module_name); d->module_name = xstrdup (m); d->is_header_unit = is_header_unit; + d->is_exported = is_exported; d->cmi_name = xstrdup (cmi); } @@ -395,10 +418,15 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) if (colmax && colmax < 34) colmax = 34; + /* Write out C++ modules information if no other `-fdeps-format=` + * option is given. */ + cpp_deps_format deps_format = CPP_OPTION (pfile, deps.format); + bool write_make_modules_deps = deps_format == DEPS_FMT_NONE; + if (d->deps.size ()) { column = make_write_vec (d->targets, fp, 0, colmax, d->quote_lwm); - if (CPP_OPTION (pfile, deps.modules) && d->cmi_name) + if (write_make_modules_deps && CPP_OPTION (pfile, deps.modules) && d->cmi_name) column = make_write_name (d->cmi_name, fp, column, colmax); fputs (":", fp); column++; @@ -412,7 +440,7 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) if (!CPP_OPTION (pfile, deps.modules)) return; - if (d->modules.size ()) + if (write_make_modules_deps && d->modules.size ()) { column = make_write_vec (d->targets, fp, 0, colmax, d->quote_lwm); if (d->cmi_name) @@ -423,7 +451,7 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) fputs ("\n", fp); } - if (d->module_name) + if (write_make_modules_deps && d->module_name) { if (d->cmi_name) { @@ -455,7 +483,7 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) } } - if (d->modules.size ()) + if (write_make_modules_deps && d->modules.size ()) { column = fprintf (fp, "CXX_IMPORTS +="); make_write_vec (d->modules, fp, column, colmax, 0, ".c++m"); @@ -468,11 +496,118 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) /* Really we should be opening fp here. */ void -deps_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) +deps_write (const struct cpp_reader *pfile, FILE *fp, unsigned int colmax) { make_write (pfile, fp, colmax); } +static void +p1689r5_write_filepath (const char *name, FILE *fp) +{ + if (_cpp_valid_utf8_str (name)) + { + fputc ('"', fp); + for (const char* c = name; *c; c++) + { + // Escape control characters. + if (ISCNTRL (*c)) + fprintf (fp, "\\u%04x", *c); + // JSON escape characters. + else if (*c == '"' || *c == '\\') + { + fputc ('\\', fp); + fputc (*c, fp); + } + // Everything else. + else + fputc (*c, fp); + } + fputc ('"', fp); + } + else + { + // TODO: print an error + } +} + +static void +p1689r5_write_vec (const mkdeps::vec &vec, FILE *fp) +{ + for (unsigned ix = 0; ix != vec.size (); ix++) + { + p1689r5_write_filepath (vec[ix], fp); + if (ix < vec.size () - 1) + fputc (',', fp); + fputc ('\n', fp); + } +} + +void +deps_write_p1689r5 (const struct mkdeps *d, FILE *fp) +{ + fputs ("{\n", fp); + + fputs ("\"rules\": [\n", fp); + fputs ("{\n", fp); + + if (d->primary_output) + { + fputs ("\"primary-output\": ", fp); + p1689r5_write_filepath (d->primary_output, fp); + fputs (",\n", fp); + } + + if (d->outputs.size ()) + { + fputs ("\"outputs\": [\n", fp); + p1689r5_write_vec (d->outputs, fp); + fputs ("],\n", fp); + } + + if (d->module_name) + { + fputs ("\"provides\": [\n", fp); + fputs ("{\n", fp); + + fputs ("\"logical-name\": ", fp); + p1689r5_write_filepath (d->module_name, fp); + fputs (",\n", fp); + + fprintf (fp, "\"is-interface\": %s\n", d->is_exported ? "true" : "false"); + + // TODO: header-unit information + + fputs ("}\n", fp); + fputs ("],\n", fp); + } + + fputs ("\"requires\": [\n", fp); + for (size_t i = 0; i < d->modules.size (); i++) + { + if (i != 0) + fputs (",\n", fp); + fputs ("{\n", fp); + + fputs ("\"logical-name\": ", fp); + p1689r5_write_filepath (d->modules[i], fp); + fputs ("\n", fp); + + // TODO: header-unit information + + fputs ("}\n", fp); + } + fputs ("]\n", fp); + + fputs ("}\n", fp); + + fputs ("],\n", fp); + + fputs ("\"version\": 0,\n", fp); + fputs ("\"revision\": 0\n", fp); + + fputs ("}\n", fp); +} + /* Write out a deps buffer to a file, in a form that can be read back with deps_restore. Returns nonzero on error, in which case the error number will be in errno. */