From patchwork Sun Jul 2 16:32:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Boeckel X-Patchwork-Id: 115102 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp46272vqx; Sun, 2 Jul 2023 09:34:27 -0700 (PDT) X-Google-Smtp-Source: APBJJlETiumVXAOJwzSFOZ0w0sAOuDr+QKJuc4s+sWeaEC1QIsMdZg1xhD+raqnPVHeAUH5YdAFh X-Received: by 2002:a17:906:10dc:b0:992:2f67:cd34 with SMTP id v28-20020a17090610dc00b009922f67cd34mr9771215ejv.22.1688315667735; Sun, 02 Jul 2023 09:34:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688315667; cv=none; d=google.com; s=arc-20160816; b=HJaS52GTJx56EBD0lKSKOLYaK2cVPdDcYEsa1pFuap7VxJdlVjP3zAYYSeAyocAiUJ qgkwx/nr+tNDANZ/QrZBMLXiG4FLsAy9Gy9+esxiGV0Plw+oYhQkDONam60nfAHMBMAi P0RyECS+ElxjcXCzhtFf0vvKKppCwoHsOyjFk9wL6c+X4yex2iAeDimrjX1CU6Z9qzsA v27Xqi8trtfqiT/idnuHaYbxUL5m8GhfhyRL1b1KUv3PHvxM9G5donIeFoMi3szlExNu yP7VjlbgVEhUJtO9R28ZM+CzG78c4Ix8CBpxXQe+YVfmGCzKEpxR7pvv3D+IQzCaejM0 s89Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=DcitMMybUfPBuzj7MYfk8s1f8Mo7g5RL3ya9lCEzb3I=; fh=WmOQlEQshjW1IyckmJV5Nh43dgdMAM2UcuXDxiXBGYo=; b=nDo8yuF5mV+E7Ia2K6jEdYhl3FSL6VWD8wHl9VMpvSe7JgwJJZZjkjezh+XdR95Wy6 CbWpf2kuWL+F6cgaNHVhynerrxUCoTCXjqe0yAgJ21MnBTzfV85BjxGlnBcvNDaxyYTI FSxqgkBho7BYrAEOKRabQKd5zenDQE7IB1Ia0w0XErKVJACXLBpKWzVkcM15VCaT5e4/ rTrA/Bag5TQ5sc3r1j2ocaTVtEegVUm4YjA3EYGX4FJa8iS8KDXAZ63zXsJ7qBdhGJwJ ZYRUr07T7fDIBuYdjpTHOeTEoIlXgCwfIHHfR+IpEWdOFsbBmoYqmMo+TmdN9TKin96o aq3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Vcpc+zEg; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id qq17-20020a17090720d100b009930c925bd9si2043430ejb.945.2023.07.02.09.34.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 09:34:27 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Vcpc+zEg; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 727ED3857343 for ; Sun, 2 Jul 2023 16:33:15 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 727ED3857343 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1688315595; bh=DcitMMybUfPBuzj7MYfk8s1f8Mo7g5RL3ya9lCEzb3I=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=Vcpc+zEgIWUwtTb3LmIW9cqhXjpQkzLzoSkAwcAnM6dT/i5eTbhIKLrsVdYMV0vJW xybwKTxi2rDMo3mLBkUqCNUjZtk2NcYwBTco6IhHDzMtDQ935ip2uCAoU5MJekYcD1 Kr64FHJtwG0NRylRkQiD7sUQTIltxN2JsliXyN1g= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com [IPv6:2607:f8b0:4864:20::f2b]) by sourceware.org (Postfix) with ESMTPS id 79B903857B98 for ; Sun, 2 Jul 2023 16:32:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 79B903857B98 Received: by mail-qv1-xf2b.google.com with SMTP id 6a1803df08f44-634a3682c25so28797496d6.3 for ; Sun, 02 Jul 2023 09:32:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688315542; x=1690907542; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DcitMMybUfPBuzj7MYfk8s1f8Mo7g5RL3ya9lCEzb3I=; b=hYnSGQXb0lITH3fW+3XmD3CPjS8+tbmRtDnNj9yTgXr0MqhiuKjFH58pIpr4pXfHzt InL6KxzSNl6CBuY3JK2hHrpUikjeYovGDJI3Bq68UaHEKTe5EQBTrmn9AM7p9utV4jda rPADI32ttFY2HFRqRYmlXuwXNbvsCcjmu1npErYjCZbRQKMz2ynxFGyuRp3mOTk/5/0o 5++2aOXWZ3kbOwkkGDEEmA5kivKHa/0wxr5wosuSxvGAh3U5LnX7fuQIzPaX5mFkyjzr K0OPI0cxJCy8f8d1olqel5nX4JE8tCtDz+G8uLQbP9OXQwHUnXsQCrPcZxRWoYbeCi44 kraA== X-Gm-Message-State: ABy/qLaZt5HaAT3UwuaWuuahhCjyMR95ge0+jLouuipHM1O9Tt3nHiy2 yNQC0ZipOsIy861cN2xSlmp567oZpZ2jkr7sOcWeHg== X-Received: by 2002:a05:6214:2aa3:b0:631:eb39:b7a6 with SMTP id js3-20020a0562142aa300b00631eb39b7a6mr10093886qvb.33.1688315542708; Sun, 02 Jul 2023 09:32:22 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id i4-20020a05620a144400b0076728319575sm4552839qkl.27.2023.07.02.09.32.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 09:32:22 -0700 (PDT) To: gcc-patches@gcc.gnu.org Cc: Ben Boeckel , jason@redhat.com, nathan@acm.org, fortran@gcc.gnu.org, gcc@gcc.gnu.org, brad.king@kitware.com Subject: [PATCH v7 3/4] c++modules: report imported CMI files as dependencies Date: Sun, 2 Jul 2023 12:32:10 -0400 Message-Id: <20230702163211.3396210-4-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230702163211.3396210-1-ben.boeckel@kitware.com> References: <20230702163211.3396210-1-ben.boeckel@kitware.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Boeckel via Gcc-patches From: Ben Boeckel Reply-To: Ben Boeckel Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770327289523750710?= X-GMAIL-MSGID: =?utf-8?q?1770327289523750710?= They affect the build, so report them via `-MF` mechanisms. gcc/cp/ * module.cc (do_import): Report imported CMI files as dependencies. gcc/testsuite/ * g++.dg/modules/depreport-1_a.C: New test. * g++.dg/modules/depreport-1_b.C: New test. * g++.dg/modules/test-depfile.py: New tool for validating depfile information. * lib/modules.exp: Support for validating depfile contents. Signed-off-by: Ben Boeckel --- gcc/cp/module.cc | 3 + gcc/testsuite/g++.dg/modules/depreport-1_a.C | 10 + gcc/testsuite/g++.dg/modules/depreport-1_b.C | 12 ++ gcc/testsuite/g++.dg/modules/test-depfile.py | 187 +++++++++++++++++++ gcc/testsuite/lib/modules.exp | 29 +++ 5 files changed, 241 insertions(+) create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_a.C create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_b.C create mode 100644 gcc/testsuite/g++.dg/modules/test-depfile.py diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc index 9df60d695b1..f3acc4e02fe 100644 --- a/gcc/cp/module.cc +++ b/gcc/cp/module.cc @@ -18968,6 +18968,9 @@ module_state::do_import (cpp_reader *reader, bool outermost) dump () && dump ("CMI is %s", file); if (note_module_cmi_yes || inform_cmi_p) inform (loc, "reading CMI %qs", file); + /* Add the CMI file to the dependency tracking. */ + if (cpp_get_deps (reader)) + deps_add_dep (cpp_get_deps (reader), file); fd = open (file, O_RDONLY | O_CLOEXEC | O_BINARY); e = errno; } diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_a.C b/gcc/testsuite/g++.dg/modules/depreport-1_a.C new file mode 100644 index 00000000000..241701728a2 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_a.C @@ -0,0 +1,10 @@ +// { dg-additional-options -fmodules-ts } + +export module Foo; +// { dg-module-cmi Foo } + +export class Base +{ +public: + int m; +}; diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_b.C b/gcc/testsuite/g++.dg/modules/depreport-1_b.C new file mode 100644 index 00000000000..b6e317c6703 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_b.C @@ -0,0 +1,12 @@ +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -MD } +// { dg-additional-options "-MF depreport-1.d" } + +import Foo; + +void foo () +{ + Base b; +} + +// { dg-final { run-check-module-dep-expect-input "depreport-1.d" "gcm.cache/Foo.gcm" } } diff --git a/gcc/testsuite/g++.dg/modules/test-depfile.py b/gcc/testsuite/g++.dg/modules/test-depfile.py new file mode 100644 index 00000000000..ea4edb61434 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/test-depfile.py @@ -0,0 +1,187 @@ +import json + + +# Parameters. +ALL_ERRORS = False + + +def _report_error(msg): + '''Report an error.''' + full_msg = 'ERROR: ' + msg + if ALL_ERRORS: + print(full_msg) + else: + raise RuntimeError(full_msg) + + +class Token(object): + pass + + +class Output(Token): + def __init__(self, path): + self.path = path + + +class Input(Token): + def __init__(self, path): + self.path = path + + +class Colon(Token): + pass + + +class Append(Token): + pass + + +class Variable(Token): + def __init__(self, name): + self.name = name + + +class Word(Token): + def __init__(self, name): + self.name = name + + +def validate_depfile(depfile, expect_input=None): + '''Validate a depfile contains some information + + Returns `False` if the information is not found. + ''' + with open(depfile, 'r') as fin: + depfile_content = fin.read() + + real_lines = [] + join_line = False + for line in depfile_content.split('\n'): + # Join the line if needed. + if join_line: + line = real_lines.pop() + line + + # Detect line continuations. + join_line = line.endswith('\\') + # Strip line continuation characters. + if join_line: + line = line[:-1] + + # Add to the real line set. + real_lines.append(line) + + # Perform tokenization. + tokenized_lines = [] + for line in real_lines: + tokenized = [] + join_word = False + for word in line.split(' '): + if join_word: + word = tokenized.pop() + ' ' + word + + # Detect word joins. + join_word = word.endswith('\\') + # Strip escape character. + if join_word: + word = word[:-1] + + # Detect `:` at the end of a word. + if word.endswith(':'): + tokenized.append(word[:-1]) + word = word[-1] + + # Add word to the tokenized set. + tokenized.append(word) + + tokenized_lines.append(tokenized) + + # Parse. + ast = [] + for line in tokenized_lines: + kind = None + for token in line: + if token == ':': + kind = 'dependency' + elif token == '+=': + kind = 'append' + if line == ['']: + kind = 'empty' + + if kind is None: + _report_error('unknown line kind: %s' % line) + + line_parse = [] + if kind == 'dependency': + after_colon = False + for token in line: + if token == ':': + after_colon = True + elif after_colon: + line_parse.append(Input(token)) + else: + line_parse.append(Output(token)) + elif kind == 'append': + after_op = False + for token in line: + if token == '+=': + after_op = True + elif after_op: + line_parse.append(Word(token)) + else: + line_parse.append(Variable(token)) + + ast.append(line_parse) + + for node in ast: + for token in node: + if expect_input is not None: + # If the input is found, clear the expectation. + if isinstance(token, Input) and token.path == expect_input: + expect_input = None + + result = True + if expect_input: + _report_error('missing input: %s' % expect_input) + result = False + + return result + + +if __name__ == '__main__': + import sys + + depfile = None + have_expect = False + expect_input = None + + # Parse arguments. + args = sys.argv[1:] + while args: + # Take an argument. + arg = args.pop(0) + + # Flag to change how errors are reported. + if arg == '-A' or arg == '--all': + ALL_ERRORS = True + # Required arguments. + elif arg == '-d' or arg == '--depfile': + depfile = args.pop(0) + elif arg == '-i' or arg == '--expect-input': + expect_input = args.pop(0) + have_expect = True + + # Validate that we have the required arguments. + if depfile is None: + raise RuntimeError('missing "depfile" file') + if have_expect is None: + raise RuntimeError('missing an "expect" argument') + + # Do the actual work. + try: + is_ok = validate_depfile(depfile, expect_input=expect_input) + except BaseException as e: + _report_error('exception: %s' % e) + + # Fail if errors are found. + if not is_ok: + sys.exit(1) diff --git a/gcc/testsuite/lib/modules.exp b/gcc/testsuite/lib/modules.exp index d466a73cbe1..e24ee7618d9 100644 --- a/gcc/testsuite/lib/modules.exp +++ b/gcc/testsuite/lib/modules.exp @@ -69,3 +69,32 @@ proc run-check-p1689-valid { depfile template } { clean-p1689 $testcase } + +proc run-check-module-dep { depfile flag expected } { + global srcdir subdir + # Extract the test file name from the arguments. + set testcase [file rootname [file tail $depfile]] + + verbose "Verifying dependencies for $testcase in $srcdir/$subdir" 2 + set testcase [remote_download host $testcase] + + set pytest_script "test-depfile.py" + if { ![check_effective_target_recent_python3] } { + unsupported "$pytest_script python3 is missing" + return + } + + verbose "running script test-depfile.py" 1 + spawn -noecho python3 $srcdir/$subdir/$pytest_script --all --depfile $depfile $flag $expected + + expect { + -re "ERROR: (\[^\r\n\]*)" { + fail $expect_out(0,string) + exp_continue + } + } +} + +proc run-check-module-dep-expect-input { depfile expected } { + run-check-module-dep $depfile "--expect-input" $expected +}