From patchwork Fri Sep 1 13:04:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Boeckel X-Patchwork-Id: 137394 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c792:0:b0:3f2:4152:657d with SMTP id b18csp877737vqu; Fri, 1 Sep 2023 06:14:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGG4zQqdzxP0Yoo8DEJ+5o2TFbWPYC9kTkfN29gXaYwp1rJBMpuI7E86cWL84gkyTk6Bot+ X-Received: by 2002:a17:906:224a:b0:9a1:f415:7c24 with SMTP id 10-20020a170906224a00b009a1f4157c24mr2031629ejr.46.1693574047664; Fri, 01 Sep 2023 06:14:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693574047; cv=none; d=google.com; s=arc-20160816; b=qtl0h0xyYUUHfRrc98fEDI2jPMLTSAHVCvZ2+/VF2O4tlhKiwvvMFBO+OCOBo6X50P L5p2smj72/98u0B+H0p6pQVEt55LZcAb7bFqUXParUVq99TC1CSkLZqys1FnVXDsWcb9 7zvdKvm8+XoaLzxMkTOqyD21CwMfh3QHmWJpFtfO7DwUB3CW3PD4fTCNROCiFNzHfbTN tzIXQ8K2ZbjdYcQ4nmpmvIGzxXfG+wkD4abvcHv9gv9dLK+lYFurgdiBJSG13pnxJgmO X/TPQjuzHEj3/nJeV9pue6HTIeooPYj8VaTF/p2Fk2QC3WYFo/yS7DADdSyeDMYssmOE Z3JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:dmarc-filter:delivered-to:dkim-signature :dkim-filter; bh=4WOpSU81BFy+TUWfWtmlVIfS2uHQn9f9fi5pb17KBU4=; fh=k/Ceyxcdi7bE5nnOFo1POLZehPadJc8HBydZ6eliaUc=; b=P3eyUpvdjsZlgDWrGgQ2ns0QGATjIu5pM/fscOK2JSnCFk8JLRLrUYsPGZW1lHF/+c Ga+znszkozmW+jMO+Nxxk8Hw0lTXjMjYVwj6ng1/vnLdZP74/x7aSSJPzpVfJftCjJCF ZsUsiUEtqZ51Ai6Ngcr+Tm8qP17EtbXGSgjDzeX1k9Y3CrW66Dn3viMh9zTi6f6wfdnv ZmpS4V7HApTtUVLXWcmxDwvMbXTSaIufERPt+9pf97iKwq6XvwK5blOly7AaIy9x8ONG TwY1Y1vt7WUV5SNq9y5WKkPxS80G7/Rgpw+tuzrYc5R4dT/KCrkJxfYoGYaj9EVViUWU WNzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=BU16R+ru; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id qq21-20020a17090720d500b0098df1cbe2aesi2324569ejb.1008.2023.09.01.06.14.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Sep 2023 06:14:07 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=BU16R+ru; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 83E093889E36 for ; Fri, 1 Sep 2023 13:05:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 83E093889E36 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1693573534; bh=4WOpSU81BFy+TUWfWtmlVIfS2uHQn9f9fi5pb17KBU4=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=BU16R+ruNH+5tkj/BCdF35fCL31Mgaq5e+VDTRez15Ajh5isK/ivxvdP494Xc43KP 33o7P4x8amInMfCeonfomLYeR6h9pAC05ZvGEIpIsfnBVyW0C8A8TxM1oB3895vuQ8 zjvrXXPBrD7Wu4pC1a82uZeE8MacXpTL+8Ko5Pvw= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com [IPv6:2607:f8b0:4864:20::832]) by sourceware.org (Postfix) with ESMTPS id 36259388265F for ; Fri, 1 Sep 2023 13:04:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 36259388265F Received: by mail-qt1-x832.google.com with SMTP id d75a77b69052e-412137ae07aso11977951cf.2 for ; Fri, 01 Sep 2023 06:04:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693573460; x=1694178260; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4WOpSU81BFy+TUWfWtmlVIfS2uHQn9f9fi5pb17KBU4=; b=gkUIkeCT3djxF/D4GMuwiva0X7cSexw8s1XYj8mN1gquo9IWKDpdL1/vU5i1zvTeaq 27AQpFAieaqRz78uFKHd6U/u2KNlB25LoddVMA4lT96PHjS+7GJ5fdasG9zXR6MEh0tJ qFOC8Tixlh6ghqnUol4mbcsKVWwhaKR0MO043IjRAwfB7uvkKzMHoX3WQk0M557dozw1 uuTF5usfZ7FOUWZbKI/Ga1bGN8zIEE4IKF4eTzNdZy8h7GqWDY8tfcS30N7AWoLKJ/At ZnfTzjzipLoyV8Snvu+/RYQp+/fMRYnQVMhkCI++E2qjm9ie+mhmZuGeT/O6hnZBw4Qe qfYg== X-Gm-Message-State: AOJu0YytANIe3Yb9QvQKV4BN6QVnpJEC/vs3HJLn+Fvk62J58QIxYU4v almKH1q2cSbXP0lBZ0e76C9lNm216rmR9Mak5HXAEw== X-Received: by 2002:a0c:8e41:0:b0:641:e7b8:6a18 with SMTP id w1-20020a0c8e41000000b00641e7b86a18mr2240182qvb.56.1693573460113; Fri, 01 Sep 2023 06:04:20 -0700 (PDT) Received: from localhost (cpe-142-105-146-128.nycap.res.rr.com. [142.105.146.128]) by smtp.gmail.com with ESMTPSA id v5-20020a0ce1c5000000b0064f4ec872dfsm1434448qvl.0.2023.09.01.06.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Sep 2023 06:04:19 -0700 (PDT) To: gcc-patches@gcc.gnu.org Subject: [PATCH v8 3/4] c++modules: report imported CMI files as dependencies Date: Fri, 1 Sep 2023 09:04:03 -0400 Message-ID: <20230901130407.259768-4-ben.boeckel@kitware.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230901130407.259768-1-ben.boeckel@kitware.com> References: <20230901130407.259768-1-ben.boeckel@kitware.com> MIME-Version: 1.0 X-Spam-Status: No, score=-13.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Boeckel via Gcc-patches From: Ben Boeckel Reply-To: Ben Boeckel Cc: gcc@gcc.gnu.org, brad.king@kitware.com, fortran@gcc.gnu.org, Ben Boeckel , nathan@acm.org Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1775841100258997365 X-GMAIL-MSGID: 1775841100258997365 They affect the build, so report them via `-MF` mechanisms. gcc/cp/ * module.cc (do_import): Report imported CMI files as dependencies. gcc/testsuite/ * g++.dg/modules/depreport-1_a.C: New test. * g++.dg/modules/depreport-1_b.C: New test. * g++.dg/modules/test-depfile.py: New tool for validating depfile information. * lib/modules.exp: Support for validating depfile contents. Signed-off-by: Ben Boeckel --- gcc/cp/module.cc | 3 + gcc/testsuite/g++.dg/modules/depreport-1_a.C | 10 + gcc/testsuite/g++.dg/modules/depreport-1_b.C | 12 ++ gcc/testsuite/g++.dg/modules/test-depfile.py | 187 +++++++++++++++++++ gcc/testsuite/lib/modules.exp | 29 +++ 5 files changed, 241 insertions(+) create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_a.C create mode 100644 gcc/testsuite/g++.dg/modules/depreport-1_b.C create mode 100644 gcc/testsuite/g++.dg/modules/test-depfile.py diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc index 9df60d695b1..f3acc4e02fe 100644 --- a/gcc/cp/module.cc +++ b/gcc/cp/module.cc @@ -18968,6 +18968,9 @@ module_state::do_import (cpp_reader *reader, bool outermost) dump () && dump ("CMI is %s", file); if (note_module_cmi_yes || inform_cmi_p) inform (loc, "reading CMI %qs", file); + /* Add the CMI file to the dependency tracking. */ + if (cpp_get_deps (reader)) + deps_add_dep (cpp_get_deps (reader), file); fd = open (file, O_RDONLY | O_CLOEXEC | O_BINARY); e = errno; } diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_a.C b/gcc/testsuite/g++.dg/modules/depreport-1_a.C new file mode 100644 index 00000000000..241701728a2 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_a.C @@ -0,0 +1,10 @@ +// { dg-additional-options -fmodules-ts } + +export module Foo; +// { dg-module-cmi Foo } + +export class Base +{ +public: + int m; +}; diff --git a/gcc/testsuite/g++.dg/modules/depreport-1_b.C b/gcc/testsuite/g++.dg/modules/depreport-1_b.C new file mode 100644 index 00000000000..b6e317c6703 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/depreport-1_b.C @@ -0,0 +1,12 @@ +// { dg-additional-options -fmodules-ts } +// { dg-additional-options -MD } +// { dg-additional-options "-MF depreport-1.d" } + +import Foo; + +void foo () +{ + Base b; +} + +// { dg-final { run-check-module-dep-expect-input "depreport-1.d" "gcm.cache/Foo.gcm" } } diff --git a/gcc/testsuite/g++.dg/modules/test-depfile.py b/gcc/testsuite/g++.dg/modules/test-depfile.py new file mode 100644 index 00000000000..ea4edb61434 --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/test-depfile.py @@ -0,0 +1,187 @@ +import json + + +# Parameters. +ALL_ERRORS = False + + +def _report_error(msg): + '''Report an error.''' + full_msg = 'ERROR: ' + msg + if ALL_ERRORS: + print(full_msg) + else: + raise RuntimeError(full_msg) + + +class Token(object): + pass + + +class Output(Token): + def __init__(self, path): + self.path = path + + +class Input(Token): + def __init__(self, path): + self.path = path + + +class Colon(Token): + pass + + +class Append(Token): + pass + + +class Variable(Token): + def __init__(self, name): + self.name = name + + +class Word(Token): + def __init__(self, name): + self.name = name + + +def validate_depfile(depfile, expect_input=None): + '''Validate a depfile contains some information + + Returns `False` if the information is not found. + ''' + with open(depfile, 'r') as fin: + depfile_content = fin.read() + + real_lines = [] + join_line = False + for line in depfile_content.split('\n'): + # Join the line if needed. + if join_line: + line = real_lines.pop() + line + + # Detect line continuations. + join_line = line.endswith('\\') + # Strip line continuation characters. + if join_line: + line = line[:-1] + + # Add to the real line set. + real_lines.append(line) + + # Perform tokenization. + tokenized_lines = [] + for line in real_lines: + tokenized = [] + join_word = False + for word in line.split(' '): + if join_word: + word = tokenized.pop() + ' ' + word + + # Detect word joins. + join_word = word.endswith('\\') + # Strip escape character. + if join_word: + word = word[:-1] + + # Detect `:` at the end of a word. + if word.endswith(':'): + tokenized.append(word[:-1]) + word = word[-1] + + # Add word to the tokenized set. + tokenized.append(word) + + tokenized_lines.append(tokenized) + + # Parse. + ast = [] + for line in tokenized_lines: + kind = None + for token in line: + if token == ':': + kind = 'dependency' + elif token == '+=': + kind = 'append' + if line == ['']: + kind = 'empty' + + if kind is None: + _report_error('unknown line kind: %s' % line) + + line_parse = [] + if kind == 'dependency': + after_colon = False + for token in line: + if token == ':': + after_colon = True + elif after_colon: + line_parse.append(Input(token)) + else: + line_parse.append(Output(token)) + elif kind == 'append': + after_op = False + for token in line: + if token == '+=': + after_op = True + elif after_op: + line_parse.append(Word(token)) + else: + line_parse.append(Variable(token)) + + ast.append(line_parse) + + for node in ast: + for token in node: + if expect_input is not None: + # If the input is found, clear the expectation. + if isinstance(token, Input) and token.path == expect_input: + expect_input = None + + result = True + if expect_input: + _report_error('missing input: %s' % expect_input) + result = False + + return result + + +if __name__ == '__main__': + import sys + + depfile = None + have_expect = False + expect_input = None + + # Parse arguments. + args = sys.argv[1:] + while args: + # Take an argument. + arg = args.pop(0) + + # Flag to change how errors are reported. + if arg == '-A' or arg == '--all': + ALL_ERRORS = True + # Required arguments. + elif arg == '-d' or arg == '--depfile': + depfile = args.pop(0) + elif arg == '-i' or arg == '--expect-input': + expect_input = args.pop(0) + have_expect = True + + # Validate that we have the required arguments. + if depfile is None: + raise RuntimeError('missing "depfile" file') + if have_expect is None: + raise RuntimeError('missing an "expect" argument') + + # Do the actual work. + try: + is_ok = validate_depfile(depfile, expect_input=expect_input) + except BaseException as e: + _report_error('exception: %s' % e) + + # Fail if errors are found. + if not is_ok: + sys.exit(1) diff --git a/gcc/testsuite/lib/modules.exp b/gcc/testsuite/lib/modules.exp index d466a73cbe1..e24ee7618d9 100644 --- a/gcc/testsuite/lib/modules.exp +++ b/gcc/testsuite/lib/modules.exp @@ -69,3 +69,32 @@ proc run-check-p1689-valid { depfile template } { clean-p1689 $testcase } + +proc run-check-module-dep { depfile flag expected } { + global srcdir subdir + # Extract the test file name from the arguments. + set testcase [file rootname [file tail $depfile]] + + verbose "Verifying dependencies for $testcase in $srcdir/$subdir" 2 + set testcase [remote_download host $testcase] + + set pytest_script "test-depfile.py" + if { ![check_effective_target_recent_python3] } { + unsupported "$pytest_script python3 is missing" + return + } + + verbose "running script test-depfile.py" 1 + spawn -noecho python3 $srcdir/$subdir/$pytest_script --all --depfile $depfile $flag $expected + + expect { + -re "ERROR: (\[^\r\n\]*)" { + fail $expect_out(0,string) + exp_continue + } + } +} + +proc run-check-module-dep-expect-input { depfile expected } { + run-check-module-dep $depfile "--expect-input" $expected +}