From patchwork Thu Nov 16 14:28:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Malcolm X-Patchwork-Id: 165837 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp3248428vqg; Thu, 16 Nov 2023 06:29:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IHGCh9FZ+QebmggffSm/9ieCl2cz5NmzmDk8oEP2GwImJNsrEbju7tnSAKYvqi4P3RIZ8hE X-Received: by 2002:a05:620a:269a:b0:77a:1d5b:2ccb with SMTP id c26-20020a05620a269a00b0077a1d5b2ccbmr9538494qkp.6.1700144993192; Thu, 16 Nov 2023 06:29:53 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1700144993; cv=pass; d=google.com; s=arc-20160816; b=qe0/406BBSg4gI8Wo+cLydnLFBFZ7pEKdrhSP3flrDWfj5iV7NBL+bf0jUMJKboWKI hcDX0fCqBdVp6FEqGu1IQrpXyV8MnjboYdLnfOBbHv9vvNAW3BSLuZIK4P0v9MIuQkDU LbpsizgSYyhU5Rf93SZqqFDEOYn44ckFkArM2BK40jga2fT0hGQcs7OlPHWo5ZuusPnP 9AUS7xl+Zgw62Ftdc39DNM/TEyvPKwMD8iCZJsvdCA9LmbQpcyBqiO1/YJyOVcIdYS1v XwmMNuo/Kk2wvAP9kH7U9ETLF3wEnfFZYLEYRKn7ipWrDiPzdmCBXI2E2ghLsFzFy3Wh 3s0w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=4G77EQk84/NR0aFxYO+B1j7QaLCQZ/umcny4HPISna4=; fh=uGqK9H3hQo0aQWEcGijEy2GFuf/ntLTfnhK7DtoWgU0=; b=hZGGhxizfvtY87eyk/r2gGlt0CaL08uwRnSYdAo/OSriKKiwBi14U/T5RWBT5QIded V9pc5aC5ZIplzdVObFHKXL2LuzVN4d2cQWNRzlBgsctAdr+DpygxPSqa/wt9wLWWncGK kLlW/CWFOChF0HSd5Zn0HEzMuP+G6k0N7XjSeSmzNOebtj7QIiSzIMZyaKFRsjRSmKgf KsZzGazGv250ABam6crLai4O6v5pYKrMo9c4aR9cnGllTYoNp27tWKUyi9UvK4WnFIOE a4sf3UWtOq1YDksJDla1CoGtKwFg9Qza2WZiQ3tZDpQOqn/qzZ/gBKigJil+iO5S2Xi/ KjFA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=B+Z6pfXy; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id qp1-20020a05620a388100b00775c180abd3si10730140qkn.13.2023.11.16.06.29.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Nov 2023 06:29:53 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=B+Z6pfXy; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 85D50385DC05 for ; Thu, 16 Nov 2023 14:29:35 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 968793856975 for ; Thu, 16 Nov 2023 14:29:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 968793856975 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 968793856975 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700144946; cv=none; b=KLZq7ZGfYGnyyYudBoFfEzN2lsZCD+Q5I32L9S8y7t9e8Yn1c24WsEob7an67D4u7oSkt4FRfsJgBlCECaDvpbAokIHTMQP1cJI2Yrjjpon/XOGpQE446EfTlL0ETEkc80uZKyrrMmQzKCG3mLi29Ao2kuQ0Lr3FfSqDZPwQ3sg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700144946; c=relaxed/simple; bh=cbgK+CLGdjecojGpXfVsKPuyzae7IDVkbQScDJFDnOw=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=qmGqXbf7GQBwT8SZnTPwjPLO2qvkk4IEIhTP0TVTULRLU4f9IWn5khF3s2e1vCphDVSw7mMb5O3QZXzvDr9dKVg34RKlAc96DQvUhg9VVrWPF9GLoRja74tUDUppF+dQq9ldB0YUgkDahEioa0ioSSvPEKofj22x0p+UXKaCiv0= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700144943; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4G77EQk84/NR0aFxYO+B1j7QaLCQZ/umcny4HPISna4=; b=B+Z6pfXyGMoTGIzf7L7MZ1ju7pWM4thXnyeE5jstjSKtZOdt4cZ6yJxhsaYea1mey9dAOB X8xqARVTXYGkkwktQ4RDUfvhKSIkc7nL+CSrs/V2l0tMY62gfVhxuipjdYoTVyfQsOP6hq Z3IvM8ISv4KEP55Ft5jRmaleiLPGLdE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-279-gWSL-nBMM96-TMuTACaPFw-1; Thu, 16 Nov 2023 09:29:01 -0500 X-MC-Unique: gWSL-nBMM96-TMuTACaPFw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A0201101A598; Thu, 16 Nov 2023 14:29:01 +0000 (UTC) Received: from t14s.localdomain.com (unknown [10.22.10.115]) by smtp.corp.redhat.com (Postfix) with ESMTP id 68E141121306; Thu, 16 Nov 2023 14:29:01 +0000 (UTC) From: David Malcolm To: Joseph Myers Cc: gcc-patches@gcc.gnu.org, David Malcolm Subject: [PATCH 1/4] options: add gcc/regenerate-opt-urls.py Date: Thu, 16 Nov 2023 09:28:55 -0500 Message-Id: <20231116142858.3996740-2-dmalcolm@redhat.com> In-Reply-To: <20231116142858.3996740-1-dmalcolm@redhat.com> References: <9e5d6710-84e-15b9-d955-5381a6dc18a9@codesourcery.com> <20231116142858.3996740-1-dmalcolm@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782731236263431451 X-GMAIL-MSGID: 1782731236263431451 Changed in v2: - added convenience targets to Makefile for regenerating the .opt.urls files, and for running unit tests for the generation code - parse gdc and gfortran documentation, and create LangUrlSuffix_{lang} directives for language-specific URLs. - add documentation to sourcebuild.texi gcc/ChangeLog: * Makefile.in (regenerate-opt-urls): New target. (regenerate-opt-urls-unit-test): New target. * doc/options.texi (Option properties): Add UrlSuffix and description of regenerate-opt-urls.py. Add LangUrlSuffix_*. * doc/sourcebuild.texi (Anatomy of a Target Back End): Add reference to regenerate-opt-urls.py's TARGET_SPECIFIC_PAGES. * regenerate-opt-urls.py: New file. --- gcc/Makefile.in | 11 + gcc/doc/options.texi | 26 +++ gcc/doc/sourcebuild.texi | 4 + gcc/regenerate-opt-urls.py | 408 +++++++++++++++++++++++++++++++++++++ 4 files changed, 449 insertions(+) create mode 100755 gcc/regenerate-opt-urls.py diff --git a/gcc/Makefile.in b/gcc/Makefile.in index 7f2df4b5925f..f3b79b8a2663 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -3606,6 +3606,17 @@ $(build_htmldir)/gccinstall/index.html: $(TEXI_GCCINSTALL_FILES) DESTDIR=$(@D) \ $(SHELL) $(srcdir)/doc/install.texi2html +# Regenerate the .opt.urls files from the generated html, and from the .opt +# files. +.PHONY: regenerate-opt-urls +regenerate-opt-urls: + $(srcdir)/regenerate-opt-urls.py $(build_htmldir) $(shell dirname $(srcdir)) + +# Run the unit tests for regenerate-opt-urls.py +.PHONY: regenerate-opt-urls-unit-test +regenerate-opt-urls-unit-test: + $(srcdir)/regenerate-opt-urls.py $(build_htmldir) $(shell dirname $(srcdir)) --unit-test + MANFILES = doc/gcov.1 doc/cpp.1 doc/gcc.1 doc/gfdl.7 doc/gpl.7 \ doc/fsf-funding.7 doc/gcov-tool.1 doc/gcov-dump.1 \ $(if $(filter yes,@enable_lto@),doc/lto-dump.1) diff --git a/gcc/doc/options.texi b/gcc/doc/options.texi index 715f0a1479c7..37d7ecc1477d 100644 --- a/gcc/doc/options.texi +++ b/gcc/doc/options.texi @@ -597,4 +597,30 @@ This warning option corresponds to @code{cpplib.h} warning reason code @var{CPP_W_Enum}. This should only be used for warning options of the C-family front-ends. +@item UrlSuffix(@var{url_suffix}) +Adjacent to each human-written @code{.opt} file in the source tree is +a corresponding file with a @code{.opt.urls} extension. These files +contain @code{UrlSuffix} directives giving the ending part of the URL +for the documentation of the option, such as: + +@smallexample +Wabi-tag +UrlSuffix(gcc/C_002b_002b-Dialect-Options.html#index-Wabi-tag) +@end smallexample + +These URL suffixes are relative to @code{DOCUMENTATION_ROOT_URL}. + +There files are generated from the @code{.opt} files and the generated +HTML documentation by @code{regenerate-opt-urls.py}, and should be +regenerated when adding new options, via manually invoking +@code{make regenerate-opt-urls}. + +@item LangUrlSuffix_@var{lang}(@var{url_suffix}) +In addition to @code{UrlSuffix} directives, @code{regenerate-opt-urls.py} +can generate language-specific URLs, such as: + +@smallexample +LangUrlSuffix_D(gdc/Code-Generation.html#index-MMD) +@end smallexample + @end table diff --git a/gcc/doc/sourcebuild.texi b/gcc/doc/sourcebuild.texi index eaa75f00f5ce..f51db17a086c 100644 --- a/gcc/doc/sourcebuild.texi +++ b/gcc/doc/sourcebuild.texi @@ -813,6 +813,10 @@ options supported by this target (@pxref{Run-time Target, , Run-time Target Specification}). This means both entries in the summary table of options and details of the individual options. @item +An entry in @file{gcc/regenerate-opt-urls.py}'s TARGET_SPECIFIC_PAGES +dictionary mapping from target-specific HTML documentation pages +to the target specific source directory. +@item Documentation in @file{gcc/doc/extend.texi} for any target-specific attributes supported (@pxref{Target Attributes, , Defining target-specific uses of @code{__attribute__}}), including where the diff --git a/gcc/regenerate-opt-urls.py b/gcc/regenerate-opt-urls.py new file mode 100755 index 000000000000..b123fc57c7b9 --- /dev/null +++ b/gcc/regenerate-opt-urls.py @@ -0,0 +1,408 @@ +#!/usr/bin/env python3 + +# Copyright (C) 2023 Free Software Foundation, Inc. +# +# Script to regenerate FOO.opt.urls files for each FOO.opt in the +# source tree. +# +# This file is part of GCC. +# +# GCC is free software; you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free +# Software Foundation; either version 3, or (at your option) any later +# version. +# +# GCC is distributed in the hope that it will be useful, but WITHOUT ANY +# WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# for more details. +# +# You should have received a copy of the GNU General Public License +# along with GCC; see the file COPYING3. If not see +# . */ + +DESCRIPTION = """ +Parses the generated HTML (from "make html") to locate anchors +for options, then parses the .opt files within the source tree, +and generates a .opt.urls in the source tree for each .opt file, +giving URLs for each option, where it can. + +Usage (from build/gcc subdirectory): + ../../src/gcc/regenerate-opt-urls.py HTML/gcc-14.0.0/ ../../src + +To run unit tests: + ../../src/gcc/regenerate-opt-urls.py HTML/gcc-14.0.0/ ../../src --unit-test +""" + +import argparse +import json +import os +from pathlib import Path +from pprint import pprint +import sys +import re +import unittest + +def canonicalize_option_name(option_name): + if option_name.endswith('='): + option_name = option_name[0:-1] + return option_name + + +def canonicalize_url_suffix(url_suffix): + """ + Various options have anchors for both the positive and + negative form. For example -Wcpp has both: + 'gcc/Warning-Options.html#index-Wno-cpp' + 'gcc/Warning-Options.html#index-Wcpp' + + Return a canonicalized version of the url_suffix that + strips out any "no-" prefixes, for use in deduplication. + Note that the resulting url suffix might not correspond to + an actual anchor in the HTML. + """ + url_suffix = re.sub('index-Wno-', 'index-W', url_suffix) + url_suffix = re.sub('index-fno-', 'index-f', url_suffix) + url_suffix = re.sub('_003d$', '', url_suffix) + url_suffix = re.sub('-([0-9]+)$', '', url_suffix) + return url_suffix + + +class Index: + def __init__(self): + # Map from language (or None) to map from option name to set of URL suffixes + self.entries = {} + + def add_entry(self, matched_text, url_suffix, language, verbose=False): + # TODO: use language + if 'Attributes.html' in url_suffix: + return + matched_text = canonicalize_option_name(matched_text) + if language not in self.entries: + self.entries[language] = {} + per_lang_entries = self.entries[language] + if matched_text in per_lang_entries: + # Partition by canonicalized url_suffixes; add the + # first url_suffix in each such partition. + c_new = canonicalize_url_suffix(url_suffix) + for entry in per_lang_entries[matched_text]: + c_entry = canonicalize_url_suffix(entry) + if c_new == c_entry: + return + per_lang_entries[matched_text].add(url_suffix) + else: + per_lang_entries[matched_text] = set([url_suffix]) + + def get_languages(self): + return self.entries.keys() + + def get_url_suffixes(self, text, language=None): + text = canonicalize_option_name(text) + per_lang_entries = self.entries.get(language) + if per_lang_entries: + return per_lang_entries.get(text) + + def parse_option_index(self, input_filename, language, verbose=False): + with open(input_filename) as f: + dirname = input_filename.parent.name + for line in f: + self.parse_html_line_option_index(dirname, line, language, verbose) + + def parse_html_line_option_index(self, dirname, line, language, verbose=False): + if verbose: + print(repr(line)) + + # Update for this in the GCC website's bin/preprocess process_html_file: + # | sed -e 's/_002d/-/g' -e 's/_002a/*/g' \ + line = line.replace('_002d', '-') + line = line.replace('_002a', '*') + + # e.g. fmodulo-sched + m = re.search(r'([\S]+)', line) + if not m: + return + if verbose: + print(m.groups()) + url_suffix, index_text = m.groups() + #print(f'{url_suffix=} {index_text=}') + option = '-' + index_text + + # Strip off "no-" prefixes from options + if option[:5] == '-Wno-': + option = '-W' + option[5:] + if option[:5] == '-fno-': + option = '-f' + option[5:] + + url_suffix = dirname + '/' + url_suffix + self.add_entry(option, url_suffix, language, verbose) + + +class TestParsingIndex(unittest.TestCase): + def test_parse_line(self): + index = Index() + index.parse_html_line_option_index('gcc', + 'fmodulo-sched', + None) + self.assertEqual(index.get_url_suffixes('-fmodulo-sched'), + {'gcc/Optimize-Options.html#index-fmodulo-sched'}) + + def test_negated_flag(self): + index = Index() + index.parse_html_line_option_index('gcc', + 'fno-analyzer: Static Analyzer Options\n', + None) + self.assertEqual(index.get_url_suffixes('-fno-analyzer'), None) + self.assertEqual(index.get_url_suffixes('-fanalyzer'), + {'gcc/Static-Analyzer-Options.html#index-fno-analyzer'}) + + def test_negated_warning(self): + index = Index() + index.parse_html_line_option_index('gcc', + 'Wno-alloca: Warning Options\n', + None) + self.assertEqual(index.get_url_suffixes('-Wno-alloca'), + None) + self.assertEqual(index.get_url_suffixes('-Walloca'), + {'gcc/Warning-Options.html#index-Wno-alloca'}) + + def test_parse_option_index(self): + index = Index() + index.parse_option_index(INPUT_HTML_PATH / 'gcc/Option-Index.html', + language=None) + self.assertEqual(index.get_url_suffixes('-fmodulo-sched'), + {'gcc/Optimize-Options.html#index-fmodulo-sched'}) + self.assertEqual(index.get_url_suffixes('-O'), + {'gcc/Optimize-Options.html#index-O'}) + self.assertEqual(index.get_url_suffixes('-O0'), + {'gcc/Optimize-Options.html#index-O0'}) + self.assertEqual(index.get_url_suffixes('-Wframe-larger-than='), + {'gcc/Warning-Options.html#index-Wframe-larger-than_003d'}) + + # Check an option with duplicates: '-march' + # The url_suffixes will be of the form + # 'gcc/HPPA-Options.html#index-march-5', + # 'gcc/LoongArch-Options.html#index-march-7', + # etc, where the trailing number is, unfortunately, likely to + # change from release to release. + # Replace them with 'NN' for the purpose of this test: + em_arch_url_suffixes = [re.sub('(-[0-9]+)', '-NN', s) + for s in index.get_url_suffixes('-march')] + if 0: + print(em_arch_url_suffixes) + self.assertIn('gcc/ARM-Options.html#index-march-NN', em_arch_url_suffixes) + self.assertIn('gcc/x86-Options.html#index-march-NN', em_arch_url_suffixes) + + self.assertEqual(index.get_url_suffixes('-Wcpp'), + {'gcc/Warning-Options.html#index-Wcpp'}) + + self.assertNotEqual(index.get_url_suffixes('-march'), None) + self.assertNotEqual(index.get_url_suffixes('-march='), None) + +class OptFile: + def __init__(self, opt_path, rel_path): + """ + Parse a .opt file. Similar to opt-gather.awk. + """ + self.rel_path = rel_path + assert rel_path.startswith('gcc') + # self.filename = os.path.basename(path) + self.records = [] + with open(opt_path) as f: + flag = 0 + for line in f: + #print(repr(line)) + if re.match(r'[ \t]*(;|$)', line): + flag = 0 + else: + if flag == 0: + self.records.append([line]) + flag = 1 + else: + self.records[-1].append(line) + +# Mapping from target-specific page to subdirectory containing .opt files +# documented on that page. + +TARGET_SPECIFIC_PAGES = { + 'gcc/AArch64-Options.html' : 'gcc/config/aarch64/', + 'gcc/AMD-GCN-Options.html' : 'gcc/config/gcn/', + 'gcc/ARC-Options.html' : 'gcc/config/arc/', + 'gcc/ARC-Options.html' : 'gcc/config/arc/', + 'gcc/ARM-Options.html' : 'gcc/config/arm/', + 'gcc/AVR-Options.html' : 'gcc/config/avr/', + 'gcc/Adapteva-Epiphany-Options.html' : 'gcc/config/epiphany/', + 'gcc/Blackfin-Options.html' : 'gcc/config/bfin/', + 'gcc/C-SKY-Options.html' : 'gcc/config/csky/', + 'gcc/C6X-Options.html' : 'gcc/config/c6x/', + 'gcc/CRIS-Options.html' : 'gcc/config/cris/', + 'gcc/DEC-Alpha-Options.html' : 'gcc/config/alpha/', + 'gcc/FR30-Options.html' : 'gcc/config/fr30/', + 'gcc/FRV-Options.html' : 'gcc/config/frv/', + 'gcc/FT32-Options.html' : 'gcc/config/ft32/', + 'gcc/H8_002f300-Options.html' : 'gcc/config/h8300/', + 'gcc/HPPA-Options.html' : 'gcc/config/pa/', + 'gcc/IA-64-Options.html' : 'gcc/config/ia64/', + 'gcc/LoongArch-Options.html' : 'gcc/config/loongarch/', + 'gcc/M32C-Options.html' : 'gcc/config/m32c/', + 'gcc/M32R_002fD-Options.html' : 'gcc/config/m32r/', + 'gcc/M680x0-Options.html' : 'gcc/config/m68k/', + 'gcc/MCore-Options.html' : 'gcc/config/mcore/', + 'gcc/MIPS-Options.html' : 'gcc/config/mips/', + 'gcc/MMIX-Options.html' : 'gcc/config/mmix/', + 'gcc/MN10300-Options.html' : 'gcc/config/mn10300/', + 'gcc/MSP430-Options.html' : 'gcc/config/msp430/', + 'gcc/MicroBlaze-Options.html' : 'gcc/config/microblaze/', + 'gcc/Moxie-Options.html' : 'gcc/config/moxie/', + 'gcc/NDS32-Options.html' : 'gcc/config/nds32/', + 'gcc/Nios-II-Options.html' : 'gcc/config/nios2/', + 'gcc/Nvidia-PTX-Options.html' : 'gcc/config/nvptx/', + 'gcc/OpenRISC-Options.html' : 'gcc/config/or1k/', + 'gcc/PDP-11-Options.html' : 'gcc/config/pdp11', + 'gcc/PRU-Options.html' : 'gcc/config/pru/', + 'gcc/RISC-V-Options.html' : 'gcc/config/riscv/', + 'gcc/RL78-Options.html' : 'gcc/config/rl78/', + 'gcc/RS_002f6000-and-PowerPC-Options.html' : 'gcc/config/rs6000/', + 'gcc/RX-Options.html' : 'gcc/config/rx/', + 'gcc/SH-Options.html' : 'gcc/config/sh/', + 'gcc/SPARC-Options.html' : 'gcc/config/sparc/', + 'gcc/S_002f390-and-zSeries-Options.html' : 'gcc/config/s390', + 'gcc/V850-Options.html' : 'gcc/config/vax/', + 'gcc/VAX-Options.html' : 'gcc/config/v850/', + 'gcc/Visium-Options.html' : 'gcc/config/visium/', + 'gcc/Xstormy16-Options.html' : 'gcc/config/stormy16/', + 'gcc/Xtensa-Options.html' : 'gcc/config/xtensa/', + 'gcc/eBPF-Options.html' : 'gcc/config/bpf/', + 'gcc/x86-Options.html' : 'gcc/config/i386/', +} + +def target_specific(url_suffix): + for page_prefix, subdir in TARGET_SPECIFIC_PAGES.items(): + if url_suffix.startswith(page_prefix): + return subdir + +def filter_urlsuffixes_for_optfile(optfile, url_suffixes): + """ + Filter out target-specific options for the wrong target. + """ + result = set() + for url_suffix in url_suffixes: + subdir = target_specific(url_suffix) + if subdir: + if 0: + print(f'{optfile.rel_path=}') + print(f'{url_suffixes=}') + print(f'{subdir=}') + if not optfile.rel_path.startswith(subdir): + # Skip this + continue + result.add(url_suffix) + return result + + +class TestFiltering(unittest.TestCase): + def test_target_specific(self): + self.assertEqual(target_specific('gcc/Preprocessor-Options.html#index-A'), + None) + self.assertEqual(target_specific('gcc/MMIX-Options.html#index-mknuthdiv'), + 'gcc/config/mmix/') + + def test_filter(self): + s = {'gcc/MIPS-Options.html#index-munaligned-access-1', + 'gcc/ARM-Options.html#index-munaligned-access'} + arm_optfile = OptFile('/dev/null', 'gcc/config/arm/arm.opt') + mips_optfile = OptFile('/dev/null', 'gcc/config/mips/mips.opt') + self.assertEqual( + filter_urlsuffixes_for_optfile(arm_optfile, s), + {'gcc/ARM-Options.html#index-munaligned-access'}) + self.assertEqual( + filter_urlsuffixes_for_optfile(mips_optfile, s), + {'gcc/MIPS-Options.html#index-munaligned-access-1'}) + + +def write_url_file(index, optfile, dstfile): + dstfile.write('; Autogenerated by regenerate-opt-urls.py from %s' + ' and generated HTML\n\n' + % optfile.rel_path) + for record in optfile.records: + opt = '-' + record[0].strip() + if 0: + dstfile.write('; entry for %s\n' % record) + dstfile.write('; opt=%r\n' % opt) + url_suffixes_per_lang = {} + count = 0 + for lang in index.get_languages(): + this_lang_suffixes = index.get_url_suffixes(opt, language=lang) + url_suffixes_per_lang[lang] = this_lang_suffixes + if this_lang_suffixes: + count += len(this_lang_suffixes) + if not count: + continue + directives = [] + for lang in index.get_languages(): + if lang: + directive = 'LangUrlSuffix_%s for %r' % (lang, opt[1:]) + else: + directive = 'UrlSuffix for %r' % opt[1:] + url_suffixes = url_suffixes_per_lang[lang] + if 0: + dstfile.write('; lang=%r url_suffixes=%r\n' % (lang, url_suffixes)) + if url_suffixes: + url_suffixes = filter_urlsuffixes_for_optfile(optfile, url_suffixes) + if url_suffixes: + if len(url_suffixes) == 1: + if lang: + directives.append('LangUrlSuffix_%s(%s)' % (lang, list(url_suffixes)[0])) + else: + directives.append('UrlSuffix(%s)' % list(url_suffixes)[0]) + else: + dstfile.write('; skipping %s due to multiple URLs:\n' + % directive) + for u in sorted(url_suffixes): + dstfile.write('; duplicate: %r\n' % u) + else: + dstfile.write('; skipping %s due to finding no URLs\n' + % directive) + if directives: + dstfile.write('%s\n' % opt[1:]) + dstfile.write(' '.join(directives) + '\n') + dstfile.write('\n') + + +def main(args): + index = Index() + index.parse_option_index(args.base_html_dir / 'gcc/Option-Index.html', + language=None) + index.parse_option_index(args.base_html_dir / 'gdc/Option-Index.html', + language='D') + index.parse_option_index(args.base_html_dir / 'gfortran/Option-Index.html', + language='Fortran') + if 0: + pprint(index.entries) + for root, dirs, files in os.walk(args.src_gcc_dir): + for f in files: + if f.endswith('.opt'): + opt_path = os.path.join(root, f) + rel_path = os.path.relpath(opt_path, args.src_gcc_dir) + optfile = OptFile(opt_path, rel_path) + if 0: + pprint(optfile.path) + pprint(optfile.records) + dstname = f + '.urls' + urlfile = os.path.join(root, dstname) + with open(urlfile, 'w') as dstfile: + write_url_file(index, optfile, dstfile) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=DESCRIPTION, + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('base_html_dir', type=Path) + parser.add_argument('src_gcc_dir', type=Path) + parser.add_argument('--unit-test', action='store_true') + args = parser.parse_args() + + if args.unit_test: + INPUT_HTML_PATH = args.base_html_dir + unittest.main(argv=[sys.argv[0], '-v']) + else: + main(args)