[Rust,front-end,v3,15/46] gccrs: Add expansion pass for the Rust front-end
Checks
Commit Message
From: Arthur Cohen <arthur.cohen@embecosm.com>
The expansion pass is responsible for two actions on our AST:
1. Expanding macro calls
2. Performing conditional compilation
Calls to macros should be checked and expanded into an AST fragment based on
the context they've been called in. This is similar to token substitution, with
a lot of intricacies and checks being performed. A single invocation can result
in an AST fragment containing multiple statements or multiple expressions,
which need to be handled as well. Furthermore, Rust macros can contain
repetitions relying on Kleine operators, similar to regular expression
patterns, that also need to be expanded properly.
Finally, Rust code can be hidden behind `cfg` directives, which allow the user
to perform conditional compilation. If a `cfg` predicate is not met, the
expression or statement it refers to should be marked for strip and removed
from the AST.
Co-authored-by: Philip Herron <philip.herron@embecosm.com>
Co-authored-by: The Other <simplytheother@gmail.com>
---
gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++
gcc/rust/expand/rust-attribute-visitor.h | 316 ++
gcc/rust/expand/rust-macro-builtins.cc | 484 +++
gcc/rust/expand/rust-macro-builtins.h | 107 +
gcc/rust/expand/rust-macro-expand.cc | 1012 +++++
gcc/rust/expand/rust-macro-expand.h | 366 ++
gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 +
gcc/rust/expand/rust-macro-invoc-lexer.h | 64 +
gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++
gcc/rust/expand/rust-macro-substitute-ctx.h | 93 +
10 files changed, 6228 insertions(+)
create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc
create mode 100644 gcc/rust/expand/rust-attribute-visitor.h
create mode 100644 gcc/rust/expand/rust-macro-builtins.cc
create mode 100644 gcc/rust/expand/rust-macro-builtins.h
create mode 100644 gcc/rust/expand/rust-macro-expand.cc
create mode 100644 gcc/rust/expand/rust-macro-expand.h
create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc
create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h
create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc
create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h
Comments
On Wed, Oct 26, 2022 at 10:28 AM <arthur.cohen@embecosm.com> wrote:
>
> From: Arthur Cohen <arthur.cohen@embecosm.com>
>
> The expansion pass is responsible for two actions on our AST:
>
> 1. Expanding macro calls
> 2. Performing conditional compilation
>
> Calls to macros should be checked and expanded into an AST fragment based on
> the context they've been called in. This is similar to token substitution, with
> a lot of intricacies and checks being performed. A single invocation can result
> in an AST fragment containing multiple statements or multiple expressions,
> which need to be handled as well. Furthermore, Rust macros can contain
> repetitions relying on Kleine operators, similar to regular expression
> patterns, that also need to be expanded properly.
>
> Finally, Rust code can be hidden behind `cfg` directives, which allow the user
> to perform conditional compilation. If a `cfg` predicate is not met, the
> expression or statement it refers to should be marked for strip and removed
> from the AST.
I'm just picking one of the purely rust frontend specific patches as example and
do not plan to look on others - I don't think meaningful review can happen by
somebody not deeply understanding rust or this frontend can be done which means
approval of those parts shouldn't be needed but would be covered (even
the initial
contribution) by your maintainership of the code.
The commit comment above is quite helpful in getting an idea of what this part
of the rust frontend is doing. I'd appreciate if something along these lines
would be added to each of the individual source files (preferably the
implementation,
not the header). Documentation of the sub-directory structure would
also be nice,
we have (had ..) sourcebuild.texi which contains some of that but not really for
specific frontends (sofar).
Some comments on this particular patch that probably apply to all others as well
inline below.
> Co-authored-by: Philip Herron <philip.herron@embecosm.com>
> Co-authored-by: The Other <simplytheother@gmail.com>
> ---
> gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++
> gcc/rust/expand/rust-attribute-visitor.h | 316 ++
> gcc/rust/expand/rust-macro-builtins.cc | 484 +++
> gcc/rust/expand/rust-macro-builtins.h | 107 +
> gcc/rust/expand/rust-macro-expand.cc | 1012 +++++
> gcc/rust/expand/rust-macro-expand.h | 366 ++
> gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 +
> gcc/rust/expand/rust-macro-invoc-lexer.h | 64 +
> gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++
> gcc/rust/expand/rust-macro-substitute-ctx.h | 93 +
> 10 files changed, 6228 insertions(+)
> create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc
> create mode 100644 gcc/rust/expand/rust-attribute-visitor.h
> create mode 100644 gcc/rust/expand/rust-macro-builtins.cc
> create mode 100644 gcc/rust/expand/rust-macro-builtins.h
> create mode 100644 gcc/rust/expand/rust-macro-expand.cc
> create mode 100644 gcc/rust/expand/rust-macro-expand.h
> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc
> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h
> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc
> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h
>
> diff --git a/gcc/rust/expand/rust-attribute-visitor.cc b/gcc/rust/expand/rust-attribute-visitor.cc
> new file mode 100644
> index 00000000000..8016f9430eb
> --- /dev/null
> +++ b/gcc/rust/expand/rust-attribute-visitor.cc
> @@ -0,0 +1,3445 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "rust-attribute-visitor.h"
> +#include "rust-session-manager.h"
> +
> +namespace Rust {
> +
> +// Visitor used to expand attributes.
> +void
> +AttrVisitor::expand_struct_fields (std::vector<AST::StructField> &fields)
> +{
> + for (auto it = fields.begin (); it != fields.end ();)
> + {
> + auto &field = *it;
> +
> + auto &field_attrs = field.get_outer_attrs ();
> + expander.expand_cfg_attrs (field_attrs);
> + if (expander.fails_cfg_with_expand (field_attrs))
> + {
> + it = fields.erase (it);
> + continue;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // expand sub-types of type, but can't strip type itself
> + auto &type = field.get_field_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + // if nothing else happens, increment
> + ++it;
> + }
> +}
> +
The coding conventions warrant a function level comment for each function.
If there's anything meaningful to say (in this case "// expand tuple fields"
wouldn't be meaningful) please consider adding such to all functions. In
my view every function that doesn't fit in a 80x24 terminal warrants one.
> +void
> +AttrVisitor::expand_tuple_fields (std::vector<AST::TupleField> &fields)
> +{
> + for (auto it = fields.begin (); it != fields.end ();)
> + {
> + auto &field = *it;
> +
> + auto &field_attrs = field.get_outer_attrs ();
> + expander.expand_cfg_attrs (field_attrs);
> + if (expander.fails_cfg_with_expand (field_attrs))
> + {
> + it = fields.erase (it);
> + continue;
> + }
> +
> + // expand sub-types of type, but can't strip type itself
> + auto &type = field.get_field_type ();
> + type->accept_vis (*this);
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
When reading the other patches I wondered whether rust_error_at are
user-facing diagnostics or internal errors?
> +
> + // if nothing else happens, increment
> + ++it;
> + }
> +}
> +
> +void
> +AttrVisitor::expand_function_params (std::vector<AST::FunctionParam> ¶ms)
> +{
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + for (auto it = params.begin (); it != params.end ();)
> + {
> + auto ¶m = *it;
> +
> + auto ¶m_attrs = param.get_outer_attrs ();
> + expander.expand_cfg_attrs (param_attrs);
> + if (expander.fails_cfg_with_expand (param_attrs))
> + {
> + it = params.erase (it);
> + continue;
> + }
> +
> + // TODO: should an unwanted strip lead to break out of loop?
> + auto &pattern = param.get_pattern ();
'auto' is used a lot in the frontend, it does make reading code harder because
critical information on context is lost. Here I can at least 2nd guess
'param' being of type AST::FunctionParam and thus find get_pattern (),
but I will have to find that to be able to find accept_vis () below.
In my mind 'auto' is useful when it hides information that's not used as in
for (auto it = ...)
{
AST::FunctionParam ¶m = *it;
but not so much for being lazy when typing.
At least with (some of) the uses the identifiers of the methods look
unique enough so grep will be of help, OTOH is_marked_for_strip
is defined a lot.
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> +
> + auto &type = param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + // increment
> + ++it;
> + }
> +
> + expander.pop_context ();
> +}
> +
> +void
> +AttrVisitor::expand_generic_args (AST::GenericArgs &args)
> +{
> + // lifetime args can't be expanded
> + // FIXME: Can we have macro invocations for lifetimes?
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // expand type args - strip sub-types only
> + for (auto &arg : args.get_generic_args ())
> + {
> + switch (arg.get_kind ())
> + {
> + case AST::GenericArg::Kind::Type: {
> + auto &type = arg.get_type ();
> + type->accept_vis (*this);
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> + break;
> + }
> + case AST::GenericArg::Kind::Const: {
> + auto &expr = arg.get_expression ();
> + expr->accept_vis (*this);
> + maybe_expand_expr (expr);
> +
> + if (expr->is_marked_for_strip ())
> + rust_error_at (expr->get_locus (),
> + "cannot strip expression in this position");
> + break;
> + }
> + default:
> + break;
> + // FIXME: Figure out what to do here if there is ambiguity. Since the
> + // resolver comes after the expansion, we need to figure out a way to
> + // strip ambiguous values here
> + // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg`
> + // or something. This would clean up this whole thing
> + }
> + }
> +
> + expander.pop_context ();
> +
> + // FIXME: Can we have macro invocations in generic type bindings?
> + // expand binding args - strip sub-types only
> + for (auto &binding : args.get_binding_args ())
> + {
> + auto &type = binding.get_type ();
> + type->accept_vis (*this);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> + }
> +}
> +
> +void
> +AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type)
> +{
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = path_type.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + expander.pop_context ();
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + if (path_type.has_as_clause ())
> + {
> + auto &type_path = path_type.get_as_type_path ();
> + visit (type_path);
> + if (type_path.is_marked_for_strip ())
> + rust_error_at (type_path.get_locus (),
> + "cannot strip type path in this position");
> + }
> +}
> +
> +void
> +AttrVisitor::AttrVisitor::expand_closure_params (
> + std::vector<AST::ClosureParam> ¶ms)
> +{
> + for (auto it = params.begin (); it != params.end ();)
> + {
> + auto ¶m = *it;
> +
> + auto ¶m_attrs = param.get_outer_attrs ();
> + expander.expand_cfg_attrs (param_attrs);
> + if (expander.fails_cfg_with_expand (param_attrs))
> + {
> + it = params.erase (it);
> + continue;
> + }
> +
> + auto &pattern = param.get_pattern ();
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> +
> + if (param.has_type_given ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> + auto &type = param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + // increment if found nothing else so far
> + ++it;
> + }
> +}
> +
> +void
> +AttrVisitor::expand_self_param (AST::SelfParam &self_param)
> +{
> + if (self_param.has_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> + auto &type = self_param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> + /* TODO: maybe check for invariants being violated - e.g. both type and
> + * lifetime? */
C-style comment where I see C++-style comments everywhere else.
> +}
> +
> +void
> +AttrVisitor::expand_where_clause (AST::WhereClause &where_clause)
> +{
> + // items cannot be stripped conceptually, so just accept visitor
> + for (auto &item : where_clause.get_items ())
> + item->accept_vis (*this);
> +}
> +
> +void
> +AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl)
> +{
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : decl.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* strip function parameters if required - this is specifically
> + * allowed by spec */
> + expand_function_params (decl.get_function_params ());
> +
> + if (decl.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = decl.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + if (decl.has_where_clause ())
> + expand_where_clause (decl.get_where_clause ());
> +}
> +
> +void
> +AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl)
> +{
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : decl.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* assuming you can't strip self param - wouldn't be a method
> + * anymore. spec allows outer attrs on self param, but doesn't
> + * specify whether cfg is used. */
> + expand_self_param (decl.get_self_param ());
> +
> + /* strip function parameters if required - this is specifically
> + * allowed by spec */
> + expand_function_params (decl.get_function_params ());
> +
> + if (decl.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = decl.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + if (decl.has_where_clause ())
> + expand_where_clause (decl.get_where_clause ());
> +}
> +
> +void
> +AttrVisitor::visit (AST::Token &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::DelimTokenTree &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::AttrInputMetaItemContainer &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::IdentifierExpr &ident_expr)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (ident_expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ()))
> + {
> + ident_expr.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::Lifetime &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::LifetimeParam &)
> +{
> + // supposedly does not require - cfg does nothing
> +}
> +void
> +AttrVisitor::visit (AST::ConstGenericParam &)
> +{
> + // likewise
> +}
> +
> +void
> +AttrVisitor::visit (AST::MacroInvocation ¯o_invoc)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ()))
> + {
> + macro_invoc.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip simple path
> +
> + // I don't think any macro token trees can be stripped in any way
> +
> + // TODO: maybe have cfg! macro stripping behaviour here?
> + expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ());
> +}
> +
> +void
> +AttrVisitor::visit (AST::PathInExpression &path)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (path.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
> + {
> + path.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &segment : path.get_segments ())
> + {
> + if (segment.has_generic_args ())
> + expand_generic_args (segment.get_generic_args ());
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TypePathSegment &)
> +{
> + // shouldn't require
> +}
> +void
> +AttrVisitor::visit (AST::TypePathSegmentGeneric &segment)
> +{
> + // TODO: strip inside generic args
> +
> + if (!segment.has_generic_args ())
> + return;
> +
> + expand_generic_args (segment.get_generic_args ());
> +}
> +void
> +AttrVisitor::visit (AST::TypePathSegmentFunction &segment)
> +{
> + auto &type_path_function = segment.get_type_path_function ();
> +
> + for (auto &type : type_path_function.get_params ())
> + {
> + type->accept_vis (*this);
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> + }
> +
> + if (type_path_function.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = type_path_function.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TypePath &path)
> +{
> + // this shouldn't strip any segments, but can strip inside them
> + for (auto &segment : path.get_segments ())
> + segment->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::QualifiedPathInExpression &path)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (path.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
> + {
> + path.mark_for_strip ();
> + return;
> + }
> +
> + expand_qualified_path_type (path.get_qualified_path_type ());
> +
> + for (auto &segment : path.get_segments ())
> + {
> + if (segment.has_generic_args ())
> + expand_generic_args (segment.get_generic_args ());
> + }
> +}
> +void
> +AttrVisitor::visit (AST::QualifiedPathInType &path)
> +{
> + expand_qualified_path_type (path.get_qualified_path_type ());
> +
> + // this shouldn't strip any segments, but can strip inside them
> + for (auto &segment : path.get_segments ())
> + segment->accept_vis (*this);
> +}
> +
> +void
> +AttrVisitor::visit (AST::LiteralExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::AttrInputLiteral &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::MetaItemLitExpr &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::MetaItemPathLit &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::BorrowExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &borrowed_expr = expr.get_borrowed_expr ();
> + borrowed_expr->accept_vis (*this);
> + if (borrowed_expr->is_marked_for_strip ())
> + rust_error_at (borrowed_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::DereferenceExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &dereferenced_expr = expr.get_dereferenced_expr ();
> + dereferenced_expr->accept_vis (*this);
> + if (dereferenced_expr->is_marked_for_strip ())
> + rust_error_at (dereferenced_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ErrorPropagationExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &propagating_expr = expr.get_propagating_expr ();
> + propagating_expr->accept_vis (*this);
> + if (propagating_expr->is_marked_for_strip ())
> + rust_error_at (propagating_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::NegationExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &negated_expr = expr.get_negated_expr ();
> + negated_expr->accept_vis (*this);
> + if (negated_expr->is_marked_for_strip ())
> + rust_error_at (negated_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &l_expr = expr.get_left_expr ();
> + l_expr->accept_vis (*this);
> + maybe_expand_expr (l_expr);
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &r_expr = expr.get_right_expr ();
> + r_expr->accept_vis (*this);
> + maybe_expand_expr (r_expr);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_left_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_left_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before binary op exprs");
> + if (expr.get_right_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_right_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ComparisonExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &l_expr = expr.get_left_expr ();
> + l_expr->accept_vis (*this);
> + maybe_expand_expr (l_expr);
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &r_expr = expr.get_right_expr ();
> + r_expr->accept_vis (*this);
> + maybe_expand_expr (r_expr);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_left_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_left_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before binary op exprs");
> + if (expr.get_right_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_right_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
vertical space between function definitions missing here (and above and below)
> +void
> +AttrVisitor::visit (AST::LazyBooleanExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &l_expr = expr.get_left_expr ();
> + l_expr->accept_vis (*this);
> + maybe_expand_expr (l_expr);
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &r_expr = expr.get_right_expr ();
> + r_expr->accept_vis (*this);
> + maybe_expand_expr (r_expr);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_left_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_left_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before binary op exprs");
> + if (expr.get_right_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_right_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::TypeCastExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * direct descendant expression, can strip ones below that */
> +
> + auto &casted_expr = expr.get_casted_expr ();
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + casted_expr->accept_vis (*this);
> +
> + // ensure that they are not marked for strip
> + if (casted_expr->is_marked_for_strip ())
> + rust_error_at (casted_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed before cast exprs");
> +
> + // TODO: strip sub-types of type
> + auto &type = expr.get_type_to_cast_to ();
> + type->accept_vis (*this);
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::AssignmentExpr &expr)
> +{
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &l_expr = expr.get_left_expr ();
> + l_expr->accept_vis (*this);
> + maybe_expand_expr (l_expr);
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &r_expr = expr.get_right_expr ();
> + r_expr->accept_vis (*this);
> + maybe_expand_expr (r_expr);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_left_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_left_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before binary op exprs");
> + if (expr.get_right_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_right_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::CompoundAssignmentExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &l_expr = expr.get_left_expr ();
> + l_expr->accept_vis (*this);
> + maybe_expand_expr (l_expr);
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &r_expr = expr.get_right_expr ();
> + r_expr->accept_vis (*this);
> + maybe_expand_expr (r_expr);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_left_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_left_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before binary op exprs");
> + if (expr.get_right_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_right_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::GroupedExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says these are inner
> + * attributes, not outer attributes of inner expr */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &inner_expr = expr.get_expr_in_parens ();
> + inner_expr->accept_vis (*this);
> + if (inner_expr->is_marked_for_strip ())
> + rust_error_at (inner_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ArrayElemsValues &elems)
> +{
> + /* apparently outer attributes are allowed in "elements of array
> + * expressions" according to spec */
> + expand_pointer_allow_strip (elems.get_values ());
> +}
> +void
> +AttrVisitor::visit (AST::ArrayElemsCopied &elems)
> +{
> + /* apparently outer attributes are allowed in "elements of array
> + * expressions" according to spec. on the other hand, it would not
> + * make conceptual sense to be able to remove either expression. As
> + * such, not implementing. TODO clear up the ambiguity here */
> +
> + // only intend stripping for internal sub-expressions
> + auto &copied_expr = elems.get_elem_to_copy ();
> + copied_expr->accept_vis (*this);
> + if (copied_expr->is_marked_for_strip ())
> + rust_error_at (copied_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + auto ©_count = elems.get_num_copies ();
> + copy_count->accept_vis (*this);
> + if (copy_count->is_marked_for_strip ())
> + rust_error_at (copy_count->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ArrayExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says there are separate
> + * inner attributes, not just outer attributes of inner exprs */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* assuming you can't strip away the ArrayElems type, but can strip
> + * internal expressions and whatever */
> + expr.get_array_elems ()->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::ArrayIndexExpr &expr)
> +{
> + /* it is unclear whether outer attributes are supposed to be
> + * allowed, but conceptually it wouldn't make much sense, but
> + * having expansion code anyway. TODO */
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &array_expr = expr.get_array_expr ();
> + array_expr->accept_vis (*this);
> + if (array_expr->is_marked_for_strip ())
> + rust_error_at (array_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + auto &index_expr = expr.get_index_expr ();
> + index_expr->accept_vis (*this);
> + if (index_expr->is_marked_for_strip ())
> + rust_error_at (index_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::TupleExpr &expr)
> +{
> + /* according to spec, outer attributes are allowed on "elements of
> + * tuple expressions" */
> +
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says these are inner
> + * attributes, not outer attributes of inner expr */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* apparently outer attributes are allowed in "elements of tuple
> + * expressions" according to spec */
> + expand_pointer_allow_strip (expr.get_tuple_elems ());
> +}
> +void
> +AttrVisitor::visit (AST::TupleIndexExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* wouldn't strip this directly (as outer attrs should be
> + * associated with this level), but any sub-expressions would be
> + * stripped. Thus, no need to erase when strip check called. */
> + auto &tuple_expr = expr.get_tuple_expr ();
> + tuple_expr->accept_vis (*this);
> + if (tuple_expr->is_marked_for_strip ())
> + rust_error_at (tuple_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::StructExprStruct &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says these are inner
> + * attributes, not outer attributes of inner expr */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-exprs of path
> + auto &struct_name = expr.get_struct_name ();
> + visit (struct_name);
> + if (struct_name.is_marked_for_strip ())
> + rust_error_at (struct_name.get_locus (),
> + "cannot strip path in this position");
> +}
> +void
> +AttrVisitor::visit (AST::StructExprFieldIdentifier &)
> +{
> + // as no attrs (at moment, at least), no stripping possible
has
is there a reason to not put a gcc_unreachable () here?
> +}
> +void
> +AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field)
> +{
> + /* as no attrs possible (at moment, at least), only sub-expression
> + * stripping is possible */
> + auto &value = field.get_value ();
> + value->accept_vis (*this);
> + if (value->is_marked_for_strip ())
> + rust_error_at (value->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::StructExprFieldIndexValue &field)
> +{
> + /* as no attrs possible (at moment, at least), only sub-expression
> + * stripping is possible */
> + auto &value = field.get_value ();
> + value->accept_vis (*this);
> + if (value->is_marked_for_strip ())
> + rust_error_at (value->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::StructExprStructFields &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says these are inner
> + * attributes, not outer attributes of inner expr */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-exprs of path
> + auto &struct_name = expr.get_struct_name ();
> + visit (struct_name);
> + if (struct_name.is_marked_for_strip ())
> + rust_error_at (struct_name.get_locus (),
> + "cannot strip path in this position");
> +
> + /* spec does not specify whether expressions are allowed to be
> + * stripped at top level of struct fields, but I wouldn't think
> + * that they would be, so operating under the assumption that only
> + * sub-expressions can be stripped. */
> + for (auto &field : expr.get_fields ())
> + {
> + field->accept_vis (*this);
> + // shouldn't strip in this
> + }
> +
> + /* struct base presumably can't be stripped, as the '..' is before
> + * the expression. as such, can only strip sub-expressions. */
> + if (expr.has_struct_base ())
> + {
> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
> + base_struct_expr->accept_vis (*this);
> + if (base_struct_expr->is_marked_for_strip ())
> + rust_error_at (base_struct_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::StructExprStructBase &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says these are inner
> + * attributes, not outer attributes of inner expr */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-exprs of path
> + auto &struct_name = expr.get_struct_name ();
> + visit (struct_name);
> + if (struct_name.is_marked_for_strip ())
> + rust_error_at (struct_name.get_locus (),
> + "cannot strip path in this position");
> +
> + /* struct base presumably can't be stripped, as the '..' is before
> + * the expression. as such, can only strip sub-expressions. */
> + rust_assert (!expr.get_struct_base ().is_invalid ());
> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
> + base_struct_expr->accept_vis (*this);
> + if (base_struct_expr->is_marked_for_strip ())
> + rust_error_at (base_struct_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::CallExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* should not be outer attrs on "function" expression - outer attrs
> + * should be associated with call expr as a whole. only sub-expr
> + * expansion is possible. */
> + auto &function = expr.get_function_expr ();
> + function->accept_vis (*this);
> + if (function->is_marked_for_strip ())
> + rust_error_at (function->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + /* spec says outer attributes are specifically allowed for elements
> + * of call expressions, so full stripping possible */
> + // FIXME: Arthur: Figure out how to refactor this - This is similar to
> + // expanding items in the crate or stmts in blocks
> + expand_pointer_allow_strip (expr.get_params ());
> + auto ¶ms = expr.get_params ();
> + for (auto it = params.begin (); it != params.end ();)
> + {
> + auto &stmt = *it;
> +
> + stmt->accept_vis (*this);
> +
> + auto final_fragment = expand_macro_fragment_recursive ();
> + if (final_fragment.should_expand ())
> + {
> + // Remove the current expanded invocation
> + it = params.erase (it);
> + for (auto &node : final_fragment.get_nodes ())
> + {
> + it = params.insert (it, node.take_expr ());
> + it++;
> + }
> + }
> + else if (stmt->is_marked_for_strip ())
> + it = params.erase (it);
> + else
> + it++;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::MethodCallExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* should not be outer attrs on "receiver" expression - outer attrs
> + * should be associated with call expr as a whole. only sub-expr
> + * expansion is possible. */
> + auto &receiver = expr.get_receiver_expr ();
> + receiver->accept_vis (*this);
> + if (receiver->is_marked_for_strip ())
> + rust_error_at (receiver->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + auto &method_name = expr.get_method_name ();
> + if (method_name.has_generic_args ())
> + expand_generic_args (method_name.get_generic_args ());
> +
> + /* spec says outer attributes are specifically allowed for elements
> + * of method call expressions, so full stripping possible */
> + expand_pointer_allow_strip (expr.get_params ());
> +}
> +void
> +AttrVisitor::visit (AST::FieldAccessExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* should not be outer attrs on "receiver" expression - outer attrs
> + * should be associated with field expr as a whole. only sub-expr
> + * expansion is possible. */
> + auto &receiver = expr.get_receiver_expr ();
> + receiver->accept_vis (*this);
> + if (receiver->is_marked_for_strip ())
> + rust_error_at (receiver->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ClosureExprInner &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip closure parameters if required - this is specifically
> + * allowed by spec */
> + expand_closure_params (expr.get_params ());
> +
> + // can't strip expression itself, but can strip sub-expressions
> + auto &definition_expr = expr.get_definition_expr ();
> + definition_expr->accept_vis (*this);
> + if (definition_expr->is_marked_for_strip ())
> + rust_error_at (definition_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +
> +void
> +AttrVisitor::visit (AST::BlockExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip test based on inner attrs - spec says there are inner
> + * attributes, not just outer attributes of inner stmts */
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + std::function<std::unique_ptr<AST::Stmt> (AST::SingleASTNode)> extractor
> + = [] (AST::SingleASTNode node) { return node.take_stmt (); };
> +
> + expand_macro_children (MacroExpander::BLOCK, expr.get_statements (),
> + extractor);
> +
> + expander.push_context (MacroExpander::BLOCK);
> +
> + // strip tail expression if exists - can actually fully remove it
> + if (expr.has_tail_expr ())
> + {
> + auto &tail_expr = expr.get_tail_expr ();
> +
> + tail_expr->accept_vis (*this);
> + maybe_expand_expr (tail_expr);
> +
> + if (tail_expr->is_marked_for_strip ())
> + expr.strip_tail_expr ();
> + }
> + expander.pop_context ();
> +}
> +
> +void
> +AttrVisitor::visit (AST::ClosureExprInnerTyped &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* strip closure parameters if required - this is specifically
> + * allowed by spec */
> + expand_closure_params (expr.get_params ());
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // can't strip return type, but can strip sub-types
> + auto &type = expr.get_return_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + // can't strip expression itself, but can strip sub-expressions
> + auto &definition_block = expr.get_definition_block ();
> + definition_block->accept_vis (*this);
> + if (definition_block->is_marked_for_strip ())
> + rust_error_at (definition_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ContinueExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::BreakExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* spec does not say that you can have outer attributes on
> + * expression, so assuming you can't. stripping for sub-expressions
> + * is the only thing that can be done */
> + if (expr.has_break_expr ())
> + {
> + auto &break_expr = expr.get_break_expr ();
> +
> + break_expr->accept_vis (*this);
> +
> + if (break_expr->is_marked_for_strip ())
> + rust_error_at (break_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::RangeFromToExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + expr.get_from_expr ()->accept_vis (*this);
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + expr.get_to_expr ()->accept_vis (*this);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_from_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_from_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before range exprs");
> + if (expr.get_to_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_to_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::RangeFromExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * direct descendant expression, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + auto &from_expr = expr.get_from_expr ();
> +
> + from_expr->accept_vis (*this);
> +
> + if (from_expr->is_marked_for_strip ())
> + rust_error_at (from_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed before range exprs");
> +}
> +void
> +AttrVisitor::visit (AST::RangeToExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * direct descendant expression, can strip ones below that */
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &to_expr = expr.get_to_expr ();
> +
> + to_expr->accept_vis (*this);
> +
> + if (to_expr->is_marked_for_strip ())
> + rust_error_at (to_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::RangeFullExpr &)
> +{
> + // outer attributes never allowed before these, so no stripping
> +}
> +void
> +AttrVisitor::visit (AST::RangeFromToInclExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * two direct descendant expressions, can strip ones below that */
> +
> + /* should have no possibility for outer attrs as would be parsed
> + * with outer expr */
> + expr.get_from_expr ()->accept_vis (*this);
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + expr.get_to_expr ()->accept_vis (*this);
> +
> + // ensure that they are not marked for strip
> + if (expr.get_from_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_from_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes are never allowed "
> + "before range exprs");
> + if (expr.get_to_expr ()->is_marked_for_strip ())
> + rust_error_at (expr.get_to_expr ()->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::RangeToInclExpr &expr)
> +{
> + /* outer attributes never allowed before these. while cannot strip
> + * direct descendant expression, can strip ones below that */
> +
> + /* should syntactically not have outer attributes, though this may
> + * not have worked in practice */
> + auto &to_expr = expr.get_to_expr ();
> +
> + to_expr->accept_vis (*this);
> +
> + if (to_expr->is_marked_for_strip ())
> + rust_error_at (to_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ReturnExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* spec does not say that you can have outer attributes on
> + * expression, so assuming you can't. stripping for sub-expressions
> + * is the only thing that can be done */
> + if (expr.has_returned_expr ())
> + {
> + auto &returned_expr = expr.get_returned_expr ();
> +
> + returned_expr->accept_vis (*this);
> +
> + if (returned_expr->is_marked_for_strip ())
> + rust_error_at (returned_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> + }
> + /* TODO: conceptually, you would maybe be able to remove a returned
> + * expr - e.g. if you had conditional compilation returning void or
> + * returning a type. On the other hand, I think that function
> + * return type cannot be conditionally compiled, so I assumed you
> + * can't do this either. */
> +}
> +void
> +AttrVisitor::visit (AST::UnsafeBlockExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &block_expr = expr.get_block_expr ();
> + block_expr->accept_vis (*this);
> + if (block_expr->is_marked_for_strip ())
> + rust_error_at (block_expr->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::LoopExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &loop_block = expr.get_loop_block ();
> + loop_block->accept_vis (*this);
> + if (loop_block->is_marked_for_strip ())
> + rust_error_at (loop_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::WhileLoopExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip predicate expr itself, but can strip sub-expressions
> + auto &predicate_expr = expr.get_predicate_expr ();
> + predicate_expr->accept_vis (*this);
> + if (predicate_expr->is_marked_for_strip ())
> + rust_error_at (predicate_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &loop_block = expr.get_loop_block ();
> + loop_block->accept_vis (*this);
> + if (loop_block->is_marked_for_strip ())
> + rust_error_at (loop_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::WhileLetLoopExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &pattern : expr.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + // can't strip scrutinee expr itself, but can strip sub-expressions
> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
> + scrutinee_expr->accept_vis (*this);
> + if (scrutinee_expr->is_marked_for_strip ())
> + rust_error_at (scrutinee_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &loop_block = expr.get_loop_block ();
> + loop_block->accept_vis (*this);
> + if (loop_block->is_marked_for_strip ())
> + rust_error_at (loop_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::ForLoopExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-patterns of pattern
> + auto &pattern = expr.get_pattern ();
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> +
> + // can't strip scrutinee expr itself, but can strip sub-expressions
> + auto &iterator_expr = expr.get_iterator_expr ();
> + iterator_expr->accept_vis (*this);
> + if (iterator_expr->is_marked_for_strip ())
> + rust_error_at (iterator_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &loop_block = expr.get_loop_block ();
> + loop_block->accept_vis (*this);
> + if (loop_block->is_marked_for_strip ())
> + rust_error_at (loop_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfExpr &expr)
> +{
> + // rust playground test shows that IfExpr does support outer attrs, at least
> + // when used as statement
> +
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip condition expr itself, but can strip sub-expressions
> + auto &condition_expr = expr.get_condition_expr ();
> + condition_expr->accept_vis (*this);
> + maybe_expand_expr (condition_expr);
> + if (condition_expr->is_marked_for_strip ())
> + rust_error_at (condition_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfExprConseqElse &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip condition expr itself, but can strip sub-expressions
> + auto &condition_expr = expr.get_condition_expr ();
> + condition_expr->accept_vis (*this);
> + maybe_expand_expr (condition_expr);
> + if (condition_expr->is_marked_for_strip ())
> + rust_error_at (condition_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip else block itself, but can strip sub-expressions
> + auto &else_block = expr.get_else_block ();
> + else_block->accept_vis (*this);
> + if (else_block->is_marked_for_strip ())
> + rust_error_at (else_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfExprConseqIf &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip condition expr itself, but can strip sub-expressions
> + auto &condition_expr = expr.get_condition_expr ();
> + condition_expr->accept_vis (*this);
> + maybe_expand_expr (condition_expr);
> + if (condition_expr->is_marked_for_strip ())
> + rust_error_at (condition_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if expr itself, but can strip sub-expressions
> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
> + conseq_if_expr->accept_vis (*this);
> + if (conseq_if_expr->is_marked_for_strip ())
> + rust_error_at (conseq_if_expr->get_locus (),
> + "cannot strip consequent if expression in this "
> + "position - outer attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfExprConseqIfLet &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip condition expr itself, but can strip sub-expressions
> + auto &condition_expr = expr.get_condition_expr ();
> + condition_expr->accept_vis (*this);
> + maybe_expand_expr (condition_expr);
> + if (condition_expr->is_marked_for_strip ())
> + rust_error_at (condition_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if let expr itself, but can strip sub-expressions
> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
> + conseq_if_let_expr->accept_vis (*this);
> + if (conseq_if_let_expr->is_marked_for_strip ())
> + rust_error_at (conseq_if_let_expr->get_locus (),
> + "cannot strip consequent if let expression in this "
> + "position - outer attributes not "
> + "allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfLetExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &pattern : expr.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + // can't strip value expr itself, but can strip sub-expressions
> + auto &value_expr = expr.get_value_expr ();
> + value_expr->accept_vis (*this);
> + if (value_expr->is_marked_for_strip ())
> + rust_error_at (value_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfLetExprConseqElse &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &pattern : expr.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + // can't strip value expr itself, but can strip sub-expressions
> + auto &value_expr = expr.get_value_expr ();
> + value_expr->accept_vis (*this);
> + if (value_expr->is_marked_for_strip ())
> + rust_error_at (value_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip else block itself, but can strip sub-expressions
> + auto &else_block = expr.get_else_block ();
> + else_block->accept_vis (*this);
> + if (else_block->is_marked_for_strip ())
> + rust_error_at (else_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfLetExprConseqIf &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &pattern : expr.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + // can't strip value expr itself, but can strip sub-expressions
> + auto &value_expr = expr.get_value_expr ();
> + value_expr->accept_vis (*this);
> + if (value_expr->is_marked_for_strip ())
> + rust_error_at (value_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if expr itself, but can strip sub-expressions
> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
> + conseq_if_expr->accept_vis (*this);
> + if (conseq_if_expr->is_marked_for_strip ())
> + rust_error_at (conseq_if_expr->get_locus (),
> + "cannot strip consequent if expression in this "
> + "position - outer attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + for (auto &pattern : expr.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + // can't strip value expr itself, but can strip sub-expressions
> + auto &value_expr = expr.get_value_expr ();
> + value_expr->accept_vis (*this);
> + if (value_expr->is_marked_for_strip ())
> + rust_error_at (value_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if block itself, but can strip sub-expressions
> + auto &if_block = expr.get_if_block ();
> + if_block->accept_vis (*this);
> + if (if_block->is_marked_for_strip ())
> + rust_error_at (if_block->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +
> + // can't strip if let expr itself, but can strip sub-expressions
> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
> + conseq_if_let_expr->accept_vis (*this);
> + if (conseq_if_let_expr->is_marked_for_strip ())
> + rust_error_at (conseq_if_let_expr->get_locus (),
> + "cannot strip consequent if let expression in this "
> + "position - outer attributes not "
> + "allowed");
> +}
> +void
> +AttrVisitor::visit (AST::MatchExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // inner attr strip test
> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip scrutinee expr itself, but can strip sub-expressions
> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
> + scrutinee_expr->accept_vis (*this);
> + if (scrutinee_expr->is_marked_for_strip ())
> + rust_error_at (scrutinee_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // strip match cases
> + auto &match_cases = expr.get_match_cases ();
> + for (auto it = match_cases.begin (); it != match_cases.end ();)
> + {
> + auto &match_case = *it;
> +
> + // strip match case based on outer attributes in match arm
> + auto &match_arm = match_case.get_arm ();
> + expander.expand_cfg_attrs (match_arm.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ()))
> + {
> + // strip match case
> + it = match_cases.erase (it);
> + continue;
> + }
> +
> + for (auto &pattern : match_arm.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + }
> +
> + /* assuming that guard expression cannot be stripped as
> + * strictly speaking you would have to strip the whole guard to
> + * make syntactical sense, which you can't do. as such, only
> + * strip sub-expressions */
> + if (match_arm.has_match_arm_guard ())
> + {
> + auto &guard_expr = match_arm.get_guard_expr ();
> + guard_expr->accept_vis (*this);
> + if (guard_expr->is_marked_for_strip ())
> + rust_error_at (guard_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> + }
> +
> + // strip sub-expressions from match cases
> + auto &case_expr = match_case.get_expr ();
> + case_expr->accept_vis (*this);
> + if (case_expr->is_marked_for_strip ())
> + rust_error_at (case_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + // increment to next case if haven't continued
> + ++it;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::AwaitExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + /* can't strip awaited expr itself, but can strip sub-expressions
> + * - this is because you can't have no expr to await */
> + auto &awaited_expr = expr.get_awaited_expr ();
> + awaited_expr->accept_vis (*this);
> + if (awaited_expr->is_marked_for_strip ())
> + rust_error_at (awaited_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::AsyncBlockExpr &expr)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> + {
> + expr.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip block itself, but can strip sub-expressions
> + auto &block_expr = expr.get_block_expr ();
> + block_expr->accept_vis (*this);
> + if (block_expr->is_marked_for_strip ())
> + rust_error_at (block_expr->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +
> +void
> +AttrVisitor::visit (AST::TypeParam ¶m)
> +{
> + // outer attributes don't actually do anything, so ignore them
> +
> + if (param.has_type_param_bounds ())
> + {
> + // don't strip directly, only components of bounds
> + for (auto &bound : param.get_type_param_bounds ())
> + bound->accept_vis (*this);
> + }
> +
> + if (param.has_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> + auto &type = param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +}
> +void
> +AttrVisitor::visit (AST::LifetimeWhereClauseItem &)
> +{
> + // shouldn't require
> +}
> +void
> +AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item)
> +{
> + // for lifetimes shouldn't require
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = item.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + // don't strip directly, only components of bounds
> + for (auto &bound : item.get_type_param_bounds ())
> + bound->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::Method &method)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (method.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (method.get_outer_attrs ()))
> + {
> + method.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : method.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* assuming you can't strip self param - wouldn't be a method
> + * anymore. spec allows outer attrs on self param, but doesn't
> + * specify whether cfg is used. */
> + expand_self_param (method.get_self_param ());
> +
> + /* strip method parameters if required - this is specifically
> + * allowed by spec */
> + expand_function_params (method.get_function_params ());
> +
> + if (method.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = method.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + if (method.has_where_clause ())
> + expand_where_clause (method.get_where_clause ());
> +
> + /* body should always exist - if error state, should have returned
> + * before now */
> + // can't strip block itself, but can strip sub-expressions
> + auto &block_expr = method.get_definition ();
> + block_expr->accept_vis (*this);
> + if (block_expr->is_marked_for_strip ())
> + rust_error_at (block_expr->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::Module &module)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (module.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (module.get_outer_attrs ()))
> + {
> + module.mark_for_strip ();
> + return;
> + }
> +
> + // A loaded module might have inner attributes
> + if (module.get_kind () == AST::Module::ModuleKind::LOADED)
> + {
> + // strip test based on inner attrs
> + expander.expand_cfg_attrs (module.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (module.get_inner_attrs ()))
> + {
> + module.mark_for_strip ();
> + return;
> + }
> + }
> +
> + // Parse the module's items if they haven't been expanded and the file
> + // should be parsed (i.e isn't hidden behind an untrue or impossible cfg
> + // directive)
> + if (!module.is_marked_for_strip ()
> + && module.get_kind () == AST::Module::ModuleKind::UNLOADED)
> + {
> + module.load_items ();
> + }
> +
> + // strip items if required
> + expand_pointer_allow_strip (module.get_items ());
> +}
> +void
> +AttrVisitor::visit (AST::ExternCrate &extern_crate)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (extern_crate.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ()))
> + {
> + extern_crate.mark_for_strip ();
> + return;
> + }
> +
> + if (!extern_crate.references_self ())
> + {
> + Session &session = Session::get_instance ();
> + session.load_extern_crate (extern_crate.get_referenced_crate (),
> + extern_crate.get_locus ());
> + }
> +}
> +void
> +AttrVisitor::visit (AST::UseTreeGlob &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::UseTreeList &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::UseTreeRebind &)
> +{
> + // shouldn't require?
> +}
> +void
> +AttrVisitor::visit (AST::UseDeclaration &use_decl)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (use_decl.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ()))
> + {
> + use_decl.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::Function &function)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (function.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (function.get_outer_attrs ()))
> + {
> + function.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : function.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* strip function parameters if required - this is specifically
> + * allowed by spec */
> + expand_function_params (function.get_function_params ());
> +
> + if (function.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = function.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + if (function.has_where_clause ())
> + expand_where_clause (function.get_where_clause ());
> +
> + /* body should always exist - if error state, should have returned
> + * before now */
> + // can't strip block itself, but can strip sub-expressions
> + auto &block_expr = function.get_definition ();
> + block_expr->accept_vis (*this);
> + if (block_expr->is_marked_for_strip ())
> + rust_error_at (block_expr->get_locus (),
> + "cannot strip block expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::TypeAlias &type_alias)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (type_alias.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ()))
> + {
> + type_alias.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : type_alias.get_generic_params ())
> + param->accept_vis (*this);
> +
> + if (type_alias.has_where_clause ())
> + expand_where_clause (type_alias.get_where_clause ());
> +
> + auto &type = type_alias.get_type_aliased ();
> + type->accept_vis (*this);
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::StructStruct &struct_item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (struct_item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ()))
> + {
> + struct_item.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : struct_item.get_generic_params ())
> + param->accept_vis (*this);
> +
> + if (struct_item.has_where_clause ())
> + expand_where_clause (struct_item.get_where_clause ());
> +
> + /* strip struct fields if required - this is presumably
> + * allowed by spec */
> + expand_struct_fields (struct_item.get_fields ());
> +}
> +void
> +AttrVisitor::visit (AST::TupleStruct &tuple_struct)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ()))
> + {
> + tuple_struct.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : tuple_struct.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* strip struct fields if required - this is presumably
> + * allowed by spec */
> + expand_tuple_fields (tuple_struct.get_fields ());
> +
> + if (tuple_struct.has_where_clause ())
> + expand_where_clause (tuple_struct.get_where_clause ());
> +}
> +void
> +AttrVisitor::visit (AST::EnumItem &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::EnumItemTuple &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + /* strip item fields if required - this is presumably
> + * allowed by spec */
> + expand_tuple_fields (item.get_tuple_fields ());
> +}
> +void
> +AttrVisitor::visit (AST::EnumItemStruct &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + /* strip item fields if required - this is presumably
> + * allowed by spec */
> + expand_struct_fields (item.get_struct_fields ());
> +}
> +void
> +AttrVisitor::visit (AST::EnumItemDiscriminant &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &expr = item.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + rust_error_at (expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::Enum &enum_item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (enum_item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ()))
> + {
> + enum_item.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : enum_item.get_generic_params ())
> + param->accept_vis (*this);
> +
> + if (enum_item.has_where_clause ())
> + expand_where_clause (enum_item.get_where_clause ());
> +
> + /* strip enum fields if required - this is presumably
> + * allowed by spec */
> + expand_pointer_allow_strip (enum_item.get_variants ());
> +}
> +void
> +AttrVisitor::visit (AST::Union &union_item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (union_item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ()))
> + {
> + union_item.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : union_item.get_generic_params ())
> + param->accept_vis (*this);
> +
> + if (union_item.has_where_clause ())
> + expand_where_clause (union_item.get_where_clause ());
> +
> + /* strip union fields if required - this is presumably
> + * allowed by spec */
> + expand_struct_fields (union_item.get_variants ());
> +}
> +void
> +AttrVisitor::visit (AST::ConstantItem &const_item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (const_item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ()))
> + {
> + const_item.mark_for_strip ();
> + return;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // strip any sub-types
> + auto &type = const_item.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &expr = const_item.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + rust_error_at (expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::StaticItem &static_item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (static_item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ()))
> + {
> + static_item.mark_for_strip ();
> + return;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // strip any sub-types
> + auto &type = static_item.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &expr = static_item.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + rust_error_at (expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +}
> +void
> +AttrVisitor::visit (AST::TraitItemFunc &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + expand_trait_function_decl (item.get_trait_function_decl ());
> +
> + if (item.has_definition ())
> + {
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &block = item.get_definition ();
> + block->accept_vis (*this);
> + if (block->is_marked_for_strip ())
> + rust_error_at (block->get_locus (),
> + "cannot strip block expression in this "
> + "position - outer attributes not allowed");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TraitItemMethod &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + expand_trait_method_decl (item.get_trait_method_decl ());
> +
> + if (item.has_definition ())
> + {
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped. */
> + auto &block = item.get_definition ();
> + block->accept_vis (*this);
> + if (block->is_marked_for_strip ())
> + rust_error_at (block->get_locus (),
> + "cannot strip block expression in this "
> + "position - outer attributes not allowed");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TraitItemConst &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + // strip any sub-types
> + auto &type = item.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped */
> + if (item.has_expression ())
> + {
> + auto &expr = item.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + rust_error_at (expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TraitItemType &item)
> +{
> + // initial test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + if (item.has_type_param_bounds ())
> + {
> + // don't strip directly, only components of bounds
> + for (auto &bound : item.get_type_param_bounds ())
> + bound->accept_vis (*this);
> + }
> +}
> +void
> +AttrVisitor::visit (AST::Trait &trait)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (trait.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (trait.get_outer_attrs ()))
> + {
> + trait.mark_for_strip ();
> + return;
> + }
> +
> + // strip test based on inner attrs
> + expander.expand_cfg_attrs (trait.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (trait.get_inner_attrs ()))
> + {
> + trait.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : trait.get_generic_params ())
> + param->accept_vis (*this);
> +
> + if (trait.has_type_param_bounds ())
> + {
> + // don't strip directly, only components of bounds
> + for (auto &bound : trait.get_type_param_bounds ())
> + bound->accept_vis (*this);
> + }
> +
> + if (trait.has_where_clause ())
> + expand_where_clause (trait.get_where_clause ());
> +
> + std::function<std::unique_ptr<AST::TraitItem> (AST::SingleASTNode)> extractor
> + = [] (AST::SingleASTNode node) { return node.take_trait_item (); };
> +
> + expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (),
> + extractor);
> +}
> +void
> +AttrVisitor::visit (AST::InherentImpl &impl)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
> + {
> + impl.mark_for_strip ();
> + return;
> + }
> +
> + // strip test based on inner attrs
> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
> + {
> + impl.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : impl.get_generic_params ())
> + param->accept_vis (*this);
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = impl.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + if (impl.has_where_clause ())
> + expand_where_clause (impl.get_where_clause ());
> +
> + std::function<std::unique_ptr<AST::InherentImplItem> (AST::SingleASTNode)>
> + extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); };
> +
> + expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (),
> + extractor);
> +}
> +void
> +AttrVisitor::visit (AST::TraitImpl &impl)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
> + {
> + impl.mark_for_strip ();
> + return;
> + }
> +
> + // strip test based on inner attrs
> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
> + {
> + impl.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : impl.get_generic_params ())
> + param->accept_vis (*this);
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = impl.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + auto &trait_path = impl.get_trait_path ();
> + visit (trait_path);
> + if (trait_path.is_marked_for_strip ())
> + rust_error_at (trait_path.get_locus (),
> + "cannot strip typepath in this position");
> +
> + if (impl.has_where_clause ())
> + expand_where_clause (impl.get_where_clause ());
> +
> + std::function<std::unique_ptr<AST::TraitImplItem> (AST::SingleASTNode)>
> + extractor
> + = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); };
> +
> + expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (),
> + extractor);
> +}
> +void
> +AttrVisitor::visit (AST::ExternalStaticItem &item)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = item.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> +
> + expander.pop_context ();
> +}
> +void
> +AttrVisitor::visit (AST::ExternalFunctionItem &item)
> +{
> + // strip test based on outer attrs
> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> + {
> + item.mark_for_strip ();
> + return;
> + }
> +
> + // just expand sub-stuff - can't actually strip generic params themselves
> + for (auto ¶m : item.get_generic_params ())
> + param->accept_vis (*this);
> +
> + /* strip function parameters if required - this is specifically
> + * allowed by spec */
> + auto ¶ms = item.get_function_params ();
> + for (auto it = params.begin (); it != params.end ();)
> + {
> + auto ¶m = *it;
> +
> + auto ¶m_attrs = param.get_outer_attrs ();
> + expander.expand_cfg_attrs (param_attrs);
> + if (expander.fails_cfg_with_expand (param_attrs))
> + {
> + it = params.erase (it);
> + continue;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + // increment if nothing else happens
> + ++it;
> + }
> + /* NOTE: these are extern function params, which may have different
> + * rules and restrictions to "normal" function params. So expansion
> + * handled separately. */
> +
> + /* TODO: assuming that variadic nature cannot be stripped. If this
> + * is not true, then have code here to do so. */
> +
> + if (item.has_return_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &return_type = item.get_return_type ();
> + return_type->accept_vis (*this);
> +
> + maybe_expand_type (return_type);
> +
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + if (item.has_where_clause ())
> + expand_where_clause (item.get_where_clause ());
> +}
> +
> +void
> +AttrVisitor::visit (AST::ExternBlock &block)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (block.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (block.get_outer_attrs ()))
> + {
> + block.mark_for_strip ();
> + return;
> + }
> +
> + // strip test based on inner attrs
> + expander.expand_cfg_attrs (block.get_inner_attrs ());
> + if (expander.fails_cfg_with_expand (block.get_inner_attrs ()))
> + {
> + block.mark_for_strip ();
> + return;
> + }
> +
> + std::function<std::unique_ptr<AST::ExternalItem> (AST::SingleASTNode)>
> + extractor
> + = [] (AST::SingleASTNode node) { return node.take_external_item (); };
> +
> + expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (),
> + extractor);
> +}
> +
> +// I don't think it would be possible to strip macros without expansion
> +void
> +AttrVisitor::visit (AST::MacroMatchFragment &)
> +{}
> +void
> +AttrVisitor::visit (AST::MacroMatchRepetition &)
> +{}
> +void
> +AttrVisitor::visit (AST::MacroMatcher &)
> +{}
> +void
> +AttrVisitor::visit (AST::MacroRulesDefinition &rules_def)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (rules_def.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ()))
> + {
> + rules_def.mark_for_strip ();
> + return;
> + }
> +
> + // I don't think any macro rules can be stripped in any way
> +
> + auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (),
> + rules_def.get_rule_name ());
> + expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (),
> + rules_def.get_locus ());
> + expander.mappings->insert_macro_def (&rules_def);
> + rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]",
> + path.get ().c_str ());
> +}
> +
> +void
> +AttrVisitor::visit (AST::MetaItemPath &)
> +{}
> +void
> +AttrVisitor::visit (AST::MetaItemSeq &)
> +{}
> +void
> +AttrVisitor::visit (AST::MetaWord &)
> +{}
> +void
> +AttrVisitor::visit (AST::MetaNameValueStr &)
> +{}
> +void
> +AttrVisitor::visit (AST::MetaListPaths &)
> +{}
> +void
> +AttrVisitor::visit (AST::MetaListNameValueStr &)
> +{}
> +
> +void
> +AttrVisitor::visit (AST::LiteralPattern &)
> +{
> + // not possible
> +}
> +void
> +AttrVisitor::visit (AST::IdentifierPattern &pattern)
> +{
> + // can only strip sub-patterns of the inner pattern to bind
> + if (!pattern.has_pattern_to_bind ())
> + return;
> +
> + auto &sub_pattern = pattern.get_pattern_to_bind ();
> + sub_pattern->accept_vis (*this);
> + if (sub_pattern->is_marked_for_strip ())
> + rust_error_at (sub_pattern->get_locus (),
> + "cannot strip pattern in this position");
> +}
> +void
> +AttrVisitor::visit (AST::WildcardPattern &)
> +{
> + // not possible
> +}
> +void
> +AttrVisitor::visit (AST::RangePatternBoundLiteral &)
> +{
> + // not possible
> +}
> +void
> +AttrVisitor::visit (AST::RangePatternBoundPath &bound)
> +{
> + // can expand path, but not strip it directly
> + auto &path = bound.get_path ();
> + visit (path);
> + if (path.is_marked_for_strip ())
> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> +}
> +void
> +AttrVisitor::visit (AST::RangePatternBoundQualPath &bound)
> +{
> + // can expand path, but not strip it directly
> + auto &path = bound.get_qualified_path ();
> + visit (path);
> + if (path.is_marked_for_strip ())
> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> +}
> +void
> +AttrVisitor::visit (AST::RangePattern &pattern)
> +{
> + // should have no capability to strip lower or upper bounds, only expand
> + pattern.get_lower_bound ()->accept_vis (*this);
> + pattern.get_upper_bound ()->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::ReferencePattern &pattern)
> +{
> + auto &sub_pattern = pattern.get_referenced_pattern ();
> + sub_pattern->accept_vis (*this);
> + if (sub_pattern->is_marked_for_strip ())
> + rust_error_at (sub_pattern->get_locus (),
> + "cannot strip pattern in this position");
> +}
> +void
> +AttrVisitor::visit (AST::StructPatternFieldTuplePat &field)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> + {
> + field.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-patterns (can't strip top-level pattern)
> + auto &sub_pattern = field.get_index_pattern ();
> + sub_pattern->accept_vis (*this);
> + if (sub_pattern->is_marked_for_strip ())
> + rust_error_at (sub_pattern->get_locus (),
> + "cannot strip pattern in this position");
> +}
> +void
> +AttrVisitor::visit (AST::StructPatternFieldIdentPat &field)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> + {
> + field.mark_for_strip ();
> + return;
> + }
> +
> + // strip sub-patterns (can't strip top-level pattern)
> + auto &sub_pattern = field.get_ident_pattern ();
> + sub_pattern->accept_vis (*this);
> + if (sub_pattern->is_marked_for_strip ())
> + rust_error_at (sub_pattern->get_locus (),
> + "cannot strip pattern in this position");
> +}
> +void
> +AttrVisitor::visit (AST::StructPatternFieldIdent &field)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> + {
> + field.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::StructPattern &pattern)
> +{
> + // expand (but don't strip) path
> + auto &path = pattern.get_path ();
> + visit (path);
> + if (path.is_marked_for_strip ())
> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> +
> + /* TODO: apparently struct pattern fields can have outer attrs. so can they
> + * be stripped? */
> + if (!pattern.has_struct_pattern_elems ())
> + return;
> +
> + auto &elems = pattern.get_struct_pattern_elems ();
> +
> + // assuming you can strip struct pattern fields
> + expand_pointer_allow_strip (elems.get_struct_pattern_fields ());
> +
> + // assuming you can strip the ".." part
> + if (elems.has_etc ())
> + {
> + expander.expand_cfg_attrs (elems.get_etc_outer_attrs ());
> + if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ()))
> + elems.strip_etc ();
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items)
> +{
> + // can't strip individual patterns, only sub-patterns
> + for (auto &pattern : tuple_items.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> +
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items)
> +{
> + // can't strip individual patterns, only sub-patterns
> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
> + {
> + lower_pattern->accept_vis (*this);
> +
> + if (lower_pattern->is_marked_for_strip ())
> + rust_error_at (lower_pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
> + {
> + upper_pattern->accept_vis (*this);
> +
> + if (upper_pattern->is_marked_for_strip ())
> + rust_error_at (upper_pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TupleStructPattern &pattern)
> +{
> + // expand (but don't strip) path
> + auto &path = pattern.get_path ();
> + visit (path);
> + if (path.is_marked_for_strip ())
> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> +
> + if (pattern.has_items ())
> + pattern.get_items ()->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items)
> +{
> + // can't strip individual patterns, only sub-patterns
> + for (auto &pattern : tuple_items.get_patterns ())
> + {
> + pattern->accept_vis (*this);
> +
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items)
> +{
> + // can't strip individual patterns, only sub-patterns
> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
> + {
> + lower_pattern->accept_vis (*this);
> +
> + if (lower_pattern->is_marked_for_strip ())
> + rust_error_at (lower_pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
> + {
> + upper_pattern->accept_vis (*this);
> +
> + if (upper_pattern->is_marked_for_strip ())
> + rust_error_at (upper_pattern->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> +}
> +void
> +AttrVisitor::visit (AST::TuplePattern &pattern)
> +{
> + if (pattern.has_tuple_pattern_items ())
> + pattern.get_items ()->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::GroupedPattern &pattern)
> +{
> + // can't strip inner pattern, only sub-patterns
> + auto &pattern_in_parens = pattern.get_pattern_in_parens ();
> +
> + pattern_in_parens->accept_vis (*this);
> +
> + if (pattern_in_parens->is_marked_for_strip ())
> + rust_error_at (pattern_in_parens->get_locus (),
> + "cannot strip pattern in this position");
> +}
> +void
> +AttrVisitor::visit (AST::SlicePattern &pattern)
> +{
> + // can't strip individual patterns, only sub-patterns
> + for (auto &item : pattern.get_items ())
> + {
> + item->accept_vis (*this);
> +
> + if (item->is_marked_for_strip ())
> + rust_error_at (item->get_locus (),
> + "cannot strip pattern in this position");
> + // TODO: quit stripping now? or keep going?
> + }
> +}
> +
> +void
> +AttrVisitor::visit (AST::EmptyStmt &)
> +{
> + // assuming no outer attributes, so nothing can happen
> +}
> +void
> +AttrVisitor::visit (AST::LetStmt &stmt)
> +{
> + // initial strip test based on outer attrs
> + expander.expand_cfg_attrs (stmt.get_outer_attrs ());
> + if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ()))
> + {
> + stmt.mark_for_strip ();
> + return;
> + }
> +
> + // can't strip pattern, but call for sub-patterns
> + auto &pattern = stmt.get_pattern ();
> + pattern->accept_vis (*this);
> + if (pattern->is_marked_for_strip ())
> + rust_error_at (pattern->get_locus (),
> + "cannot strip pattern in this position");
> +
> + // similar for type
> + if (stmt.has_type ())
> + {
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = stmt.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> + }
> +
> + /* strip any internal sub-expressions - expression itself isn't
> + * allowed to have external attributes in this position so can't be
> + * stripped */
> + if (stmt.has_init_expr ())
> + {
> + auto &init_expr = stmt.get_init_expr ();
> + init_expr->accept_vis (*this);
> +
> + if (init_expr->is_marked_for_strip ())
> + rust_error_at (init_expr->get_locus (),
> + "cannot strip expression in this position - outer "
> + "attributes not allowed");
> +
> + maybe_expand_expr (init_expr);
> + }
> +}
> +void
> +AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt)
> +{
> + // outer attributes associated with expr, so rely on expr
> +
> + // guard - should prevent null pointer expr
> + if (stmt.is_marked_for_strip ())
> + return;
> +
> + // strip if expr is to be stripped
> + auto &expr = stmt.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + {
> + stmt.mark_for_strip ();
> + return;
> + }
> +}
> +void
> +AttrVisitor::visit (AST::ExprStmtWithBlock &stmt)
> +{
> + // outer attributes associated with expr, so rely on expr
> +
> + // guard - should prevent null pointer expr
> + if (stmt.is_marked_for_strip ())
> + return;
> +
> + // strip if expr is to be stripped
> + auto &expr = stmt.get_expr ();
> + expr->accept_vis (*this);
> + if (expr->is_marked_for_strip ())
> + {
> + stmt.mark_for_strip ();
> + return;
> + }
> +}
> +
> +void
> +AttrVisitor::visit (AST::TraitBound &bound)
> +{
> + // nothing in for lifetimes to strip
> +
> + // expand but don't strip type path
> + auto &path = bound.get_type_path ();
> + visit (path);
> + if (path.is_marked_for_strip ())
> + rust_error_at (path.get_locus (),
> + "cannot strip type path in this position");
> +}
> +void
> +AttrVisitor::visit (AST::ImplTraitType &type)
> +{
> + // don't strip directly, only components of bounds
> + for (auto &bound : type.get_type_param_bounds ())
> + bound->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::TraitObjectType &type)
> +{
> + // don't strip directly, only components of bounds
> + for (auto &bound : type.get_type_param_bounds ())
> + bound->accept_vis (*this);
> +}
> +void
> +AttrVisitor::visit (AST::ParenthesisedType &type)
> +{
> + // expand but don't strip inner type
> + auto &inner_type = type.get_type_in_parens ();
> + inner_type->accept_vis (*this);
> + if (inner_type->is_marked_for_strip ())
> + rust_error_at (inner_type->get_locus (),
> + "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::ImplTraitTypeOneBound &type)
> +{
> + // no stripping possible
> + visit (type.get_trait_bound ());
> +}
> +void
> +AttrVisitor::visit (AST::TraitObjectTypeOneBound &type)
> +{
> + // no stripping possible
> + visit (type.get_trait_bound ());
> +}
> +void
> +AttrVisitor::visit (AST::TupleType &type)
> +{
> + // TODO: assuming that types can't be stripped as types don't have outer
> + // attributes
> + for (auto &elem_type : type.get_elems ())
> + {
> + elem_type->accept_vis (*this);
> + if (elem_type->is_marked_for_strip ())
> + rust_error_at (elem_type->get_locus (),
> + "cannot strip type in this position");
> + }
> +}
> +void
> +AttrVisitor::visit (AST::NeverType &)
> +{
> + // no stripping possible
> +}
> +void
> +AttrVisitor::visit (AST::RawPointerType &type)
> +{
> + // expand but don't strip type pointed to
> + auto &pointed_type = type.get_type_pointed_to ();
> + pointed_type->accept_vis (*this);
> + if (pointed_type->is_marked_for_strip ())
> + rust_error_at (pointed_type->get_locus (),
> + "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::ReferenceType &type)
> +{
> + // expand but don't strip type referenced
> + auto &referenced_type = type.get_type_referenced ();
> + referenced_type->accept_vis (*this);
> + if (referenced_type->is_marked_for_strip ())
> + rust_error_at (referenced_type->get_locus (),
> + "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::ArrayType &type)
> +{
> + // expand but don't strip type referenced
> + auto &base_type = type.get_elem_type ();
> + base_type->accept_vis (*this);
> + if (base_type->is_marked_for_strip ())
> + rust_error_at (base_type->get_locus (),
> + "cannot strip type in this position");
> +
> + // same for expression
> + auto &size_expr = type.get_size_expr ();
> + size_expr->accept_vis (*this);
> + if (size_expr->is_marked_for_strip ())
> + rust_error_at (size_expr->get_locus (),
> + "cannot strip expression in this position");
> +}
> +void
> +AttrVisitor::visit (AST::SliceType &type)
> +{
> + // expand but don't strip elem type
> + auto &elem_type = type.get_elem_type ();
> + elem_type->accept_vis (*this);
> + if (elem_type->is_marked_for_strip ())
> + rust_error_at (elem_type->get_locus (),
> + "cannot strip type in this position");
> +}
> +void
> +AttrVisitor::visit (AST::InferredType &)
> +{
> + // none possible
> +}
> +void
> +AttrVisitor::visit (AST::BareFunctionType &type)
> +{
> + // seem to be no generics
> +
> + // presumably function params can be stripped
> + auto ¶ms = type.get_function_params ();
> + for (auto it = params.begin (); it != params.end ();)
> + {
> + auto ¶m = *it;
> +
> + auto ¶m_attrs = param.get_outer_attrs ();
> + expander.expand_cfg_attrs (param_attrs);
> + if (expander.fails_cfg_with_expand (param_attrs))
> + {
> + it = params.erase (it);
> + continue;
> + }
> +
> + expander.push_context (MacroExpander::ContextType::TYPE);
> +
> + auto &type = param.get_type ();
> + type->accept_vis (*this);
> +
> + maybe_expand_type (type);
> +
> + if (type->is_marked_for_strip ())
> + rust_error_at (type->get_locus (),
> + "cannot strip type in this position");
> +
> + expander.pop_context ();
> +
> + // increment if nothing else happens
> + ++it;
> + }
> +
> + /* TODO: assuming that variadic nature cannot be stripped. If this
> + * is not true, then have code here to do so. */
> +
> + if (type.has_return_type ())
> + {
> + // FIXME: Can we have type expansion in this position?
> + // In that case, we need to handle AST::TypeNoBounds on top of just
> + // AST::Types
> + auto &return_type = type.get_return_type ();
> + return_type->accept_vis (*this);
> + if (return_type->is_marked_for_strip ())
> + rust_error_at (return_type->get_locus (),
> + "cannot strip type in this position");
> + }
> +
> + // no where clause, apparently
> +}
> +void
> +AttrVisitor::maybe_expand_expr (std::unique_ptr<AST::Expr> &expr)
> +{
> + auto final_fragment = expand_macro_fragment_recursive ();
> + if (final_fragment.should_expand ())
> + expr = final_fragment.take_expression_fragment ();
> +}
> +
> +void
> +AttrVisitor::maybe_expand_type (std::unique_ptr<AST::Type> &type)
> +{
> + auto final_fragment = expand_macro_fragment_recursive ();
> + if (final_fragment.should_expand ())
> + type = final_fragment.take_type_fragment ();
> +}
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-attribute-visitor.h b/gcc/rust/expand/rust-attribute-visitor.h
> new file mode 100644
> index 00000000000..0f9d1065334
> --- /dev/null
> +++ b/gcc/rust/expand/rust-attribute-visitor.h
> @@ -0,0 +1,316 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "rust-ast-visitor.h"
> +#include "rust-ast.h"
> +#include "rust-macro-expand.h"
> +
> +namespace Rust {
> +// Visitor used to expand attributes.
> +class AttrVisitor : public AST::ASTVisitor
> +{
> +private:
> + MacroExpander &expander;
> + void maybe_expand_expr (std::unique_ptr<AST::Expr> &expr);
> + void maybe_expand_type (std::unique_ptr<AST::Type> &expr);
> +
> +public:
> + AttrVisitor (MacroExpander &expander) : expander (expander) {}
> +
> + void expand_struct_fields (std::vector<AST::StructField> &fields);
> + void expand_tuple_fields (std::vector<AST::TupleField> &fields);
> + void expand_function_params (std::vector<AST::FunctionParam> ¶ms);
> + void expand_generic_args (AST::GenericArgs &args);
> + void expand_qualified_path_type (AST::QualifiedPathType &path_type);
> + void expand_closure_params (std::vector<AST::ClosureParam> ¶ms);
> + void expand_self_param (AST::SelfParam &self_param);
> + void expand_where_clause (AST::WhereClause &where_clause);
> + void expand_trait_function_decl (AST::TraitFunctionDecl &decl);
> + void expand_trait_method_decl (AST::TraitMethodDecl &decl);
> +
> + /**
> + * Expand The current macro fragment recursively until it could not be
> + * expanded further.
> + *
> + * The return value checking works because correctly
> + * expanded fragment can never be an error (if the fragment can not be
> + * expanded, a stand-in error fragment will be returned; for fragments that
> + * could not be further expanded: the fragment prior to the expansion failure
> + * will be returned).
> + *
> + * @return Either the expanded fragment or an empty errored-out fragment
> + * indicating an expansion failure.
> + */
> + AST::ASTFragment expand_macro_fragment_recursive ()
Why is this in the header?
> + {
> + auto fragment = expander.take_expanded_fragment (*this);
> + unsigned int original_depth = expander.expansion_depth;
> + auto final_fragment = AST::ASTFragment ({}, true);
> +
> + while (fragment.should_expand ())
> + {
> + final_fragment = std::move (fragment);
> + expander.expansion_depth++;
> + // further expand the previously expanded macro fragment
> + auto new_fragment = expander.take_expanded_fragment (*this);
> + if (new_fragment.is_error ())
> + break;
> + fragment = std::move (new_fragment);
> + }
> + expander.expansion_depth = original_depth;
> + return final_fragment;
> + }
> +
> + /**
> + * Expand a set of values, erasing them if they are marked for strip, and
> + * replacing them with expanded macro nodes if necessary.
> + * This function is slightly different from `expand_pointer_allow_strip` as
> + * it can only be called in certain expansion contexts - where macro
> + * invocations are allowed.
> + *
> + * @param ctx Context to use for macro expansion
> + * @param values Iterable reference over values to replace or erase
> + * @param extractor Function to call when replacing values with the content
> + * of an expanded AST node
> + */
> + template <typename T, typename U>
> + void expand_macro_children (MacroExpander::ContextType ctx, T &values,
> + std::function<U (AST::SingleASTNode)> extractor)
> + {
> + expander.push_context (ctx);
> +
> + for (auto it = values.begin (); it != values.end ();)
> + {
> + auto &value = *it;
> +
> + // mark for stripping if required
> + value->accept_vis (*this);
> +
> + // recursively expand the children
> + auto final_fragment = expand_macro_fragment_recursive ();
> +
> + if (final_fragment.should_expand ())
> + {
> + it = values.erase (it);
> + for (auto &node : final_fragment.get_nodes ())
> + {
> + auto new_node = extractor (node);
> + if (new_node != nullptr && !new_node->is_marked_for_strip ())
> + {
> + it = values.insert (it, std::move (new_node));
> + it++;
> + }
> + }
> + }
> + else if (value->is_marked_for_strip ())
> + {
> + it = values.erase (it);
> + }
> + else
> + {
> + ++it;
> + }
> + }
> +
> + expander.pop_context ();
> + }
> +
> + template <typename T> void expand_pointer_allow_strip (T &values)
> + {
> + for (auto it = values.begin (); it != values.end ();)
> + {
> + auto &value = *it;
> +
> + // mark for stripping if required
> + value->accept_vis (*this);
> + if (value->is_marked_for_strip ())
> + {
> + it = values.erase (it);
> + }
> + else
> + {
> + ++it;
> + }
> + }
> + }
> +
> + void visit (AST::Token &) override;
> + void visit (AST::DelimTokenTree &) override;
> + void visit (AST::AttrInputMetaItemContainer &) override;
> + void visit (AST::IdentifierExpr &ident_expr) override;
> + void visit (AST::Lifetime &) override;
> + void visit (AST::LifetimeParam &) override;
> + void visit (AST::ConstGenericParam &) override;
> +
> + void visit (AST::MacroInvocation ¯o_invoc) override;
> +
> + void visit (AST::PathInExpression &path) override;
> + void visit (AST::TypePathSegment &) override;
> + void visit (AST::TypePathSegmentGeneric &segment) override;
> + void visit (AST::TypePathSegmentFunction &segment) override;
> + void visit (AST::TypePath &path) override;
> + void visit (AST::QualifiedPathInExpression &path) override;
> + void visit (AST::QualifiedPathInType &path) override;
> +
> + void visit (AST::LiteralExpr &expr) override;
> + void visit (AST::AttrInputLiteral &) override;
> + void visit (AST::MetaItemLitExpr &) override;
> + void visit (AST::MetaItemPathLit &) override;
> + void visit (AST::BorrowExpr &expr) override;
> + void visit (AST::DereferenceExpr &expr) override;
> + void visit (AST::ErrorPropagationExpr &expr) override;
> + void visit (AST::NegationExpr &expr) override;
> + void visit (AST::ArithmeticOrLogicalExpr &expr) override;
> + void visit (AST::ComparisonExpr &expr) override;
> + void visit (AST::LazyBooleanExpr &expr) override;
> + void visit (AST::TypeCastExpr &expr) override;
> + void visit (AST::AssignmentExpr &expr) override;
> + void visit (AST::CompoundAssignmentExpr &expr) override;
> + void visit (AST::GroupedExpr &expr) override;
> + void visit (AST::ArrayElemsValues &elems) override;
> + void visit (AST::ArrayElemsCopied &elems) override;
> + void visit (AST::ArrayExpr &expr) override;
> + void visit (AST::ArrayIndexExpr &expr) override;
> + void visit (AST::TupleExpr &expr) override;
> + void visit (AST::TupleIndexExpr &expr) override;
> + void visit (AST::StructExprStruct &expr) override;
> + void visit (AST::StructExprFieldIdentifier &) override;
> + void visit (AST::StructExprFieldIdentifierValue &field) override;
> +
> + void visit (AST::StructExprFieldIndexValue &field) override;
> + void visit (AST::StructExprStructFields &expr) override;
> + void visit (AST::StructExprStructBase &expr) override;
> + void visit (AST::CallExpr &expr) override;
> + void visit (AST::MethodCallExpr &expr) override;
> + void visit (AST::FieldAccessExpr &expr) override;
> + void visit (AST::ClosureExprInner &expr) override;
> +
> + void visit (AST::BlockExpr &expr) override;
> +
> + void visit (AST::ClosureExprInnerTyped &expr) override;
> + void visit (AST::ContinueExpr &expr) override;
> + void visit (AST::BreakExpr &expr) override;
> + void visit (AST::RangeFromToExpr &expr) override;
> + void visit (AST::RangeFromExpr &expr) override;
> + void visit (AST::RangeToExpr &expr) override;
> + void visit (AST::RangeFullExpr &) override;
> + void visit (AST::RangeFromToInclExpr &expr) override;
> + void visit (AST::RangeToInclExpr &expr) override;
> + void visit (AST::ReturnExpr &expr) override;
> + void visit (AST::UnsafeBlockExpr &expr) override;
> + void visit (AST::LoopExpr &expr) override;
> + void visit (AST::WhileLoopExpr &expr) override;
> + void visit (AST::WhileLetLoopExpr &expr) override;
> + void visit (AST::ForLoopExpr &expr) override;
> + void visit (AST::IfExpr &expr) override;
> + void visit (AST::IfExprConseqElse &expr) override;
> + void visit (AST::IfExprConseqIf &expr) override;
> + void visit (AST::IfExprConseqIfLet &expr) override;
> + void visit (AST::IfLetExpr &expr) override;
> + void visit (AST::IfLetExprConseqElse &expr) override;
> + void visit (AST::IfLetExprConseqIf &expr) override;
> + void visit (AST::IfLetExprConseqIfLet &expr) override;
> + void visit (AST::MatchExpr &expr) override;
> + void visit (AST::AwaitExpr &expr) override;
> + void visit (AST::AsyncBlockExpr &expr) override;
> + void visit (AST::TypeParam ¶m) override;
> + void visit (AST::LifetimeWhereClauseItem &) override;
> + void visit (AST::TypeBoundWhereClauseItem &item) override;
> + void visit (AST::Method &method) override;
> + void visit (AST::Module &module) override;
> + void visit (AST::ExternCrate &crate) override;
> + void visit (AST::UseTreeGlob &) override;
> + void visit (AST::UseTreeList &) override;
> + void visit (AST::UseTreeRebind &) override;
> + void visit (AST::UseDeclaration &use_decl) override;
> + void visit (AST::Function &function) override;
> + void visit (AST::TypeAlias &type_alias) override;
> + void visit (AST::StructStruct &struct_item) override;
> + void visit (AST::TupleStruct &tuple_struct) override;
> + void visit (AST::EnumItem &item) override;
> + void visit (AST::EnumItemTuple &item) override;
> + void visit (AST::EnumItemStruct &item) override;
> + void visit (AST::EnumItemDiscriminant &item) override;
> + void visit (AST::Enum &enum_item) override;
> + void visit (AST::Union &union_item) override;
> + void visit (AST::ConstantItem &const_item) override;
> + void visit (AST::StaticItem &static_item) override;
> + void visit (AST::TraitItemFunc &item) override;
> + void visit (AST::TraitItemMethod &item) override;
> + void visit (AST::TraitItemConst &item) override;
> + void visit (AST::TraitItemType &item) override;
> + void visit (AST::Trait &trait) override;
> + void visit (AST::InherentImpl &impl) override;
> + void visit (AST::TraitImpl &impl) override;
> + void visit (AST::ExternalStaticItem &item) override;
> + void visit (AST::ExternalFunctionItem &item) override;
> + void visit (AST::ExternBlock &block) override;
> +
> + // I don't think it would be possible to strip macros without expansion
> + void visit (AST::MacroMatchFragment &) override;
> + void visit (AST::MacroMatchRepetition &) override;
> + void visit (AST::MacroMatcher &) override;
> + void visit (AST::MacroRulesDefinition &rules_def) override;
> + void visit (AST::MetaItemPath &) override;
> + void visit (AST::MetaItemSeq &) override;
> + void visit (AST::MetaWord &) override;
> + void visit (AST::MetaNameValueStr &) override;
> + void visit (AST::MetaListPaths &) override;
> + void visit (AST::MetaListNameValueStr &) override;
> + void visit (AST::LiteralPattern &) override;
> + void visit (AST::IdentifierPattern &pattern) override;
> + void visit (AST::WildcardPattern &) override;
> + void visit (AST::RangePatternBoundLiteral &) override;
> + void visit (AST::RangePatternBoundPath &bound) override;
> + void visit (AST::RangePatternBoundQualPath &bound) override;
> + void visit (AST::RangePattern &pattern) override;
> + void visit (AST::ReferencePattern &pattern) override;
> + void visit (AST::StructPatternFieldTuplePat &field) override;
> + void visit (AST::StructPatternFieldIdentPat &field) override;
> + void visit (AST::StructPatternFieldIdent &field) override;
> + void visit (AST::StructPattern &pattern) override;
> + void visit (AST::TupleStructItemsNoRange &tuple_items) override;
> + void visit (AST::TupleStructItemsRange &tuple_items) override;
> + void visit (AST::TupleStructPattern &pattern) override;
> + void visit (AST::TuplePatternItemsMultiple &tuple_items) override;
> + void visit (AST::TuplePatternItemsRanged &tuple_items) override;
> + void visit (AST::TuplePattern &pattern) override;
> + void visit (AST::GroupedPattern &pattern) override;
> + void visit (AST::SlicePattern &pattern) override;
> +
> + void visit (AST::EmptyStmt &) override;
> + void visit (AST::LetStmt &stmt) override;
> + void visit (AST::ExprStmtWithoutBlock &stmt) override;
> + void visit (AST::ExprStmtWithBlock &stmt) override;
> +
> + void visit (AST::TraitBound &bound) override;
> + void visit (AST::ImplTraitType &type) override;
> + void visit (AST::TraitObjectType &type) override;
> + void visit (AST::ParenthesisedType &type) override;
> + void visit (AST::ImplTraitTypeOneBound &type) override;
> + void visit (AST::TraitObjectTypeOneBound &type) override;
> + void visit (AST::TupleType &type) override;
> + void visit (AST::NeverType &) override;
> + void visit (AST::RawPointerType &type) override;
> + void visit (AST::ReferenceType &type) override;
> + void visit (AST::ArrayType &type) override;
> + void visit (AST::SliceType &type) override;
> + void visit (AST::InferredType &) override;
> + void visit (AST::BareFunctionType &type) override;
> +};
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-macro-builtins.cc b/gcc/rust/expand/rust-macro-builtins.cc
> new file mode 100644
> index 00000000000..5eace13d197
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-builtins.cc
> @@ -0,0 +1,484 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "rust-macro-builtins.h"
> +#include "rust-diagnostics.h"
> +#include "rust-expr.h"
> +#include "rust-session-manager.h"
> +#include "rust-macro-invoc-lexer.h"
> +#include "rust-lex.h"
> +#include "rust-parse.h"
> +
> +namespace Rust {
> +namespace {
> +std::unique_ptr<AST::Expr>
> +make_string (Location locus, std::string value)
> +{
> + return std::unique_ptr<AST::Expr> (
> + new AST::LiteralExpr (value, AST::Literal::STRING,
> + PrimitiveCoreType::CORETYPE_STR, {}, locus));
> +}
> +
> +/* Match the end token of a macro given the start delimiter of the macro */
> +
> +static inline TokenId
> +macro_end_token (AST::DelimTokenTree &invoc_token_tree,
> + Parser<MacroInvocLexer> &parser)
> +{
> + auto last_token_id = TokenId::RIGHT_CURLY;
> + switch (invoc_token_tree.get_delim_type ())
> + {
> + case AST::DelimType::PARENS:
> + last_token_id = TokenId::RIGHT_PAREN;
> + rust_assert (parser.skip_token (LEFT_PAREN));
> + break;
> +
> + case AST::DelimType::CURLY:
> + rust_assert (parser.skip_token (LEFT_CURLY));
> + break;
> +
> + case AST::DelimType::SQUARE:
> + last_token_id = TokenId::RIGHT_SQUARE;
> + rust_assert (parser.skip_token (LEFT_SQUARE));
> + break;
> + }
> +
> + return last_token_id;
> +}
> +
> +/* Parse a single string literal from the given delimited token tree,
> + and return the LiteralExpr for it. Allow for an optional trailing comma,
> + but otherwise enforce that these are the only tokens. */
> +
> +std::unique_ptr<AST::LiteralExpr>
> +parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree,
> + Location invoc_locus)
> +{
> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> + Parser<MacroInvocLexer> parser (lex);
> +
> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> +
> + std::unique_ptr<AST::LiteralExpr> lit_expr = nullptr;
> +
> + if (parser.peek_current_token ()->get_id () == STRING_LITERAL)
> + {
> + lit_expr = parser.parse_literal_expr ();
> + parser.maybe_skip_token (COMMA);
> + if (parser.peek_current_token ()->get_id () != last_token_id)
> + {
> + lit_expr = nullptr;
> + rust_error_at (invoc_locus, "macro takes 1 argument");
> + }
> + }
> + else if (parser.peek_current_token ()->get_id () == last_token_id)
> + rust_error_at (invoc_locus, "macro takes 1 argument");
> + else
> + rust_error_at (invoc_locus, "argument must be a string literal");
> +
> + parser.skip_token (last_token_id);
> +
> + return lit_expr;
> +}
> +
> +/* Treat PATH as a path relative to the source file currently being
> + compiled, and return the absolute path for it. */
> +
> +std::string
> +source_relative_path (std::string path, Location locus)
> +{
> + std::string compile_fname
> + = Session::get_instance ().linemap->location_file (locus);
> +
> + auto dir_separator_pos = compile_fname.rfind (file_separator);
> +
> + /* If there is no file_separator in the path, use current dir ('.'). */
> + std::string dirname;
> + if (dir_separator_pos == std::string::npos)
> + dirname = std::string (".") + file_separator;
> + else
> + dirname = compile_fname.substr (0, dir_separator_pos) + file_separator;
> +
> + return dirname + path;
> +}
> +
> +/* Read the full contents of the file FILENAME and return them in a vector.
> + FIXME: platform specific. */
> +
> +std::vector<uint8_t>
> +load_file_bytes (const char *filename)
> +{
> + RAIIFile file_wrap (filename);
> + if (file_wrap.get_raw () == nullptr)
> + {
> + rust_error_at (Location (), "cannot open filename %s: %m", filename);
> + return std::vector<uint8_t> ();
> + }
> +
> + FILE *f = file_wrap.get_raw ();
> + fseek (f, 0L, SEEK_END);
> + long fsize = ftell (f);
> + fseek (f, 0L, SEEK_SET);
> +
> + std::vector<uint8_t> buf (fsize);
> +
> + if (fread (&buf[0], fsize, 1, f) != 1)
> + {
> + rust_error_at (Location (), "error reading file %s: %m", filename);
> + return std::vector<uint8_t> ();
> + }
> +
> + return buf;
> +}
> +} // namespace
> +
> +AST::ASTFragment
> +MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + rust_debug ("assert!() called");
> +
> + return AST::ASTFragment::create_error ();
> +}
> +
> +AST::ASTFragment
> +MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto current_file
> + = Session::get_instance ().linemap->location_file (invoc_locus);
> + auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file));
> +
> + return AST::ASTFragment ({file_str});
> +}
> +
> +AST::ASTFragment
> +MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto current_column
> + = Session::get_instance ().linemap->location_to_column (invoc_locus);
> +
> + auto column_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> + new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT,
> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
> +
> + return AST::ASTFragment ({column_no});
> +}
> +
> +/* Expand builtin macro include_bytes!("filename"), which includes the contents
> + of the given file as reference to a byte array. Yields an expression of type
> + &'static [u8; N]. */
> +
> +AST::ASTFragment
> +MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + /* Get target filename from the macro invocation, which is treated as a path
> + relative to the include!-ing file (currently being compiled). */
> + auto lit_expr
> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> + if (lit_expr == nullptr)
> + return AST::ASTFragment::create_error ();
> +
> + std::string target_filename
> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> +
> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
> +
> + /* Is there a more efficient way to do this? */
Well, first of all you don't need to read all of the file at once, second you
could pre-allocate 'elts' to the correct size and final you could obviously
choose a cheaper representation ;)
> + std::vector<std::unique_ptr<AST::Expr>> elts;
> + for (uint8_t b : bytes)
> + {
> + elts.emplace_back (
> + new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE,
> + PrimitiveCoreType::CORETYPE_U8,
> + {} /* outer_attrs */, invoc_locus));
> + }
> +
> + auto elems = std::unique_ptr<AST::ArrayElems> (
> + new AST::ArrayElemsValues (std::move (elts), invoc_locus));
> +
> + auto array = std::unique_ptr<AST::Expr> (
> + new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus));
> +
> + auto borrow = std::unique_ptr<AST::Expr> (
> + new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus));
> +
> + auto node = AST::SingleASTNode (std::move (borrow));
> + return AST::ASTFragment ({node});
> +}
> +
> +/* Expand builtin macro include_str!("filename"), which includes the contents
> + of the given file as a string. The file must be UTF-8 encoded. Yields an
> + expression of type &'static str. */
> +
> +AST::ASTFragment
> +MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + /* Get target filename from the macro invocation, which is treated as a path
> + relative to the include!-ing file (currently being compiled). */
> + auto lit_expr
> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> + if (lit_expr == nullptr)
> + return AST::ASTFragment::create_error ();
> +
> + std::string target_filename
> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> +
> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
> +
> + /* FIXME: Enforce that the file contents are valid UTF-8. */
> + std::string str ((const char *) &bytes[0], bytes.size ());
> +
> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
> + return AST::ASTFragment ({node});
> +}
> +
> +/* Expand builtin macro compile_error!("error"), which forces a compile error
> + during the compile time. */
> +AST::ASTFragment
> +MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto lit_expr
> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> + if (lit_expr == nullptr)
> + return AST::ASTFragment::create_error ();
> +
> + std::string error_string = lit_expr->as_string ();
> + rust_error_at (invoc_locus, "%s", error_string.c_str ());
> +
> + return AST::ASTFragment::create_error ();
> +}
> +
> +/* Expand builtin macro concat!(), which joins all the literal parameters
> + into a string with no delimiter. */
> +
> +AST::ASTFragment
> +MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> + Parser<MacroInvocLexer> parser (lex);
> +
> + auto str = std::string ();
> + bool has_error = false;
> +
> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> +
> + /* NOTE: concat! could accept no argument, so we don't have any checks here */
> + while (parser.peek_current_token ()->get_id () != last_token_id)
> + {
> + auto lit_expr = parser.parse_literal_expr ();
> + if (lit_expr)
> + {
> + str += lit_expr->as_string ();
> + }
> + else
> + {
> + auto current_token = parser.peek_current_token ();
> + rust_error_at (current_token->get_locus (),
> + "argument must be a constant literal");
> + has_error = true;
> + // Just crash if the current token can't be skipped
> + rust_assert (parser.skip_token (current_token->get_id ()));
> + }
> + parser.maybe_skip_token (COMMA);
> + }
> +
> + parser.skip_token (last_token_id);
> +
> + if (has_error)
> + return AST::ASTFragment::create_error ();
> +
> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
> + return AST::ASTFragment ({node});
> +}
> +
> +/* Expand builtin macro env!(), which inspects an environment variable at
> + compile time. */
> +
> +AST::ASTFragment
> +MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> + Parser<MacroInvocLexer> parser (lex);
> +
> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> +
> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
> + {
> + if (parser.peek_current_token ()->get_id () == last_token_id)
> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
> + else
> + rust_error_at (parser.peek_current_token ()->get_locus (),
> + "argument must be a string literal");
> + return AST::ASTFragment::create_error ();
> + }
> +
> + auto lit_expr = parser.parse_literal_expr ();
> + auto comma_skipped = parser.maybe_skip_token (COMMA);
> +
> + std::unique_ptr<AST::LiteralExpr> error_expr = nullptr;
> +
> + if (parser.peek_current_token ()->get_id () != last_token_id)
> + {
> + if (!comma_skipped)
> + {
> + rust_error_at (parser.peek_current_token ()->get_locus (),
> + "expected token: %<,%>");
> + return AST::ASTFragment::create_error ();
> + }
> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
> + {
> + rust_error_at (parser.peek_current_token ()->get_locus (),
> + "argument must be a string literal");
> + return AST::ASTFragment::create_error ();
> + }
> +
> + error_expr = parser.parse_literal_expr ();
> + parser.maybe_skip_token (COMMA);
> + }
> +
> + if (parser.peek_current_token ()->get_id () != last_token_id)
> + {
> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
> + return AST::ASTFragment::create_error ();
> + }
> +
> + parser.skip_token (last_token_id);
> +
> + auto env_value = getenv (lit_expr->as_string ().c_str ());
> +
> + if (env_value == nullptr)
> + {
> + if (error_expr == nullptr)
> + rust_error_at (invoc_locus, "environment variable %qs not defined",
> + lit_expr->as_string ().c_str ());
> + else
> + rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ());
> + return AST::ASTFragment::create_error ();
> + }
> +
> + auto node = AST::SingleASTNode (make_string (invoc_locus, env_value));
> + return AST::ASTFragment ({node});
> +}
> +
> +AST::ASTFragment
> +MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + // only parse if not already parsed
> + if (!invoc.is_parsed ())
> + {
> + std::unique_ptr<AST::AttrInputMetaItemContainer> converted_input (
> + invoc.get_delim_tok_tree ().parse_to_meta_item ());
> +
> + if (converted_input == nullptr)
> + {
> + rust_debug ("DEBUG: failed to parse macro to meta item");
> + // TODO: do something now? is this an actual error?
> + }
> + else
> + {
> + std::vector<std::unique_ptr<AST::MetaItemInner>> meta_items (
> + std::move (converted_input->get_items ()));
> + invoc.set_meta_item_output (std::move (meta_items));
> + }
> + }
> +
> + /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg
> + * attributes */
> + if (invoc.get_meta_items ().size () != 1)
> + return AST::ASTFragment::create_error ();
> +
> + bool result = invoc.get_meta_items ()[0]->check_cfg_predicate (
> + Session::get_instance ());
> + auto literal_exp = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> + new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL,
> + PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus)));
> +
> + return AST::ASTFragment ({literal_exp});
> +}
> +
> +/* Expand builtin macro include!(), which includes a source file at the current
> + scope compile time. */
> +
> +AST::ASTFragment
> +MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + /* Get target filename from the macro invocation, which is treated as a path
> + relative to the include!-ing file (currently being compiled). */
> + auto lit_expr
> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> + if (lit_expr == nullptr)
> + return AST::ASTFragment::create_error ();
> +
> + std::string filename
> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> + auto target_filename
> + = Rust::Session::get_instance ().include_extra_file (std::move (filename));
> +
> + RAIIFile target_file (target_filename);
> + Linemap *linemap = Session::get_instance ().linemap;
> +
> + if (!target_file.ok ())
> + {
> + rust_error_at (lit_expr->get_locus (),
> + "cannot open included file %qs: %m", target_filename);
> + return AST::ASTFragment::create_error ();
> + }
> +
> + rust_debug ("Attempting to parse included file %s", target_filename);
> +
> + Lexer lex (target_filename, std::move (target_file), linemap);
> + Parser<Lexer> parser (lex);
> +
> + auto parsed_items = parser.parse_items ();
> + bool has_error = !parser.get_errors ().empty ();
> +
> + for (const auto &error : parser.get_errors ())
> + error.emit_error ();
> +
> + if (has_error)
> + {
> + // inform the user that the errors above are from a included file
> + rust_inform (invoc_locus, "included from here");
> + return AST::ASTFragment::create_error ();
> + }
> +
> + std::vector<AST::SingleASTNode> nodes{};
> + for (auto &item : parsed_items)
> + {
> + AST::SingleASTNode node (std::move (item));
> + nodes.push_back (node);
> + }
> +
> + return AST::ASTFragment (nodes);
> +}
> +
> +AST::ASTFragment
> +MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc)
> +{
> + auto current_line
> + = Session::get_instance ().linemap->location_to_line (invoc_locus);
> +
> + auto line_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> + new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT,
> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
> +
> + return AST::ASTFragment ({line_no});
> +}
> +
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-macro-builtins.h b/gcc/rust/expand/rust-macro-builtins.h
> new file mode 100644
> index 00000000000..91f3727d450
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-builtins.h
> @@ -0,0 +1,107 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef RUST_MACRO_BUILTINS_H
> +#define RUST_MACRO_BUILTINS_H
> +
> +#include "rust-ast.h"
> +#include "rust-location.h"
> +
> +/**
> + * This class provides a list of builtin macros implemented by the compiler.
> + * The functions defined are called "builtin transcribers" in that they replace
> + * the transcribing part of a macro definition.
> + *
> + * Like regular macro transcribers, they are responsible for building and
> + * returning an AST fragment: basically a vector of AST nodes put together.
> + *
> + * Unlike regular declarative macros where each match arm has its own associated
> + * transcriber, builtin transcribers are responsible for handling all match arms
> + * of the macro. This means that you should take extra care when implementing a
> + * builtin containing multiple match arms: You will probably need to do some
> + * lookahead in order to determine which match arm the user intended to use.
> + *
> + * An example of this is the `assert!()` macro:
> + *
> + * ```
> + * macro_rules! assert {
> + * ($cond:expr $(,)?) => {{ ... }};
> + * ($cond : expr, $ ($arg : tt) +) = > {{ ... }};
> + * }
> + * ```
> + *
> + * If more tokens exist beyond the optional comma, they need to be handled as
> + * a token-tree for a custom panic message.
> + *
> + * These builtin macros with empty transcribers are defined in the standard
> + * library. They are marked with a special attribute, `#[rustc_builtin_macro]`.
> + * When this attribute is present on a macro definition, the compiler should
> + * look for an associated transcriber in the mappings. Meaning that you must
> + * remember to insert your transcriber in the `builtin_macros` map of the
> + *`Mappings`.
> + *
> + * This map is built as a static variable in the `insert_macro_def()` method
> + * of the `Mappings` class.
> + */
> +
> +/* If assert is defined as a macro this file will not parse, so undefine this
> + before continuing. */
> +#ifdef assert
> +#undef assert
> +#endif
> +
> +namespace Rust {
> +class MacroBuiltin
> +{
> +public:
> + static AST::ASTFragment assert (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment file (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment column (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment include_bytes (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment include_str (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment compile_error (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment concat (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment env (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment cfg (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment include (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +
> + static AST::ASTFragment line (Location invoc_locus,
> + AST::MacroInvocData &invoc);
> +};
> +} // namespace Rust
> +
> +#endif // RUST_MACRO_BUILTINS_H
> diff --git a/gcc/rust/expand/rust-macro-expand.cc b/gcc/rust/expand/rust-macro-expand.cc
> new file mode 100644
> index 00000000000..1d57e394220
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-expand.cc
> @@ -0,0 +1,1012 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "rust-macro-expand.h"
> +#include "rust-macro-substitute-ctx.h"
> +#include "rust-ast-full.h"
> +#include "rust-ast-visitor.h"
> +#include "rust-diagnostics.h"
> +#include "rust-parse.h"
> +#include "rust-attribute-visitor.h"
> +
> +namespace Rust {
> +AST::ASTFragment
> +MacroExpander::expand_decl_macro (Location invoc_locus,
> + AST::MacroInvocData &invoc,
> + AST::MacroRulesDefinition &rules_def,
> + bool semicolon)
> +{
> + // ensure that both invocation and rules are in a valid state
> + rust_assert (!invoc.is_marked_for_strip ());
> + rust_assert (!rules_def.is_marked_for_strip ());
> + rust_assert (rules_def.get_macro_rules ().size () > 0);
> +
> + /* probably something here about parsing invoc and rules def token trees to
> + * token stream. if not, how would parser handle the captures of exprs and
> + * stuff? on the other hand, token trees may be kind of useful in rules def as
> + * creating a point where recursion can occur (like having
> + * "compare_macro_match" and then it calling itself when it finds delimiters)
> + */
> +
> + /* find matching rule to invoc token tree, based on macro rule's matcher. if
> + * none exist, error.
> + * - specifically, check each matcher in order. if one fails to match, move
> + * onto next. */
> + /* TODO: does doing this require parsing expressions and whatever in the
> + * invoc? if so, might as well save the results if referenced using $ or
> + * whatever. If not, do another pass saving them. Except this is probably
> + * useless as different rules could have different starting points for exprs
> + * or whatever. Decision trees could avoid this, but they have their own
> + * issues. */
> + /* TODO: will need to modify the parser so that it can essentially "catch"
> + * errors - maybe "try_parse_expr" or whatever methods. */
> + // this technically creates a back-tracking parser - this will be the
> + // implementation style
> +
> + /* then, after results are saved, generate the macro output from the
> + * transcriber token tree. if i understand this correctly, the macro
> + * invocation gets replaced by the transcriber tokens, except with
> + * substitutions made (e.g. for $i variables) */
> +
> + /* TODO: it is probably better to modify AST::Token to store a pointer to a
> + * Lexer::Token (rather than being converted) - i.e. not so much have
> + * AST::Token as a Token but rather a TokenContainer (as it is another type of
> + * TokenTree). This will prevent re-conversion of Tokens between each type
> + * all the time, while still allowing the heterogenous storage of token trees.
> + */
> +
> + AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree ();
> +
> + // find matching arm
> + AST::MacroRule *matched_rule = nullptr;
> + std::map<std::string, MatchedFragmentContainer> matched_fragments;
> + for (auto &rule : rules_def.get_rules ())
> + {
> + sub_stack.push ();
> + bool did_match_rule = try_match_rule (rule, invoc_token_tree);
> + matched_fragments = sub_stack.pop ();
> +
> + if (did_match_rule)
> + {
> + // // Debugging
> + // for (auto &kv : matched_fragments)
> + // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (),
> + // kv.second.get_fragments ().size (),
> + // kv.second.get_kind ()
> + // == MatchedFragmentContainer::Kind::Repetition
> + // ? "repetition"
> + // : "metavar");
> +
> + matched_rule = &rule;
> + break;
> + }
> + }
> +
> + if (matched_rule == nullptr)
> + {
> + RichLocation r (invoc_locus);
> + r.add_range (rules_def.get_locus ());
> + rust_error_at (r, "Failed to match any rule within macro");
> + return AST::ASTFragment::create_error ();
> + }
> +
> + return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments,
> + semicolon, peek_context ());
> +}
> +
> +void
> +MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon)
> +{
> + if (depth_exceeds_recursion_limit ())
> + {
> + rust_error_at (invoc.get_locus (), "reached recursion limit");
> + return;
> + }
> +
> + AST::MacroInvocData &invoc_data = invoc.get_invoc_data ();
> +
> + // ??
> + // switch on type of macro:
> + // - '!' syntax macro (inner switch)
> + // - procedural macro - "A token-based function-like macro"
> + // - 'macro_rules' (by example/pattern-match) macro? or not? "an
> + // AST-based function-like macro"
> + // - else is unreachable
> + // - attribute syntax macro (inner switch)
> + // - procedural macro attribute syntax - "A token-based attribute
> + // macro"
> + // - legacy macro attribute syntax? - "an AST-based attribute macro"
> + // - non-macro attribute: mark known
> + // - else is unreachable
> + // - derive macro (inner switch)
> + // - derive or legacy derive - "token-based" vs "AST-based"
> + // - else is unreachable
> + // - derive container macro - unreachable
> +
> + // lookup the rules for this macro
> + NodeId resolved_node = UNKNOWN_NODEID;
> + NodeId source_node = UNKNOWN_NODEID;
> + if (has_semicolon)
> + source_node = invoc.get_macro_node_id ();
> + else
> + source_node = invoc.get_pattern_node_id ();
> + auto seg
> + = Resolver::CanonicalPath::new_seg (source_node,
> + invoc_data.get_path ().as_string ());
> +
> + bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node);
> + if (!found)
> + {
> + rust_error_at (invoc.get_locus (), "unknown macro: [%s]",
> + seg.get ().c_str ());
> + return;
> + }
> +
> + // lookup the rules
> + AST::MacroRulesDefinition *rules_def = nullptr;
> + bool ok = mappings->lookup_macro_def (resolved_node, &rules_def);
> + rust_assert (ok);
> +
> + auto fragment = AST::ASTFragment::create_error ();
> +
> + if (rules_def->is_builtin ())
> + fragment
> + = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data);
> + else
> + fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def,
> + has_semicolon);
> +
> + set_expanded_fragment (std::move (fragment));
> +}
> +
> +/* Determines whether any cfg predicate is false and hence item with attributes
> + * should be stripped. Note that attributes must be expanded before calling. */
> +bool
> +MacroExpander::fails_cfg (const AST::AttrVec &attrs) const
> +{
> + for (const auto &attr : attrs)
> + {
> + if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session))
> + return true;
> + }
> + return false;
> +}
> +
> +/* Determines whether any cfg predicate is false and hence item with attributes
> + * should be stripped. Will expand attributes as well. */
> +bool
> +MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const
> +{
> + // TODO: maybe have something that strips cfg attributes that evaluate true?
> + for (auto &attr : attrs)
> + {
> + if (attr.get_path () == "cfg")
> + {
> + if (!attr.is_parsed_to_meta_item ())
> + attr.parse_attr_to_meta_item ();
> +
> + // DEBUG
> + if (!attr.is_parsed_to_meta_item ())
> + rust_debug ("failed to parse attr to meta item, right before "
> + "cfg predicate check");
> + else
> + rust_debug ("attr has been successfully parsed to meta item, "
> + "right before cfg predicate check");
> +
> + if (!attr.check_cfg_predicate (session))
> + {
> + // DEBUG
> + rust_debug (
> + "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m",
> + attr.as_string ().c_str ());
> +
> + return true;
> + }
> + else
> + {
> + // DEBUG
> + rust_debug ("cfg predicate succeeded for attribute: "
> + "\033[0;31m'%s'\033[0m",
> + attr.as_string ().c_str ());
> + }
> + }
> + }
> + return false;
> +}
> +
> +// Expands cfg_attr attributes.
> +void
> +MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs)
> +{
> + for (std::size_t i = 0; i < attrs.size (); i++)
> + {
> + auto &attr = attrs[i];
> + if (attr.get_path () == "cfg_attr")
> + {
> + if (!attr.is_parsed_to_meta_item ())
> + attr.parse_attr_to_meta_item ();
> +
> + if (attr.check_cfg_predicate (session))
> + {
> + // split off cfg_attr
> + AST::AttrVec new_attrs = attr.separate_cfg_attrs ();
> +
> + // remove attr from vector
> + attrs.erase (attrs.begin () + i);
> +
> + // add new attrs to vector
> + attrs.insert (attrs.begin () + i,
> + std::make_move_iterator (new_attrs.begin ()),
> + std::make_move_iterator (new_attrs.end ()));
> + }
> +
> + /* do something - if feature (first token in tree) is in fact enabled,
> + * make tokens listed afterwards into attributes. i.e.: for
> + * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add
> + * attributes [wow1] and [wow2] to attribute list. This can also be
> + * recursive, so check for expanded attributes being recursive and
> + * possibly recursively call the expand_attrs? */
> + }
> + else
> + {
> + i++;
> + }
> + }
> + attrs.shrink_to_fit ();
> +}
> +
> +void
> +MacroExpander::expand_crate ()
> +{
> + NodeId scope_node_id = crate.get_node_id ();
> + resolver->get_macro_scope ().push (scope_node_id);
> +
> + /* fill macro/decorator map from init list? not sure where init list comes
> + * from? */
> +
> + // TODO: does cfg apply for inner attributes? research.
> + // the apparent answer (from playground test) is yes
> +
> + // expand crate cfg_attr attributes
> + expand_cfg_attrs (crate.inner_attrs);
> +
> + if (fails_cfg_with_expand (crate.inner_attrs))
> + {
> + // basically, delete whole crate
> + crate.strip_crate ();
> + // TODO: maybe create warning here? probably not desired behaviour
> + }
> + // expand module attributes?
> +
> + push_context (ITEM);
> +
> + // expand attributes recursively and strip items if required
> + AttrVisitor attr_visitor (*this);
> + auto &items = crate.items;
> + for (auto it = items.begin (); it != items.end ();)
> + {
> + auto &item = *it;
> +
> + // mark for stripping if required
> + item->accept_vis (attr_visitor);
> +
> + auto fragment = take_expanded_fragment (attr_visitor);
> + if (fragment.should_expand ())
> + {
> + // Remove the current expanded invocation
> + it = items.erase (it);
> + for (auto &node : fragment.get_nodes ())
> + {
> + it = items.insert (it, node.take_item ());
> + it++;
> + }
> + }
> + else if (item->is_marked_for_strip ())
> + it = items.erase (it);
> + else
> + it++;
> + }
> +
> + pop_context ();
> +
> + // TODO: should recursive attribute and macro expansion be done in the same
> + // transversal? Or in separate ones like currently?
> +
> + // expand module tree recursively
> +
> + // post-process
> +
> + // extract exported macros?
> +}
> +
> +bool
> +MacroExpander::depth_exceeds_recursion_limit () const
> +{
> + return expansion_depth >= cfg.recursion_limit;
> +}
> +
> +bool
> +MacroExpander::try_match_rule (AST::MacroRule &match_rule,
> + AST::DelimTokenTree &invoc_token_tree)
> +{
> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> + Parser<MacroInvocLexer> parser (lex);
> +
> + AST::MacroMatcher &matcher = match_rule.get_matcher ();
> +
> + expansion_depth++;
> + if (!match_matcher (parser, matcher))
> + {
> + expansion_depth--;
> + return false;
> + }
> + expansion_depth--;
> +
> + bool used_all_input_tokens = parser.skip_token (END_OF_FILE);
> + return used_all_input_tokens;
> +}
> +
> +bool
> +MacroExpander::match_fragment (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchFragment &fragment)
> +{
> + switch (fragment.get_frag_spec ().get_kind ())
> + {
> + case AST::MacroFragSpec::EXPR:
> + parser.parse_expr ();
> + break;
> +
> + case AST::MacroFragSpec::BLOCK:
> + parser.parse_block_expr ();
> + break;
> +
> + case AST::MacroFragSpec::IDENT:
> + parser.parse_identifier_pattern ();
> + break;
> +
> + case AST::MacroFragSpec::LITERAL:
> + parser.parse_literal_expr ();
> + break;
> +
> + case AST::MacroFragSpec::ITEM:
> + parser.parse_item (false);
> + break;
> +
> + case AST::MacroFragSpec::TY:
> + parser.parse_type ();
> + break;
> +
> + case AST::MacroFragSpec::PAT:
> + parser.parse_pattern ();
> + break;
> +
> + case AST::MacroFragSpec::PATH:
> + parser.parse_path_in_expression ();
> + break;
> +
> + case AST::MacroFragSpec::VIS:
> + parser.parse_visibility ();
> + break;
> +
> + case AST::MacroFragSpec::STMT: {
> + auto restrictions = ParseRestrictions ();
> + restrictions.consume_semi = false;
> + parser.parse_stmt (restrictions);
> + break;
> + }
> +
> + case AST::MacroFragSpec::LIFETIME:
> + parser.parse_lifetime_params ();
> + break;
> +
> + // is meta attributes?
> + case AST::MacroFragSpec::META:
> + parser.parse_attribute_body ();
> + break;
> +
> + case AST::MacroFragSpec::TT:
> + parser.parse_token_tree ();
> + break;
> +
> + // i guess we just ignore invalid and just error out
> + case AST::MacroFragSpec::INVALID:
> + return false;
> + }
> +
> + // it matches if the parser did not produce errors trying to parse that type
> + // of item
> + return !parser.has_errors ();
> +}
> +
> +bool
> +MacroExpander::match_matcher (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatcher &matcher)
> +{
> + if (depth_exceeds_recursion_limit ())
> + {
> + rust_error_at (matcher.get_match_locus (), "reached recursion limit");
> + return false;
> + }
> +
> + auto delimiter = parser.peek_current_token ();
> +
> + // this is used so we can check that we delimit the stream correctly.
> + switch (delimiter->get_id ())
> + {
> + case LEFT_PAREN: {
> + if (!parser.skip_token (LEFT_PAREN))
> + return false;
> + }
> + break;
> +
> + case LEFT_SQUARE: {
> + if (!parser.skip_token (LEFT_SQUARE))
> + return false;
> + }
> + break;
> +
> + case LEFT_CURLY: {
> + if (!parser.skip_token (LEFT_CURLY))
> + return false;
> + }
> + break;
> + default:
> + gcc_unreachable ();
> + }
> +
> + const MacroInvocLexer &source = parser.get_token_source ();
> +
> + for (auto &match : matcher.get_matches ())
> + {
> + size_t offs_begin = source.get_offs ();
> +
> + switch (match->get_macro_match_type ())
> + {
> + case AST::MacroMatch::MacroMatchType::Fragment: {
> + AST::MacroMatchFragment *fragment
> + = static_cast<AST::MacroMatchFragment *> (match.get ());
> + if (!match_fragment (parser, *fragment))
> + return false;
> +
> + // matched fragment get the offset in the token stream
> + size_t offs_end = source.get_offs ();
> + sub_stack.insert_metavar (
> + MatchedFragment (fragment->get_ident (), offs_begin, offs_end));
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Tok: {
> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
> + if (!match_token (parser, *tok))
> + return false;
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Repetition: {
> + AST::MacroMatchRepetition *rep
> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
> + if (!match_repetition (parser, *rep))
> + return false;
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Matcher: {
> + AST::MacroMatcher *m
> + = static_cast<AST::MacroMatcher *> (match.get ());
> + expansion_depth++;
> + if (!match_matcher (parser, *m))
> + {
> + expansion_depth--;
> + return false;
> + }
> + expansion_depth--;
> + }
> + break;
> + }
> + }
> +
> + switch (delimiter->get_id ())
> + {
> + case LEFT_PAREN: {
> + if (!parser.skip_token (RIGHT_PAREN))
> + return false;
> + }
> + break;
> +
> + case LEFT_SQUARE: {
> + if (!parser.skip_token (RIGHT_SQUARE))
> + return false;
> + }
> + break;
> +
> + case LEFT_CURLY: {
> + if (!parser.skip_token (RIGHT_CURLY))
> + return false;
> + }
> + break;
> + default:
> + gcc_unreachable ();
> + }
> +
> + return true;
> +}
> +
> +bool
> +MacroExpander::match_token (Parser<MacroInvocLexer> &parser, AST::Token &token)
> +{
> + // FIXME this needs to actually match the content and the type
> + return parser.skip_token (token.get_id ());
> +}
> +
> +bool
> +MacroExpander::match_n_matches (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchRepetition &rep,
> + size_t &match_amount, size_t lo_bound,
> + size_t hi_bound)
> +{
> + match_amount = 0;
> + auto &matches = rep.get_matches ();
> +
> + const MacroInvocLexer &source = parser.get_token_source ();
> + while (true)
> + {
> + // If the current token is a closing macro delimiter, break away.
> + // TODO: Is this correct?
> + auto t_id = parser.peek_current_token ()->get_id ();
> + if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY)
> + break;
> +
> + // Skip parsing a separator on the first match, otherwise consume it.
> + // If it isn't present, this is an error
> + if (rep.has_sep () && match_amount > 0)
> + if (!match_token (parser, *rep.get_sep ()))
> + break;
> +
> + bool valid_current_match = false;
> + for (auto &match : matches)
> + {
> + size_t offs_begin = source.get_offs ();
> + switch (match->get_macro_match_type ())
> + {
> + case AST::MacroMatch::MacroMatchType::Fragment: {
> + AST::MacroMatchFragment *fragment
> + = static_cast<AST::MacroMatchFragment *> (match.get ());
> + valid_current_match = match_fragment (parser, *fragment);
> +
> + // matched fragment get the offset in the token stream
> + size_t offs_end = source.get_offs ();
> +
> + // The main difference with match_matcher happens here: Instead
> + // of inserting a new fragment, we append to one. If that
> + // fragment does not exist, then the operation is similar to
> + // `insert_fragment` with the difference that we are not
> + // creating a metavariable, but a repetition of one, which is
> + // really different.
> + sub_stack.append_fragment (
> + MatchedFragment (fragment->get_ident (), offs_begin,
> + offs_end));
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Tok: {
> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
> + valid_current_match = match_token (parser, *tok);
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Repetition: {
> + AST::MacroMatchRepetition *rep
> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
> + valid_current_match = match_repetition (parser, *rep);
> + }
> + break;
> +
> + case AST::MacroMatch::MacroMatchType::Matcher: {
> + AST::MacroMatcher *m
> + = static_cast<AST::MacroMatcher *> (match.get ());
> + valid_current_match = match_matcher (parser, *m);
> + }
> + break;
> + }
> + }
> + // If we've encountered an error once, stop trying to match more
> + // repetitions
> + if (!valid_current_match)
> + break;
> +
> + match_amount++;
> +
> + // Break early if we notice there's too many expressions already
> + if (hi_bound && match_amount > hi_bound)
> + break;
> + }
> +
> + // Check if the amount of matches we got is valid: Is it more than the lower
> + // bound and less than the higher bound?
> + bool did_meet_lo_bound = match_amount >= lo_bound;
> + bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true;
> +
> + // If the end-result is valid, then we can clear the parse errors: Since
> + // repetitions are parsed eagerly, it is okay to fail in some cases
> + auto res = did_meet_lo_bound && did_meet_hi_bound;
> + if (res)
> + parser.clear_errors ();
> +
> + return res;
> +}
> +
> +bool
> +MacroExpander::match_repetition (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchRepetition &rep)
> +{
> + size_t match_amount = 0;
> + bool res = false;
> +
> + std::string lo_str;
> + std::string hi_str;
> + switch (rep.get_op ())
> + {
> + case AST::MacroMatchRepetition::MacroRepOp::ANY:
> + lo_str = "0";
> + hi_str = "+inf";
> + res = match_n_matches (parser, rep, match_amount);
> + break;
> + case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE:
> + lo_str = "1";
> + hi_str = "+inf";
> + res = match_n_matches (parser, rep, match_amount, 1);
> + break;
> + case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE:
> + lo_str = "0";
> + hi_str = "1";
> + res = match_n_matches (parser, rep, match_amount, 0, 1);
> + break;
> + default:
> + gcc_unreachable ();
> + }
> +
> + if (!res)
> + rust_error_at (rep.get_match_locus (),
> + "invalid amount of matches for macro invocation. Expected "
> + "between %s and %s, got %lu",
> + lo_str.c_str (), hi_str.c_str (),
> + (unsigned long) match_amount);
> +
> + rust_debug_loc (rep.get_match_locus (), "%s matched %lu times",
> + res ? "successfully" : "unsuccessfully",
> + (unsigned long) match_amount);
> +
> + // We have to handle zero fragments differently: They will not have been
> + // "matched" but they are still valid and should be inserted as a special
> + // case. So we go through the stack map, and for every fragment which doesn't
> + // exist, insert a zero-matched fragment.
> + auto &stack_map = sub_stack.peek ();
> + for (auto &match : rep.get_matches ())
> + {
> + if (match->get_macro_match_type ()
> + == AST::MacroMatch::MacroMatchType::Fragment)
> + {
> + auto fragment = static_cast<AST::MacroMatchFragment *> (match.get ());
> + auto it = stack_map.find (fragment->get_ident ());
> +
> + if (it == stack_map.end ())
> + sub_stack.insert_matches (fragment->get_ident (),
> + MatchedFragmentContainer::zero ());
> + }
> + }
> +
> + return res;
> +}
> +
> +/**
> + * Helper function to refactor calling a parsing function 0 or more times
> + */
> +static AST::ASTFragment
> +parse_many (Parser<MacroInvocLexer> &parser, TokenId &delimiter,
> + std::function<AST::SingleASTNode ()> parse_fn)
> +{
> + std::vector<AST::SingleASTNode> nodes;
> + while (true)
> + {
> + if (parser.peek_current_token ()->get_id () == delimiter)
> + break;
> +
> + auto node = parse_fn ();
> + nodes.emplace_back (std::move (node));
> + }
> +
> + return AST::ASTFragment (std::move (nodes));
> +}
> +
> +/**
> + * Transcribe 0 or more items from a macro invocation
> + *
> + * @param parser Parser to extract items from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> +{
> + return parse_many (parser, delimiter, [&parser] () {
> + auto item = parser.parse_item (true);
> + return AST::SingleASTNode (std::move (item));
> + });
> +}
> +
> +/**
> + * Transcribe 0 or more external items from a macro invocation
> + *
> + * @param parser Parser to extract items from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_ext (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> +{
> + return parse_many (parser, delimiter, [&parser] () {
> + auto item = parser.parse_external_item ();
> + return AST::SingleASTNode (std::move (item));
> + });
> +}
> +
> +/**
> + * Transcribe 0 or more trait items from a macro invocation
> + *
> + * @param parser Parser to extract items from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_trait_items (Parser<MacroInvocLexer> &parser,
> + TokenId &delimiter)
> +{
> + return parse_many (parser, delimiter, [&parser] () {
> + auto item = parser.parse_trait_item ();
> + return AST::SingleASTNode (std::move (item));
> + });
> +}
> +
> +/**
> + * Transcribe 0 or more impl items from a macro invocation
> + *
> + * @param parser Parser to extract items from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_impl_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> +{
> + return parse_many (parser, delimiter, [&parser] () {
> + auto item = parser.parse_inherent_impl_item ();
> + return AST::SingleASTNode (std::move (item));
> + });
> +}
> +
> +/**
> + * Transcribe 0 or more trait impl items from a macro invocation
> + *
> + * @param parser Parser to extract items from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_trait_impl_items (Parser<MacroInvocLexer> &parser,
> + TokenId &delimiter)
> +{
> + return parse_many (parser, delimiter, [&parser] () {
> + auto item = parser.parse_trait_impl_item ();
> + return AST::SingleASTNode (std::move (item));
> + });
> +}
> +
> +/**
> + * Transcribe 0 or more statements from a macro invocation
> + *
> + * @param parser Parser to extract statements from
> + * @param delimiter Id of the token on which parsing should stop
> + */
> +static AST::ASTFragment
> +transcribe_many_stmts (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> +{
> + auto restrictions = ParseRestrictions ();
> + restrictions.consume_semi = false;
> +
> + // FIXME: This is invalid! It needs to also handle cases where the macro
> + // transcriber is an expression, but since the macro call is followed by
> + // a semicolon, it's a valid ExprStmt
> + return parse_many (parser, delimiter, [&parser, restrictions] () {
> + auto stmt = parser.parse_stmt (restrictions);
> + return AST::SingleASTNode (std::move (stmt));
> + });
> +}
> +
> +/**
> + * Transcribe one expression from a macro invocation
> + *
> + * @param parser Parser to extract statements from
> + */
> +static AST::ASTFragment
> +transcribe_expression (Parser<MacroInvocLexer> &parser)
> +{
> + auto expr = parser.parse_expr ();
> +
> + return AST::ASTFragment ({std::move (expr)});
> +}
> +
> +/**
> + * Transcribe one type from a macro invocation
> + *
> + * @param parser Parser to extract statements from
> + */
> +static AST::ASTFragment
> +transcribe_type (Parser<MacroInvocLexer> &parser)
> +{
> + auto type = parser.parse_type ();
> +
> + return AST::ASTFragment ({std::move (type)});
> +}
> +
> +static AST::ASTFragment
> +transcribe_on_delimiter (Parser<MacroInvocLexer> &parser, bool semicolon,
> + AST::DelimType delimiter, TokenId last_token_id)
> +{
> + if (semicolon || delimiter == AST::DelimType::CURLY)
> + return transcribe_many_stmts (parser, last_token_id);
> + else
> + return transcribe_expression (parser);
> +} // namespace Rust
> +
> +static AST::ASTFragment
> +transcribe_context (MacroExpander::ContextType ctx,
> + Parser<MacroInvocLexer> &parser, bool semicolon,
> + AST::DelimType delimiter, TokenId last_token_id)
> +{
> + // The flow-chart in order to choose a parsing function is as follows:
> + //
> + // [switch special context]
> + // -- Item --> parser.parse_item();
> + // -- Trait --> parser.parse_trait_item();
> + // -- Impl --> parser.parse_impl_item();
> + // -- Extern --> parser.parse_extern_item();
> + // -- None --> [has semicolon?]
> + // -- Yes --> parser.parse_stmt();
> + // -- No --> [switch invocation.delimiter()]
> + // -- { } --> parser.parse_stmt();
> + // -- _ --> parser.parse_expr(); // once!
> +
> + // If there is a semicolon OR we are expanding a MacroInvocationSemi, then
> + // we can parse multiple items. Otherwise, parse *one* expression
> +
> + switch (ctx)
> + {
> + case MacroExpander::ContextType::ITEM:
> + return transcribe_many_items (parser, last_token_id);
> + break;
> + case MacroExpander::ContextType::TRAIT:
> + return transcribe_many_trait_items (parser, last_token_id);
> + break;
> + case MacroExpander::ContextType::IMPL:
> + return transcribe_many_impl_items (parser, last_token_id);
> + break;
> + case MacroExpander::ContextType::TRAIT_IMPL:
> + return transcribe_many_trait_impl_items (parser, last_token_id);
> + break;
> + case MacroExpander::ContextType::EXTERN:
> + return transcribe_many_ext (parser, last_token_id);
> + break;
> + case MacroExpander::ContextType::TYPE:
> + return transcribe_type (parser);
> + break;
> + default:
> + return transcribe_on_delimiter (parser, semicolon, delimiter,
> + last_token_id);
> + }
> +}
> +
> +static std::string
> +tokens_to_str (std::vector<std::unique_ptr<AST::Token>> &tokens)
> +{
> + std::string str;
> + if (!tokens.empty ())
> + {
> + str += tokens[0]->as_string ();
> + for (size_t i = 1; i < tokens.size (); i++)
> + str += " " + tokens[i]->as_string ();
> + }
> +
> + return str;
> +}
> +
> +AST::ASTFragment
> +MacroExpander::transcribe_rule (
> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
> + bool semicolon, ContextType ctx)
> +{
> + // we can manipulate the token tree to substitute the dollar identifiers so
> + // that when we call parse its already substituted for us
> + AST::MacroTranscriber &transcriber = match_rule.get_transcriber ();
> + AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree ();
> +
> + auto invoc_stream = invoc_token_tree.to_token_stream ();
> + auto macro_rule_tokens = transcribe_tree.to_token_stream ();
> +
> + auto substitute_context
> + = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments);
> + std::vector<std::unique_ptr<AST::Token>> substituted_tokens
> + = substitute_context.substitute_tokens ();
> +
> + rust_debug ("substituted tokens: %s",
> + tokens_to_str (substituted_tokens).c_str ());
> +
> + // parse it to an ASTFragment
> + MacroInvocLexer lex (std::move (substituted_tokens));
> + Parser<MacroInvocLexer> parser (lex);
> +
> + auto last_token_id = TokenId::RIGHT_CURLY;
> +
> + // this is used so we can check that we delimit the stream correctly.
> + switch (transcribe_tree.get_delim_type ())
> + {
> + case AST::DelimType::PARENS:
> + last_token_id = TokenId::RIGHT_PAREN;
> + rust_assert (parser.skip_token (LEFT_PAREN));
> + break;
> +
> + case AST::DelimType::CURLY:
> + rust_assert (parser.skip_token (LEFT_CURLY));
> + break;
> +
> + case AST::DelimType::SQUARE:
> + last_token_id = TokenId::RIGHT_SQUARE;
> + rust_assert (parser.skip_token (LEFT_SQUARE));
> + break;
> + }
> +
> + // see https://github.com/Rust-GCC/gccrs/issues/22
> + // TL;DR:
> + // - Treat all macro invocations with parentheses, (), or square brackets,
> + // [], as expressions.
> + // - If the macro invocation has curly brackets, {}, it may be parsed as a
> + // statement depending on the context.
> + // - If the macro invocation has a semicolon at the end, it must be parsed
> + // as a statement (either via ExpressionStatement or
> + // MacroInvocationWithSemi)
> +
> + auto fragment
> + = transcribe_context (ctx, parser, semicolon,
> + invoc_token_tree.get_delim_type (), last_token_id);
> +
> + // emit any errors
> + if (parser.has_errors ())
> + {
> + for (auto &err : parser.get_errors ())
> + rust_error_at (err.locus, "%s", err.message.c_str ());
> + return AST::ASTFragment::create_error ();
> + }
> +
> + // are all the tokens used?
> + bool did_delimit = parser.skip_token (last_token_id);
> +
> + bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE);
> + if (!reached_end_of_stream)
> + {
> + const_TokenPtr current_token = parser.peek_current_token ();
> + rust_error_at (current_token->get_locus (),
> + "tokens here and after are unparsed");
> + }
> +
> + return fragment;
> +}
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-macro-expand.h b/gcc/rust/expand/rust-macro-expand.h
> new file mode 100644
> index 00000000000..94d6702ecb8
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-expand.h
> @@ -0,0 +1,366 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef RUST_MACRO_EXPAND_H
> +#define RUST_MACRO_EXPAND_H
> +
> +#include "rust-buffered-queue.h"
> +#include "rust-parse.h"
> +#include "rust-token.h"
> +#include "rust-ast.h"
> +#include "rust-macro.h"
> +#include "rust-hir-map.h"
> +#include "rust-name-resolver.h"
> +#include "rust-macro-invoc-lexer.h"
> +
> +// Provides objects and method prototypes for macro expansion
> +
> +namespace Rust {
> +// forward decls for AST
> +namespace AST {
> +class MacroInvocation;
> +}
> +
> +// Object used to store configuration data for macro expansion.
> +// NOTE: Keep all these items complying with the latest rustc.
> +struct ExpansionCfg
> +{
> + // features?
> + // TODO: Add `features' when we have it.
> + unsigned int recursion_limit = 1024;
> + bool trace_mac = false; // trace macro
> + bool should_test = false; // strip #[test] nodes if false
> + bool keep_macs = false; // keep macro definitions
> + std::string crate_name = "";
> +};
> +
> +struct MatchedFragment
> +{
> + std::string fragment_ident;
> + size_t token_offset_begin;
> + size_t token_offset_end;
> +
> + MatchedFragment (std::string identifier, size_t token_offset_begin,
> + size_t token_offset_end)
> + : fragment_ident (identifier), token_offset_begin (token_offset_begin),
> + token_offset_end (token_offset_end)
> + {}
> +
> + /**
> + * Empty constructor for uninitialized fragments
> + */
> + MatchedFragment () : MatchedFragment ("", 0, 0) {}
> +
> + std::string as_string () const
> + {
> + return fragment_ident + "=" + std::to_string (token_offset_begin) + ":"
> + + std::to_string (token_offset_end);
> + }
> +};
> +
> +class MatchedFragmentContainer
> +{
> +public:
> + // Does the container refer to a simple metavariable, different from a
> + // repetition repeated once
> + enum class Kind
> + {
> + MetaVar,
> + Repetition,
> + };
> +
> + MatchedFragmentContainer (std::vector<MatchedFragment> fragments,
> + Kind kind = Kind::Repetition)
> + : fragments (fragments), kind (kind)
> + {}
> +
> + /**
> + * Create a valid fragment matched zero times. This is useful for repetitions
> + * which allow the absence of a fragment, such as * and ?
> + */
> + static MatchedFragmentContainer zero ()
> + {
> + return MatchedFragmentContainer ({});
> + }
> +
> + /**
> + * Create a valid fragment matched one time
> + */
> + static MatchedFragmentContainer metavar (MatchedFragment fragment)
> + {
> + return MatchedFragmentContainer ({fragment}, Kind::MetaVar);
> + }
> +
> + /**
> + * Add a matched fragment to the container
> + */
> + void add_fragment (MatchedFragment fragment)
> + {
> + rust_assert (!is_single_fragment ());
> +
> + fragments.emplace_back (fragment);
> + }
> +
> + size_t get_match_amount () const { return fragments.size (); }
> + const std::vector<MatchedFragment> &get_fragments () const
> + {
> + return fragments;
> + }
> + // const std::string &get_fragment_name () const { return fragment_name; }
> +
> + bool is_single_fragment () const
> + {
> + return get_match_amount () == 1 && kind == Kind::MetaVar;
> + }
> +
> + const MatchedFragment get_single_fragment () const
> + {
> + rust_assert (is_single_fragment ());
> +
> + return fragments[0];
> + }
> +
> + const Kind &get_kind () const { return kind; }
> +
> +private:
> + /**
> + * Fragments matched `match_amount` times. This can be an empty vector
> + * in case having zero matches is allowed (i.e ? or * operators)
> + */
> + std::vector<MatchedFragment> fragments;
> + Kind kind;
> +};
> +
> +class SubstitutionScope
> +{
> +public:
> + SubstitutionScope () : stack () {}
> +
> + void push () { stack.push_back ({}); }
> +
> + std::map<std::string, MatchedFragmentContainer> pop ()
> + {
> + auto top = stack.back ();
> + stack.pop_back ();
> + return top;
> + }
> +
> + std::map<std::string, MatchedFragmentContainer> &peek ()
> + {
> + return stack.back ();
> + }
> +
> + /**
> + * Insert a new matched metavar into the current substitution map
> + */
> + void insert_metavar (MatchedFragment fragment)
> + {
> + auto ¤t_map = stack.back ();
> + auto it = current_map.find (fragment.fragment_ident);
> +
> + if (it == current_map.end ())
> + current_map.insert ({fragment.fragment_ident,
> + MatchedFragmentContainer::metavar (fragment)});
> + else
> + gcc_unreachable ();
> + }
> +
> + /**
> + * Append a new matched fragment to a repetition into the current substitution
> + * map
> + */
> + void append_fragment (MatchedFragment fragment)
> + {
> + auto ¤t_map = stack.back ();
> + auto it = current_map.find (fragment.fragment_ident);
> +
> + if (it == current_map.end ())
> + current_map.insert (
> + {fragment.fragment_ident, MatchedFragmentContainer ({fragment})});
> + else
> + it->second.add_fragment (fragment);
> + }
> +
> + void insert_matches (std::string key, MatchedFragmentContainer matches)
> + {
> + auto ¤t_map = stack.back ();
> + auto it = current_map.find (key);
> + rust_assert (it == current_map.end ());
> +
> + current_map.insert ({key, matches});
> + }
> +
> +private:
> + std::vector<std::map<std::string, MatchedFragmentContainer>> stack;
> +};
> +
> +// Object used to store shared data (between functions) for macro expansion.
> +struct MacroExpander
> +{
> + enum ContextType
> + {
> + ITEM,
> + BLOCK,
> + EXTERN,
> + TYPE,
> + TRAIT,
> + IMPL,
> + TRAIT_IMPL,
> + };
> +
> + ExpansionCfg cfg;
> + unsigned int expansion_depth = 0;
> +
> + MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session)
> + : cfg (cfg), crate (crate), session (session),
> + sub_stack (SubstitutionScope ()),
> + expanded_fragment (AST::ASTFragment::create_error ()),
> + resolver (Resolver::Resolver::get ()),
> + mappings (Analysis::Mappings::get ())
> + {}
> +
> + ~MacroExpander () = default;
> +
> + // Expands all macros in the crate passed in.
> + void expand_crate ();
> +
> + /* Expands a macro invocation - possibly make both
> + * have similar duck-typed interface and use templates?*/
> + // should this be public or private?
> + void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon);
> +
> + // Expands a single declarative macro.
> + AST::ASTFragment expand_decl_macro (Location locus,
> + AST::MacroInvocData &invoc,
> + AST::MacroRulesDefinition &rules_def,
> + bool semicolon);
> +
> + void expand_cfg_attrs (AST::AttrVec &attrs);
> + bool fails_cfg (const AST::AttrVec &attr) const;
> + bool fails_cfg_with_expand (AST::AttrVec &attrs) const;
> +
> + bool depth_exceeds_recursion_limit () const;
> +
> + bool try_match_rule (AST::MacroRule &match_rule,
> + AST::DelimTokenTree &invoc_token_tree);
> +
> + AST::ASTFragment transcribe_rule (
> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
> + bool semicolon, ContextType ctx);
> +
> + bool match_fragment (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchFragment &fragment);
> +
> + bool match_token (Parser<MacroInvocLexer> &parser, AST::Token &token);
> +
> + bool match_repetition (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchRepetition &rep);
> +
> + bool match_matcher (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatcher &matcher);
> +
> + /**
> + * Match any amount of matches
> + *
> + * @param parser Parser to use for matching
> + * @param rep Repetition to try and match
> + * @param match_amount Reference in which to store the ammount of succesful
> + * and valid matches
> + *
> + * @param lo_bound Lower bound of the matcher. When specified, the matcher
> + * will only succeed if it parses at *least* `lo_bound` fragments. If
> + * unspecified, the matcher could succeed when parsing 0 fragments.
> + *
> + * @param hi_bound Higher bound of the matcher. When specified, the matcher
> + * will only succeed if it parses *less than* `hi_bound` fragments. If
> + * unspecified, the matcher could succeed when parsing an infinity of
> + * fragments.
> + *
> + * @return true if matching was successful and within the given limits, false
> + * otherwise
> + */
> + bool match_n_matches (Parser<MacroInvocLexer> &parser,
> + AST::MacroMatchRepetition &rep, size_t &match_amount,
> + size_t lo_bound = 0, size_t hi_bound = 0);
> +
> + void push_context (ContextType t) { context.push_back (t); }
> +
> + ContextType pop_context ()
> + {
> + rust_assert (!context.empty ());
> +
> + ContextType t = context.back ();
> + context.pop_back ();
> +
> + return t;
> + }
> +
> + ContextType peek_context () { return context.back (); }
> +
> + void set_expanded_fragment (AST::ASTFragment &&fragment)
> + {
> + expanded_fragment = std::move (fragment);
> + }
> +
> + AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis)
> + {
> + AST::ASTFragment old_fragment = std::move (expanded_fragment);
> + auto accumulator = std::vector<AST::SingleASTNode> ();
> + expanded_fragment = AST::ASTFragment::create_error ();
> +
> + for (auto &node : old_fragment.get_nodes ())
> + {
> + expansion_depth++;
> + node.accept_vis (vis);
> + // we'll decide the next move according to the outcome of the macro
> + // expansion
> + if (expanded_fragment.is_error ())
> + accumulator.push_back (node); // if expansion fails, there might be a
> + // non-macro expression we need to keep
> + else
> + {
> + // if expansion succeeded, then we need to merge the fragment with
> + // the contents in the accumulator, so that our final expansion
> + // result will contain non-macro nodes as it should
> + auto new_nodes = expanded_fragment.get_nodes ();
> + std::move (new_nodes.begin (), new_nodes.end (),
> + std::back_inserter (accumulator));
> + expanded_fragment = AST::ASTFragment (accumulator);
> + }
> + expansion_depth--;
> + }
> +
> + return old_fragment;
> + }
> +
> +private:
> + AST::Crate &crate;
> + Session &session;
> + SubstitutionScope sub_stack;
> + std::vector<ContextType> context;
> + AST::ASTFragment expanded_fragment;
> +
> +public:
> + Resolver::Resolver *resolver;
> + Analysis::Mappings *mappings;
> +};
> +
> +} // namespace Rust
> +
> +#endif
> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.cc b/gcc/rust/expand/rust-macro-invoc-lexer.cc
> new file mode 100644
> index 00000000000..8a43d29e0d1
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.cc
> @@ -0,0 +1,29 @@
> +#include "rust-macro-invoc-lexer.h"
> +
> +namespace Rust {
> +
> +const_TokenPtr
> +MacroInvocLexer::peek_token (int n)
> +{
> + if ((offs + n) >= token_stream.size ())
> + return Token::make (END_OF_FILE, Location ());
> +
> + return token_stream.at (offs + n)->get_tok_ptr ();
> +}
> +
> +// Advances current token to n + 1 tokens ahead of current position.
> +void
> +MacroInvocLexer::skip_token (int n)
> +{
> + offs += (n + 1);
> +}
> +
> +void
> +MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)),
> + TokenId new_right
> + __attribute__ ((unused)))
> +{
> + // FIXME
> + gcc_unreachable ();
> +}
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.h b/gcc/rust/expand/rust-macro-invoc-lexer.h
> new file mode 100644
> index 00000000000..0fd4554d02f
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.h
> @@ -0,0 +1,64 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#ifndef RUST_MACRO_INVOC_LEXER_H
> +#define RUST_MACRO_INVOC_LEXER_H
> +
> +#include "rust-ast.h"
> +
> +namespace Rust {
> +class MacroInvocLexer
> +{
> +public:
> + MacroInvocLexer (std::vector<std::unique_ptr<AST::Token>> stream)
> + : offs (0), token_stream (std::move (stream))
> + {}
> +
> + // Returns token n tokens ahead of current position.
> + const_TokenPtr peek_token (int n);
> +
> + // Peeks the current token.
> + const_TokenPtr peek_token () { return peek_token (0); }
> +
> + // Advances current token to n + 1 tokens ahead of current position.
> + void skip_token (int n);
> +
> + // Skips the current token.
> + void skip_token () { skip_token (0); }
> +
> + // Splits the current token into two. Intended for use with nested generics
> + // closes (i.e. T<U<X>> where >> is wrongly lexed as one token). Note that
> + // this will only work with "simple" tokens like punctuation.
> + void split_current_token (TokenId new_left, TokenId new_right);
> +
> + std::string get_filename () const
> + {
> + // FIXME
> + gcc_unreachable ();
> + return "FIXME";
> + }
> +
> + size_t get_offs () const { return offs; }
> +
> +private:
> + size_t offs;
> + std::vector<std::unique_ptr<AST::Token>> token_stream;
> +};
> +} // namespace Rust
> +
> +#endif // RUST_MACRO_INVOC_LEXER_H
> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
> new file mode 100644
> index 00000000000..9592d2d2a9e
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
> @@ -0,0 +1,312 @@
> +#include "rust-macro-substitute-ctx.h"
> +
> +namespace Rust {
> +
> +std::vector<std::unique_ptr<AST::Token>>
> +SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
> +{
> + auto metavar_name = metavar->get_str ();
> +
> + std::vector<std::unique_ptr<AST::Token>> expanded;
> + auto it = fragments.find (metavar_name);
> + if (it == fragments.end ())
> + {
> + // Return a copy of the original token
> + expanded.push_back (metavar->clone_token ());
> + }
> + else
> + {
> + // If we are expanding a metavar which has a lof of matches, we are
> + // currently expanding a repetition metavar - not a simple metavar. We
> + // need to error out and inform the user.
> + // Associated test case for an example: compile/macro-issue1224.rs
> + if (it->second.get_match_amount () != 1)
> + {
> + rust_error_at (metavar->get_locus (),
> + "metavariable is still repeating at this depth");
> + rust_inform (
> + metavar->get_locus (),
> + "you probably forgot the repetition operator: %<%s%s%s%>", "$(",
> + metavar->as_string ().c_str (), ")*");
> + return expanded;
> + }
> +
> + // We only care about the vector when expanding repetitions.
> + // Just access the first element of the vector.
> + auto &frag = it->second.get_single_fragment ();
> + for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
> + offs++)
> + {
> + auto &tok = input.at (offs);
> + expanded.push_back (tok->clone_token ());
> + }
> + }
> +
> + return expanded;
> +}
> +
> +bool
> +SubstituteCtx::check_repetition_amount (size_t pattern_start,
> + size_t pattern_end,
> + size_t &expected_repetition_amount)
> +{
> + bool first_fragment_found = false;
> + bool is_valid = true;
> +
> + for (size_t i = pattern_start; i < pattern_end; i++)
> + {
> + if (macro.at (i)->get_id () == DOLLAR_SIGN)
> + {
> + auto &frag_token = macro.at (i + 1);
> + if (frag_token->get_id () == IDENTIFIER)
> + {
> + auto it = fragments.find (frag_token->get_str ());
> + if (it == fragments.end ())
> + {
> + // If the repetition is not anything we know (ie no declared
> + // metavars, or metavars which aren't present in the
> + // fragment), we can just error out. No need to paste the
> + // tokens as if nothing had happened.
> + rust_error_at (frag_token->get_locus (),
> + "metavar %s used in repetition does not exist",
> + frag_token->get_str ().c_str ());
> +
> + is_valid = false;
> + }
> +
> + auto &fragment = it->second;
> +
> + size_t repeat_amount = fragment.get_match_amount ();
> + if (!first_fragment_found)
> + {
> + first_fragment_found = true;
> + expected_repetition_amount = repeat_amount;
> + }
> + else
> + {
> + if (repeat_amount != expected_repetition_amount
> + && !fragment.is_single_fragment ())
> + {
> + rust_error_at (
> + frag_token->get_locus (),
> + "different amount of matches used in merged "
> + "repetitions: expected %lu, got %lu",
> + (unsigned long) expected_repetition_amount,
> + (unsigned long) repeat_amount);
> + is_valid = false;
> + }
> + }
> + }
> + }
> + }
> +
> + return is_valid;
> +}
> +
> +std::vector<std::unique_ptr<AST::Token>>
> +SubstituteCtx::substitute_repetition (
> + size_t pattern_start, size_t pattern_end,
> + std::unique_ptr<AST::Token> separator_token)
> +{
> + rust_assert (pattern_end < macro.size ());
> +
> + size_t repeat_amount = 0;
> + if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount))
> + return {};
> +
> + rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount);
> + std::vector<std::unique_ptr<AST::Token>> expanded;
> + std::vector<std::unique_ptr<AST::Token>> new_macro;
> +
> + // We want to generate a "new macro" to substitute with. This new macro
> + // should contain only the tokens inside the pattern
> + for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++)
> + new_macro.emplace_back (macro.at (tok_idx)->clone_token ());
> +
> + // Then, we want to create a subset of the matches so that
> + // `substitute_tokens()` can only see one fragment per metavar. Let's say we
> + // have the following user input: (1 145 'h')
> + // on the following match arm: ($($lit:literal)*)
> + // which causes the following matches: { "lit": [1, 145, 'h'] }
> + //
> + // The pattern (new_macro) is `$lit:literal`
> + // The first time we expand it, we want $lit to have the following token: 1
> + // The second time, 145
> + // The third and final time, 'h'
> + //
> + // In order to do so we must create "sub maps", which only contain parts of
> + // the original matches
> + // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ]
> + //
> + // and give them to `substitute_tokens` one by one.
> +
> + for (size_t i = 0; i < repeat_amount; i++)
> + {
> + std::map<std::string, MatchedFragmentContainer> sub_map;
> + for (auto &kv_match : fragments)
> + {
> + MatchedFragment sub_fragment;
> +
> + // FIXME: Hack: If a fragment is not repeated, how does it fit in the
> + // submap? Do we really want to expand it? Is this normal behavior?
> + if (kv_match.second.is_single_fragment ())
> + sub_fragment = kv_match.second.get_single_fragment ();
> + else
> + sub_fragment = kv_match.second.get_fragments ()[i];
> +
> + sub_map.insert (
> + {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)});
> + }
> +
> + auto substitute_context = SubstituteCtx (input, new_macro, sub_map);
> + auto new_tokens = substitute_context.substitute_tokens ();
> +
> + // Skip the first repetition, but add the separator to the expanded
> + // tokens if it is present
> + if (i != 0 && separator_token)
> + expanded.emplace_back (separator_token->clone_token ());
> +
> + for (auto &new_token : new_tokens)
> + expanded.emplace_back (new_token->clone_token ());
> + }
> +
> + // FIXME: We also need to make sure that all subsequent fragments
> + // contain the same amount of repetitions as the first one
> +
> + return expanded;
> +}
> +
> +static bool
> +is_rep_op (std::unique_ptr<AST::Token> &tok)
> +{
> + auto id = tok->get_id ();
> + return id == QUESTION_MARK || id == ASTERISK || id == PLUS;
> +}
> +
> +std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
> +SubstituteCtx::substitute_token (size_t token_idx)
> +{
> + auto &token = macro.at (token_idx);
> + switch (token->get_id ())
> + {
> + case IDENTIFIER:
> + rust_debug ("expanding metavar: %s", token->get_str ().c_str ());
> + return {substitute_metavar (token), 1};
> + case LEFT_PAREN: {
> + // We need to parse up until the closing delimiter and expand this
> + // fragment->n times.
> + rust_debug ("expanding repetition");
> +
> + // We're in a context where macro repetitions have already been
> + // parsed and validated: This means that
> + // 1/ There will be no delimiters as that is an error
> + // 2/ There are no fragment specifiers anymore, which prevents us
> + // from reusing parser functions.
> + //
> + // Repetition patterns are also special in that they cannot contain
> + // "rogue" delimiters: For example, this is invalid, as they are
> + // parsed as MacroMatches and must contain a correct amount of
> + // delimiters.
> + // `$($e:expr ) )`
> + // ^ rogue closing parenthesis
> + //
> + // With all of that in mind, we can simply skip ahead from one
> + // parenthesis to the other to find the pattern to expand. Of course,
> + // pairs of delimiters, including parentheses, are allowed.
> + // `$($e:expr ( ) )`
> + // Parentheses are the sole delimiter for which we need a special
> + // behavior since they delimit the repetition pattern
> +
> + size_t pattern_start = token_idx + 1;
> + size_t pattern_end = pattern_start;
> + auto parentheses_stack = 0;
> + for (size_t idx = pattern_start; idx < macro.size (); idx++)
> + {
> + if (macro.at (idx)->get_id () == LEFT_PAREN)
> + {
> + parentheses_stack++;
> + }
> + else if (macro.at (idx)->get_id () == RIGHT_PAREN)
> + {
> + if (parentheses_stack == 0)
> + {
> + pattern_end = idx;
> + break;
> + }
> + parentheses_stack--;
> + }
> + }
> +
> + // Unreachable case, but let's make sure we don't ever run into it
> + rust_assert (pattern_end != pattern_start);
> +
> + std::unique_ptr<AST::Token> separator_token = nullptr;
> + if (pattern_end + 1 <= macro.size ())
> + {
> + auto &post_pattern_token = macro.at (pattern_end + 1);
> + if (!is_rep_op (post_pattern_token))
> + separator_token = post_pattern_token->clone_token ();
> + }
> +
> + // Amount of tokens to skip
> + auto to_skip = 0;
> + // Parentheses
> + to_skip += 2;
> + // Repetition operator
> + to_skip += 1;
> + // Separator
> + if (separator_token)
> + to_skip += 1;
> +
> + return {substitute_repetition (pattern_start, pattern_end,
> + std::move (separator_token)),
> + pattern_end - pattern_start + to_skip};
> + }
> + // TODO: We need to check if the $ was alone. In that case, do
> + // not error out: Simply act as if there was an empty identifier
> + // with no associated fragment and paste the dollar sign in the
> + // transcription. Unsure how to do that since we always have at
> + // least the closing curly brace after an empty $...
> + default:
> + rust_error_at (token->get_locus (),
> + "unexpected token in macro transcribe: expected "
> + "%<(%> or identifier after %<$%>, got %<%s%>",
> + get_token_description (token->get_id ()));
> + }
> +
> + // FIXME: gcc_unreachable() error case?
> + return {std::vector<std::unique_ptr<AST::Token>> (), 0};
> +}
> +
> +std::vector<std::unique_ptr<AST::Token>>
> +SubstituteCtx::substitute_tokens ()
> +{
> + std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
> + rust_debug ("expanding tokens");
> +
> + for (size_t i = 0; i < macro.size (); i++)
> + {
> + auto &tok = macro.at (i);
> + if (tok->get_id () == DOLLAR_SIGN)
> + {
> + // Aaaaah, if only we had C++17 :)
> + // auto [expanded, tok_to_skip] = ...
> + auto p = substitute_token (i + 1);
> + auto expanded = std::move (p.first);
> + auto tok_to_skip = p.second;
> +
> + i += tok_to_skip;
> +
> + for (auto &token : expanded)
> + replaced_tokens.emplace_back (token->clone_token ());
> + }
> + else
> + {
> + replaced_tokens.emplace_back (tok->clone_token ());
> + }
> + }
> +
> + return replaced_tokens;
> +}
> +
> +} // namespace Rust
> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.h b/gcc/rust/expand/rust-macro-substitute-ctx.h
> new file mode 100644
> index 00000000000..81dcab7643b
> --- /dev/null
> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.h
> @@ -0,0 +1,93 @@
> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> +
> +// This file is part of GCC.
> +
> +// GCC is free software; you can redistribute it and/or modify it under
> +// the terms of the GNU General Public License as published by the Free
> +// Software Foundation; either version 3, or (at your option) any later
> +// version.
> +
> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +// for more details.
> +
> +// You should have received a copy of the GNU General Public License
> +// along with GCC; see the file COPYING3. If not see
> +// <http://www.gnu.org/licenses/>.
> +
> +#include "rust-ast.h"
> +#include "rust-macro-expand.h"
> +
> +namespace Rust {
> +class SubstituteCtx
> +{
> + std::vector<std::unique_ptr<AST::Token>> &input;
> + std::vector<std::unique_ptr<AST::Token>> ¯o;
> + std::map<std::string, MatchedFragmentContainer> &fragments;
> +
> + /**
> + * Find the repetition amount to use when expanding a repetition, and
> + * check that all fragments used respect that repetition amount
> + *
> + * @param pattern_start Start of the repetition pattern
> + * @param pattern_end End of the repetition pattern
> + * @param repeat_amount Reference to fill with the matched repetition amount
> + */
> + bool check_repetition_amount (size_t pattern_start, size_t pattern_end,
> + size_t &repeat_amount);
> +
> +public:
> + SubstituteCtx (std::vector<std::unique_ptr<AST::Token>> &input,
> + std::vector<std::unique_ptr<AST::Token>> ¯o,
> + std::map<std::string, MatchedFragmentContainer> &fragments)
> + : input (input), macro (macro), fragments (fragments)
> + {}
> +
> + /**
> + * Substitute a metavariable by its given fragment in a transcribing context,
> + * i.e. replacing $var with the associated fragment.
> + *
> + * @param metavar Metavariable to try and replace
> + *
> + * @return A token containing the associated fragment expanded into tokens if
> + * any, or the cloned token if no fragment was associated
> + */
> + std::vector<std::unique_ptr<AST::Token>>
> + substitute_metavar (std::unique_ptr<AST::Token> &metavar);
> +
> + /**
> + * Substitute a macro repetition by its given fragments
> + *
> + * @param pattern_start Start index of the pattern tokens
> + * @param pattern_end End index of the patterns tokens
> + * @param separator Optional separator to include when expanding tokens
> + *
> + * @return A vector containing the repeated pattern
> + */
> + std::vector<std::unique_ptr<AST::Token>>
> + substitute_repetition (size_t pattern_start, size_t pattern_end,
> + std::unique_ptr<AST::Token> separator);
> +
> + /**
> + * Substitute a given token by its appropriate representation
> + *
> + * @param token_idx Current token to try and substitute
> + *
> + * @return A token containing the associated fragment expanded into tokens if
> + * any, or the cloned token if no fragment was associated, as well as the
> + * amount of tokens that should be skipped before the next invocation. Since
> + * this function may consume more than just one token, it is important to skip
> + * ahead of the input to avoid mis-substitutions
> + */
> + std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
> + substitute_token (size_t token_idx);
> +
> + /**
> + * Substitute all tokens by their appropriate representation
> + *
> + * @return A vector containing the substituted tokens
> + */
> + std::vector<std::unique_ptr<AST::Token>> substitute_tokens ();
> +};
> +} // namespace Rust
> --
> 2.37.2
>
On 11/10/22 11:49, Richard Biener wrote:
> On Wed, Oct 26, 2022 at 10:28 AM <arthur.cohen@embecosm.com> wrote:
>>
>> From: Arthur Cohen <arthur.cohen@embecosm.com>
>>
>> The expansion pass is responsible for two actions on our AST:
>>
>> 1. Expanding macro calls
>> 2. Performing conditional compilation
>>
>> Calls to macros should be checked and expanded into an AST fragment based on
>> the context they've been called in. This is similar to token substitution, with
>> a lot of intricacies and checks being performed. A single invocation can result
>> in an AST fragment containing multiple statements or multiple expressions,
>> which need to be handled as well. Furthermore, Rust macros can contain
>> repetitions relying on Kleine operators, similar to regular expression
>> patterns, that also need to be expanded properly.
>>
>> Finally, Rust code can be hidden behind `cfg` directives, which allow the user
>> to perform conditional compilation. If a `cfg` predicate is not met, the
>> expression or statement it refers to should be marked for strip and removed
>> from the AST.
>
> I'm just picking one of the purely rust frontend specific patches as example and
> do not plan to look on others - I don't think meaningful review can happen by
> somebody not deeply understanding rust or this frontend can be done which means
> approval of those parts shouldn't be needed but would be covered (even
> the initial
> contribution) by your maintainership of the code.
>
> The commit comment above is quite helpful in getting an idea of what this part
> of the rust frontend is doing. I'd appreciate if something along these lines
> would be added to each of the individual source files (preferably the
> implementation, not the header).
Is there any reason as to why the implementation should be prefered over
the header? This is a comment you also make down the line on one of our
function's documentation. My view is that the header should be kept as
simple and possible, with as little code as possible, and that it is
easy to find documentation around function declarations, much easier
than to find it in the implementation when code blocks are present all
around.
To be clear, I'm not opposed to switching the documentation over to our
implementation - just looking to understand. We'll make the necessary
changes.
> Documentation of the sub-directory structure would
> also be nice,
> we have (had ..) sourcebuild.texi which contains some of that but not really for
> specific frontends (sofar).
Right. We will work on adding proper documentation ASAP.
> Some comments on this particular patch that probably apply to all others as well
> inline below.
>
>> Co-authored-by: Philip Herron <philip.herron@embecosm.com>
>> Co-authored-by: The Other <simplytheother@gmail.com>
>> ---
>> gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++
>> gcc/rust/expand/rust-attribute-visitor.h | 316 ++
>> gcc/rust/expand/rust-macro-builtins.cc | 484 +++
>> gcc/rust/expand/rust-macro-builtins.h | 107 +
>> gcc/rust/expand/rust-macro-expand.cc | 1012 +++++
>> gcc/rust/expand/rust-macro-expand.h | 366 ++
>> gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 +
>> gcc/rust/expand/rust-macro-invoc-lexer.h | 64 +
>> gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++
>> gcc/rust/expand/rust-macro-substitute-ctx.h | 93 +
>> 10 files changed, 6228 insertions(+)
>> create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc
>> create mode 100644 gcc/rust/expand/rust-attribute-visitor.h
>> create mode 100644 gcc/rust/expand/rust-macro-builtins.cc
>> create mode 100644 gcc/rust/expand/rust-macro-builtins.h
>> create mode 100644 gcc/rust/expand/rust-macro-expand.cc
>> create mode 100644 gcc/rust/expand/rust-macro-expand.h
>> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc
>> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h
>> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc
>> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h
>>
>> diff --git a/gcc/rust/expand/rust-attribute-visitor.cc b/gcc/rust/expand/rust-attribute-visitor.cc
>> new file mode 100644
>> index 00000000000..8016f9430eb
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-attribute-visitor.cc
>> @@ -0,0 +1,3445 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#include "rust-attribute-visitor.h"
>> +#include "rust-session-manager.h"
>> +
>> +namespace Rust {
>> +
>> +// Visitor used to expand attributes.
>> +void
>> +AttrVisitor::expand_struct_fields (std::vector<AST::StructField> &fields)
>> +{
>> + for (auto it = fields.begin (); it != fields.end ();)
>> + {
>> + auto &field = *it;
>> +
>> + auto &field_attrs = field.get_outer_attrs ();
>> + expander.expand_cfg_attrs (field_attrs);
>> + if (expander.fails_cfg_with_expand (field_attrs))
>> + {
>> + it = fields.erase (it);
>> + continue;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // expand sub-types of type, but can't strip type itself
>> + auto &type = field.get_field_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + // if nothing else happens, increment
>> + ++it;
>> + }
>> +}
>> +
>
> The coding conventions warrant a function level comment for each function.
> If there's anything meaningful to say (in this case "// expand tuple fields"
> wouldn't be meaningful) please consider adding such to all functions. In
> my view every function that doesn't fit in a 80x24 terminal warrants one.
>
>> +void
>> +AttrVisitor::expand_tuple_fields (std::vector<AST::TupleField> &fields)
>> +{
>> + for (auto it = fields.begin (); it != fields.end ();)
>> + {
>> + auto &field = *it;
>> +
>> + auto &field_attrs = field.get_outer_attrs ();
>> + expander.expand_cfg_attrs (field_attrs);
>> + if (expander.fails_cfg_with_expand (field_attrs))
>> + {
>> + it = fields.erase (it);
>> + continue;
>> + }
>> +
>> + // expand sub-types of type, but can't strip type itself
>> + auto &type = field.get_field_type ();
>> + type->accept_vis (*this);
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>
> When reading the other patches I wondered whether rust_error_at are
> user-facing diagnostics or internal errors?
>
>> +
>> + // if nothing else happens, increment
>> + ++it;
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::expand_function_params (std::vector<AST::FunctionParam> ¶ms)
>> +{
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + for (auto it = params.begin (); it != params.end ();)
>> + {
>> + auto ¶m = *it;
>> +
>> + auto ¶m_attrs = param.get_outer_attrs ();
>> + expander.expand_cfg_attrs (param_attrs);
>> + if (expander.fails_cfg_with_expand (param_attrs))
>> + {
>> + it = params.erase (it);
>> + continue;
>> + }
>> +
>> + // TODO: should an unwanted strip lead to break out of loop?
>> + auto &pattern = param.get_pattern ();
>
> 'auto' is used a lot in the frontend, it does make reading code harder because
> critical information on context is lost. Here I can at least 2nd guess
> 'param' being of type AST::FunctionParam and thus find get_pattern (),
> but I will have to find that to be able to find accept_vis () below.
>
> In my mind 'auto' is useful when it hides information that's not used as in
>
> for (auto it = ...)
> {
> AST::FunctionParam ¶m = *it;
>
> but not so much for being lazy when typing.
>
> At least with (some of) the uses the identifiers of the methods look
> unique enough so grep will be of help, OTOH is_marked_for_strip
> is defined a lot.
>
I think removing all the `auto`s from the gccrs codebase would be a
herculean task at this point, as a quick grep for 'auto ' in our
frontend reveals around 2700 matches.
From the little research I did, it seems the C++ frontend also uses it
somewhat, as well as the D frontend (but that frontend is written in D,
so that might be moot).
Paired with descriptive variable names and short functions, I personally
don't have an issue with reviewing code that uses `auto` but I see how
that might be the case, especially for someone who is not familiar with
our frontend.
So I think the question I'm asking is as follows: considering Philip and
I will be reviewers of the Rust frontend part specifically, is it okay
to not make such a huge change to the frontend which would cost us a lot
of time while not necessarily making a difference to us? Again, I
understand that this makes a difference to some people, but as you
mentioned I'm not sure anyone would be willing to review all of our
frontend code, especially as it contains around 160 000 lines of code.
Furthering my point on the D frontend: it's also no secret that we would
like to rewrite some parts of the frontend in Rust in the future, as a
functional programming language is well suited to the compilation of
Rust. When that is the case, we will use what I would consider
"idiomatic Rust", which makes heavy use of the `let` keyword (similar in
many ways to `auto`) and type inference. Types are rarely explicitly
specified for variables, but are always specified for function arguments
or function return types (as we do presently in the C++ implementation
of our frontend).
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +
>> + auto &type = param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + // increment
>> + ++it;
>> + }
>> +
>> + expander.pop_context ();
>> +}
>> +
>> +void
>> +AttrVisitor::expand_generic_args (AST::GenericArgs &args)
>> +{
>> + // lifetime args can't be expanded
>> + // FIXME: Can we have macro invocations for lifetimes?
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // expand type args - strip sub-types only
>> + for (auto &arg : args.get_generic_args ())
>> + {
>> + switch (arg.get_kind ())
>> + {
>> + case AST::GenericArg::Kind::Type: {
>> + auto &type = arg.get_type ();
>> + type->accept_vis (*this);
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> + break;
>> + }
>> + case AST::GenericArg::Kind::Const: {
>> + auto &expr = arg.get_expression ();
>> + expr->accept_vis (*this);
>> + maybe_expand_expr (expr);
>> +
>> + if (expr->is_marked_for_strip ())
>> + rust_error_at (expr->get_locus (),
>> + "cannot strip expression in this position");
>> + break;
>> + }
>> + default:
>> + break;
>> + // FIXME: Figure out what to do here if there is ambiguity. Since the
>> + // resolver comes after the expansion, we need to figure out a way to
>> + // strip ambiguous values here
>> + // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg`
>> + // or something. This would clean up this whole thing
>> + }
>> + }
>> +
>> + expander.pop_context ();
>> +
>> + // FIXME: Can we have macro invocations in generic type bindings?
>> + // expand binding args - strip sub-types only
>> + for (auto &binding : args.get_binding_args ())
>> + {
>> + auto &type = binding.get_type ();
>> + type->accept_vis (*this);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type)
>> +{
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = path_type.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + expander.pop_context ();
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + if (path_type.has_as_clause ())
>> + {
>> + auto &type_path = path_type.get_as_type_path ();
>> + visit (type_path);
>> + if (type_path.is_marked_for_strip ())
>> + rust_error_at (type_path.get_locus (),
>> + "cannot strip type path in this position");
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::AttrVisitor::expand_closure_params (
>> + std::vector<AST::ClosureParam> ¶ms)
>> +{
>> + for (auto it = params.begin (); it != params.end ();)
>> + {
>> + auto ¶m = *it;
>> +
>> + auto ¶m_attrs = param.get_outer_attrs ();
>> + expander.expand_cfg_attrs (param_attrs);
>> + if (expander.fails_cfg_with_expand (param_attrs))
>> + {
>> + it = params.erase (it);
>> + continue;
>> + }
>> +
>> + auto &pattern = param.get_pattern ();
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +
>> + if (param.has_type_given ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> + auto &type = param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + // increment if found nothing else so far
>> + ++it;
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::expand_self_param (AST::SelfParam &self_param)
>> +{
>> + if (self_param.has_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> + auto &type = self_param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> + /* TODO: maybe check for invariants being violated - e.g. both type and
>> + * lifetime? */
>
> C-style comment where I see C++-style comments everywhere else.
>
>> +}
>> +
>> +void
>> +AttrVisitor::expand_where_clause (AST::WhereClause &where_clause)
>> +{
>> + // items cannot be stripped conceptually, so just accept visitor
>> + for (auto &item : where_clause.get_items ())
>> + item->accept_vis (*this);
>> +}
>> +
>> +void
>> +AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl)
>> +{
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : decl.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* strip function parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_function_params (decl.get_function_params ());
>> +
>> + if (decl.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = decl.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + if (decl.has_where_clause ())
>> + expand_where_clause (decl.get_where_clause ());
>> +}
>> +
>> +void
>> +AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl)
>> +{
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : decl.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* assuming you can't strip self param - wouldn't be a method
>> + * anymore. spec allows outer attrs on self param, but doesn't
>> + * specify whether cfg is used. */
>> + expand_self_param (decl.get_self_param ());
>> +
>> + /* strip function parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_function_params (decl.get_function_params ());
>> +
>> + if (decl.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = decl.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + if (decl.has_where_clause ())
>> + expand_where_clause (decl.get_where_clause ());
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::Token &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::DelimTokenTree &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::AttrInputMetaItemContainer &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::IdentifierExpr &ident_expr)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (ident_expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ()))
>> + {
>> + ident_expr.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::Lifetime &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::LifetimeParam &)
>> +{
>> + // supposedly does not require - cfg does nothing
>> +}
>> +void
>> +AttrVisitor::visit (AST::ConstGenericParam &)
>> +{
>> + // likewise
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::MacroInvocation ¯o_invoc)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ()))
>> + {
>> + macro_invoc.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip simple path
>> +
>> + // I don't think any macro token trees can be stripped in any way
>> +
>> + // TODO: maybe have cfg! macro stripping behaviour here?
>> + expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ());
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::PathInExpression &path)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (path.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
>> + {
>> + path.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &segment : path.get_segments ())
>> + {
>> + if (segment.has_generic_args ())
>> + expand_generic_args (segment.get_generic_args ());
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypePathSegment &)
>> +{
>> + // shouldn't require
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypePathSegmentGeneric &segment)
>> +{
>> + // TODO: strip inside generic args
>> +
>> + if (!segment.has_generic_args ())
>> + return;
>> +
>> + expand_generic_args (segment.get_generic_args ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypePathSegmentFunction &segment)
>> +{
>> + auto &type_path_function = segment.get_type_path_function ();
>> +
>> + for (auto &type : type_path_function.get_params ())
>> + {
>> + type->accept_vis (*this);
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> + }
>> +
>> + if (type_path_function.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = type_path_function.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypePath &path)
>> +{
>> + // this shouldn't strip any segments, but can strip inside them
>> + for (auto &segment : path.get_segments ())
>> + segment->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::QualifiedPathInExpression &path)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (path.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
>> + {
>> + path.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expand_qualified_path_type (path.get_qualified_path_type ());
>> +
>> + for (auto &segment : path.get_segments ())
>> + {
>> + if (segment.has_generic_args ())
>> + expand_generic_args (segment.get_generic_args ());
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::QualifiedPathInType &path)
>> +{
>> + expand_qualified_path_type (path.get_qualified_path_type ());
>> +
>> + // this shouldn't strip any segments, but can strip inside them
>> + for (auto &segment : path.get_segments ())
>> + segment->accept_vis (*this);
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::LiteralExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::AttrInputLiteral &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::MetaItemLitExpr &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::MetaItemPathLit &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::BorrowExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &borrowed_expr = expr.get_borrowed_expr ();
>> + borrowed_expr->accept_vis (*this);
>> + if (borrowed_expr->is_marked_for_strip ())
>> + rust_error_at (borrowed_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::DereferenceExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &dereferenced_expr = expr.get_dereferenced_expr ();
>> + dereferenced_expr->accept_vis (*this);
>> + if (dereferenced_expr->is_marked_for_strip ())
>> + rust_error_at (dereferenced_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ErrorPropagationExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &propagating_expr = expr.get_propagating_expr ();
>> + propagating_expr->accept_vis (*this);
>> + if (propagating_expr->is_marked_for_strip ())
>> + rust_error_at (propagating_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::NegationExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &negated_expr = expr.get_negated_expr ();
>> + negated_expr->accept_vis (*this);
>> + if (negated_expr->is_marked_for_strip ())
>> + rust_error_at (negated_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &l_expr = expr.get_left_expr ();
>> + l_expr->accept_vis (*this);
>> + maybe_expand_expr (l_expr);
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &r_expr = expr.get_right_expr ();
>> + r_expr->accept_vis (*this);
>> + maybe_expand_expr (r_expr);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before binary op exprs");
>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ComparisonExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &l_expr = expr.get_left_expr ();
>> + l_expr->accept_vis (*this);
>> + maybe_expand_expr (l_expr);
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &r_expr = expr.get_right_expr ();
>> + r_expr->accept_vis (*this);
>> + maybe_expand_expr (r_expr);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before binary op exprs");
>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>
> vertical space between function definitions missing here (and above and below)
>
>> +void
>> +AttrVisitor::visit (AST::LazyBooleanExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &l_expr = expr.get_left_expr ();
>> + l_expr->accept_vis (*this);
>> + maybe_expand_expr (l_expr);
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &r_expr = expr.get_right_expr ();
>> + r_expr->accept_vis (*this);
>> + maybe_expand_expr (r_expr);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before binary op exprs");
>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypeCastExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * direct descendant expression, can strip ones below that */
>> +
>> + auto &casted_expr = expr.get_casted_expr ();
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + casted_expr->accept_vis (*this);
>> +
>> + // ensure that they are not marked for strip
>> + if (casted_expr->is_marked_for_strip ())
>> + rust_error_at (casted_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed before cast exprs");
>> +
>> + // TODO: strip sub-types of type
>> + auto &type = expr.get_type_to_cast_to ();
>> + type->accept_vis (*this);
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::AssignmentExpr &expr)
>> +{
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &l_expr = expr.get_left_expr ();
>> + l_expr->accept_vis (*this);
>> + maybe_expand_expr (l_expr);
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &r_expr = expr.get_right_expr ();
>> + r_expr->accept_vis (*this);
>> + maybe_expand_expr (r_expr);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before binary op exprs");
>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::CompoundAssignmentExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &l_expr = expr.get_left_expr ();
>> + l_expr->accept_vis (*this);
>> + maybe_expand_expr (l_expr);
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &r_expr = expr.get_right_expr ();
>> + r_expr->accept_vis (*this);
>> + maybe_expand_expr (r_expr);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before binary op exprs");
>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::GroupedExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says these are inner
>> + * attributes, not outer attributes of inner expr */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &inner_expr = expr.get_expr_in_parens ();
>> + inner_expr->accept_vis (*this);
>> + if (inner_expr->is_marked_for_strip ())
>> + rust_error_at (inner_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArrayElemsValues &elems)
>> +{
>> + /* apparently outer attributes are allowed in "elements of array
>> + * expressions" according to spec */
>> + expand_pointer_allow_strip (elems.get_values ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArrayElemsCopied &elems)
>> +{
>> + /* apparently outer attributes are allowed in "elements of array
>> + * expressions" according to spec. on the other hand, it would not
>> + * make conceptual sense to be able to remove either expression. As
>> + * such, not implementing. TODO clear up the ambiguity here */
>> +
>> + // only intend stripping for internal sub-expressions
>> + auto &copied_expr = elems.get_elem_to_copy ();
>> + copied_expr->accept_vis (*this);
>> + if (copied_expr->is_marked_for_strip ())
>> + rust_error_at (copied_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + auto ©_count = elems.get_num_copies ();
>> + copy_count->accept_vis (*this);
>> + if (copy_count->is_marked_for_strip ())
>> + rust_error_at (copy_count->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArrayExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says there are separate
>> + * inner attributes, not just outer attributes of inner exprs */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* assuming you can't strip away the ArrayElems type, but can strip
>> + * internal expressions and whatever */
>> + expr.get_array_elems ()->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArrayIndexExpr &expr)
>> +{
>> + /* it is unclear whether outer attributes are supposed to be
>> + * allowed, but conceptually it wouldn't make much sense, but
>> + * having expansion code anyway. TODO */
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &array_expr = expr.get_array_expr ();
>> + array_expr->accept_vis (*this);
>> + if (array_expr->is_marked_for_strip ())
>> + rust_error_at (array_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + auto &index_expr = expr.get_index_expr ();
>> + index_expr->accept_vis (*this);
>> + if (index_expr->is_marked_for_strip ())
>> + rust_error_at (index_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleExpr &expr)
>> +{
>> + /* according to spec, outer attributes are allowed on "elements of
>> + * tuple expressions" */
>> +
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says these are inner
>> + * attributes, not outer attributes of inner expr */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* apparently outer attributes are allowed in "elements of tuple
>> + * expressions" according to spec */
>> + expand_pointer_allow_strip (expr.get_tuple_elems ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleIndexExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* wouldn't strip this directly (as outer attrs should be
>> + * associated with this level), but any sub-expressions would be
>> + * stripped. Thus, no need to erase when strip check called. */
>> + auto &tuple_expr = expr.get_tuple_expr ();
>> + tuple_expr->accept_vis (*this);
>> + if (tuple_expr->is_marked_for_strip ())
>> + rust_error_at (tuple_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprStruct &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says these are inner
>> + * attributes, not outer attributes of inner expr */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-exprs of path
>> + auto &struct_name = expr.get_struct_name ();
>> + visit (struct_name);
>> + if (struct_name.is_marked_for_strip ())
>> + rust_error_at (struct_name.get_locus (),
>> + "cannot strip path in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprFieldIdentifier &)
>> +{
>> + // as no attrs (at moment, at least), no stripping possible
>
> has
>
> is there a reason to not put a gcc_unreachable () here?
>
The visitor goes through all the nodes in a crate, even if there is
nothing to do. In the case of the Attribute visitor, there currently
isn't anything to do for the following pattern (which is what
`StructExpreFieldIdentifier` is the node for)
```rust
struct Foo {
a: i32,
b: i32,
}
fn init_struct_with_a(a: i32) -> Foo {
Foo {
a, // <--- here
b: some_fn(),
}
}
```
where a struct's field is initialized by a variable of the same name
(this is a short-hand for `a: a`).
But that does not mean that the node is not visited. What happens is
that we visit all the nodes of a struct's initialization, which are of
the base class `StructExprField` and then dynamically dispatch to the
proper visitor functions, we must handle the case of a
`StructExprFieldIdentifier` being visited even if there is nothing to do.
If we add a `gcc_unreachable` here, the above test case would crash, as
it would try stripping the attributes of both field initalizations. For
the first case, there is nothing to do. For the second case, we want to
make sure we should not be stripping the sub-expression (the call to
`some_fn`)
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field)
>> +{
>> + /* as no attrs possible (at moment, at least), only sub-expression
>> + * stripping is possible */
>> + auto &value = field.get_value ();
>> + value->accept_vis (*this);
>> + if (value->is_marked_for_strip ())
>> + rust_error_at (value->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprFieldIndexValue &field)
>> +{
>> + /* as no attrs possible (at moment, at least), only sub-expression
>> + * stripping is possible */
>> + auto &value = field.get_value ();
>> + value->accept_vis (*this);
>> + if (value->is_marked_for_strip ())
>> + rust_error_at (value->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprStructFields &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says these are inner
>> + * attributes, not outer attributes of inner expr */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-exprs of path
>> + auto &struct_name = expr.get_struct_name ();
>> + visit (struct_name);
>> + if (struct_name.is_marked_for_strip ())
>> + rust_error_at (struct_name.get_locus (),
>> + "cannot strip path in this position");
>> +
>> + /* spec does not specify whether expressions are allowed to be
>> + * stripped at top level of struct fields, but I wouldn't think
>> + * that they would be, so operating under the assumption that only
>> + * sub-expressions can be stripped. */
>> + for (auto &field : expr.get_fields ())
>> + {
>> + field->accept_vis (*this);
>> + // shouldn't strip in this
>> + }
>> +
>> + /* struct base presumably can't be stripped, as the '..' is before
>> + * the expression. as such, can only strip sub-expressions. */
>> + if (expr.has_struct_base ())
>> + {
>> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
>> + base_struct_expr->accept_vis (*this);
>> + if (base_struct_expr->is_marked_for_strip ())
>> + rust_error_at (base_struct_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructExprStructBase &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says these are inner
>> + * attributes, not outer attributes of inner expr */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-exprs of path
>> + auto &struct_name = expr.get_struct_name ();
>> + visit (struct_name);
>> + if (struct_name.is_marked_for_strip ())
>> + rust_error_at (struct_name.get_locus (),
>> + "cannot strip path in this position");
>> +
>> + /* struct base presumably can't be stripped, as the '..' is before
>> + * the expression. as such, can only strip sub-expressions. */
>> + rust_assert (!expr.get_struct_base ().is_invalid ());
>> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
>> + base_struct_expr->accept_vis (*this);
>> + if (base_struct_expr->is_marked_for_strip ())
>> + rust_error_at (base_struct_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::CallExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* should not be outer attrs on "function" expression - outer attrs
>> + * should be associated with call expr as a whole. only sub-expr
>> + * expansion is possible. */
>> + auto &function = expr.get_function_expr ();
>> + function->accept_vis (*this);
>> + if (function->is_marked_for_strip ())
>> + rust_error_at (function->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + /* spec says outer attributes are specifically allowed for elements
>> + * of call expressions, so full stripping possible */
>> + // FIXME: Arthur: Figure out how to refactor this - This is similar to
>> + // expanding items in the crate or stmts in blocks
>> + expand_pointer_allow_strip (expr.get_params ());
>> + auto ¶ms = expr.get_params ();
>> + for (auto it = params.begin (); it != params.end ();)
>> + {
>> + auto &stmt = *it;
>> +
>> + stmt->accept_vis (*this);
>> +
>> + auto final_fragment = expand_macro_fragment_recursive ();
>> + if (final_fragment.should_expand ())
>> + {
>> + // Remove the current expanded invocation
>> + it = params.erase (it);
>> + for (auto &node : final_fragment.get_nodes ())
>> + {
>> + it = params.insert (it, node.take_expr ());
>> + it++;
>> + }
>> + }
>> + else if (stmt->is_marked_for_strip ())
>> + it = params.erase (it);
>> + else
>> + it++;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::MethodCallExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* should not be outer attrs on "receiver" expression - outer attrs
>> + * should be associated with call expr as a whole. only sub-expr
>> + * expansion is possible. */
>> + auto &receiver = expr.get_receiver_expr ();
>> + receiver->accept_vis (*this);
>> + if (receiver->is_marked_for_strip ())
>> + rust_error_at (receiver->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + auto &method_name = expr.get_method_name ();
>> + if (method_name.has_generic_args ())
>> + expand_generic_args (method_name.get_generic_args ());
>> +
>> + /* spec says outer attributes are specifically allowed for elements
>> + * of method call expressions, so full stripping possible */
>> + expand_pointer_allow_strip (expr.get_params ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::FieldAccessExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* should not be outer attrs on "receiver" expression - outer attrs
>> + * should be associated with field expr as a whole. only sub-expr
>> + * expansion is possible. */
>> + auto &receiver = expr.get_receiver_expr ();
>> + receiver->accept_vis (*this);
>> + if (receiver->is_marked_for_strip ())
>> + rust_error_at (receiver->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ClosureExprInner &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip closure parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_closure_params (expr.get_params ());
>> +
>> + // can't strip expression itself, but can strip sub-expressions
>> + auto &definition_expr = expr.get_definition_expr ();
>> + definition_expr->accept_vis (*this);
>> + if (definition_expr->is_marked_for_strip ())
>> + rust_error_at (definition_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::BlockExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip test based on inner attrs - spec says there are inner
>> + * attributes, not just outer attributes of inner stmts */
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + std::function<std::unique_ptr<AST::Stmt> (AST::SingleASTNode)> extractor
>> + = [] (AST::SingleASTNode node) { return node.take_stmt (); };
>> +
>> + expand_macro_children (MacroExpander::BLOCK, expr.get_statements (),
>> + extractor);
>> +
>> + expander.push_context (MacroExpander::BLOCK);
>> +
>> + // strip tail expression if exists - can actually fully remove it
>> + if (expr.has_tail_expr ())
>> + {
>> + auto &tail_expr = expr.get_tail_expr ();
>> +
>> + tail_expr->accept_vis (*this);
>> + maybe_expand_expr (tail_expr);
>> +
>> + if (tail_expr->is_marked_for_strip ())
>> + expr.strip_tail_expr ();
>> + }
>> + expander.pop_context ();
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::ClosureExprInnerTyped &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip closure parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_closure_params (expr.get_params ());
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // can't strip return type, but can strip sub-types
>> + auto &type = expr.get_return_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + // can't strip expression itself, but can strip sub-expressions
>> + auto &definition_block = expr.get_definition_block ();
>> + definition_block->accept_vis (*this);
>> + if (definition_block->is_marked_for_strip ())
>> + rust_error_at (definition_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ContinueExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::BreakExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* spec does not say that you can have outer attributes on
>> + * expression, so assuming you can't. stripping for sub-expressions
>> + * is the only thing that can be done */
>> + if (expr.has_break_expr ())
>> + {
>> + auto &break_expr = expr.get_break_expr ();
>> +
>> + break_expr->accept_vis (*this);
>> +
>> + if (break_expr->is_marked_for_strip ())
>> + rust_error_at (break_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeFromToExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + expr.get_from_expr ()->accept_vis (*this);
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + expr.get_to_expr ()->accept_vis (*this);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_from_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_from_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before range exprs");
>> + if (expr.get_to_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_to_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeFromExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * direct descendant expression, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + auto &from_expr = expr.get_from_expr ();
>> +
>> + from_expr->accept_vis (*this);
>> +
>> + if (from_expr->is_marked_for_strip ())
>> + rust_error_at (from_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed before range exprs");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeToExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * direct descendant expression, can strip ones below that */
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &to_expr = expr.get_to_expr ();
>> +
>> + to_expr->accept_vis (*this);
>> +
>> + if (to_expr->is_marked_for_strip ())
>> + rust_error_at (to_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeFullExpr &)
>> +{
>> + // outer attributes never allowed before these, so no stripping
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeFromToInclExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * two direct descendant expressions, can strip ones below that */
>> +
>> + /* should have no possibility for outer attrs as would be parsed
>> + * with outer expr */
>> + expr.get_from_expr ()->accept_vis (*this);
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + expr.get_to_expr ()->accept_vis (*this);
>> +
>> + // ensure that they are not marked for strip
>> + if (expr.get_from_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_from_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes are never allowed "
>> + "before range exprs");
>> + if (expr.get_to_expr ()->is_marked_for_strip ())
>> + rust_error_at (expr.get_to_expr ()->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangeToInclExpr &expr)
>> +{
>> + /* outer attributes never allowed before these. while cannot strip
>> + * direct descendant expression, can strip ones below that */
>> +
>> + /* should syntactically not have outer attributes, though this may
>> + * not have worked in practice */
>> + auto &to_expr = expr.get_to_expr ();
>> +
>> + to_expr->accept_vis (*this);
>> +
>> + if (to_expr->is_marked_for_strip ())
>> + rust_error_at (to_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ReturnExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* spec does not say that you can have outer attributes on
>> + * expression, so assuming you can't. stripping for sub-expressions
>> + * is the only thing that can be done */
>> + if (expr.has_returned_expr ())
>> + {
>> + auto &returned_expr = expr.get_returned_expr ();
>> +
>> + returned_expr->accept_vis (*this);
>> +
>> + if (returned_expr->is_marked_for_strip ())
>> + rust_error_at (returned_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> + }
>> + /* TODO: conceptually, you would maybe be able to remove a returned
>> + * expr - e.g. if you had conditional compilation returning void or
>> + * returning a type. On the other hand, I think that function
>> + * return type cannot be conditionally compiled, so I assumed you
>> + * can't do this either. */
>> +}
>> +void
>> +AttrVisitor::visit (AST::UnsafeBlockExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &block_expr = expr.get_block_expr ();
>> + block_expr->accept_vis (*this);
>> + if (block_expr->is_marked_for_strip ())
>> + rust_error_at (block_expr->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::LoopExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &loop_block = expr.get_loop_block ();
>> + loop_block->accept_vis (*this);
>> + if (loop_block->is_marked_for_strip ())
>> + rust_error_at (loop_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::WhileLoopExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip predicate expr itself, but can strip sub-expressions
>> + auto &predicate_expr = expr.get_predicate_expr ();
>> + predicate_expr->accept_vis (*this);
>> + if (predicate_expr->is_marked_for_strip ())
>> + rust_error_at (predicate_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &loop_block = expr.get_loop_block ();
>> + loop_block->accept_vis (*this);
>> + if (loop_block->is_marked_for_strip ())
>> + rust_error_at (loop_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::WhileLetLoopExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &pattern : expr.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
>> + scrutinee_expr->accept_vis (*this);
>> + if (scrutinee_expr->is_marked_for_strip ())
>> + rust_error_at (scrutinee_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &loop_block = expr.get_loop_block ();
>> + loop_block->accept_vis (*this);
>> + if (loop_block->is_marked_for_strip ())
>> + rust_error_at (loop_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ForLoopExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-patterns of pattern
>> + auto &pattern = expr.get_pattern ();
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +
>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>> + auto &iterator_expr = expr.get_iterator_expr ();
>> + iterator_expr->accept_vis (*this);
>> + if (iterator_expr->is_marked_for_strip ())
>> + rust_error_at (iterator_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &loop_block = expr.get_loop_block ();
>> + loop_block->accept_vis (*this);
>> + if (loop_block->is_marked_for_strip ())
>> + rust_error_at (loop_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfExpr &expr)
>> +{
>> + // rust playground test shows that IfExpr does support outer attrs, at least
>> + // when used as statement
>> +
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip condition expr itself, but can strip sub-expressions
>> + auto &condition_expr = expr.get_condition_expr ();
>> + condition_expr->accept_vis (*this);
>> + maybe_expand_expr (condition_expr);
>> + if (condition_expr->is_marked_for_strip ())
>> + rust_error_at (condition_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfExprConseqElse &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip condition expr itself, but can strip sub-expressions
>> + auto &condition_expr = expr.get_condition_expr ();
>> + condition_expr->accept_vis (*this);
>> + maybe_expand_expr (condition_expr);
>> + if (condition_expr->is_marked_for_strip ())
>> + rust_error_at (condition_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip else block itself, but can strip sub-expressions
>> + auto &else_block = expr.get_else_block ();
>> + else_block->accept_vis (*this);
>> + if (else_block->is_marked_for_strip ())
>> + rust_error_at (else_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfExprConseqIf &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip condition expr itself, but can strip sub-expressions
>> + auto &condition_expr = expr.get_condition_expr ();
>> + condition_expr->accept_vis (*this);
>> + maybe_expand_expr (condition_expr);
>> + if (condition_expr->is_marked_for_strip ())
>> + rust_error_at (condition_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if expr itself, but can strip sub-expressions
>> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
>> + conseq_if_expr->accept_vis (*this);
>> + if (conseq_if_expr->is_marked_for_strip ())
>> + rust_error_at (conseq_if_expr->get_locus (),
>> + "cannot strip consequent if expression in this "
>> + "position - outer attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfExprConseqIfLet &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip condition expr itself, but can strip sub-expressions
>> + auto &condition_expr = expr.get_condition_expr ();
>> + condition_expr->accept_vis (*this);
>> + maybe_expand_expr (condition_expr);
>> + if (condition_expr->is_marked_for_strip ())
>> + rust_error_at (condition_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if let expr itself, but can strip sub-expressions
>> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
>> + conseq_if_let_expr->accept_vis (*this);
>> + if (conseq_if_let_expr->is_marked_for_strip ())
>> + rust_error_at (conseq_if_let_expr->get_locus (),
>> + "cannot strip consequent if let expression in this "
>> + "position - outer attributes not "
>> + "allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfLetExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &pattern : expr.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + // can't strip value expr itself, but can strip sub-expressions
>> + auto &value_expr = expr.get_value_expr ();
>> + value_expr->accept_vis (*this);
>> + if (value_expr->is_marked_for_strip ())
>> + rust_error_at (value_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfLetExprConseqElse &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &pattern : expr.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + // can't strip value expr itself, but can strip sub-expressions
>> + auto &value_expr = expr.get_value_expr ();
>> + value_expr->accept_vis (*this);
>> + if (value_expr->is_marked_for_strip ())
>> + rust_error_at (value_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip else block itself, but can strip sub-expressions
>> + auto &else_block = expr.get_else_block ();
>> + else_block->accept_vis (*this);
>> + if (else_block->is_marked_for_strip ())
>> + rust_error_at (else_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfLetExprConseqIf &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &pattern : expr.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + // can't strip value expr itself, but can strip sub-expressions
>> + auto &value_expr = expr.get_value_expr ();
>> + value_expr->accept_vis (*this);
>> + if (value_expr->is_marked_for_strip ())
>> + rust_error_at (value_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if expr itself, but can strip sub-expressions
>> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
>> + conseq_if_expr->accept_vis (*this);
>> + if (conseq_if_expr->is_marked_for_strip ())
>> + rust_error_at (conseq_if_expr->get_locus (),
>> + "cannot strip consequent if expression in this "
>> + "position - outer attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + for (auto &pattern : expr.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + // can't strip value expr itself, but can strip sub-expressions
>> + auto &value_expr = expr.get_value_expr ();
>> + value_expr->accept_vis (*this);
>> + if (value_expr->is_marked_for_strip ())
>> + rust_error_at (value_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if block itself, but can strip sub-expressions
>> + auto &if_block = expr.get_if_block ();
>> + if_block->accept_vis (*this);
>> + if (if_block->is_marked_for_strip ())
>> + rust_error_at (if_block->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // can't strip if let expr itself, but can strip sub-expressions
>> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
>> + conseq_if_let_expr->accept_vis (*this);
>> + if (conseq_if_let_expr->is_marked_for_strip ())
>> + rust_error_at (conseq_if_let_expr->get_locus (),
>> + "cannot strip consequent if let expression in this "
>> + "position - outer attributes not "
>> + "allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::MatchExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // inner attr strip test
>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
>> + scrutinee_expr->accept_vis (*this);
>> + if (scrutinee_expr->is_marked_for_strip ())
>> + rust_error_at (scrutinee_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // strip match cases
>> + auto &match_cases = expr.get_match_cases ();
>> + for (auto it = match_cases.begin (); it != match_cases.end ();)
>> + {
>> + auto &match_case = *it;
>> +
>> + // strip match case based on outer attributes in match arm
>> + auto &match_arm = match_case.get_arm ();
>> + expander.expand_cfg_attrs (match_arm.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ()))
>> + {
>> + // strip match case
>> + it = match_cases.erase (it);
>> + continue;
>> + }
>> +
>> + for (auto &pattern : match_arm.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + }
>> +
>> + /* assuming that guard expression cannot be stripped as
>> + * strictly speaking you would have to strip the whole guard to
>> + * make syntactical sense, which you can't do. as such, only
>> + * strip sub-expressions */
>> + if (match_arm.has_match_arm_guard ())
>> + {
>> + auto &guard_expr = match_arm.get_guard_expr ();
>> + guard_expr->accept_vis (*this);
>> + if (guard_expr->is_marked_for_strip ())
>> + rust_error_at (guard_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> + }
>> +
>> + // strip sub-expressions from match cases
>> + auto &case_expr = match_case.get_expr ();
>> + case_expr->accept_vis (*this);
>> + if (case_expr->is_marked_for_strip ())
>> + rust_error_at (case_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + // increment to next case if haven't continued
>> + ++it;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::AwaitExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* can't strip awaited expr itself, but can strip sub-expressions
>> + * - this is because you can't have no expr to await */
>> + auto &awaited_expr = expr.get_awaited_expr ();
>> + awaited_expr->accept_vis (*this);
>> + if (awaited_expr->is_marked_for_strip ())
>> + rust_error_at (awaited_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::AsyncBlockExpr &expr)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>> + {
>> + expr.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &block_expr = expr.get_block_expr ();
>> + block_expr->accept_vis (*this);
>> + if (block_expr->is_marked_for_strip ())
>> + rust_error_at (block_expr->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::TypeParam ¶m)
>> +{
>> + // outer attributes don't actually do anything, so ignore them
>> +
>> + if (param.has_type_param_bounds ())
>> + {
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : param.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> + }
>> +
>> + if (param.has_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> + auto &type = param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::LifetimeWhereClauseItem &)
>> +{
>> + // shouldn't require
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item)
>> +{
>> + // for lifetimes shouldn't require
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = item.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : item.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::Method &method)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (method.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (method.get_outer_attrs ()))
>> + {
>> + method.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : method.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* assuming you can't strip self param - wouldn't be a method
>> + * anymore. spec allows outer attrs on self param, but doesn't
>> + * specify whether cfg is used. */
>> + expand_self_param (method.get_self_param ());
>> +
>> + /* strip method parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_function_params (method.get_function_params ());
>> +
>> + if (method.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = method.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + if (method.has_where_clause ())
>> + expand_where_clause (method.get_where_clause ());
>> +
>> + /* body should always exist - if error state, should have returned
>> + * before now */
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &block_expr = method.get_definition ();
>> + block_expr->accept_vis (*this);
>> + if (block_expr->is_marked_for_strip ())
>> + rust_error_at (block_expr->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::Module &module)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (module.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (module.get_outer_attrs ()))
>> + {
>> + module.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // A loaded module might have inner attributes
>> + if (module.get_kind () == AST::Module::ModuleKind::LOADED)
>> + {
>> + // strip test based on inner attrs
>> + expander.expand_cfg_attrs (module.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (module.get_inner_attrs ()))
>> + {
>> + module.mark_for_strip ();
>> + return;
>> + }
>> + }
>> +
>> + // Parse the module's items if they haven't been expanded and the file
>> + // should be parsed (i.e isn't hidden behind an untrue or impossible cfg
>> + // directive)
>> + if (!module.is_marked_for_strip ()
>> + && module.get_kind () == AST::Module::ModuleKind::UNLOADED)
>> + {
>> + module.load_items ();
>> + }
>> +
>> + // strip items if required
>> + expand_pointer_allow_strip (module.get_items ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::ExternCrate &extern_crate)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (extern_crate.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ()))
>> + {
>> + extern_crate.mark_for_strip ();
>> + return;
>> + }
>> +
>> + if (!extern_crate.references_self ())
>> + {
>> + Session &session = Session::get_instance ();
>> + session.load_extern_crate (extern_crate.get_referenced_crate (),
>> + extern_crate.get_locus ());
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::UseTreeGlob &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::UseTreeList &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::UseTreeRebind &)
>> +{
>> + // shouldn't require?
>> +}
>> +void
>> +AttrVisitor::visit (AST::UseDeclaration &use_decl)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (use_decl.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ()))
>> + {
>> + use_decl.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::Function &function)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (function.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (function.get_outer_attrs ()))
>> + {
>> + function.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : function.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* strip function parameters if required - this is specifically
>> + * allowed by spec */
>> + expand_function_params (function.get_function_params ());
>> +
>> + if (function.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = function.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + if (function.has_where_clause ())
>> + expand_where_clause (function.get_where_clause ());
>> +
>> + /* body should always exist - if error state, should have returned
>> + * before now */
>> + // can't strip block itself, but can strip sub-expressions
>> + auto &block_expr = function.get_definition ();
>> + block_expr->accept_vis (*this);
>> + if (block_expr->is_marked_for_strip ())
>> + rust_error_at (block_expr->get_locus (),
>> + "cannot strip block expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::TypeAlias &type_alias)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (type_alias.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ()))
>> + {
>> + type_alias.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : type_alias.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + if (type_alias.has_where_clause ())
>> + expand_where_clause (type_alias.get_where_clause ());
>> +
>> + auto &type = type_alias.get_type_aliased ();
>> + type->accept_vis (*this);
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructStruct &struct_item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (struct_item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ()))
>> + {
>> + struct_item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : struct_item.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + if (struct_item.has_where_clause ())
>> + expand_where_clause (struct_item.get_where_clause ());
>> +
>> + /* strip struct fields if required - this is presumably
>> + * allowed by spec */
>> + expand_struct_fields (struct_item.get_fields ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleStruct &tuple_struct)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ()))
>> + {
>> + tuple_struct.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : tuple_struct.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* strip struct fields if required - this is presumably
>> + * allowed by spec */
>> + expand_tuple_fields (tuple_struct.get_fields ());
>> +
>> + if (tuple_struct.has_where_clause ())
>> + expand_where_clause (tuple_struct.get_where_clause ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::EnumItem &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::EnumItemTuple &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip item fields if required - this is presumably
>> + * allowed by spec */
>> + expand_tuple_fields (item.get_tuple_fields ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::EnumItemStruct &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip item fields if required - this is presumably
>> + * allowed by spec */
>> + expand_struct_fields (item.get_struct_fields ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::EnumItemDiscriminant &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &expr = item.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + rust_error_at (expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::Enum &enum_item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (enum_item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ()))
>> + {
>> + enum_item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : enum_item.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + if (enum_item.has_where_clause ())
>> + expand_where_clause (enum_item.get_where_clause ());
>> +
>> + /* strip enum fields if required - this is presumably
>> + * allowed by spec */
>> + expand_pointer_allow_strip (enum_item.get_variants ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::Union &union_item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (union_item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ()))
>> + {
>> + union_item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : union_item.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + if (union_item.has_where_clause ())
>> + expand_where_clause (union_item.get_where_clause ());
>> +
>> + /* strip union fields if required - this is presumably
>> + * allowed by spec */
>> + expand_struct_fields (union_item.get_variants ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::ConstantItem &const_item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (const_item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ()))
>> + {
>> + const_item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // strip any sub-types
>> + auto &type = const_item.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &expr = const_item.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + rust_error_at (expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StaticItem &static_item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (static_item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ()))
>> + {
>> + static_item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // strip any sub-types
>> + auto &type = static_item.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &expr = static_item.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + rust_error_at (expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitItemFunc &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expand_trait_function_decl (item.get_trait_function_decl ());
>> +
>> + if (item.has_definition ())
>> + {
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &block = item.get_definition ();
>> + block->accept_vis (*this);
>> + if (block->is_marked_for_strip ())
>> + rust_error_at (block->get_locus (),
>> + "cannot strip block expression in this "
>> + "position - outer attributes not allowed");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitItemMethod &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expand_trait_method_decl (item.get_trait_method_decl ());
>> +
>> + if (item.has_definition ())
>> + {
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped. */
>> + auto &block = item.get_definition ();
>> + block->accept_vis (*this);
>> + if (block->is_marked_for_strip ())
>> + rust_error_at (block->get_locus (),
>> + "cannot strip block expression in this "
>> + "position - outer attributes not allowed");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitItemConst &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + // strip any sub-types
>> + auto &type = item.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped */
>> + if (item.has_expression ())
>> + {
>> + auto &expr = item.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + rust_error_at (expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitItemType &item)
>> +{
>> + // initial test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + if (item.has_type_param_bounds ())
>> + {
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : item.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::Trait &trait)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (trait.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (trait.get_outer_attrs ()))
>> + {
>> + trait.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip test based on inner attrs
>> + expander.expand_cfg_attrs (trait.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (trait.get_inner_attrs ()))
>> + {
>> + trait.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : trait.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + if (trait.has_type_param_bounds ())
>> + {
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : trait.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> + }
>> +
>> + if (trait.has_where_clause ())
>> + expand_where_clause (trait.get_where_clause ());
>> +
>> + std::function<std::unique_ptr<AST::TraitItem> (AST::SingleASTNode)> extractor
>> + = [] (AST::SingleASTNode node) { return node.take_trait_item (); };
>> +
>> + expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (),
>> + extractor);
>> +}
>> +void
>> +AttrVisitor::visit (AST::InherentImpl &impl)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
>> + {
>> + impl.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip test based on inner attrs
>> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
>> + {
>> + impl.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : impl.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = impl.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + if (impl.has_where_clause ())
>> + expand_where_clause (impl.get_where_clause ());
>> +
>> + std::function<std::unique_ptr<AST::InherentImplItem> (AST::SingleASTNode)>
>> + extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); };
>> +
>> + expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (),
>> + extractor);
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitImpl &impl)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
>> + {
>> + impl.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip test based on inner attrs
>> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
>> + {
>> + impl.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : impl.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = impl.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + auto &trait_path = impl.get_trait_path ();
>> + visit (trait_path);
>> + if (trait_path.is_marked_for_strip ())
>> + rust_error_at (trait_path.get_locus (),
>> + "cannot strip typepath in this position");
>> +
>> + if (impl.has_where_clause ())
>> + expand_where_clause (impl.get_where_clause ());
>> +
>> + std::function<std::unique_ptr<AST::TraitImplItem> (AST::SingleASTNode)>
>> + extractor
>> + = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); };
>> +
>> + expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (),
>> + extractor);
>> +}
>> +void
>> +AttrVisitor::visit (AST::ExternalStaticItem &item)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = item.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +}
>> +void
>> +AttrVisitor::visit (AST::ExternalFunctionItem &item)
>> +{
>> + // strip test based on outer attrs
>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>> + {
>> + item.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // just expand sub-stuff - can't actually strip generic params themselves
>> + for (auto ¶m : item.get_generic_params ())
>> + param->accept_vis (*this);
>> +
>> + /* strip function parameters if required - this is specifically
>> + * allowed by spec */
>> + auto ¶ms = item.get_function_params ();
>> + for (auto it = params.begin (); it != params.end ();)
>> + {
>> + auto ¶m = *it;
>> +
>> + auto ¶m_attrs = param.get_outer_attrs ();
>> + expander.expand_cfg_attrs (param_attrs);
>> + if (expander.fails_cfg_with_expand (param_attrs))
>> + {
>> + it = params.erase (it);
>> + continue;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + // increment if nothing else happens
>> + ++it;
>> + }
>> + /* NOTE: these are extern function params, which may have different
>> + * rules and restrictions to "normal" function params. So expansion
>> + * handled separately. */
>> +
>> + /* TODO: assuming that variadic nature cannot be stripped. If this
>> + * is not true, then have code here to do so. */
>> +
>> + if (item.has_return_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &return_type = item.get_return_type ();
>> + return_type->accept_vis (*this);
>> +
>> + maybe_expand_type (return_type);
>> +
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + if (item.has_where_clause ())
>> + expand_where_clause (item.get_where_clause ());
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::ExternBlock &block)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (block.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (block.get_outer_attrs ()))
>> + {
>> + block.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip test based on inner attrs
>> + expander.expand_cfg_attrs (block.get_inner_attrs ());
>> + if (expander.fails_cfg_with_expand (block.get_inner_attrs ()))
>> + {
>> + block.mark_for_strip ();
>> + return;
>> + }
>> +
>> + std::function<std::unique_ptr<AST::ExternalItem> (AST::SingleASTNode)>
>> + extractor
>> + = [] (AST::SingleASTNode node) { return node.take_external_item (); };
>> +
>> + expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (),
>> + extractor);
>> +}
>> +
>> +// I don't think it would be possible to strip macros without expansion
>> +void
>> +AttrVisitor::visit (AST::MacroMatchFragment &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MacroMatchRepetition &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MacroMatcher &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MacroRulesDefinition &rules_def)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (rules_def.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ()))
>> + {
>> + rules_def.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // I don't think any macro rules can be stripped in any way
>> +
>> + auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (),
>> + rules_def.get_rule_name ());
>> + expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (),
>> + rules_def.get_locus ());
>> + expander.mappings->insert_macro_def (&rules_def);
>> + rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]",
>> + path.get ().c_str ());
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::MetaItemPath &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MetaItemSeq &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MetaWord &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MetaNameValueStr &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MetaListPaths &)
>> +{}
>> +void
>> +AttrVisitor::visit (AST::MetaListNameValueStr &)
>> +{}
>> +
>> +void
>> +AttrVisitor::visit (AST::LiteralPattern &)
>> +{
>> + // not possible
>> +}
>> +void
>> +AttrVisitor::visit (AST::IdentifierPattern &pattern)
>> +{
>> + // can only strip sub-patterns of the inner pattern to bind
>> + if (!pattern.has_pattern_to_bind ())
>> + return;
>> +
>> + auto &sub_pattern = pattern.get_pattern_to_bind ();
>> + sub_pattern->accept_vis (*this);
>> + if (sub_pattern->is_marked_for_strip ())
>> + rust_error_at (sub_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::WildcardPattern &)
>> +{
>> + // not possible
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangePatternBoundLiteral &)
>> +{
>> + // not possible
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangePatternBoundPath &bound)
>> +{
>> + // can expand path, but not strip it directly
>> + auto &path = bound.get_path ();
>> + visit (path);
>> + if (path.is_marked_for_strip ())
>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangePatternBoundQualPath &bound)
>> +{
>> + // can expand path, but not strip it directly
>> + auto &path = bound.get_qualified_path ();
>> + visit (path);
>> + if (path.is_marked_for_strip ())
>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::RangePattern &pattern)
>> +{
>> + // should have no capability to strip lower or upper bounds, only expand
>> + pattern.get_lower_bound ()->accept_vis (*this);
>> + pattern.get_upper_bound ()->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::ReferencePattern &pattern)
>> +{
>> + auto &sub_pattern = pattern.get_referenced_pattern ();
>> + sub_pattern->accept_vis (*this);
>> + if (sub_pattern->is_marked_for_strip ())
>> + rust_error_at (sub_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructPatternFieldTuplePat &field)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>> + {
>> + field.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-patterns (can't strip top-level pattern)
>> + auto &sub_pattern = field.get_index_pattern ();
>> + sub_pattern->accept_vis (*this);
>> + if (sub_pattern->is_marked_for_strip ())
>> + rust_error_at (sub_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructPatternFieldIdentPat &field)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>> + {
>> + field.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // strip sub-patterns (can't strip top-level pattern)
>> + auto &sub_pattern = field.get_ident_pattern ();
>> + sub_pattern->accept_vis (*this);
>> + if (sub_pattern->is_marked_for_strip ())
>> + rust_error_at (sub_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructPatternFieldIdent &field)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>> + {
>> + field.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::StructPattern &pattern)
>> +{
>> + // expand (but don't strip) path
>> + auto &path = pattern.get_path ();
>> + visit (path);
>> + if (path.is_marked_for_strip ())
>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>> +
>> + /* TODO: apparently struct pattern fields can have outer attrs. so can they
>> + * be stripped? */
>> + if (!pattern.has_struct_pattern_elems ())
>> + return;
>> +
>> + auto &elems = pattern.get_struct_pattern_elems ();
>> +
>> + // assuming you can strip struct pattern fields
>> + expand_pointer_allow_strip (elems.get_struct_pattern_fields ());
>> +
>> + // assuming you can strip the ".." part
>> + if (elems.has_etc ())
>> + {
>> + expander.expand_cfg_attrs (elems.get_etc_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ()))
>> + elems.strip_etc ();
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items)
>> +{
>> + // can't strip individual patterns, only sub-patterns
>> + for (auto &pattern : tuple_items.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> +
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items)
>> +{
>> + // can't strip individual patterns, only sub-patterns
>> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
>> + {
>> + lower_pattern->accept_vis (*this);
>> +
>> + if (lower_pattern->is_marked_for_strip ())
>> + rust_error_at (lower_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
>> + {
>> + upper_pattern->accept_vis (*this);
>> +
>> + if (upper_pattern->is_marked_for_strip ())
>> + rust_error_at (upper_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleStructPattern &pattern)
>> +{
>> + // expand (but don't strip) path
>> + auto &path = pattern.get_path ();
>> + visit (path);
>> + if (path.is_marked_for_strip ())
>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>> +
>> + if (pattern.has_items ())
>> + pattern.get_items ()->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items)
>> +{
>> + // can't strip individual patterns, only sub-patterns
>> + for (auto &pattern : tuple_items.get_patterns ())
>> + {
>> + pattern->accept_vis (*this);
>> +
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items)
>> +{
>> + // can't strip individual patterns, only sub-patterns
>> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
>> + {
>> + lower_pattern->accept_vis (*this);
>> +
>> + if (lower_pattern->is_marked_for_strip ())
>> + rust_error_at (lower_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
>> + {
>> + upper_pattern->accept_vis (*this);
>> +
>> + if (upper_pattern->is_marked_for_strip ())
>> + rust_error_at (upper_pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::TuplePattern &pattern)
>> +{
>> + if (pattern.has_tuple_pattern_items ())
>> + pattern.get_items ()->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::GroupedPattern &pattern)
>> +{
>> + // can't strip inner pattern, only sub-patterns
>> + auto &pattern_in_parens = pattern.get_pattern_in_parens ();
>> +
>> + pattern_in_parens->accept_vis (*this);
>> +
>> + if (pattern_in_parens->is_marked_for_strip ())
>> + rust_error_at (pattern_in_parens->get_locus (),
>> + "cannot strip pattern in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::SlicePattern &pattern)
>> +{
>> + // can't strip individual patterns, only sub-patterns
>> + for (auto &item : pattern.get_items ())
>> + {
>> + item->accept_vis (*this);
>> +
>> + if (item->is_marked_for_strip ())
>> + rust_error_at (item->get_locus (),
>> + "cannot strip pattern in this position");
>> + // TODO: quit stripping now? or keep going?
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::EmptyStmt &)
>> +{
>> + // assuming no outer attributes, so nothing can happen
>> +}
>> +void
>> +AttrVisitor::visit (AST::LetStmt &stmt)
>> +{
>> + // initial strip test based on outer attrs
>> + expander.expand_cfg_attrs (stmt.get_outer_attrs ());
>> + if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ()))
>> + {
>> + stmt.mark_for_strip ();
>> + return;
>> + }
>> +
>> + // can't strip pattern, but call for sub-patterns
>> + auto &pattern = stmt.get_pattern ();
>> + pattern->accept_vis (*this);
>> + if (pattern->is_marked_for_strip ())
>> + rust_error_at (pattern->get_locus (),
>> + "cannot strip pattern in this position");
>> +
>> + // similar for type
>> + if (stmt.has_type ())
>> + {
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = stmt.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + /* strip any internal sub-expressions - expression itself isn't
>> + * allowed to have external attributes in this position so can't be
>> + * stripped */
>> + if (stmt.has_init_expr ())
>> + {
>> + auto &init_expr = stmt.get_init_expr ();
>> + init_expr->accept_vis (*this);
>> +
>> + if (init_expr->is_marked_for_strip ())
>> + rust_error_at (init_expr->get_locus (),
>> + "cannot strip expression in this position - outer "
>> + "attributes not allowed");
>> +
>> + maybe_expand_expr (init_expr);
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt)
>> +{
>> + // outer attributes associated with expr, so rely on expr
>> +
>> + // guard - should prevent null pointer expr
>> + if (stmt.is_marked_for_strip ())
>> + return;
>> +
>> + // strip if expr is to be stripped
>> + auto &expr = stmt.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + {
>> + stmt.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::ExprStmtWithBlock &stmt)
>> +{
>> + // outer attributes associated with expr, so rely on expr
>> +
>> + // guard - should prevent null pointer expr
>> + if (stmt.is_marked_for_strip ())
>> + return;
>> +
>> + // strip if expr is to be stripped
>> + auto &expr = stmt.get_expr ();
>> + expr->accept_vis (*this);
>> + if (expr->is_marked_for_strip ())
>> + {
>> + stmt.mark_for_strip ();
>> + return;
>> + }
>> +}
>> +
>> +void
>> +AttrVisitor::visit (AST::TraitBound &bound)
>> +{
>> + // nothing in for lifetimes to strip
>> +
>> + // expand but don't strip type path
>> + auto &path = bound.get_type_path ();
>> + visit (path);
>> + if (path.is_marked_for_strip ())
>> + rust_error_at (path.get_locus (),
>> + "cannot strip type path in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ImplTraitType &type)
>> +{
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : type.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitObjectType &type)
>> +{
>> + // don't strip directly, only components of bounds
>> + for (auto &bound : type.get_type_param_bounds ())
>> + bound->accept_vis (*this);
>> +}
>> +void
>> +AttrVisitor::visit (AST::ParenthesisedType &type)
>> +{
>> + // expand but don't strip inner type
>> + auto &inner_type = type.get_type_in_parens ();
>> + inner_type->accept_vis (*this);
>> + if (inner_type->is_marked_for_strip ())
>> + rust_error_at (inner_type->get_locus (),
>> + "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ImplTraitTypeOneBound &type)
>> +{
>> + // no stripping possible
>> + visit (type.get_trait_bound ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::TraitObjectTypeOneBound &type)
>> +{
>> + // no stripping possible
>> + visit (type.get_trait_bound ());
>> +}
>> +void
>> +AttrVisitor::visit (AST::TupleType &type)
>> +{
>> + // TODO: assuming that types can't be stripped as types don't have outer
>> + // attributes
>> + for (auto &elem_type : type.get_elems ())
>> + {
>> + elem_type->accept_vis (*this);
>> + if (elem_type->is_marked_for_strip ())
>> + rust_error_at (elem_type->get_locus (),
>> + "cannot strip type in this position");
>> + }
>> +}
>> +void
>> +AttrVisitor::visit (AST::NeverType &)
>> +{
>> + // no stripping possible
>> +}
>> +void
>> +AttrVisitor::visit (AST::RawPointerType &type)
>> +{
>> + // expand but don't strip type pointed to
>> + auto &pointed_type = type.get_type_pointed_to ();
>> + pointed_type->accept_vis (*this);
>> + if (pointed_type->is_marked_for_strip ())
>> + rust_error_at (pointed_type->get_locus (),
>> + "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ReferenceType &type)
>> +{
>> + // expand but don't strip type referenced
>> + auto &referenced_type = type.get_type_referenced ();
>> + referenced_type->accept_vis (*this);
>> + if (referenced_type->is_marked_for_strip ())
>> + rust_error_at (referenced_type->get_locus (),
>> + "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::ArrayType &type)
>> +{
>> + // expand but don't strip type referenced
>> + auto &base_type = type.get_elem_type ();
>> + base_type->accept_vis (*this);
>> + if (base_type->is_marked_for_strip ())
>> + rust_error_at (base_type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + // same for expression
>> + auto &size_expr = type.get_size_expr ();
>> + size_expr->accept_vis (*this);
>> + if (size_expr->is_marked_for_strip ())
>> + rust_error_at (size_expr->get_locus (),
>> + "cannot strip expression in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::SliceType &type)
>> +{
>> + // expand but don't strip elem type
>> + auto &elem_type = type.get_elem_type ();
>> + elem_type->accept_vis (*this);
>> + if (elem_type->is_marked_for_strip ())
>> + rust_error_at (elem_type->get_locus (),
>> + "cannot strip type in this position");
>> +}
>> +void
>> +AttrVisitor::visit (AST::InferredType &)
>> +{
>> + // none possible
>> +}
>> +void
>> +AttrVisitor::visit (AST::BareFunctionType &type)
>> +{
>> + // seem to be no generics
>> +
>> + // presumably function params can be stripped
>> + auto ¶ms = type.get_function_params ();
>> + for (auto it = params.begin (); it != params.end ();)
>> + {
>> + auto ¶m = *it;
>> +
>> + auto ¶m_attrs = param.get_outer_attrs ();
>> + expander.expand_cfg_attrs (param_attrs);
>> + if (expander.fails_cfg_with_expand (param_attrs))
>> + {
>> + it = params.erase (it);
>> + continue;
>> + }
>> +
>> + expander.push_context (MacroExpander::ContextType::TYPE);
>> +
>> + auto &type = param.get_type ();
>> + type->accept_vis (*this);
>> +
>> + maybe_expand_type (type);
>> +
>> + if (type->is_marked_for_strip ())
>> + rust_error_at (type->get_locus (),
>> + "cannot strip type in this position");
>> +
>> + expander.pop_context ();
>> +
>> + // increment if nothing else happens
>> + ++it;
>> + }
>> +
>> + /* TODO: assuming that variadic nature cannot be stripped. If this
>> + * is not true, then have code here to do so. */
>> +
>> + if (type.has_return_type ())
>> + {
>> + // FIXME: Can we have type expansion in this position?
>> + // In that case, we need to handle AST::TypeNoBounds on top of just
>> + // AST::Types
>> + auto &return_type = type.get_return_type ();
>> + return_type->accept_vis (*this);
>> + if (return_type->is_marked_for_strip ())
>> + rust_error_at (return_type->get_locus (),
>> + "cannot strip type in this position");
>> + }
>> +
>> + // no where clause, apparently
>> +}
>> +void
>> +AttrVisitor::maybe_expand_expr (std::unique_ptr<AST::Expr> &expr)
>> +{
>> + auto final_fragment = expand_macro_fragment_recursive ();
>> + if (final_fragment.should_expand ())
>> + expr = final_fragment.take_expression_fragment ();
>> +}
>> +
>> +void
>> +AttrVisitor::maybe_expand_type (std::unique_ptr<AST::Type> &type)
>> +{
>> + auto final_fragment = expand_macro_fragment_recursive ();
>> + if (final_fragment.should_expand ())
>> + type = final_fragment.take_type_fragment ();
>> +}
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-attribute-visitor.h b/gcc/rust/expand/rust-attribute-visitor.h
>> new file mode 100644
>> index 00000000000..0f9d1065334
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-attribute-visitor.h
>> @@ -0,0 +1,316 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#include "rust-ast-visitor.h"
>> +#include "rust-ast.h"
>> +#include "rust-macro-expand.h"
>> +
>> +namespace Rust {
>> +// Visitor used to expand attributes.
>> +class AttrVisitor : public AST::ASTVisitor
>> +{
>> +private:
>> + MacroExpander &expander;
>> + void maybe_expand_expr (std::unique_ptr<AST::Expr> &expr);
>> + void maybe_expand_type (std::unique_ptr<AST::Type> &expr);
>> +
>> +public:
>> + AttrVisitor (MacroExpander &expander) : expander (expander) {}
>> +
>> + void expand_struct_fields (std::vector<AST::StructField> &fields);
>> + void expand_tuple_fields (std::vector<AST::TupleField> &fields);
>> + void expand_function_params (std::vector<AST::FunctionParam> ¶ms);
>> + void expand_generic_args (AST::GenericArgs &args);
>> + void expand_qualified_path_type (AST::QualifiedPathType &path_type);
>> + void expand_closure_params (std::vector<AST::ClosureParam> ¶ms);
>> + void expand_self_param (AST::SelfParam &self_param);
>> + void expand_where_clause (AST::WhereClause &where_clause);
>> + void expand_trait_function_decl (AST::TraitFunctionDecl &decl);
>> + void expand_trait_method_decl (AST::TraitMethodDecl &decl);
>> +
>> + /**
>> + * Expand The current macro fragment recursively until it could not be
>> + * expanded further.
>> + *
>> + * The return value checking works because correctly
>> + * expanded fragment can never be an error (if the fragment can not be
>> + * expanded, a stand-in error fragment will be returned; for fragments that
>> + * could not be further expanded: the fragment prior to the expansion failure
>> + * will be returned).
>> + *
>> + * @return Either the expanded fragment or an empty errored-out fragment
>> + * indicating an expansion failure.
>> + */
>> + AST::ASTFragment expand_macro_fragment_recursive ()
>
> Why is this in the header?
Do you mean the definition or the documentation? If you refer to the
definition, then that is a mistake and should be in the implementation.
>> + {
>> + auto fragment = expander.take_expanded_fragment (*this);
>> + unsigned int original_depth = expander.expansion_depth;
>> + auto final_fragment = AST::ASTFragment ({}, true);
>> +
>> + while (fragment.should_expand ())
>> + {
>> + final_fragment = std::move (fragment);
>> + expander.expansion_depth++;
>> + // further expand the previously expanded macro fragment
>> + auto new_fragment = expander.take_expanded_fragment (*this);
>> + if (new_fragment.is_error ())
>> + break;
>> + fragment = std::move (new_fragment);
>> + }
>> + expander.expansion_depth = original_depth;
>> + return final_fragment;
>> + }
>> +
>> + /**
>> + * Expand a set of values, erasing them if they are marked for strip, and
>> + * replacing them with expanded macro nodes if necessary.
>> + * This function is slightly different from `expand_pointer_allow_strip` as
>> + * it can only be called in certain expansion contexts - where macro
>> + * invocations are allowed.
>> + *
>> + * @param ctx Context to use for macro expansion
>> + * @param values Iterable reference over values to replace or erase
>> + * @param extractor Function to call when replacing values with the content
>> + * of an expanded AST node
>> + */
>> + template <typename T, typename U>
>> + void expand_macro_children (MacroExpander::ContextType ctx, T &values,
>> + std::function<U (AST::SingleASTNode)> extractor)
>> + {
>> + expander.push_context (ctx);
>> +
>> + for (auto it = values.begin (); it != values.end ();)
>> + {
>> + auto &value = *it;
>> +
>> + // mark for stripping if required
>> + value->accept_vis (*this);
>> +
>> + // recursively expand the children
>> + auto final_fragment = expand_macro_fragment_recursive ();
>> +
>> + if (final_fragment.should_expand ())
>> + {
>> + it = values.erase (it);
>> + for (auto &node : final_fragment.get_nodes ())
>> + {
>> + auto new_node = extractor (node);
>> + if (new_node != nullptr && !new_node->is_marked_for_strip ())
>> + {
>> + it = values.insert (it, std::move (new_node));
>> + it++;
>> + }
>> + }
>> + }
>> + else if (value->is_marked_for_strip ())
>> + {
>> + it = values.erase (it);
>> + }
>> + else
>> + {
>> + ++it;
>> + }
>> + }
>> +
>> + expander.pop_context ();
>> + }
>> +
>> + template <typename T> void expand_pointer_allow_strip (T &values)
>> + {
>> + for (auto it = values.begin (); it != values.end ();)
>> + {
>> + auto &value = *it;
>> +
>> + // mark for stripping if required
>> + value->accept_vis (*this);
>> + if (value->is_marked_for_strip ())
>> + {
>> + it = values.erase (it);
>> + }
>> + else
>> + {
>> + ++it;
>> + }
>> + }
>> + }
>> +
>> + void visit (AST::Token &) override;
>> + void visit (AST::DelimTokenTree &) override;
>> + void visit (AST::AttrInputMetaItemContainer &) override;
>> + void visit (AST::IdentifierExpr &ident_expr) override;
>> + void visit (AST::Lifetime &) override;
>> + void visit (AST::LifetimeParam &) override;
>> + void visit (AST::ConstGenericParam &) override;
>> +
>> + void visit (AST::MacroInvocation ¯o_invoc) override;
>> +
>> + void visit (AST::PathInExpression &path) override;
>> + void visit (AST::TypePathSegment &) override;
>> + void visit (AST::TypePathSegmentGeneric &segment) override;
>> + void visit (AST::TypePathSegmentFunction &segment) override;
>> + void visit (AST::TypePath &path) override;
>> + void visit (AST::QualifiedPathInExpression &path) override;
>> + void visit (AST::QualifiedPathInType &path) override;
>> +
>> + void visit (AST::LiteralExpr &expr) override;
>> + void visit (AST::AttrInputLiteral &) override;
>> + void visit (AST::MetaItemLitExpr &) override;
>> + void visit (AST::MetaItemPathLit &) override;
>> + void visit (AST::BorrowExpr &expr) override;
>> + void visit (AST::DereferenceExpr &expr) override;
>> + void visit (AST::ErrorPropagationExpr &expr) override;
>> + void visit (AST::NegationExpr &expr) override;
>> + void visit (AST::ArithmeticOrLogicalExpr &expr) override;
>> + void visit (AST::ComparisonExpr &expr) override;
>> + void visit (AST::LazyBooleanExpr &expr) override;
>> + void visit (AST::TypeCastExpr &expr) override;
>> + void visit (AST::AssignmentExpr &expr) override;
>> + void visit (AST::CompoundAssignmentExpr &expr) override;
>> + void visit (AST::GroupedExpr &expr) override;
>> + void visit (AST::ArrayElemsValues &elems) override;
>> + void visit (AST::ArrayElemsCopied &elems) override;
>> + void visit (AST::ArrayExpr &expr) override;
>> + void visit (AST::ArrayIndexExpr &expr) override;
>> + void visit (AST::TupleExpr &expr) override;
>> + void visit (AST::TupleIndexExpr &expr) override;
>> + void visit (AST::StructExprStruct &expr) override;
>> + void visit (AST::StructExprFieldIdentifier &) override;
>> + void visit (AST::StructExprFieldIdentifierValue &field) override;
>> +
>> + void visit (AST::StructExprFieldIndexValue &field) override;
>> + void visit (AST::StructExprStructFields &expr) override;
>> + void visit (AST::StructExprStructBase &expr) override;
>> + void visit (AST::CallExpr &expr) override;
>> + void visit (AST::MethodCallExpr &expr) override;
>> + void visit (AST::FieldAccessExpr &expr) override;
>> + void visit (AST::ClosureExprInner &expr) override;
>> +
>> + void visit (AST::BlockExpr &expr) override;
>> +
>> + void visit (AST::ClosureExprInnerTyped &expr) override;
>> + void visit (AST::ContinueExpr &expr) override;
>> + void visit (AST::BreakExpr &expr) override;
>> + void visit (AST::RangeFromToExpr &expr) override;
>> + void visit (AST::RangeFromExpr &expr) override;
>> + void visit (AST::RangeToExpr &expr) override;
>> + void visit (AST::RangeFullExpr &) override;
>> + void visit (AST::RangeFromToInclExpr &expr) override;
>> + void visit (AST::RangeToInclExpr &expr) override;
>> + void visit (AST::ReturnExpr &expr) override;
>> + void visit (AST::UnsafeBlockExpr &expr) override;
>> + void visit (AST::LoopExpr &expr) override;
>> + void visit (AST::WhileLoopExpr &expr) override;
>> + void visit (AST::WhileLetLoopExpr &expr) override;
>> + void visit (AST::ForLoopExpr &expr) override;
>> + void visit (AST::IfExpr &expr) override;
>> + void visit (AST::IfExprConseqElse &expr) override;
>> + void visit (AST::IfExprConseqIf &expr) override;
>> + void visit (AST::IfExprConseqIfLet &expr) override;
>> + void visit (AST::IfLetExpr &expr) override;
>> + void visit (AST::IfLetExprConseqElse &expr) override;
>> + void visit (AST::IfLetExprConseqIf &expr) override;
>> + void visit (AST::IfLetExprConseqIfLet &expr) override;
>> + void visit (AST::MatchExpr &expr) override;
>> + void visit (AST::AwaitExpr &expr) override;
>> + void visit (AST::AsyncBlockExpr &expr) override;
>> + void visit (AST::TypeParam ¶m) override;
>> + void visit (AST::LifetimeWhereClauseItem &) override;
>> + void visit (AST::TypeBoundWhereClauseItem &item) override;
>> + void visit (AST::Method &method) override;
>> + void visit (AST::Module &module) override;
>> + void visit (AST::ExternCrate &crate) override;
>> + void visit (AST::UseTreeGlob &) override;
>> + void visit (AST::UseTreeList &) override;
>> + void visit (AST::UseTreeRebind &) override;
>> + void visit (AST::UseDeclaration &use_decl) override;
>> + void visit (AST::Function &function) override;
>> + void visit (AST::TypeAlias &type_alias) override;
>> + void visit (AST::StructStruct &struct_item) override;
>> + void visit (AST::TupleStruct &tuple_struct) override;
>> + void visit (AST::EnumItem &item) override;
>> + void visit (AST::EnumItemTuple &item) override;
>> + void visit (AST::EnumItemStruct &item) override;
>> + void visit (AST::EnumItemDiscriminant &item) override;
>> + void visit (AST::Enum &enum_item) override;
>> + void visit (AST::Union &union_item) override;
>> + void visit (AST::ConstantItem &const_item) override;
>> + void visit (AST::StaticItem &static_item) override;
>> + void visit (AST::TraitItemFunc &item) override;
>> + void visit (AST::TraitItemMethod &item) override;
>> + void visit (AST::TraitItemConst &item) override;
>> + void visit (AST::TraitItemType &item) override;
>> + void visit (AST::Trait &trait) override;
>> + void visit (AST::InherentImpl &impl) override;
>> + void visit (AST::TraitImpl &impl) override;
>> + void visit (AST::ExternalStaticItem &item) override;
>> + void visit (AST::ExternalFunctionItem &item) override;
>> + void visit (AST::ExternBlock &block) override;
>> +
>> + // I don't think it would be possible to strip macros without expansion
>> + void visit (AST::MacroMatchFragment &) override;
>> + void visit (AST::MacroMatchRepetition &) override;
>> + void visit (AST::MacroMatcher &) override;
>> + void visit (AST::MacroRulesDefinition &rules_def) override;
>> + void visit (AST::MetaItemPath &) override;
>> + void visit (AST::MetaItemSeq &) override;
>> + void visit (AST::MetaWord &) override;
>> + void visit (AST::MetaNameValueStr &) override;
>> + void visit (AST::MetaListPaths &) override;
>> + void visit (AST::MetaListNameValueStr &) override;
>> + void visit (AST::LiteralPattern &) override;
>> + void visit (AST::IdentifierPattern &pattern) override;
>> + void visit (AST::WildcardPattern &) override;
>> + void visit (AST::RangePatternBoundLiteral &) override;
>> + void visit (AST::RangePatternBoundPath &bound) override;
>> + void visit (AST::RangePatternBoundQualPath &bound) override;
>> + void visit (AST::RangePattern &pattern) override;
>> + void visit (AST::ReferencePattern &pattern) override;
>> + void visit (AST::StructPatternFieldTuplePat &field) override;
>> + void visit (AST::StructPatternFieldIdentPat &field) override;
>> + void visit (AST::StructPatternFieldIdent &field) override;
>> + void visit (AST::StructPattern &pattern) override;
>> + void visit (AST::TupleStructItemsNoRange &tuple_items) override;
>> + void visit (AST::TupleStructItemsRange &tuple_items) override;
>> + void visit (AST::TupleStructPattern &pattern) override;
>> + void visit (AST::TuplePatternItemsMultiple &tuple_items) override;
>> + void visit (AST::TuplePatternItemsRanged &tuple_items) override;
>> + void visit (AST::TuplePattern &pattern) override;
>> + void visit (AST::GroupedPattern &pattern) override;
>> + void visit (AST::SlicePattern &pattern) override;
>> +
>> + void visit (AST::EmptyStmt &) override;
>> + void visit (AST::LetStmt &stmt) override;
>> + void visit (AST::ExprStmtWithoutBlock &stmt) override;
>> + void visit (AST::ExprStmtWithBlock &stmt) override;
>> +
>> + void visit (AST::TraitBound &bound) override;
>> + void visit (AST::ImplTraitType &type) override;
>> + void visit (AST::TraitObjectType &type) override;
>> + void visit (AST::ParenthesisedType &type) override;
>> + void visit (AST::ImplTraitTypeOneBound &type) override;
>> + void visit (AST::TraitObjectTypeOneBound &type) override;
>> + void visit (AST::TupleType &type) override;
>> + void visit (AST::NeverType &) override;
>> + void visit (AST::RawPointerType &type) override;
>> + void visit (AST::ReferenceType &type) override;
>> + void visit (AST::ArrayType &type) override;
>> + void visit (AST::SliceType &type) override;
>> + void visit (AST::InferredType &) override;
>> + void visit (AST::BareFunctionType &type) override;
>> +};
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-macro-builtins.cc b/gcc/rust/expand/rust-macro-builtins.cc
>> new file mode 100644
>> index 00000000000..5eace13d197
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-builtins.cc
>> @@ -0,0 +1,484 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#include "rust-macro-builtins.h"
>> +#include "rust-diagnostics.h"
>> +#include "rust-expr.h"
>> +#include "rust-session-manager.h"
>> +#include "rust-macro-invoc-lexer.h"
>> +#include "rust-lex.h"
>> +#include "rust-parse.h"
>> +
>> +namespace Rust {
>> +namespace {
>> +std::unique_ptr<AST::Expr>
>> +make_string (Location locus, std::string value)
>> +{
>> + return std::unique_ptr<AST::Expr> (
>> + new AST::LiteralExpr (value, AST::Literal::STRING,
>> + PrimitiveCoreType::CORETYPE_STR, {}, locus));
>> +}
>> +
>> +/* Match the end token of a macro given the start delimiter of the macro */
>> +
>> +static inline TokenId
>> +macro_end_token (AST::DelimTokenTree &invoc_token_tree,
>> + Parser<MacroInvocLexer> &parser)
>> +{
>> + auto last_token_id = TokenId::RIGHT_CURLY;
>> + switch (invoc_token_tree.get_delim_type ())
>> + {
>> + case AST::DelimType::PARENS:
>> + last_token_id = TokenId::RIGHT_PAREN;
>> + rust_assert (parser.skip_token (LEFT_PAREN));
>> + break;
>> +
>> + case AST::DelimType::CURLY:
>> + rust_assert (parser.skip_token (LEFT_CURLY));
>> + break;
>> +
>> + case AST::DelimType::SQUARE:
>> + last_token_id = TokenId::RIGHT_SQUARE;
>> + rust_assert (parser.skip_token (LEFT_SQUARE));
>> + break;
>> + }
>> +
>> + return last_token_id;
>> +}
>> +
>> +/* Parse a single string literal from the given delimited token tree,
>> + and return the LiteralExpr for it. Allow for an optional trailing comma,
>> + but otherwise enforce that these are the only tokens. */
>> +
>> +std::unique_ptr<AST::LiteralExpr>
>> +parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree,
>> + Location invoc_locus)
>> +{
>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>> + Parser<MacroInvocLexer> parser (lex);
>> +
>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>> +
>> + std::unique_ptr<AST::LiteralExpr> lit_expr = nullptr;
>> +
>> + if (parser.peek_current_token ()->get_id () == STRING_LITERAL)
>> + {
>> + lit_expr = parser.parse_literal_expr ();
>> + parser.maybe_skip_token (COMMA);
>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>> + {
>> + lit_expr = nullptr;
>> + rust_error_at (invoc_locus, "macro takes 1 argument");
>> + }
>> + }
>> + else if (parser.peek_current_token ()->get_id () == last_token_id)
>> + rust_error_at (invoc_locus, "macro takes 1 argument");
>> + else
>> + rust_error_at (invoc_locus, "argument must be a string literal");
>> +
>> + parser.skip_token (last_token_id);
>> +
>> + return lit_expr;
>> +}
>> +
>> +/* Treat PATH as a path relative to the source file currently being
>> + compiled, and return the absolute path for it. */
>> +
>> +std::string
>> +source_relative_path (std::string path, Location locus)
>> +{
>> + std::string compile_fname
>> + = Session::get_instance ().linemap->location_file (locus);
>> +
>> + auto dir_separator_pos = compile_fname.rfind (file_separator);
>> +
>> + /* If there is no file_separator in the path, use current dir ('.'). */
>> + std::string dirname;
>> + if (dir_separator_pos == std::string::npos)
>> + dirname = std::string (".") + file_separator;
>> + else
>> + dirname = compile_fname.substr (0, dir_separator_pos) + file_separator;
>> +
>> + return dirname + path;
>> +}
>> +
>> +/* Read the full contents of the file FILENAME and return them in a vector.
>> + FIXME: platform specific. */
>> +
>> +std::vector<uint8_t>
>> +load_file_bytes (const char *filename)
>> +{
>> + RAIIFile file_wrap (filename);
>> + if (file_wrap.get_raw () == nullptr)
>> + {
>> + rust_error_at (Location (), "cannot open filename %s: %m", filename);
>> + return std::vector<uint8_t> ();
>> + }
>> +
>> + FILE *f = file_wrap.get_raw ();
>> + fseek (f, 0L, SEEK_END);
>> + long fsize = ftell (f);
>> + fseek (f, 0L, SEEK_SET);
>> +
>> + std::vector<uint8_t> buf (fsize);
>> +
>> + if (fread (&buf[0], fsize, 1, f) != 1)
>> + {
>> + rust_error_at (Location (), "error reading file %s: %m", filename);
>> + return std::vector<uint8_t> ();
>> + }
>> +
>> + return buf;
>> +}
>> +} // namespace
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + rust_debug ("assert!() called");
>> +
>> + return AST::ASTFragment::create_error ();
>> +}
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto current_file
>> + = Session::get_instance ().linemap->location_file (invoc_locus);
>> + auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file));
>> +
>> + return AST::ASTFragment ({file_str});
>> +}
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto current_column
>> + = Session::get_instance ().linemap->location_to_column (invoc_locus);
>> +
>> + auto column_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>> + new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT,
>> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
>> +
>> + return AST::ASTFragment ({column_no});
>> +}
>> +
>> +/* Expand builtin macro include_bytes!("filename"), which includes the contents
>> + of the given file as reference to a byte array. Yields an expression of type
>> + &'static [u8; N]. */
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + /* Get target filename from the macro invocation, which is treated as a path
>> + relative to the include!-ing file (currently being compiled). */
>> + auto lit_expr
>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>> + if (lit_expr == nullptr)
>> + return AST::ASTFragment::create_error ();
>> +
>> + std::string target_filename
>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>> +
>> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
>> +
>> + /* Is there a more efficient way to do this? */
>
> Well, first of all you don't need to read all of the file at once, second you
> could pre-allocate 'elts' to the correct size and final you could obviously
> choose a cheaper representation ;)
>
>> + std::vector<std::unique_ptr<AST::Expr>> elts;
>> + for (uint8_t b : bytes)
>> + {
>> + elts.emplace_back (
>> + new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE,
>> + PrimitiveCoreType::CORETYPE_U8,
>> + {} /* outer_attrs */, invoc_locus));
>> + }
>> +
>> + auto elems = std::unique_ptr<AST::ArrayElems> (
>> + new AST::ArrayElemsValues (std::move (elts), invoc_locus));
>> +
>> + auto array = std::unique_ptr<AST::Expr> (
>> + new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus));
>> +
>> + auto borrow = std::unique_ptr<AST::Expr> (
>> + new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus));
>> +
>> + auto node = AST::SingleASTNode (std::move (borrow));
>> + return AST::ASTFragment ({node});
>> +}
>> +
>> +/* Expand builtin macro include_str!("filename"), which includes the contents
>> + of the given file as a string. The file must be UTF-8 encoded. Yields an
>> + expression of type &'static str. */
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + /* Get target filename from the macro invocation, which is treated as a path
>> + relative to the include!-ing file (currently being compiled). */
>> + auto lit_expr
>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>> + if (lit_expr == nullptr)
>> + return AST::ASTFragment::create_error ();
>> +
>> + std::string target_filename
>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>> +
>> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
>> +
>> + /* FIXME: Enforce that the file contents are valid UTF-8. */
>> + std::string str ((const char *) &bytes[0], bytes.size ());
>> +
>> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
>> + return AST::ASTFragment ({node});
>> +}
>> +
>> +/* Expand builtin macro compile_error!("error"), which forces a compile error
>> + during the compile time. */
>> +AST::ASTFragment
>> +MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto lit_expr
>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>> + if (lit_expr == nullptr)
>> + return AST::ASTFragment::create_error ();
>> +
>> + std::string error_string = lit_expr->as_string ();
>> + rust_error_at (invoc_locus, "%s", error_string.c_str ());
>> +
>> + return AST::ASTFragment::create_error ();
>> +}
>> +
>> +/* Expand builtin macro concat!(), which joins all the literal parameters
>> + into a string with no delimiter. */
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>> + Parser<MacroInvocLexer> parser (lex);
>> +
>> + auto str = std::string ();
>> + bool has_error = false;
>> +
>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>> +
>> + /* NOTE: concat! could accept no argument, so we don't have any checks here */
>> + while (parser.peek_current_token ()->get_id () != last_token_id)
>> + {
>> + auto lit_expr = parser.parse_literal_expr ();
>> + if (lit_expr)
>> + {
>> + str += lit_expr->as_string ();
>> + }
>> + else
>> + {
>> + auto current_token = parser.peek_current_token ();
>> + rust_error_at (current_token->get_locus (),
>> + "argument must be a constant literal");
>> + has_error = true;
>> + // Just crash if the current token can't be skipped
>> + rust_assert (parser.skip_token (current_token->get_id ()));
>> + }
>> + parser.maybe_skip_token (COMMA);
>> + }
>> +
>> + parser.skip_token (last_token_id);
>> +
>> + if (has_error)
>> + return AST::ASTFragment::create_error ();
>> +
>> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
>> + return AST::ASTFragment ({node});
>> +}
>> +
>> +/* Expand builtin macro env!(), which inspects an environment variable at
>> + compile time. */
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>> + Parser<MacroInvocLexer> parser (lex);
>> +
>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>> +
>> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
>> + {
>> + if (parser.peek_current_token ()->get_id () == last_token_id)
>> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
>> + else
>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>> + "argument must be a string literal");
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + auto lit_expr = parser.parse_literal_expr ();
>> + auto comma_skipped = parser.maybe_skip_token (COMMA);
>> +
>> + std::unique_ptr<AST::LiteralExpr> error_expr = nullptr;
>> +
>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>> + {
>> + if (!comma_skipped)
>> + {
>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>> + "expected token: %<,%>");
>> + return AST::ASTFragment::create_error ();
>> + }
>> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
>> + {
>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>> + "argument must be a string literal");
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + error_expr = parser.parse_literal_expr ();
>> + parser.maybe_skip_token (COMMA);
>> + }
>> +
>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>> + {
>> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + parser.skip_token (last_token_id);
>> +
>> + auto env_value = getenv (lit_expr->as_string ().c_str ());
>> +
>> + if (env_value == nullptr)
>> + {
>> + if (error_expr == nullptr)
>> + rust_error_at (invoc_locus, "environment variable %qs not defined",
>> + lit_expr->as_string ().c_str ());
>> + else
>> + rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ());
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + auto node = AST::SingleASTNode (make_string (invoc_locus, env_value));
>> + return AST::ASTFragment ({node});
>> +}
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + // only parse if not already parsed
>> + if (!invoc.is_parsed ())
>> + {
>> + std::unique_ptr<AST::AttrInputMetaItemContainer> converted_input (
>> + invoc.get_delim_tok_tree ().parse_to_meta_item ());
>> +
>> + if (converted_input == nullptr)
>> + {
>> + rust_debug ("DEBUG: failed to parse macro to meta item");
>> + // TODO: do something now? is this an actual error?
>> + }
>> + else
>> + {
>> + std::vector<std::unique_ptr<AST::MetaItemInner>> meta_items (
>> + std::move (converted_input->get_items ()));
>> + invoc.set_meta_item_output (std::move (meta_items));
>> + }
>> + }
>> +
>> + /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg
>> + * attributes */
>> + if (invoc.get_meta_items ().size () != 1)
>> + return AST::ASTFragment::create_error ();
>> +
>> + bool result = invoc.get_meta_items ()[0]->check_cfg_predicate (
>> + Session::get_instance ());
>> + auto literal_exp = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>> + new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL,
>> + PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus)));
>> +
>> + return AST::ASTFragment ({literal_exp});
>> +}
>> +
>> +/* Expand builtin macro include!(), which includes a source file at the current
>> + scope compile time. */
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + /* Get target filename from the macro invocation, which is treated as a path
>> + relative to the include!-ing file (currently being compiled). */
>> + auto lit_expr
>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>> + if (lit_expr == nullptr)
>> + return AST::ASTFragment::create_error ();
>> +
>> + std::string filename
>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>> + auto target_filename
>> + = Rust::Session::get_instance ().include_extra_file (std::move (filename));
>> +
>> + RAIIFile target_file (target_filename);
>> + Linemap *linemap = Session::get_instance ().linemap;
>> +
>> + if (!target_file.ok ())
>> + {
>> + rust_error_at (lit_expr->get_locus (),
>> + "cannot open included file %qs: %m", target_filename);
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + rust_debug ("Attempting to parse included file %s", target_filename);
>> +
>> + Lexer lex (target_filename, std::move (target_file), linemap);
>> + Parser<Lexer> parser (lex);
>> +
>> + auto parsed_items = parser.parse_items ();
>> + bool has_error = !parser.get_errors ().empty ();
>> +
>> + for (const auto &error : parser.get_errors ())
>> + error.emit_error ();
>> +
>> + if (has_error)
>> + {
>> + // inform the user that the errors above are from a included file
>> + rust_inform (invoc_locus, "included from here");
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + std::vector<AST::SingleASTNode> nodes{};
>> + for (auto &item : parsed_items)
>> + {
>> + AST::SingleASTNode node (std::move (item));
>> + nodes.push_back (node);
>> + }
>> +
>> + return AST::ASTFragment (nodes);
>> +}
>> +
>> +AST::ASTFragment
>> +MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc)
>> +{
>> + auto current_line
>> + = Session::get_instance ().linemap->location_to_line (invoc_locus);
>> +
>> + auto line_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>> + new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT,
>> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
>> +
>> + return AST::ASTFragment ({line_no});
>> +}
>> +
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-macro-builtins.h b/gcc/rust/expand/rust-macro-builtins.h
>> new file mode 100644
>> index 00000000000..91f3727d450
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-builtins.h
>> @@ -0,0 +1,107 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#ifndef RUST_MACRO_BUILTINS_H
>> +#define RUST_MACRO_BUILTINS_H
>> +
>> +#include "rust-ast.h"
>> +#include "rust-location.h"
>> +
>> +/**
>> + * This class provides a list of builtin macros implemented by the compiler.
>> + * The functions defined are called "builtin transcribers" in that they replace
>> + * the transcribing part of a macro definition.
>> + *
>> + * Like regular macro transcribers, they are responsible for building and
>> + * returning an AST fragment: basically a vector of AST nodes put together.
>> + *
>> + * Unlike regular declarative macros where each match arm has its own associated
>> + * transcriber, builtin transcribers are responsible for handling all match arms
>> + * of the macro. This means that you should take extra care when implementing a
>> + * builtin containing multiple match arms: You will probably need to do some
>> + * lookahead in order to determine which match arm the user intended to use.
>> + *
>> + * An example of this is the `assert!()` macro:
>> + *
>> + * ```
>> + * macro_rules! assert {
>> + * ($cond:expr $(,)?) => {{ ... }};
>> + * ($cond : expr, $ ($arg : tt) +) = > {{ ... }};
>> + * }
>> + * ```
>> + *
>> + * If more tokens exist beyond the optional comma, they need to be handled as
>> + * a token-tree for a custom panic message.
>> + *
>> + * These builtin macros with empty transcribers are defined in the standard
>> + * library. They are marked with a special attribute, `#[rustc_builtin_macro]`.
>> + * When this attribute is present on a macro definition, the compiler should
>> + * look for an associated transcriber in the mappings. Meaning that you must
>> + * remember to insert your transcriber in the `builtin_macros` map of the
>> + *`Mappings`.
>> + *
>> + * This map is built as a static variable in the `insert_macro_def()` method
>> + * of the `Mappings` class.
>> + */
>> +
>> +/* If assert is defined as a macro this file will not parse, so undefine this
>> + before continuing. */
>> +#ifdef assert
>> +#undef assert
>> +#endif
>> +
>> +namespace Rust {
>> +class MacroBuiltin
>> +{
>> +public:
>> + static AST::ASTFragment assert (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment file (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment column (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment include_bytes (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment include_str (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment compile_error (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment concat (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment env (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment cfg (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment include (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +
>> + static AST::ASTFragment line (Location invoc_locus,
>> + AST::MacroInvocData &invoc);
>> +};
>> +} // namespace Rust
>> +
>> +#endif // RUST_MACRO_BUILTINS_H
>> diff --git a/gcc/rust/expand/rust-macro-expand.cc b/gcc/rust/expand/rust-macro-expand.cc
>> new file mode 100644
>> index 00000000000..1d57e394220
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-expand.cc
>> @@ -0,0 +1,1012 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#include "rust-macro-expand.h"
>> +#include "rust-macro-substitute-ctx.h"
>> +#include "rust-ast-full.h"
>> +#include "rust-ast-visitor.h"
>> +#include "rust-diagnostics.h"
>> +#include "rust-parse.h"
>> +#include "rust-attribute-visitor.h"
>> +
>> +namespace Rust {
>> +AST::ASTFragment
>> +MacroExpander::expand_decl_macro (Location invoc_locus,
>> + AST::MacroInvocData &invoc,
>> + AST::MacroRulesDefinition &rules_def,
>> + bool semicolon)
>> +{
>> + // ensure that both invocation and rules are in a valid state
>> + rust_assert (!invoc.is_marked_for_strip ());
>> + rust_assert (!rules_def.is_marked_for_strip ());
>> + rust_assert (rules_def.get_macro_rules ().size () > 0);
>> +
>> + /* probably something here about parsing invoc and rules def token trees to
>> + * token stream. if not, how would parser handle the captures of exprs and
>> + * stuff? on the other hand, token trees may be kind of useful in rules def as
>> + * creating a point where recursion can occur (like having
>> + * "compare_macro_match" and then it calling itself when it finds delimiters)
>> + */
>> +
>> + /* find matching rule to invoc token tree, based on macro rule's matcher. if
>> + * none exist, error.
>> + * - specifically, check each matcher in order. if one fails to match, move
>> + * onto next. */
>> + /* TODO: does doing this require parsing expressions and whatever in the
>> + * invoc? if so, might as well save the results if referenced using $ or
>> + * whatever. If not, do another pass saving them. Except this is probably
>> + * useless as different rules could have different starting points for exprs
>> + * or whatever. Decision trees could avoid this, but they have their own
>> + * issues. */
>> + /* TODO: will need to modify the parser so that it can essentially "catch"
>> + * errors - maybe "try_parse_expr" or whatever methods. */
>> + // this technically creates a back-tracking parser - this will be the
>> + // implementation style
>> +
>> + /* then, after results are saved, generate the macro output from the
>> + * transcriber token tree. if i understand this correctly, the macro
>> + * invocation gets replaced by the transcriber tokens, except with
>> + * substitutions made (e.g. for $i variables) */
>> +
>> + /* TODO: it is probably better to modify AST::Token to store a pointer to a
>> + * Lexer::Token (rather than being converted) - i.e. not so much have
>> + * AST::Token as a Token but rather a TokenContainer (as it is another type of
>> + * TokenTree). This will prevent re-conversion of Tokens between each type
>> + * all the time, while still allowing the heterogenous storage of token trees.
>> + */
>> +
>> + AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree ();
>> +
>> + // find matching arm
>> + AST::MacroRule *matched_rule = nullptr;
>> + std::map<std::string, MatchedFragmentContainer> matched_fragments;
>> + for (auto &rule : rules_def.get_rules ())
>> + {
>> + sub_stack.push ();
>> + bool did_match_rule = try_match_rule (rule, invoc_token_tree);
>> + matched_fragments = sub_stack.pop ();
>> +
>> + if (did_match_rule)
>> + {
>> + // // Debugging
>> + // for (auto &kv : matched_fragments)
>> + // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (),
>> + // kv.second.get_fragments ().size (),
>> + // kv.second.get_kind ()
>> + // == MatchedFragmentContainer::Kind::Repetition
>> + // ? "repetition"
>> + // : "metavar");
>> +
>> + matched_rule = &rule;
>> + break;
>> + }
>> + }
>> +
>> + if (matched_rule == nullptr)
>> + {
>> + RichLocation r (invoc_locus);
>> + r.add_range (rules_def.get_locus ());
>> + rust_error_at (r, "Failed to match any rule within macro");
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments,
>> + semicolon, peek_context ());
>> +}
>> +
>> +void
>> +MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon)
>> +{
>> + if (depth_exceeds_recursion_limit ())
>> + {
>> + rust_error_at (invoc.get_locus (), "reached recursion limit");
>> + return;
>> + }
>> +
>> + AST::MacroInvocData &invoc_data = invoc.get_invoc_data ();
>> +
>> + // ??
>> + // switch on type of macro:
>> + // - '!' syntax macro (inner switch)
>> + // - procedural macro - "A token-based function-like macro"
>> + // - 'macro_rules' (by example/pattern-match) macro? or not? "an
>> + // AST-based function-like macro"
>> + // - else is unreachable
>> + // - attribute syntax macro (inner switch)
>> + // - procedural macro attribute syntax - "A token-based attribute
>> + // macro"
>> + // - legacy macro attribute syntax? - "an AST-based attribute macro"
>> + // - non-macro attribute: mark known
>> + // - else is unreachable
>> + // - derive macro (inner switch)
>> + // - derive or legacy derive - "token-based" vs "AST-based"
>> + // - else is unreachable
>> + // - derive container macro - unreachable
>> +
>> + // lookup the rules for this macro
>> + NodeId resolved_node = UNKNOWN_NODEID;
>> + NodeId source_node = UNKNOWN_NODEID;
>> + if (has_semicolon)
>> + source_node = invoc.get_macro_node_id ();
>> + else
>> + source_node = invoc.get_pattern_node_id ();
>> + auto seg
>> + = Resolver::CanonicalPath::new_seg (source_node,
>> + invoc_data.get_path ().as_string ());
>> +
>> + bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node);
>> + if (!found)
>> + {
>> + rust_error_at (invoc.get_locus (), "unknown macro: [%s]",
>> + seg.get ().c_str ());
>> + return;
>> + }
>> +
>> + // lookup the rules
>> + AST::MacroRulesDefinition *rules_def = nullptr;
>> + bool ok = mappings->lookup_macro_def (resolved_node, &rules_def);
>> + rust_assert (ok);
>> +
>> + auto fragment = AST::ASTFragment::create_error ();
>> +
>> + if (rules_def->is_builtin ())
>> + fragment
>> + = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data);
>> + else
>> + fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def,
>> + has_semicolon);
>> +
>> + set_expanded_fragment (std::move (fragment));
>> +}
>> +
>> +/* Determines whether any cfg predicate is false and hence item with attributes
>> + * should be stripped. Note that attributes must be expanded before calling. */
>> +bool
>> +MacroExpander::fails_cfg (const AST::AttrVec &attrs) const
>> +{
>> + for (const auto &attr : attrs)
>> + {
>> + if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session))
>> + return true;
>> + }
>> + return false;
>> +}
>> +
>> +/* Determines whether any cfg predicate is false and hence item with attributes
>> + * should be stripped. Will expand attributes as well. */
>> +bool
>> +MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const
>> +{
>> + // TODO: maybe have something that strips cfg attributes that evaluate true?
>> + for (auto &attr : attrs)
>> + {
>> + if (attr.get_path () == "cfg")
>> + {
>> + if (!attr.is_parsed_to_meta_item ())
>> + attr.parse_attr_to_meta_item ();
>> +
>> + // DEBUG
>> + if (!attr.is_parsed_to_meta_item ())
>> + rust_debug ("failed to parse attr to meta item, right before "
>> + "cfg predicate check");
>> + else
>> + rust_debug ("attr has been successfully parsed to meta item, "
>> + "right before cfg predicate check");
>> +
>> + if (!attr.check_cfg_predicate (session))
>> + {
>> + // DEBUG
>> + rust_debug (
>> + "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m",
>> + attr.as_string ().c_str ());
>> +
>> + return true;
>> + }
>> + else
>> + {
>> + // DEBUG
>> + rust_debug ("cfg predicate succeeded for attribute: "
>> + "\033[0;31m'%s'\033[0m",
>> + attr.as_string ().c_str ());
>> + }
>> + }
>> + }
>> + return false;
>> +}
>> +
>> +// Expands cfg_attr attributes.
>> +void
>> +MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs)
>> +{
>> + for (std::size_t i = 0; i < attrs.size (); i++)
>> + {
>> + auto &attr = attrs[i];
>> + if (attr.get_path () == "cfg_attr")
>> + {
>> + if (!attr.is_parsed_to_meta_item ())
>> + attr.parse_attr_to_meta_item ();
>> +
>> + if (attr.check_cfg_predicate (session))
>> + {
>> + // split off cfg_attr
>> + AST::AttrVec new_attrs = attr.separate_cfg_attrs ();
>> +
>> + // remove attr from vector
>> + attrs.erase (attrs.begin () + i);
>> +
>> + // add new attrs to vector
>> + attrs.insert (attrs.begin () + i,
>> + std::make_move_iterator (new_attrs.begin ()),
>> + std::make_move_iterator (new_attrs.end ()));
>> + }
>> +
>> + /* do something - if feature (first token in tree) is in fact enabled,
>> + * make tokens listed afterwards into attributes. i.e.: for
>> + * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add
>> + * attributes [wow1] and [wow2] to attribute list. This can also be
>> + * recursive, so check for expanded attributes being recursive and
>> + * possibly recursively call the expand_attrs? */
>> + }
>> + else
>> + {
>> + i++;
>> + }
>> + }
>> + attrs.shrink_to_fit ();
>> +}
>> +
>> +void
>> +MacroExpander::expand_crate ()
>> +{
>> + NodeId scope_node_id = crate.get_node_id ();
>> + resolver->get_macro_scope ().push (scope_node_id);
>> +
>> + /* fill macro/decorator map from init list? not sure where init list comes
>> + * from? */
>> +
>> + // TODO: does cfg apply for inner attributes? research.
>> + // the apparent answer (from playground test) is yes
>> +
>> + // expand crate cfg_attr attributes
>> + expand_cfg_attrs (crate.inner_attrs);
>> +
>> + if (fails_cfg_with_expand (crate.inner_attrs))
>> + {
>> + // basically, delete whole crate
>> + crate.strip_crate ();
>> + // TODO: maybe create warning here? probably not desired behaviour
>> + }
>> + // expand module attributes?
>> +
>> + push_context (ITEM);
>> +
>> + // expand attributes recursively and strip items if required
>> + AttrVisitor attr_visitor (*this);
>> + auto &items = crate.items;
>> + for (auto it = items.begin (); it != items.end ();)
>> + {
>> + auto &item = *it;
>> +
>> + // mark for stripping if required
>> + item->accept_vis (attr_visitor);
>> +
>> + auto fragment = take_expanded_fragment (attr_visitor);
>> + if (fragment.should_expand ())
>> + {
>> + // Remove the current expanded invocation
>> + it = items.erase (it);
>> + for (auto &node : fragment.get_nodes ())
>> + {
>> + it = items.insert (it, node.take_item ());
>> + it++;
>> + }
>> + }
>> + else if (item->is_marked_for_strip ())
>> + it = items.erase (it);
>> + else
>> + it++;
>> + }
>> +
>> + pop_context ();
>> +
>> + // TODO: should recursive attribute and macro expansion be done in the same
>> + // transversal? Or in separate ones like currently?
>> +
>> + // expand module tree recursively
>> +
>> + // post-process
>> +
>> + // extract exported macros?
>> +}
>> +
>> +bool
>> +MacroExpander::depth_exceeds_recursion_limit () const
>> +{
>> + return expansion_depth >= cfg.recursion_limit;
>> +}
>> +
>> +bool
>> +MacroExpander::try_match_rule (AST::MacroRule &match_rule,
>> + AST::DelimTokenTree &invoc_token_tree)
>> +{
>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>> + Parser<MacroInvocLexer> parser (lex);
>> +
>> + AST::MacroMatcher &matcher = match_rule.get_matcher ();
>> +
>> + expansion_depth++;
>> + if (!match_matcher (parser, matcher))
>> + {
>> + expansion_depth--;
>> + return false;
>> + }
>> + expansion_depth--;
>> +
>> + bool used_all_input_tokens = parser.skip_token (END_OF_FILE);
>> + return used_all_input_tokens;
>> +}
>> +
>> +bool
>> +MacroExpander::match_fragment (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchFragment &fragment)
>> +{
>> + switch (fragment.get_frag_spec ().get_kind ())
>> + {
>> + case AST::MacroFragSpec::EXPR:
>> + parser.parse_expr ();
>> + break;
>> +
>> + case AST::MacroFragSpec::BLOCK:
>> + parser.parse_block_expr ();
>> + break;
>> +
>> + case AST::MacroFragSpec::IDENT:
>> + parser.parse_identifier_pattern ();
>> + break;
>> +
>> + case AST::MacroFragSpec::LITERAL:
>> + parser.parse_literal_expr ();
>> + break;
>> +
>> + case AST::MacroFragSpec::ITEM:
>> + parser.parse_item (false);
>> + break;
>> +
>> + case AST::MacroFragSpec::TY:
>> + parser.parse_type ();
>> + break;
>> +
>> + case AST::MacroFragSpec::PAT:
>> + parser.parse_pattern ();
>> + break;
>> +
>> + case AST::MacroFragSpec::PATH:
>> + parser.parse_path_in_expression ();
>> + break;
>> +
>> + case AST::MacroFragSpec::VIS:
>> + parser.parse_visibility ();
>> + break;
>> +
>> + case AST::MacroFragSpec::STMT: {
>> + auto restrictions = ParseRestrictions ();
>> + restrictions.consume_semi = false;
>> + parser.parse_stmt (restrictions);
>> + break;
>> + }
>> +
>> + case AST::MacroFragSpec::LIFETIME:
>> + parser.parse_lifetime_params ();
>> + break;
>> +
>> + // is meta attributes?
>> + case AST::MacroFragSpec::META:
>> + parser.parse_attribute_body ();
>> + break;
>> +
>> + case AST::MacroFragSpec::TT:
>> + parser.parse_token_tree ();
>> + break;
>> +
>> + // i guess we just ignore invalid and just error out
>> + case AST::MacroFragSpec::INVALID:
>> + return false;
>> + }
>> +
>> + // it matches if the parser did not produce errors trying to parse that type
>> + // of item
>> + return !parser.has_errors ();
>> +}
>> +
>> +bool
>> +MacroExpander::match_matcher (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatcher &matcher)
>> +{
>> + if (depth_exceeds_recursion_limit ())
>> + {
>> + rust_error_at (matcher.get_match_locus (), "reached recursion limit");
>> + return false;
>> + }
>> +
>> + auto delimiter = parser.peek_current_token ();
>> +
>> + // this is used so we can check that we delimit the stream correctly.
>> + switch (delimiter->get_id ())
>> + {
>> + case LEFT_PAREN: {
>> + if (!parser.skip_token (LEFT_PAREN))
>> + return false;
>> + }
>> + break;
>> +
>> + case LEFT_SQUARE: {
>> + if (!parser.skip_token (LEFT_SQUARE))
>> + return false;
>> + }
>> + break;
>> +
>> + case LEFT_CURLY: {
>> + if (!parser.skip_token (LEFT_CURLY))
>> + return false;
>> + }
>> + break;
>> + default:
>> + gcc_unreachable ();
>> + }
>> +
>> + const MacroInvocLexer &source = parser.get_token_source ();
>> +
>> + for (auto &match : matcher.get_matches ())
>> + {
>> + size_t offs_begin = source.get_offs ();
>> +
>> + switch (match->get_macro_match_type ())
>> + {
>> + case AST::MacroMatch::MacroMatchType::Fragment: {
>> + AST::MacroMatchFragment *fragment
>> + = static_cast<AST::MacroMatchFragment *> (match.get ());
>> + if (!match_fragment (parser, *fragment))
>> + return false;
>> +
>> + // matched fragment get the offset in the token stream
>> + size_t offs_end = source.get_offs ();
>> + sub_stack.insert_metavar (
>> + MatchedFragment (fragment->get_ident (), offs_begin, offs_end));
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Tok: {
>> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
>> + if (!match_token (parser, *tok))
>> + return false;
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Repetition: {
>> + AST::MacroMatchRepetition *rep
>> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
>> + if (!match_repetition (parser, *rep))
>> + return false;
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Matcher: {
>> + AST::MacroMatcher *m
>> + = static_cast<AST::MacroMatcher *> (match.get ());
>> + expansion_depth++;
>> + if (!match_matcher (parser, *m))
>> + {
>> + expansion_depth--;
>> + return false;
>> + }
>> + expansion_depth--;
>> + }
>> + break;
>> + }
>> + }
>> +
>> + switch (delimiter->get_id ())
>> + {
>> + case LEFT_PAREN: {
>> + if (!parser.skip_token (RIGHT_PAREN))
>> + return false;
>> + }
>> + break;
>> +
>> + case LEFT_SQUARE: {
>> + if (!parser.skip_token (RIGHT_SQUARE))
>> + return false;
>> + }
>> + break;
>> +
>> + case LEFT_CURLY: {
>> + if (!parser.skip_token (RIGHT_CURLY))
>> + return false;
>> + }
>> + break;
>> + default:
>> + gcc_unreachable ();
>> + }
>> +
>> + return true;
>> +}
>> +
>> +bool
>> +MacroExpander::match_token (Parser<MacroInvocLexer> &parser, AST::Token &token)
>> +{
>> + // FIXME this needs to actually match the content and the type
>> + return parser.skip_token (token.get_id ());
>> +}
>> +
>> +bool
>> +MacroExpander::match_n_matches (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchRepetition &rep,
>> + size_t &match_amount, size_t lo_bound,
>> + size_t hi_bound)
>> +{
>> + match_amount = 0;
>> + auto &matches = rep.get_matches ();
>> +
>> + const MacroInvocLexer &source = parser.get_token_source ();
>> + while (true)
>> + {
>> + // If the current token is a closing macro delimiter, break away.
>> + // TODO: Is this correct?
>> + auto t_id = parser.peek_current_token ()->get_id ();
>> + if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY)
>> + break;
>> +
>> + // Skip parsing a separator on the first match, otherwise consume it.
>> + // If it isn't present, this is an error
>> + if (rep.has_sep () && match_amount > 0)
>> + if (!match_token (parser, *rep.get_sep ()))
>> + break;
>> +
>> + bool valid_current_match = false;
>> + for (auto &match : matches)
>> + {
>> + size_t offs_begin = source.get_offs ();
>> + switch (match->get_macro_match_type ())
>> + {
>> + case AST::MacroMatch::MacroMatchType::Fragment: {
>> + AST::MacroMatchFragment *fragment
>> + = static_cast<AST::MacroMatchFragment *> (match.get ());
>> + valid_current_match = match_fragment (parser, *fragment);
>> +
>> + // matched fragment get the offset in the token stream
>> + size_t offs_end = source.get_offs ();
>> +
>> + // The main difference with match_matcher happens here: Instead
>> + // of inserting a new fragment, we append to one. If that
>> + // fragment does not exist, then the operation is similar to
>> + // `insert_fragment` with the difference that we are not
>> + // creating a metavariable, but a repetition of one, which is
>> + // really different.
>> + sub_stack.append_fragment (
>> + MatchedFragment (fragment->get_ident (), offs_begin,
>> + offs_end));
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Tok: {
>> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
>> + valid_current_match = match_token (parser, *tok);
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Repetition: {
>> + AST::MacroMatchRepetition *rep
>> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
>> + valid_current_match = match_repetition (parser, *rep);
>> + }
>> + break;
>> +
>> + case AST::MacroMatch::MacroMatchType::Matcher: {
>> + AST::MacroMatcher *m
>> + = static_cast<AST::MacroMatcher *> (match.get ());
>> + valid_current_match = match_matcher (parser, *m);
>> + }
>> + break;
>> + }
>> + }
>> + // If we've encountered an error once, stop trying to match more
>> + // repetitions
>> + if (!valid_current_match)
>> + break;
>> +
>> + match_amount++;
>> +
>> + // Break early if we notice there's too many expressions already
>> + if (hi_bound && match_amount > hi_bound)
>> + break;
>> + }
>> +
>> + // Check if the amount of matches we got is valid: Is it more than the lower
>> + // bound and less than the higher bound?
>> + bool did_meet_lo_bound = match_amount >= lo_bound;
>> + bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true;
>> +
>> + // If the end-result is valid, then we can clear the parse errors: Since
>> + // repetitions are parsed eagerly, it is okay to fail in some cases
>> + auto res = did_meet_lo_bound && did_meet_hi_bound;
>> + if (res)
>> + parser.clear_errors ();
>> +
>> + return res;
>> +}
>> +
>> +bool
>> +MacroExpander::match_repetition (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchRepetition &rep)
>> +{
>> + size_t match_amount = 0;
>> + bool res = false;
>> +
>> + std::string lo_str;
>> + std::string hi_str;
>> + switch (rep.get_op ())
>> + {
>> + case AST::MacroMatchRepetition::MacroRepOp::ANY:
>> + lo_str = "0";
>> + hi_str = "+inf";
>> + res = match_n_matches (parser, rep, match_amount);
>> + break;
>> + case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE:
>> + lo_str = "1";
>> + hi_str = "+inf";
>> + res = match_n_matches (parser, rep, match_amount, 1);
>> + break;
>> + case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE:
>> + lo_str = "0";
>> + hi_str = "1";
>> + res = match_n_matches (parser, rep, match_amount, 0, 1);
>> + break;
>> + default:
>> + gcc_unreachable ();
>> + }
>> +
>> + if (!res)
>> + rust_error_at (rep.get_match_locus (),
>> + "invalid amount of matches for macro invocation. Expected "
>> + "between %s and %s, got %lu",
>> + lo_str.c_str (), hi_str.c_str (),
>> + (unsigned long) match_amount);
>> +
>> + rust_debug_loc (rep.get_match_locus (), "%s matched %lu times",
>> + res ? "successfully" : "unsuccessfully",
>> + (unsigned long) match_amount);
>> +
>> + // We have to handle zero fragments differently: They will not have been
>> + // "matched" but they are still valid and should be inserted as a special
>> + // case. So we go through the stack map, and for every fragment which doesn't
>> + // exist, insert a zero-matched fragment.
>> + auto &stack_map = sub_stack.peek ();
>> + for (auto &match : rep.get_matches ())
>> + {
>> + if (match->get_macro_match_type ()
>> + == AST::MacroMatch::MacroMatchType::Fragment)
>> + {
>> + auto fragment = static_cast<AST::MacroMatchFragment *> (match.get ());
>> + auto it = stack_map.find (fragment->get_ident ());
>> +
>> + if (it == stack_map.end ())
>> + sub_stack.insert_matches (fragment->get_ident (),
>> + MatchedFragmentContainer::zero ());
>> + }
>> + }
>> +
>> + return res;
>> +}
>> +
>> +/**
>> + * Helper function to refactor calling a parsing function 0 or more times
>> + */
>> +static AST::ASTFragment
>> +parse_many (Parser<MacroInvocLexer> &parser, TokenId &delimiter,
>> + std::function<AST::SingleASTNode ()> parse_fn)
>> +{
>> + std::vector<AST::SingleASTNode> nodes;
>> + while (true)
>> + {
>> + if (parser.peek_current_token ()->get_id () == delimiter)
>> + break;
>> +
>> + auto node = parse_fn ();
>> + nodes.emplace_back (std::move (node));
>> + }
>> +
>> + return AST::ASTFragment (std::move (nodes));
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more items from a macro invocation
>> + *
>> + * @param parser Parser to extract items from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>> +{
>> + return parse_many (parser, delimiter, [&parser] () {
>> + auto item = parser.parse_item (true);
>> + return AST::SingleASTNode (std::move (item));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more external items from a macro invocation
>> + *
>> + * @param parser Parser to extract items from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_ext (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>> +{
>> + return parse_many (parser, delimiter, [&parser] () {
>> + auto item = parser.parse_external_item ();
>> + return AST::SingleASTNode (std::move (item));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more trait items from a macro invocation
>> + *
>> + * @param parser Parser to extract items from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_trait_items (Parser<MacroInvocLexer> &parser,
>> + TokenId &delimiter)
>> +{
>> + return parse_many (parser, delimiter, [&parser] () {
>> + auto item = parser.parse_trait_item ();
>> + return AST::SingleASTNode (std::move (item));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more impl items from a macro invocation
>> + *
>> + * @param parser Parser to extract items from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_impl_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>> +{
>> + return parse_many (parser, delimiter, [&parser] () {
>> + auto item = parser.parse_inherent_impl_item ();
>> + return AST::SingleASTNode (std::move (item));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more trait impl items from a macro invocation
>> + *
>> + * @param parser Parser to extract items from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_trait_impl_items (Parser<MacroInvocLexer> &parser,
>> + TokenId &delimiter)
>> +{
>> + return parse_many (parser, delimiter, [&parser] () {
>> + auto item = parser.parse_trait_impl_item ();
>> + return AST::SingleASTNode (std::move (item));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe 0 or more statements from a macro invocation
>> + *
>> + * @param parser Parser to extract statements from
>> + * @param delimiter Id of the token on which parsing should stop
>> + */
>> +static AST::ASTFragment
>> +transcribe_many_stmts (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>> +{
>> + auto restrictions = ParseRestrictions ();
>> + restrictions.consume_semi = false;
>> +
>> + // FIXME: This is invalid! It needs to also handle cases where the macro
>> + // transcriber is an expression, but since the macro call is followed by
>> + // a semicolon, it's a valid ExprStmt
>> + return parse_many (parser, delimiter, [&parser, restrictions] () {
>> + auto stmt = parser.parse_stmt (restrictions);
>> + return AST::SingleASTNode (std::move (stmt));
>> + });
>> +}
>> +
>> +/**
>> + * Transcribe one expression from a macro invocation
>> + *
>> + * @param parser Parser to extract statements from
>> + */
>> +static AST::ASTFragment
>> +transcribe_expression (Parser<MacroInvocLexer> &parser)
>> +{
>> + auto expr = parser.parse_expr ();
>> +
>> + return AST::ASTFragment ({std::move (expr)});
>> +}
>> +
>> +/**
>> + * Transcribe one type from a macro invocation
>> + *
>> + * @param parser Parser to extract statements from
>> + */
>> +static AST::ASTFragment
>> +transcribe_type (Parser<MacroInvocLexer> &parser)
>> +{
>> + auto type = parser.parse_type ();
>> +
>> + return AST::ASTFragment ({std::move (type)});
>> +}
>> +
>> +static AST::ASTFragment
>> +transcribe_on_delimiter (Parser<MacroInvocLexer> &parser, bool semicolon,
>> + AST::DelimType delimiter, TokenId last_token_id)
>> +{
>> + if (semicolon || delimiter == AST::DelimType::CURLY)
>> + return transcribe_many_stmts (parser, last_token_id);
>> + else
>> + return transcribe_expression (parser);
>> +} // namespace Rust
>> +
>> +static AST::ASTFragment
>> +transcribe_context (MacroExpander::ContextType ctx,
>> + Parser<MacroInvocLexer> &parser, bool semicolon,
>> + AST::DelimType delimiter, TokenId last_token_id)
>> +{
>> + // The flow-chart in order to choose a parsing function is as follows:
>> + //
>> + // [switch special context]
>> + // -- Item --> parser.parse_item();
>> + // -- Trait --> parser.parse_trait_item();
>> + // -- Impl --> parser.parse_impl_item();
>> + // -- Extern --> parser.parse_extern_item();
>> + // -- None --> [has semicolon?]
>> + // -- Yes --> parser.parse_stmt();
>> + // -- No --> [switch invocation.delimiter()]
>> + // -- { } --> parser.parse_stmt();
>> + // -- _ --> parser.parse_expr(); // once!
>> +
>> + // If there is a semicolon OR we are expanding a MacroInvocationSemi, then
>> + // we can parse multiple items. Otherwise, parse *one* expression
>> +
>> + switch (ctx)
>> + {
>> + case MacroExpander::ContextType::ITEM:
>> + return transcribe_many_items (parser, last_token_id);
>> + break;
>> + case MacroExpander::ContextType::TRAIT:
>> + return transcribe_many_trait_items (parser, last_token_id);
>> + break;
>> + case MacroExpander::ContextType::IMPL:
>> + return transcribe_many_impl_items (parser, last_token_id);
>> + break;
>> + case MacroExpander::ContextType::TRAIT_IMPL:
>> + return transcribe_many_trait_impl_items (parser, last_token_id);
>> + break;
>> + case MacroExpander::ContextType::EXTERN:
>> + return transcribe_many_ext (parser, last_token_id);
>> + break;
>> + case MacroExpander::ContextType::TYPE:
>> + return transcribe_type (parser);
>> + break;
>> + default:
>> + return transcribe_on_delimiter (parser, semicolon, delimiter,
>> + last_token_id);
>> + }
>> +}
>> +
>> +static std::string
>> +tokens_to_str (std::vector<std::unique_ptr<AST::Token>> &tokens)
>> +{
>> + std::string str;
>> + if (!tokens.empty ())
>> + {
>> + str += tokens[0]->as_string ();
>> + for (size_t i = 1; i < tokens.size (); i++)
>> + str += " " + tokens[i]->as_string ();
>> + }
>> +
>> + return str;
>> +}
>> +
>> +AST::ASTFragment
>> +MacroExpander::transcribe_rule (
>> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
>> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
>> + bool semicolon, ContextType ctx)
>> +{
>> + // we can manipulate the token tree to substitute the dollar identifiers so
>> + // that when we call parse its already substituted for us
>> + AST::MacroTranscriber &transcriber = match_rule.get_transcriber ();
>> + AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree ();
>> +
>> + auto invoc_stream = invoc_token_tree.to_token_stream ();
>> + auto macro_rule_tokens = transcribe_tree.to_token_stream ();
>> +
>> + auto substitute_context
>> + = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments);
>> + std::vector<std::unique_ptr<AST::Token>> substituted_tokens
>> + = substitute_context.substitute_tokens ();
>> +
>> + rust_debug ("substituted tokens: %s",
>> + tokens_to_str (substituted_tokens).c_str ());
>> +
>> + // parse it to an ASTFragment
>> + MacroInvocLexer lex (std::move (substituted_tokens));
>> + Parser<MacroInvocLexer> parser (lex);
>> +
>> + auto last_token_id = TokenId::RIGHT_CURLY;
>> +
>> + // this is used so we can check that we delimit the stream correctly.
>> + switch (transcribe_tree.get_delim_type ())
>> + {
>> + case AST::DelimType::PARENS:
>> + last_token_id = TokenId::RIGHT_PAREN;
>> + rust_assert (parser.skip_token (LEFT_PAREN));
>> + break;
>> +
>> + case AST::DelimType::CURLY:
>> + rust_assert (parser.skip_token (LEFT_CURLY));
>> + break;
>> +
>> + case AST::DelimType::SQUARE:
>> + last_token_id = TokenId::RIGHT_SQUARE;
>> + rust_assert (parser.skip_token (LEFT_SQUARE));
>> + break;
>> + }
>> +
>> + // see https://github.com/Rust-GCC/gccrs/issues/22
>> + // TL;DR:
>> + // - Treat all macro invocations with parentheses, (), or square brackets,
>> + // [], as expressions.
>> + // - If the macro invocation has curly brackets, {}, it may be parsed as a
>> + // statement depending on the context.
>> + // - If the macro invocation has a semicolon at the end, it must be parsed
>> + // as a statement (either via ExpressionStatement or
>> + // MacroInvocationWithSemi)
>> +
>> + auto fragment
>> + = transcribe_context (ctx, parser, semicolon,
>> + invoc_token_tree.get_delim_type (), last_token_id);
>> +
>> + // emit any errors
>> + if (parser.has_errors ())
>> + {
>> + for (auto &err : parser.get_errors ())
>> + rust_error_at (err.locus, "%s", err.message.c_str ());
>> + return AST::ASTFragment::create_error ();
>> + }
>> +
>> + // are all the tokens used?
>> + bool did_delimit = parser.skip_token (last_token_id);
>> +
>> + bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE);
>> + if (!reached_end_of_stream)
>> + {
>> + const_TokenPtr current_token = parser.peek_current_token ();
>> + rust_error_at (current_token->get_locus (),
>> + "tokens here and after are unparsed");
>> + }
>> +
>> + return fragment;
>> +}
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-macro-expand.h b/gcc/rust/expand/rust-macro-expand.h
>> new file mode 100644
>> index 00000000000..94d6702ecb8
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-expand.h
>> @@ -0,0 +1,366 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#ifndef RUST_MACRO_EXPAND_H
>> +#define RUST_MACRO_EXPAND_H
>> +
>> +#include "rust-buffered-queue.h"
>> +#include "rust-parse.h"
>> +#include "rust-token.h"
>> +#include "rust-ast.h"
>> +#include "rust-macro.h"
>> +#include "rust-hir-map.h"
>> +#include "rust-name-resolver.h"
>> +#include "rust-macro-invoc-lexer.h"
>> +
>> +// Provides objects and method prototypes for macro expansion
>> +
>> +namespace Rust {
>> +// forward decls for AST
>> +namespace AST {
>> +class MacroInvocation;
>> +}
>> +
>> +// Object used to store configuration data for macro expansion.
>> +// NOTE: Keep all these items complying with the latest rustc.
>> +struct ExpansionCfg
>> +{
>> + // features?
>> + // TODO: Add `features' when we have it.
>> + unsigned int recursion_limit = 1024;
>> + bool trace_mac = false; // trace macro
>> + bool should_test = false; // strip #[test] nodes if false
>> + bool keep_macs = false; // keep macro definitions
>> + std::string crate_name = "";
>> +};
>> +
>> +struct MatchedFragment
>> +{
>> + std::string fragment_ident;
>> + size_t token_offset_begin;
>> + size_t token_offset_end;
>> +
>> + MatchedFragment (std::string identifier, size_t token_offset_begin,
>> + size_t token_offset_end)
>> + : fragment_ident (identifier), token_offset_begin (token_offset_begin),
>> + token_offset_end (token_offset_end)
>> + {}
>> +
>> + /**
>> + * Empty constructor for uninitialized fragments
>> + */
>> + MatchedFragment () : MatchedFragment ("", 0, 0) {}
>> +
>> + std::string as_string () const
>> + {
>> + return fragment_ident + "=" + std::to_string (token_offset_begin) + ":"
>> + + std::to_string (token_offset_end);
>> + }
>> +};
>> +
>> +class MatchedFragmentContainer
>> +{
>> +public:
>> + // Does the container refer to a simple metavariable, different from a
>> + // repetition repeated once
>> + enum class Kind
>> + {
>> + MetaVar,
>> + Repetition,
>> + };
>> +
>> + MatchedFragmentContainer (std::vector<MatchedFragment> fragments,
>> + Kind kind = Kind::Repetition)
>> + : fragments (fragments), kind (kind)
>> + {}
>> +
>> + /**
>> + * Create a valid fragment matched zero times. This is useful for repetitions
>> + * which allow the absence of a fragment, such as * and ?
>> + */
>> + static MatchedFragmentContainer zero ()
>> + {
>> + return MatchedFragmentContainer ({});
>> + }
>> +
>> + /**
>> + * Create a valid fragment matched one time
>> + */
>> + static MatchedFragmentContainer metavar (MatchedFragment fragment)
>> + {
>> + return MatchedFragmentContainer ({fragment}, Kind::MetaVar);
>> + }
>> +
>> + /**
>> + * Add a matched fragment to the container
>> + */
>> + void add_fragment (MatchedFragment fragment)
>> + {
>> + rust_assert (!is_single_fragment ());
>> +
>> + fragments.emplace_back (fragment);
>> + }
>> +
>> + size_t get_match_amount () const { return fragments.size (); }
>> + const std::vector<MatchedFragment> &get_fragments () const
>> + {
>> + return fragments;
>> + }
>> + // const std::string &get_fragment_name () const { return fragment_name; }
>> +
>> + bool is_single_fragment () const
>> + {
>> + return get_match_amount () == 1 && kind == Kind::MetaVar;
>> + }
>> +
>> + const MatchedFragment get_single_fragment () const
>> + {
>> + rust_assert (is_single_fragment ());
>> +
>> + return fragments[0];
>> + }
>> +
>> + const Kind &get_kind () const { return kind; }
>> +
>> +private:
>> + /**
>> + * Fragments matched `match_amount` times. This can be an empty vector
>> + * in case having zero matches is allowed (i.e ? or * operators)
>> + */
>> + std::vector<MatchedFragment> fragments;
>> + Kind kind;
>> +};
>> +
>> +class SubstitutionScope
>> +{
>> +public:
>> + SubstitutionScope () : stack () {}
>> +
>> + void push () { stack.push_back ({}); }
>> +
>> + std::map<std::string, MatchedFragmentContainer> pop ()
>> + {
>> + auto top = stack.back ();
>> + stack.pop_back ();
>> + return top;
>> + }
>> +
>> + std::map<std::string, MatchedFragmentContainer> &peek ()
>> + {
>> + return stack.back ();
>> + }
>> +
>> + /**
>> + * Insert a new matched metavar into the current substitution map
>> + */
>> + void insert_metavar (MatchedFragment fragment)
>> + {
>> + auto ¤t_map = stack.back ();
>> + auto it = current_map.find (fragment.fragment_ident);
>> +
>> + if (it == current_map.end ())
>> + current_map.insert ({fragment.fragment_ident,
>> + MatchedFragmentContainer::metavar (fragment)});
>> + else
>> + gcc_unreachable ();
>> + }
>> +
>> + /**
>> + * Append a new matched fragment to a repetition into the current substitution
>> + * map
>> + */
>> + void append_fragment (MatchedFragment fragment)
>> + {
>> + auto ¤t_map = stack.back ();
>> + auto it = current_map.find (fragment.fragment_ident);
>> +
>> + if (it == current_map.end ())
>> + current_map.insert (
>> + {fragment.fragment_ident, MatchedFragmentContainer ({fragment})});
>> + else
>> + it->second.add_fragment (fragment);
>> + }
>> +
>> + void insert_matches (std::string key, MatchedFragmentContainer matches)
>> + {
>> + auto ¤t_map = stack.back ();
>> + auto it = current_map.find (key);
>> + rust_assert (it == current_map.end ());
>> +
>> + current_map.insert ({key, matches});
>> + }
>> +
>> +private:
>> + std::vector<std::map<std::string, MatchedFragmentContainer>> stack;
>> +};
>> +
>> +// Object used to store shared data (between functions) for macro expansion.
>> +struct MacroExpander
>> +{
>> + enum ContextType
>> + {
>> + ITEM,
>> + BLOCK,
>> + EXTERN,
>> + TYPE,
>> + TRAIT,
>> + IMPL,
>> + TRAIT_IMPL,
>> + };
>> +
>> + ExpansionCfg cfg;
>> + unsigned int expansion_depth = 0;
>> +
>> + MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session)
>> + : cfg (cfg), crate (crate), session (session),
>> + sub_stack (SubstitutionScope ()),
>> + expanded_fragment (AST::ASTFragment::create_error ()),
>> + resolver (Resolver::Resolver::get ()),
>> + mappings (Analysis::Mappings::get ())
>> + {}
>> +
>> + ~MacroExpander () = default;
>> +
>> + // Expands all macros in the crate passed in.
>> + void expand_crate ();
>> +
>> + /* Expands a macro invocation - possibly make both
>> + * have similar duck-typed interface and use templates?*/
>> + // should this be public or private?
>> + void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon);
>> +
>> + // Expands a single declarative macro.
>> + AST::ASTFragment expand_decl_macro (Location locus,
>> + AST::MacroInvocData &invoc,
>> + AST::MacroRulesDefinition &rules_def,
>> + bool semicolon);
>> +
>> + void expand_cfg_attrs (AST::AttrVec &attrs);
>> + bool fails_cfg (const AST::AttrVec &attr) const;
>> + bool fails_cfg_with_expand (AST::AttrVec &attrs) const;
>> +
>> + bool depth_exceeds_recursion_limit () const;
>> +
>> + bool try_match_rule (AST::MacroRule &match_rule,
>> + AST::DelimTokenTree &invoc_token_tree);
>> +
>> + AST::ASTFragment transcribe_rule (
>> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
>> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
>> + bool semicolon, ContextType ctx);
>> +
>> + bool match_fragment (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchFragment &fragment);
>> +
>> + bool match_token (Parser<MacroInvocLexer> &parser, AST::Token &token);
>> +
>> + bool match_repetition (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchRepetition &rep);
>> +
>> + bool match_matcher (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatcher &matcher);
>> +
>> + /**
>> + * Match any amount of matches
>> + *
>> + * @param parser Parser to use for matching
>> + * @param rep Repetition to try and match
>> + * @param match_amount Reference in which to store the ammount of succesful
>> + * and valid matches
>> + *
>> + * @param lo_bound Lower bound of the matcher. When specified, the matcher
>> + * will only succeed if it parses at *least* `lo_bound` fragments. If
>> + * unspecified, the matcher could succeed when parsing 0 fragments.
>> + *
>> + * @param hi_bound Higher bound of the matcher. When specified, the matcher
>> + * will only succeed if it parses *less than* `hi_bound` fragments. If
>> + * unspecified, the matcher could succeed when parsing an infinity of
>> + * fragments.
>> + *
>> + * @return true if matching was successful and within the given limits, false
>> + * otherwise
>> + */
>> + bool match_n_matches (Parser<MacroInvocLexer> &parser,
>> + AST::MacroMatchRepetition &rep, size_t &match_amount,
>> + size_t lo_bound = 0, size_t hi_bound = 0);
>> +
>> + void push_context (ContextType t) { context.push_back (t); }
>> +
>> + ContextType pop_context ()
>> + {
>> + rust_assert (!context.empty ());
>> +
>> + ContextType t = context.back ();
>> + context.pop_back ();
>> +
>> + return t;
>> + }
>> +
>> + ContextType peek_context () { return context.back (); }
>> +
>> + void set_expanded_fragment (AST::ASTFragment &&fragment)
>> + {
>> + expanded_fragment = std::move (fragment);
>> + }
>> +
>> + AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis)
>> + {
>> + AST::ASTFragment old_fragment = std::move (expanded_fragment);
>> + auto accumulator = std::vector<AST::SingleASTNode> ();
>> + expanded_fragment = AST::ASTFragment::create_error ();
>> +
>> + for (auto &node : old_fragment.get_nodes ())
>> + {
>> + expansion_depth++;
>> + node.accept_vis (vis);
>> + // we'll decide the next move according to the outcome of the macro
>> + // expansion
>> + if (expanded_fragment.is_error ())
>> + accumulator.push_back (node); // if expansion fails, there might be a
>> + // non-macro expression we need to keep
>> + else
>> + {
>> + // if expansion succeeded, then we need to merge the fragment with
>> + // the contents in the accumulator, so that our final expansion
>> + // result will contain non-macro nodes as it should
>> + auto new_nodes = expanded_fragment.get_nodes ();
>> + std::move (new_nodes.begin (), new_nodes.end (),
>> + std::back_inserter (accumulator));
>> + expanded_fragment = AST::ASTFragment (accumulator);
>> + }
>> + expansion_depth--;
>> + }
>> +
>> + return old_fragment;
>> + }
>> +
>> +private:
>> + AST::Crate &crate;
>> + Session &session;
>> + SubstitutionScope sub_stack;
>> + std::vector<ContextType> context;
>> + AST::ASTFragment expanded_fragment;
>> +
>> +public:
>> + Resolver::Resolver *resolver;
>> + Analysis::Mappings *mappings;
>> +};
>> +
>> +} // namespace Rust
>> +
>> +#endif
>> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.cc b/gcc/rust/expand/rust-macro-invoc-lexer.cc
>> new file mode 100644
>> index 00000000000..8a43d29e0d1
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.cc
>> @@ -0,0 +1,29 @@
>> +#include "rust-macro-invoc-lexer.h"
>> +
>> +namespace Rust {
>> +
>> +const_TokenPtr
>> +MacroInvocLexer::peek_token (int n)
>> +{
>> + if ((offs + n) >= token_stream.size ())
>> + return Token::make (END_OF_FILE, Location ());
>> +
>> + return token_stream.at (offs + n)->get_tok_ptr ();
>> +}
>> +
>> +// Advances current token to n + 1 tokens ahead of current position.
>> +void
>> +MacroInvocLexer::skip_token (int n)
>> +{
>> + offs += (n + 1);
>> +}
>> +
>> +void
>> +MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)),
>> + TokenId new_right
>> + __attribute__ ((unused)))
>> +{
>> + // FIXME
>> + gcc_unreachable ();
>> +}
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.h b/gcc/rust/expand/rust-macro-invoc-lexer.h
>> new file mode 100644
>> index 00000000000..0fd4554d02f
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.h
>> @@ -0,0 +1,64 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#ifndef RUST_MACRO_INVOC_LEXER_H
>> +#define RUST_MACRO_INVOC_LEXER_H
>> +
>> +#include "rust-ast.h"
>> +
>> +namespace Rust {
>> +class MacroInvocLexer
>> +{
>> +public:
>> + MacroInvocLexer (std::vector<std::unique_ptr<AST::Token>> stream)
>> + : offs (0), token_stream (std::move (stream))
>> + {}
>> +
>> + // Returns token n tokens ahead of current position.
>> + const_TokenPtr peek_token (int n);
>> +
>> + // Peeks the current token.
>> + const_TokenPtr peek_token () { return peek_token (0); }
>> +
>> + // Advances current token to n + 1 tokens ahead of current position.
>> + void skip_token (int n);
>> +
>> + // Skips the current token.
>> + void skip_token () { skip_token (0); }
>> +
>> + // Splits the current token into two. Intended for use with nested generics
>> + // closes (i.e. T<U<X>> where >> is wrongly lexed as one token). Note that
>> + // this will only work with "simple" tokens like punctuation.
>> + void split_current_token (TokenId new_left, TokenId new_right);
>> +
>> + std::string get_filename () const
>> + {
>> + // FIXME
>> + gcc_unreachable ();
>> + return "FIXME";
>> + }
>> +
>> + size_t get_offs () const { return offs; }
>> +
>> +private:
>> + size_t offs;
>> + std::vector<std::unique_ptr<AST::Token>> token_stream;
>> +};
>> +} // namespace Rust
>> +
>> +#endif // RUST_MACRO_INVOC_LEXER_H
>> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
>> new file mode 100644
>> index 00000000000..9592d2d2a9e
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
>> @@ -0,0 +1,312 @@
>> +#include "rust-macro-substitute-ctx.h"
>> +
>> +namespace Rust {
>> +
>> +std::vector<std::unique_ptr<AST::Token>>
>> +SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
>> +{
>> + auto metavar_name = metavar->get_str ();
>> +
>> + std::vector<std::unique_ptr<AST::Token>> expanded;
>> + auto it = fragments.find (metavar_name);
>> + if (it == fragments.end ())
>> + {
>> + // Return a copy of the original token
>> + expanded.push_back (metavar->clone_token ());
>> + }
>> + else
>> + {
>> + // If we are expanding a metavar which has a lof of matches, we are
>> + // currently expanding a repetition metavar - not a simple metavar. We
>> + // need to error out and inform the user.
>> + // Associated test case for an example: compile/macro-issue1224.rs
>> + if (it->second.get_match_amount () != 1)
>> + {
>> + rust_error_at (metavar->get_locus (),
>> + "metavariable is still repeating at this depth");
>> + rust_inform (
>> + metavar->get_locus (),
>> + "you probably forgot the repetition operator: %<%s%s%s%>", "$(",
>> + metavar->as_string ().c_str (), ")*");
>> + return expanded;
>> + }
>> +
>> + // We only care about the vector when expanding repetitions.
>> + // Just access the first element of the vector.
>> + auto &frag = it->second.get_single_fragment ();
>> + for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
>> + offs++)
>> + {
>> + auto &tok = input.at (offs);
>> + expanded.push_back (tok->clone_token ());
>> + }
>> + }
>> +
>> + return expanded;
>> +}
>> +
>> +bool
>> +SubstituteCtx::check_repetition_amount (size_t pattern_start,
>> + size_t pattern_end,
>> + size_t &expected_repetition_amount)
>> +{
>> + bool first_fragment_found = false;
>> + bool is_valid = true;
>> +
>> + for (size_t i = pattern_start; i < pattern_end; i++)
>> + {
>> + if (macro.at (i)->get_id () == DOLLAR_SIGN)
>> + {
>> + auto &frag_token = macro.at (i + 1);
>> + if (frag_token->get_id () == IDENTIFIER)
>> + {
>> + auto it = fragments.find (frag_token->get_str ());
>> + if (it == fragments.end ())
>> + {
>> + // If the repetition is not anything we know (ie no declared
>> + // metavars, or metavars which aren't present in the
>> + // fragment), we can just error out. No need to paste the
>> + // tokens as if nothing had happened.
>> + rust_error_at (frag_token->get_locus (),
>> + "metavar %s used in repetition does not exist",
>> + frag_token->get_str ().c_str ());
>> +
>> + is_valid = false;
>> + }
>> +
>> + auto &fragment = it->second;
>> +
>> + size_t repeat_amount = fragment.get_match_amount ();
>> + if (!first_fragment_found)
>> + {
>> + first_fragment_found = true;
>> + expected_repetition_amount = repeat_amount;
>> + }
>> + else
>> + {
>> + if (repeat_amount != expected_repetition_amount
>> + && !fragment.is_single_fragment ())
>> + {
>> + rust_error_at (
>> + frag_token->get_locus (),
>> + "different amount of matches used in merged "
>> + "repetitions: expected %lu, got %lu",
>> + (unsigned long) expected_repetition_amount,
>> + (unsigned long) repeat_amount);
>> + is_valid = false;
>> + }
>> + }
>> + }
>> + }
>> + }
>> +
>> + return is_valid;
>> +}
>> +
>> +std::vector<std::unique_ptr<AST::Token>>
>> +SubstituteCtx::substitute_repetition (
>> + size_t pattern_start, size_t pattern_end,
>> + std::unique_ptr<AST::Token> separator_token)
>> +{
>> + rust_assert (pattern_end < macro.size ());
>> +
>> + size_t repeat_amount = 0;
>> + if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount))
>> + return {};
>> +
>> + rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount);
>> + std::vector<std::unique_ptr<AST::Token>> expanded;
>> + std::vector<std::unique_ptr<AST::Token>> new_macro;
>> +
>> + // We want to generate a "new macro" to substitute with. This new macro
>> + // should contain only the tokens inside the pattern
>> + for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++)
>> + new_macro.emplace_back (macro.at (tok_idx)->clone_token ());
>> +
>> + // Then, we want to create a subset of the matches so that
>> + // `substitute_tokens()` can only see one fragment per metavar. Let's say we
>> + // have the following user input: (1 145 'h')
>> + // on the following match arm: ($($lit:literal)*)
>> + // which causes the following matches: { "lit": [1, 145, 'h'] }
>> + //
>> + // The pattern (new_macro) is `$lit:literal`
>> + // The first time we expand it, we want $lit to have the following token: 1
>> + // The second time, 145
>> + // The third and final time, 'h'
>> + //
>> + // In order to do so we must create "sub maps", which only contain parts of
>> + // the original matches
>> + // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ]
>> + //
>> + // and give them to `substitute_tokens` one by one.
>> +
>> + for (size_t i = 0; i < repeat_amount; i++)
>> + {
>> + std::map<std::string, MatchedFragmentContainer> sub_map;
>> + for (auto &kv_match : fragments)
>> + {
>> + MatchedFragment sub_fragment;
>> +
>> + // FIXME: Hack: If a fragment is not repeated, how does it fit in the
>> + // submap? Do we really want to expand it? Is this normal behavior?
>> + if (kv_match.second.is_single_fragment ())
>> + sub_fragment = kv_match.second.get_single_fragment ();
>> + else
>> + sub_fragment = kv_match.second.get_fragments ()[i];
>> +
>> + sub_map.insert (
>> + {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)});
>> + }
>> +
>> + auto substitute_context = SubstituteCtx (input, new_macro, sub_map);
>> + auto new_tokens = substitute_context.substitute_tokens ();
>> +
>> + // Skip the first repetition, but add the separator to the expanded
>> + // tokens if it is present
>> + if (i != 0 && separator_token)
>> + expanded.emplace_back (separator_token->clone_token ());
>> +
>> + for (auto &new_token : new_tokens)
>> + expanded.emplace_back (new_token->clone_token ());
>> + }
>> +
>> + // FIXME: We also need to make sure that all subsequent fragments
>> + // contain the same amount of repetitions as the first one
>> +
>> + return expanded;
>> +}
>> +
>> +static bool
>> +is_rep_op (std::unique_ptr<AST::Token> &tok)
>> +{
>> + auto id = tok->get_id ();
>> + return id == QUESTION_MARK || id == ASTERISK || id == PLUS;
>> +}
>> +
>> +std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
>> +SubstituteCtx::substitute_token (size_t token_idx)
>> +{
>> + auto &token = macro.at (token_idx);
>> + switch (token->get_id ())
>> + {
>> + case IDENTIFIER:
>> + rust_debug ("expanding metavar: %s", token->get_str ().c_str ());
>> + return {substitute_metavar (token), 1};
>> + case LEFT_PAREN: {
>> + // We need to parse up until the closing delimiter and expand this
>> + // fragment->n times.
>> + rust_debug ("expanding repetition");
>> +
>> + // We're in a context where macro repetitions have already been
>> + // parsed and validated: This means that
>> + // 1/ There will be no delimiters as that is an error
>> + // 2/ There are no fragment specifiers anymore, which prevents us
>> + // from reusing parser functions.
>> + //
>> + // Repetition patterns are also special in that they cannot contain
>> + // "rogue" delimiters: For example, this is invalid, as they are
>> + // parsed as MacroMatches and must contain a correct amount of
>> + // delimiters.
>> + // `$($e:expr ) )`
>> + // ^ rogue closing parenthesis
>> + //
>> + // With all of that in mind, we can simply skip ahead from one
>> + // parenthesis to the other to find the pattern to expand. Of course,
>> + // pairs of delimiters, including parentheses, are allowed.
>> + // `$($e:expr ( ) )`
>> + // Parentheses are the sole delimiter for which we need a special
>> + // behavior since they delimit the repetition pattern
>> +
>> + size_t pattern_start = token_idx + 1;
>> + size_t pattern_end = pattern_start;
>> + auto parentheses_stack = 0;
>> + for (size_t idx = pattern_start; idx < macro.size (); idx++)
>> + {
>> + if (macro.at (idx)->get_id () == LEFT_PAREN)
>> + {
>> + parentheses_stack++;
>> + }
>> + else if (macro.at (idx)->get_id () == RIGHT_PAREN)
>> + {
>> + if (parentheses_stack == 0)
>> + {
>> + pattern_end = idx;
>> + break;
>> + }
>> + parentheses_stack--;
>> + }
>> + }
>> +
>> + // Unreachable case, but let's make sure we don't ever run into it
>> + rust_assert (pattern_end != pattern_start);
>> +
>> + std::unique_ptr<AST::Token> separator_token = nullptr;
>> + if (pattern_end + 1 <= macro.size ())
>> + {
>> + auto &post_pattern_token = macro.at (pattern_end + 1);
>> + if (!is_rep_op (post_pattern_token))
>> + separator_token = post_pattern_token->clone_token ();
>> + }
>> +
>> + // Amount of tokens to skip
>> + auto to_skip = 0;
>> + // Parentheses
>> + to_skip += 2;
>> + // Repetition operator
>> + to_skip += 1;
>> + // Separator
>> + if (separator_token)
>> + to_skip += 1;
>> +
>> + return {substitute_repetition (pattern_start, pattern_end,
>> + std::move (separator_token)),
>> + pattern_end - pattern_start + to_skip};
>> + }
>> + // TODO: We need to check if the $ was alone. In that case, do
>> + // not error out: Simply act as if there was an empty identifier
>> + // with no associated fragment and paste the dollar sign in the
>> + // transcription. Unsure how to do that since we always have at
>> + // least the closing curly brace after an empty $...
>> + default:
>> + rust_error_at (token->get_locus (),
>> + "unexpected token in macro transcribe: expected "
>> + "%<(%> or identifier after %<$%>, got %<%s%>",
>> + get_token_description (token->get_id ()));
>> + }
>> +
>> + // FIXME: gcc_unreachable() error case?
>> + return {std::vector<std::unique_ptr<AST::Token>> (), 0};
>> +}
>> +
>> +std::vector<std::unique_ptr<AST::Token>>
>> +SubstituteCtx::substitute_tokens ()
>> +{
>> + std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
>> + rust_debug ("expanding tokens");
>> +
>> + for (size_t i = 0; i < macro.size (); i++)
>> + {
>> + auto &tok = macro.at (i);
>> + if (tok->get_id () == DOLLAR_SIGN)
>> + {
>> + // Aaaaah, if only we had C++17 :)
>> + // auto [expanded, tok_to_skip] = ...
>> + auto p = substitute_token (i + 1);
>> + auto expanded = std::move (p.first);
>> + auto tok_to_skip = p.second;
>> +
>> + i += tok_to_skip;
>> +
>> + for (auto &token : expanded)
>> + replaced_tokens.emplace_back (token->clone_token ());
>> + }
>> + else
>> + {
>> + replaced_tokens.emplace_back (tok->clone_token ());
>> + }
>> + }
>> +
>> + return replaced_tokens;
>> +}
>> +
>> +} // namespace Rust
>> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.h b/gcc/rust/expand/rust-macro-substitute-ctx.h
>> new file mode 100644
>> index 00000000000..81dcab7643b
>> --- /dev/null
>> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.h
>> @@ -0,0 +1,93 @@
>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>> +
>> +// This file is part of GCC.
>> +
>> +// GCC is free software; you can redistribute it and/or modify it under
>> +// the terms of the GNU General Public License as published by the Free
>> +// Software Foundation; either version 3, or (at your option) any later
>> +// version.
>> +
>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>> +// for more details.
>> +
>> +// You should have received a copy of the GNU General Public License
>> +// along with GCC; see the file COPYING3. If not see
>> +// <http://www.gnu.org/licenses/>.
>> +
>> +#include "rust-ast.h"
>> +#include "rust-macro-expand.h"
>> +
>> +namespace Rust {
>> +class SubstituteCtx
>> +{
>> + std::vector<std::unique_ptr<AST::Token>> &input;
>> + std::vector<std::unique_ptr<AST::Token>> ¯o;
>> + std::map<std::string, MatchedFragmentContainer> &fragments;
>> +
>> + /**
>> + * Find the repetition amount to use when expanding a repetition, and
>> + * check that all fragments used respect that repetition amount
>> + *
>> + * @param pattern_start Start of the repetition pattern
>> + * @param pattern_end End of the repetition pattern
>> + * @param repeat_amount Reference to fill with the matched repetition amount
>> + */
>> + bool check_repetition_amount (size_t pattern_start, size_t pattern_end,
>> + size_t &repeat_amount);
>> +
>> +public:
>> + SubstituteCtx (std::vector<std::unique_ptr<AST::Token>> &input,
>> + std::vector<std::unique_ptr<AST::Token>> ¯o,
>> + std::map<std::string, MatchedFragmentContainer> &fragments)
>> + : input (input), macro (macro), fragments (fragments)
>> + {}
>> +
>> + /**
>> + * Substitute a metavariable by its given fragment in a transcribing context,
>> + * i.e. replacing $var with the associated fragment.
>> + *
>> + * @param metavar Metavariable to try and replace
>> + *
>> + * @return A token containing the associated fragment expanded into tokens if
>> + * any, or the cloned token if no fragment was associated
>> + */
>> + std::vector<std::unique_ptr<AST::Token>>
>> + substitute_metavar (std::unique_ptr<AST::Token> &metavar);
>> +
>> + /**
>> + * Substitute a macro repetition by its given fragments
>> + *
>> + * @param pattern_start Start index of the pattern tokens
>> + * @param pattern_end End index of the patterns tokens
>> + * @param separator Optional separator to include when expanding tokens
>> + *
>> + * @return A vector containing the repeated pattern
>> + */
>> + std::vector<std::unique_ptr<AST::Token>>
>> + substitute_repetition (size_t pattern_start, size_t pattern_end,
>> + std::unique_ptr<AST::Token> separator);
>> +
>> + /**
>> + * Substitute a given token by its appropriate representation
>> + *
>> + * @param token_idx Current token to try and substitute
>> + *
>> + * @return A token containing the associated fragment expanded into tokens if
>> + * any, or the cloned token if no fragment was associated, as well as the
>> + * amount of tokens that should be skipped before the next invocation. Since
>> + * this function may consume more than just one token, it is important to skip
>> + * ahead of the input to avoid mis-substitutions
>> + */
>> + std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
>> + substitute_token (size_t token_idx);
>> +
>> + /**
>> + * Substitute all tokens by their appropriate representation
>> + *
>> + * @return A vector containing the substituted tokens
>> + */
>> + std::vector<std::unique_ptr<AST::Token>> substitute_tokens ();
>> +};
>> +} // namespace Rust
>> --
>> 2.37.2
>>
Kindly,
Arthur
On Tue, Nov 15, 2022 at 12:32 PM Arthur Cohen <arthur.cohen@embecosm.com> wrote:
>
>
>
> On 11/10/22 11:49, Richard Biener wrote:
> > On Wed, Oct 26, 2022 at 10:28 AM <arthur.cohen@embecosm.com> wrote:
> >>
> >> From: Arthur Cohen <arthur.cohen@embecosm.com>
> >>
> >> The expansion pass is responsible for two actions on our AST:
> >>
> >> 1. Expanding macro calls
> >> 2. Performing conditional compilation
> >>
> >> Calls to macros should be checked and expanded into an AST fragment based on
> >> the context they've been called in. This is similar to token substitution, with
> >> a lot of intricacies and checks being performed. A single invocation can result
> >> in an AST fragment containing multiple statements or multiple expressions,
> >> which need to be handled as well. Furthermore, Rust macros can contain
> >> repetitions relying on Kleine operators, similar to regular expression
> >> patterns, that also need to be expanded properly.
> >>
> >> Finally, Rust code can be hidden behind `cfg` directives, which allow the user
> >> to perform conditional compilation. If a `cfg` predicate is not met, the
> >> expression or statement it refers to should be marked for strip and removed
> >> from the AST.
> >
> > I'm just picking one of the purely rust frontend specific patches as example and
> > do not plan to look on others - I don't think meaningful review can happen by
> > somebody not deeply understanding rust or this frontend can be done which means
> > approval of those parts shouldn't be needed but would be covered (even
> > the initial
> > contribution) by your maintainership of the code.
> >
> > The commit comment above is quite helpful in getting an idea of what this part
> > of the rust frontend is doing. I'd appreciate if something along these lines
> > would be added to each of the individual source files (preferably the
> > implementation, not the header).
>
> Is there any reason as to why the implementation should be prefered over
> the header?
That's how it's done elsewhere in GCC and it aligns with the function
documentation presence.
> This is a comment you also make down the line on one of our
> function's documentation. My view is that the header should be kept as
> simple and possible, with as little code as possible, and that it is
> easy to find documentation around function declarations, much easier
> than to find it in the implementation when code blocks are present all
> around.
That's true when working with an API but not when working on the implementation.
> To be clear, I'm not opposed to switching the documentation over to our
> implementation - just looking to understand. We'll make the necessary
> changes.
I guess consistency is then most important (consistency in rust/). Consistency
with GCC is nice. I'll note that when the implementation resides in the header
like for templates or inlines the documentation also resides there. That might
be an argument for documentation in the header (in one place).
> > Documentation of the sub-directory structure would
> > also be nice,
> > we have (had ..) sourcebuild.texi which contains some of that but not really for
> > specific frontends (sofar).
>
> Right. We will work on adding proper documentation ASAP.
>
> > Some comments on this particular patch that probably apply to all others as well
> > inline below.
> >
> >> Co-authored-by: Philip Herron <philip.herron@embecosm.com>
> >> Co-authored-by: The Other <simplytheother@gmail.com>
> >> ---
> >> gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++
> >> gcc/rust/expand/rust-attribute-visitor.h | 316 ++
> >> gcc/rust/expand/rust-macro-builtins.cc | 484 +++
> >> gcc/rust/expand/rust-macro-builtins.h | 107 +
> >> gcc/rust/expand/rust-macro-expand.cc | 1012 +++++
> >> gcc/rust/expand/rust-macro-expand.h | 366 ++
> >> gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 +
> >> gcc/rust/expand/rust-macro-invoc-lexer.h | 64 +
> >> gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++
> >> gcc/rust/expand/rust-macro-substitute-ctx.h | 93 +
> >> 10 files changed, 6228 insertions(+)
> >> create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc
> >> create mode 100644 gcc/rust/expand/rust-attribute-visitor.h
> >> create mode 100644 gcc/rust/expand/rust-macro-builtins.cc
> >> create mode 100644 gcc/rust/expand/rust-macro-builtins.h
> >> create mode 100644 gcc/rust/expand/rust-macro-expand.cc
> >> create mode 100644 gcc/rust/expand/rust-macro-expand.h
> >> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc
> >> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h
> >> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc
> >> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h
> >>
> >> diff --git a/gcc/rust/expand/rust-attribute-visitor.cc b/gcc/rust/expand/rust-attribute-visitor.cc
> >> new file mode 100644
> >> index 00000000000..8016f9430eb
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-attribute-visitor.cc
> >> @@ -0,0 +1,3445 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#include "rust-attribute-visitor.h"
> >> +#include "rust-session-manager.h"
> >> +
> >> +namespace Rust {
> >> +
> >> +// Visitor used to expand attributes.
> >> +void
> >> +AttrVisitor::expand_struct_fields (std::vector<AST::StructField> &fields)
> >> +{
> >> + for (auto it = fields.begin (); it != fields.end ();)
> >> + {
> >> + auto &field = *it;
> >> +
> >> + auto &field_attrs = field.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (field_attrs);
> >> + if (expander.fails_cfg_with_expand (field_attrs))
> >> + {
> >> + it = fields.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // expand sub-types of type, but can't strip type itself
> >> + auto &type = field.get_field_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // if nothing else happens, increment
> >> + ++it;
> >> + }
> >> +}
> >> +
> >
> > The coding conventions warrant a function level comment for each function.
> > If there's anything meaningful to say (in this case "// expand tuple fields"
> > wouldn't be meaningful) please consider adding such to all functions. In
> > my view every function that doesn't fit in a 80x24 terminal warrants one.
> >
> >> +void
> >> +AttrVisitor::expand_tuple_fields (std::vector<AST::TupleField> &fields)
> >> +{
> >> + for (auto it = fields.begin (); it != fields.end ();)
> >> + {
> >> + auto &field = *it;
> >> +
> >> + auto &field_attrs = field.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (field_attrs);
> >> + if (expander.fails_cfg_with_expand (field_attrs))
> >> + {
> >> + it = fields.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + // expand sub-types of type, but can't strip type itself
> >> + auto &type = field.get_field_type ();
> >> + type->accept_vis (*this);
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >
> > When reading the other patches I wondered whether rust_error_at are
> > user-facing diagnostics or internal errors?
> >
> >> +
> >> + // if nothing else happens, increment
> >> + ++it;
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_function_params (std::vector<AST::FunctionParam> ¶ms)
> >> +{
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + for (auto it = params.begin (); it != params.end ();)
> >> + {
> >> + auto ¶m = *it;
> >> +
> >> + auto ¶m_attrs = param.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (param_attrs);
> >> + if (expander.fails_cfg_with_expand (param_attrs))
> >> + {
> >> + it = params.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + // TODO: should an unwanted strip lead to break out of loop?
> >> + auto &pattern = param.get_pattern ();
> >
> > 'auto' is used a lot in the frontend, it does make reading code harder because
> > critical information on context is lost. Here I can at least 2nd guess
> > 'param' being of type AST::FunctionParam and thus find get_pattern (),
> > but I will have to find that to be able to find accept_vis () below.
> >
> > In my mind 'auto' is useful when it hides information that's not used as in
> >
> > for (auto it = ...)
> > {
> > AST::FunctionParam ¶m = *it;
> >
> > but not so much for being lazy when typing.
> >
> > At least with (some of) the uses the identifiers of the methods look
> > unique enough so grep will be of help, OTOH is_marked_for_strip
> > is defined a lot.
> >
>
> I think removing all the `auto`s from the gccrs codebase would be a
> herculean task at this point, as a quick grep for 'auto ' in our
> frontend reveals around 2700 matches.
>
> From the little research I did, it seems the C++ frontend also uses it
> somewhat, as well as the D frontend (but that frontend is written in D,
> so that might be moot).
>
> Paired with descriptive variable names and short functions, I personally
> don't have an issue with reviewing code that uses `auto` but I see how
> that might be the case, especially for someone who is not familiar with
> our frontend.
>
> So I think the question I'm asking is as follows: considering Philip and
> I will be reviewers of the Rust frontend part specifically, is it okay
> to not make such a huge change to the frontend which would cost us a lot
> of time while not necessarily making a difference to us? Again, I
> understand that this makes a difference to some people, but as you
> mentioned I'm not sure anyone would be willing to review all of our
> frontend code, especially as it contains around 160 000 lines of code.
Yes, my remark was quite general about 'auto' (I see uses creeping in
in GCC elsewhere), sorry with putting this into a rust frontend review mail ;)
It just occured to me that writing 'tree' isn't any bigger than 'auto' but it
would have helped me a lot identifying pieces that work with GENERIC.
> Furthering my point on the D frontend: it's also no secret that we would
> like to rewrite some parts of the frontend in Rust in the future, as a
> functional programming language is well suited to the compilation of
> Rust. When that is the case, we will use what I would consider
> "idiomatic Rust", which makes heavy use of the `let` keyword (similar in
> many ways to `auto`) and type inference. Types are rarely explicitly
> specified for variables, but are always specified for function arguments
> or function return types (as we do presently in the C++ implementation
> of our frontend).
>
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +
> >> + auto &type = param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + // increment
> >> + ++it;
> >> + }
> >> +
> >> + expander.pop_context ();
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_generic_args (AST::GenericArgs &args)
> >> +{
> >> + // lifetime args can't be expanded
> >> + // FIXME: Can we have macro invocations for lifetimes?
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // expand type args - strip sub-types only
> >> + for (auto &arg : args.get_generic_args ())
> >> + {
> >> + switch (arg.get_kind ())
> >> + {
> >> + case AST::GenericArg::Kind::Type: {
> >> + auto &type = arg.get_type ();
> >> + type->accept_vis (*this);
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> + break;
> >> + }
> >> + case AST::GenericArg::Kind::Const: {
> >> + auto &expr = arg.get_expression ();
> >> + expr->accept_vis (*this);
> >> + maybe_expand_expr (expr);
> >> +
> >> + if (expr->is_marked_for_strip ())
> >> + rust_error_at (expr->get_locus (),
> >> + "cannot strip expression in this position");
> >> + break;
> >> + }
> >> + default:
> >> + break;
> >> + // FIXME: Figure out what to do here if there is ambiguity. Since the
> >> + // resolver comes after the expansion, we need to figure out a way to
> >> + // strip ambiguous values here
> >> + // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg`
> >> + // or something. This would clean up this whole thing
> >> + }
> >> + }
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // FIXME: Can we have macro invocations in generic type bindings?
> >> + // expand binding args - strip sub-types only
> >> + for (auto &binding : args.get_binding_args ())
> >> + {
> >> + auto &type = binding.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type)
> >> +{
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = path_type.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + expander.pop_context ();
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + if (path_type.has_as_clause ())
> >> + {
> >> + auto &type_path = path_type.get_as_type_path ();
> >> + visit (type_path);
> >> + if (type_path.is_marked_for_strip ())
> >> + rust_error_at (type_path.get_locus (),
> >> + "cannot strip type path in this position");
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::AttrVisitor::expand_closure_params (
> >> + std::vector<AST::ClosureParam> ¶ms)
> >> +{
> >> + for (auto it = params.begin (); it != params.end ();)
> >> + {
> >> + auto ¶m = *it;
> >> +
> >> + auto ¶m_attrs = param.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (param_attrs);
> >> + if (expander.fails_cfg_with_expand (param_attrs))
> >> + {
> >> + it = params.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + auto &pattern = param.get_pattern ();
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +
> >> + if (param.has_type_given ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> + auto &type = param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + // increment if found nothing else so far
> >> + ++it;
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_self_param (AST::SelfParam &self_param)
> >> +{
> >> + if (self_param.has_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> + auto &type = self_param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> + /* TODO: maybe check for invariants being violated - e.g. both type and
> >> + * lifetime? */
> >
> > C-style comment where I see C++-style comments everywhere else.
> >
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_where_clause (AST::WhereClause &where_clause)
> >> +{
> >> + // items cannot be stripped conceptually, so just accept visitor
> >> + for (auto &item : where_clause.get_items ())
> >> + item->accept_vis (*this);
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl)
> >> +{
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : decl.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* strip function parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_function_params (decl.get_function_params ());
> >> +
> >> + if (decl.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = decl.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + if (decl.has_where_clause ())
> >> + expand_where_clause (decl.get_where_clause ());
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl)
> >> +{
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : decl.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* assuming you can't strip self param - wouldn't be a method
> >> + * anymore. spec allows outer attrs on self param, but doesn't
> >> + * specify whether cfg is used. */
> >> + expand_self_param (decl.get_self_param ());
> >> +
> >> + /* strip function parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_function_params (decl.get_function_params ());
> >> +
> >> + if (decl.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = decl.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + if (decl.has_where_clause ())
> >> + expand_where_clause (decl.get_where_clause ());
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::Token &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::DelimTokenTree &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::AttrInputMetaItemContainer &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IdentifierExpr &ident_expr)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (ident_expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ()))
> >> + {
> >> + ident_expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Lifetime &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::LifetimeParam &)
> >> +{
> >> + // supposedly does not require - cfg does nothing
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ConstGenericParam &)
> >> +{
> >> + // likewise
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::MacroInvocation ¯o_invoc)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ()))
> >> + {
> >> + macro_invoc.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip simple path
> >> +
> >> + // I don't think any macro token trees can be stripped in any way
> >> +
> >> + // TODO: maybe have cfg! macro stripping behaviour here?
> >> + expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ());
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::PathInExpression &path)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (path.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
> >> + {
> >> + path.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &segment : path.get_segments ())
> >> + {
> >> + if (segment.has_generic_args ())
> >> + expand_generic_args (segment.get_generic_args ());
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypePathSegment &)
> >> +{
> >> + // shouldn't require
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypePathSegmentGeneric &segment)
> >> +{
> >> + // TODO: strip inside generic args
> >> +
> >> + if (!segment.has_generic_args ())
> >> + return;
> >> +
> >> + expand_generic_args (segment.get_generic_args ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypePathSegmentFunction &segment)
> >> +{
> >> + auto &type_path_function = segment.get_type_path_function ();
> >> +
> >> + for (auto &type : type_path_function.get_params ())
> >> + {
> >> + type->accept_vis (*this);
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> + }
> >> +
> >> + if (type_path_function.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = type_path_function.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypePath &path)
> >> +{
> >> + // this shouldn't strip any segments, but can strip inside them
> >> + for (auto &segment : path.get_segments ())
> >> + segment->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::QualifiedPathInExpression &path)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (path.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
> >> + {
> >> + path.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expand_qualified_path_type (path.get_qualified_path_type ());
> >> +
> >> + for (auto &segment : path.get_segments ())
> >> + {
> >> + if (segment.has_generic_args ())
> >> + expand_generic_args (segment.get_generic_args ());
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::QualifiedPathInType &path)
> >> +{
> >> + expand_qualified_path_type (path.get_qualified_path_type ());
> >> +
> >> + // this shouldn't strip any segments, but can strip inside them
> >> + for (auto &segment : path.get_segments ())
> >> + segment->accept_vis (*this);
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::LiteralExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::AttrInputLiteral &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::MetaItemLitExpr &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::MetaItemPathLit &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::BorrowExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &borrowed_expr = expr.get_borrowed_expr ();
> >> + borrowed_expr->accept_vis (*this);
> >> + if (borrowed_expr->is_marked_for_strip ())
> >> + rust_error_at (borrowed_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::DereferenceExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &dereferenced_expr = expr.get_dereferenced_expr ();
> >> + dereferenced_expr->accept_vis (*this);
> >> + if (dereferenced_expr->is_marked_for_strip ())
> >> + rust_error_at (dereferenced_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ErrorPropagationExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &propagating_expr = expr.get_propagating_expr ();
> >> + propagating_expr->accept_vis (*this);
> >> + if (propagating_expr->is_marked_for_strip ())
> >> + rust_error_at (propagating_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::NegationExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &negated_expr = expr.get_negated_expr ();
> >> + negated_expr->accept_vis (*this);
> >> + if (negated_expr->is_marked_for_strip ())
> >> + rust_error_at (negated_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &l_expr = expr.get_left_expr ();
> >> + l_expr->accept_vis (*this);
> >> + maybe_expand_expr (l_expr);
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &r_expr = expr.get_right_expr ();
> >> + r_expr->accept_vis (*this);
> >> + maybe_expand_expr (r_expr);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_left_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_left_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before binary op exprs");
> >> + if (expr.get_right_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_right_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ComparisonExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &l_expr = expr.get_left_expr ();
> >> + l_expr->accept_vis (*this);
> >> + maybe_expand_expr (l_expr);
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &r_expr = expr.get_right_expr ();
> >> + r_expr->accept_vis (*this);
> >> + maybe_expand_expr (r_expr);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_left_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_left_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before binary op exprs");
> >> + if (expr.get_right_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_right_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >
> > vertical space between function definitions missing here (and above and below)
> >
> >> +void
> >> +AttrVisitor::visit (AST::LazyBooleanExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &l_expr = expr.get_left_expr ();
> >> + l_expr->accept_vis (*this);
> >> + maybe_expand_expr (l_expr);
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &r_expr = expr.get_right_expr ();
> >> + r_expr->accept_vis (*this);
> >> + maybe_expand_expr (r_expr);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_left_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_left_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before binary op exprs");
> >> + if (expr.get_right_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_right_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypeCastExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * direct descendant expression, can strip ones below that */
> >> +
> >> + auto &casted_expr = expr.get_casted_expr ();
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + casted_expr->accept_vis (*this);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (casted_expr->is_marked_for_strip ())
> >> + rust_error_at (casted_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed before cast exprs");
> >> +
> >> + // TODO: strip sub-types of type
> >> + auto &type = expr.get_type_to_cast_to ();
> >> + type->accept_vis (*this);
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::AssignmentExpr &expr)
> >> +{
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &l_expr = expr.get_left_expr ();
> >> + l_expr->accept_vis (*this);
> >> + maybe_expand_expr (l_expr);
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &r_expr = expr.get_right_expr ();
> >> + r_expr->accept_vis (*this);
> >> + maybe_expand_expr (r_expr);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_left_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_left_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before binary op exprs");
> >> + if (expr.get_right_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_right_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::CompoundAssignmentExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &l_expr = expr.get_left_expr ();
> >> + l_expr->accept_vis (*this);
> >> + maybe_expand_expr (l_expr);
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &r_expr = expr.get_right_expr ();
> >> + r_expr->accept_vis (*this);
> >> + maybe_expand_expr (r_expr);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_left_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_left_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before binary op exprs");
> >> + if (expr.get_right_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_right_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::GroupedExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says these are inner
> >> + * attributes, not outer attributes of inner expr */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &inner_expr = expr.get_expr_in_parens ();
> >> + inner_expr->accept_vis (*this);
> >> + if (inner_expr->is_marked_for_strip ())
> >> + rust_error_at (inner_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArrayElemsValues &elems)
> >> +{
> >> + /* apparently outer attributes are allowed in "elements of array
> >> + * expressions" according to spec */
> >> + expand_pointer_allow_strip (elems.get_values ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArrayElemsCopied &elems)
> >> +{
> >> + /* apparently outer attributes are allowed in "elements of array
> >> + * expressions" according to spec. on the other hand, it would not
> >> + * make conceptual sense to be able to remove either expression. As
> >> + * such, not implementing. TODO clear up the ambiguity here */
> >> +
> >> + // only intend stripping for internal sub-expressions
> >> + auto &copied_expr = elems.get_elem_to_copy ();
> >> + copied_expr->accept_vis (*this);
> >> + if (copied_expr->is_marked_for_strip ())
> >> + rust_error_at (copied_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + auto ©_count = elems.get_num_copies ();
> >> + copy_count->accept_vis (*this);
> >> + if (copy_count->is_marked_for_strip ())
> >> + rust_error_at (copy_count->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArrayExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says there are separate
> >> + * inner attributes, not just outer attributes of inner exprs */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* assuming you can't strip away the ArrayElems type, but can strip
> >> + * internal expressions and whatever */
> >> + expr.get_array_elems ()->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArrayIndexExpr &expr)
> >> +{
> >> + /* it is unclear whether outer attributes are supposed to be
> >> + * allowed, but conceptually it wouldn't make much sense, but
> >> + * having expansion code anyway. TODO */
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &array_expr = expr.get_array_expr ();
> >> + array_expr->accept_vis (*this);
> >> + if (array_expr->is_marked_for_strip ())
> >> + rust_error_at (array_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + auto &index_expr = expr.get_index_expr ();
> >> + index_expr->accept_vis (*this);
> >> + if (index_expr->is_marked_for_strip ())
> >> + rust_error_at (index_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleExpr &expr)
> >> +{
> >> + /* according to spec, outer attributes are allowed on "elements of
> >> + * tuple expressions" */
> >> +
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says these are inner
> >> + * attributes, not outer attributes of inner expr */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* apparently outer attributes are allowed in "elements of tuple
> >> + * expressions" according to spec */
> >> + expand_pointer_allow_strip (expr.get_tuple_elems ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleIndexExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* wouldn't strip this directly (as outer attrs should be
> >> + * associated with this level), but any sub-expressions would be
> >> + * stripped. Thus, no need to erase when strip check called. */
> >> + auto &tuple_expr = expr.get_tuple_expr ();
> >> + tuple_expr->accept_vis (*this);
> >> + if (tuple_expr->is_marked_for_strip ())
> >> + rust_error_at (tuple_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprStruct &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says these are inner
> >> + * attributes, not outer attributes of inner expr */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-exprs of path
> >> + auto &struct_name = expr.get_struct_name ();
> >> + visit (struct_name);
> >> + if (struct_name.is_marked_for_strip ())
> >> + rust_error_at (struct_name.get_locus (),
> >> + "cannot strip path in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprFieldIdentifier &)
> >> +{
> >> + // as no attrs (at moment, at least), no stripping possible
> >
> > has
> >
> > is there a reason to not put a gcc_unreachable () here?
> >
>
> The visitor goes through all the nodes in a crate, even if there is
> nothing to do. In the case of the Attribute visitor, there currently
> isn't anything to do for the following pattern (which is what
> `StructExpreFieldIdentifier` is the node for)
>
> ```rust
> struct Foo {
> a: i32,
> b: i32,
> }
>
> fn init_struct_with_a(a: i32) -> Foo {
> Foo {
> a, // <--- here
> b: some_fn(),
> }
> }
> ```
>
> where a struct's field is initialized by a variable of the same name
> (this is a short-hand for `a: a`).
>
> But that does not mean that the node is not visited. What happens is
> that we visit all the nodes of a struct's initialization, which are of
> the base class `StructExprField` and then dynamically dispatch to the
> proper visitor functions, we must handle the case of a
> `StructExprFieldIdentifier` being visited even if there is nothing to do.
>
> If we add a `gcc_unreachable` here, the above test case would crash, as
> it would try stripping the attributes of both field initalizations. For
> the first case, there is nothing to do. For the second case, we want to
> make sure we should not be stripping the sub-expression (the call to
> `some_fn`)
Ah - I confused 'no stripping possible' with 'no stripping needed' then
(or rather you did?)
>
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field)
> >> +{
> >> + /* as no attrs possible (at moment, at least), only sub-expression
> >> + * stripping is possible */
> >> + auto &value = field.get_value ();
> >> + value->accept_vis (*this);
> >> + if (value->is_marked_for_strip ())
> >> + rust_error_at (value->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprFieldIndexValue &field)
> >> +{
> >> + /* as no attrs possible (at moment, at least), only sub-expression
> >> + * stripping is possible */
> >> + auto &value = field.get_value ();
> >> + value->accept_vis (*this);
> >> + if (value->is_marked_for_strip ())
> >> + rust_error_at (value->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprStructFields &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says these are inner
> >> + * attributes, not outer attributes of inner expr */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-exprs of path
> >> + auto &struct_name = expr.get_struct_name ();
> >> + visit (struct_name);
> >> + if (struct_name.is_marked_for_strip ())
> >> + rust_error_at (struct_name.get_locus (),
> >> + "cannot strip path in this position");
> >> +
> >> + /* spec does not specify whether expressions are allowed to be
> >> + * stripped at top level of struct fields, but I wouldn't think
> >> + * that they would be, so operating under the assumption that only
> >> + * sub-expressions can be stripped. */
> >> + for (auto &field : expr.get_fields ())
> >> + {
> >> + field->accept_vis (*this);
> >> + // shouldn't strip in this
> >> + }
> >> +
> >> + /* struct base presumably can't be stripped, as the '..' is before
> >> + * the expression. as such, can only strip sub-expressions. */
> >> + if (expr.has_struct_base ())
> >> + {
> >> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
> >> + base_struct_expr->accept_vis (*this);
> >> + if (base_struct_expr->is_marked_for_strip ())
> >> + rust_error_at (base_struct_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructExprStructBase &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says these are inner
> >> + * attributes, not outer attributes of inner expr */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-exprs of path
> >> + auto &struct_name = expr.get_struct_name ();
> >> + visit (struct_name);
> >> + if (struct_name.is_marked_for_strip ())
> >> + rust_error_at (struct_name.get_locus (),
> >> + "cannot strip path in this position");
> >> +
> >> + /* struct base presumably can't be stripped, as the '..' is before
> >> + * the expression. as such, can only strip sub-expressions. */
> >> + rust_assert (!expr.get_struct_base ().is_invalid ());
> >> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
> >> + base_struct_expr->accept_vis (*this);
> >> + if (base_struct_expr->is_marked_for_strip ())
> >> + rust_error_at (base_struct_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::CallExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* should not be outer attrs on "function" expression - outer attrs
> >> + * should be associated with call expr as a whole. only sub-expr
> >> + * expansion is possible. */
> >> + auto &function = expr.get_function_expr ();
> >> + function->accept_vis (*this);
> >> + if (function->is_marked_for_strip ())
> >> + rust_error_at (function->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + /* spec says outer attributes are specifically allowed for elements
> >> + * of call expressions, so full stripping possible */
> >> + // FIXME: Arthur: Figure out how to refactor this - This is similar to
> >> + // expanding items in the crate or stmts in blocks
> >> + expand_pointer_allow_strip (expr.get_params ());
> >> + auto ¶ms = expr.get_params ();
> >> + for (auto it = params.begin (); it != params.end ();)
> >> + {
> >> + auto &stmt = *it;
> >> +
> >> + stmt->accept_vis (*this);
> >> +
> >> + auto final_fragment = expand_macro_fragment_recursive ();
> >> + if (final_fragment.should_expand ())
> >> + {
> >> + // Remove the current expanded invocation
> >> + it = params.erase (it);
> >> + for (auto &node : final_fragment.get_nodes ())
> >> + {
> >> + it = params.insert (it, node.take_expr ());
> >> + it++;
> >> + }
> >> + }
> >> + else if (stmt->is_marked_for_strip ())
> >> + it = params.erase (it);
> >> + else
> >> + it++;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::MethodCallExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* should not be outer attrs on "receiver" expression - outer attrs
> >> + * should be associated with call expr as a whole. only sub-expr
> >> + * expansion is possible. */
> >> + auto &receiver = expr.get_receiver_expr ();
> >> + receiver->accept_vis (*this);
> >> + if (receiver->is_marked_for_strip ())
> >> + rust_error_at (receiver->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + auto &method_name = expr.get_method_name ();
> >> + if (method_name.has_generic_args ())
> >> + expand_generic_args (method_name.get_generic_args ());
> >> +
> >> + /* spec says outer attributes are specifically allowed for elements
> >> + * of method call expressions, so full stripping possible */
> >> + expand_pointer_allow_strip (expr.get_params ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::FieldAccessExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* should not be outer attrs on "receiver" expression - outer attrs
> >> + * should be associated with field expr as a whole. only sub-expr
> >> + * expansion is possible. */
> >> + auto &receiver = expr.get_receiver_expr ();
> >> + receiver->accept_vis (*this);
> >> + if (receiver->is_marked_for_strip ())
> >> + rust_error_at (receiver->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ClosureExprInner &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip closure parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_closure_params (expr.get_params ());
> >> +
> >> + // can't strip expression itself, but can strip sub-expressions
> >> + auto &definition_expr = expr.get_definition_expr ();
> >> + definition_expr->accept_vis (*this);
> >> + if (definition_expr->is_marked_for_strip ())
> >> + rust_error_at (definition_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::BlockExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip test based on inner attrs - spec says there are inner
> >> + * attributes, not just outer attributes of inner stmts */
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + std::function<std::unique_ptr<AST::Stmt> (AST::SingleASTNode)> extractor
> >> + = [] (AST::SingleASTNode node) { return node.take_stmt (); };
> >> +
> >> + expand_macro_children (MacroExpander::BLOCK, expr.get_statements (),
> >> + extractor);
> >> +
> >> + expander.push_context (MacroExpander::BLOCK);
> >> +
> >> + // strip tail expression if exists - can actually fully remove it
> >> + if (expr.has_tail_expr ())
> >> + {
> >> + auto &tail_expr = expr.get_tail_expr ();
> >> +
> >> + tail_expr->accept_vis (*this);
> >> + maybe_expand_expr (tail_expr);
> >> +
> >> + if (tail_expr->is_marked_for_strip ())
> >> + expr.strip_tail_expr ();
> >> + }
> >> + expander.pop_context ();
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::ClosureExprInnerTyped &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip closure parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_closure_params (expr.get_params ());
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // can't strip return type, but can strip sub-types
> >> + auto &type = expr.get_return_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // can't strip expression itself, but can strip sub-expressions
> >> + auto &definition_block = expr.get_definition_block ();
> >> + definition_block->accept_vis (*this);
> >> + if (definition_block->is_marked_for_strip ())
> >> + rust_error_at (definition_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ContinueExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::BreakExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* spec does not say that you can have outer attributes on
> >> + * expression, so assuming you can't. stripping for sub-expressions
> >> + * is the only thing that can be done */
> >> + if (expr.has_break_expr ())
> >> + {
> >> + auto &break_expr = expr.get_break_expr ();
> >> +
> >> + break_expr->accept_vis (*this);
> >> +
> >> + if (break_expr->is_marked_for_strip ())
> >> + rust_error_at (break_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeFromToExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + expr.get_from_expr ()->accept_vis (*this);
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + expr.get_to_expr ()->accept_vis (*this);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_from_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_from_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before range exprs");
> >> + if (expr.get_to_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_to_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeFromExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * direct descendant expression, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + auto &from_expr = expr.get_from_expr ();
> >> +
> >> + from_expr->accept_vis (*this);
> >> +
> >> + if (from_expr->is_marked_for_strip ())
> >> + rust_error_at (from_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed before range exprs");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeToExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * direct descendant expression, can strip ones below that */
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &to_expr = expr.get_to_expr ();
> >> +
> >> + to_expr->accept_vis (*this);
> >> +
> >> + if (to_expr->is_marked_for_strip ())
> >> + rust_error_at (to_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeFullExpr &)
> >> +{
> >> + // outer attributes never allowed before these, so no stripping
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeFromToInclExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * two direct descendant expressions, can strip ones below that */
> >> +
> >> + /* should have no possibility for outer attrs as would be parsed
> >> + * with outer expr */
> >> + expr.get_from_expr ()->accept_vis (*this);
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + expr.get_to_expr ()->accept_vis (*this);
> >> +
> >> + // ensure that they are not marked for strip
> >> + if (expr.get_from_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_from_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes are never allowed "
> >> + "before range exprs");
> >> + if (expr.get_to_expr ()->is_marked_for_strip ())
> >> + rust_error_at (expr.get_to_expr ()->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangeToInclExpr &expr)
> >> +{
> >> + /* outer attributes never allowed before these. while cannot strip
> >> + * direct descendant expression, can strip ones below that */
> >> +
> >> + /* should syntactically not have outer attributes, though this may
> >> + * not have worked in practice */
> >> + auto &to_expr = expr.get_to_expr ();
> >> +
> >> + to_expr->accept_vis (*this);
> >> +
> >> + if (to_expr->is_marked_for_strip ())
> >> + rust_error_at (to_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ReturnExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* spec does not say that you can have outer attributes on
> >> + * expression, so assuming you can't. stripping for sub-expressions
> >> + * is the only thing that can be done */
> >> + if (expr.has_returned_expr ())
> >> + {
> >> + auto &returned_expr = expr.get_returned_expr ();
> >> +
> >> + returned_expr->accept_vis (*this);
> >> +
> >> + if (returned_expr->is_marked_for_strip ())
> >> + rust_error_at (returned_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> + }
> >> + /* TODO: conceptually, you would maybe be able to remove a returned
> >> + * expr - e.g. if you had conditional compilation returning void or
> >> + * returning a type. On the other hand, I think that function
> >> + * return type cannot be conditionally compiled, so I assumed you
> >> + * can't do this either. */
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::UnsafeBlockExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &block_expr = expr.get_block_expr ();
> >> + block_expr->accept_vis (*this);
> >> + if (block_expr->is_marked_for_strip ())
> >> + rust_error_at (block_expr->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::LoopExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &loop_block = expr.get_loop_block ();
> >> + loop_block->accept_vis (*this);
> >> + if (loop_block->is_marked_for_strip ())
> >> + rust_error_at (loop_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::WhileLoopExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip predicate expr itself, but can strip sub-expressions
> >> + auto &predicate_expr = expr.get_predicate_expr ();
> >> + predicate_expr->accept_vis (*this);
> >> + if (predicate_expr->is_marked_for_strip ())
> >> + rust_error_at (predicate_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &loop_block = expr.get_loop_block ();
> >> + loop_block->accept_vis (*this);
> >> + if (loop_block->is_marked_for_strip ())
> >> + rust_error_at (loop_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::WhileLetLoopExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &pattern : expr.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + // can't strip scrutinee expr itself, but can strip sub-expressions
> >> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
> >> + scrutinee_expr->accept_vis (*this);
> >> + if (scrutinee_expr->is_marked_for_strip ())
> >> + rust_error_at (scrutinee_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &loop_block = expr.get_loop_block ();
> >> + loop_block->accept_vis (*this);
> >> + if (loop_block->is_marked_for_strip ())
> >> + rust_error_at (loop_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ForLoopExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-patterns of pattern
> >> + auto &pattern = expr.get_pattern ();
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +
> >> + // can't strip scrutinee expr itself, but can strip sub-expressions
> >> + auto &iterator_expr = expr.get_iterator_expr ();
> >> + iterator_expr->accept_vis (*this);
> >> + if (iterator_expr->is_marked_for_strip ())
> >> + rust_error_at (iterator_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &loop_block = expr.get_loop_block ();
> >> + loop_block->accept_vis (*this);
> >> + if (loop_block->is_marked_for_strip ())
> >> + rust_error_at (loop_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfExpr &expr)
> >> +{
> >> + // rust playground test shows that IfExpr does support outer attrs, at least
> >> + // when used as statement
> >> +
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip condition expr itself, but can strip sub-expressions
> >> + auto &condition_expr = expr.get_condition_expr ();
> >> + condition_expr->accept_vis (*this);
> >> + maybe_expand_expr (condition_expr);
> >> + if (condition_expr->is_marked_for_strip ())
> >> + rust_error_at (condition_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfExprConseqElse &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip condition expr itself, but can strip sub-expressions
> >> + auto &condition_expr = expr.get_condition_expr ();
> >> + condition_expr->accept_vis (*this);
> >> + maybe_expand_expr (condition_expr);
> >> + if (condition_expr->is_marked_for_strip ())
> >> + rust_error_at (condition_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip else block itself, but can strip sub-expressions
> >> + auto &else_block = expr.get_else_block ();
> >> + else_block->accept_vis (*this);
> >> + if (else_block->is_marked_for_strip ())
> >> + rust_error_at (else_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfExprConseqIf &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip condition expr itself, but can strip sub-expressions
> >> + auto &condition_expr = expr.get_condition_expr ();
> >> + condition_expr->accept_vis (*this);
> >> + maybe_expand_expr (condition_expr);
> >> + if (condition_expr->is_marked_for_strip ())
> >> + rust_error_at (condition_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if expr itself, but can strip sub-expressions
> >> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
> >> + conseq_if_expr->accept_vis (*this);
> >> + if (conseq_if_expr->is_marked_for_strip ())
> >> + rust_error_at (conseq_if_expr->get_locus (),
> >> + "cannot strip consequent if expression in this "
> >> + "position - outer attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfExprConseqIfLet &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip condition expr itself, but can strip sub-expressions
> >> + auto &condition_expr = expr.get_condition_expr ();
> >> + condition_expr->accept_vis (*this);
> >> + maybe_expand_expr (condition_expr);
> >> + if (condition_expr->is_marked_for_strip ())
> >> + rust_error_at (condition_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if let expr itself, but can strip sub-expressions
> >> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
> >> + conseq_if_let_expr->accept_vis (*this);
> >> + if (conseq_if_let_expr->is_marked_for_strip ())
> >> + rust_error_at (conseq_if_let_expr->get_locus (),
> >> + "cannot strip consequent if let expression in this "
> >> + "position - outer attributes not "
> >> + "allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfLetExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &pattern : expr.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + // can't strip value expr itself, but can strip sub-expressions
> >> + auto &value_expr = expr.get_value_expr ();
> >> + value_expr->accept_vis (*this);
> >> + if (value_expr->is_marked_for_strip ())
> >> + rust_error_at (value_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfLetExprConseqElse &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &pattern : expr.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + // can't strip value expr itself, but can strip sub-expressions
> >> + auto &value_expr = expr.get_value_expr ();
> >> + value_expr->accept_vis (*this);
> >> + if (value_expr->is_marked_for_strip ())
> >> + rust_error_at (value_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip else block itself, but can strip sub-expressions
> >> + auto &else_block = expr.get_else_block ();
> >> + else_block->accept_vis (*this);
> >> + if (else_block->is_marked_for_strip ())
> >> + rust_error_at (else_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfLetExprConseqIf &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &pattern : expr.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + // can't strip value expr itself, but can strip sub-expressions
> >> + auto &value_expr = expr.get_value_expr ();
> >> + value_expr->accept_vis (*this);
> >> + if (value_expr->is_marked_for_strip ())
> >> + rust_error_at (value_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if expr itself, but can strip sub-expressions
> >> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
> >> + conseq_if_expr->accept_vis (*this);
> >> + if (conseq_if_expr->is_marked_for_strip ())
> >> + rust_error_at (conseq_if_expr->get_locus (),
> >> + "cannot strip consequent if expression in this "
> >> + "position - outer attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + for (auto &pattern : expr.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + // can't strip value expr itself, but can strip sub-expressions
> >> + auto &value_expr = expr.get_value_expr ();
> >> + value_expr->accept_vis (*this);
> >> + if (value_expr->is_marked_for_strip ())
> >> + rust_error_at (value_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if block itself, but can strip sub-expressions
> >> + auto &if_block = expr.get_if_block ();
> >> + if_block->accept_vis (*this);
> >> + if (if_block->is_marked_for_strip ())
> >> + rust_error_at (if_block->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // can't strip if let expr itself, but can strip sub-expressions
> >> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
> >> + conseq_if_let_expr->accept_vis (*this);
> >> + if (conseq_if_let_expr->is_marked_for_strip ())
> >> + rust_error_at (conseq_if_let_expr->get_locus (),
> >> + "cannot strip consequent if let expression in this "
> >> + "position - outer attributes not "
> >> + "allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::MatchExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // inner attr strip test
> >> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip scrutinee expr itself, but can strip sub-expressions
> >> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
> >> + scrutinee_expr->accept_vis (*this);
> >> + if (scrutinee_expr->is_marked_for_strip ())
> >> + rust_error_at (scrutinee_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // strip match cases
> >> + auto &match_cases = expr.get_match_cases ();
> >> + for (auto it = match_cases.begin (); it != match_cases.end ();)
> >> + {
> >> + auto &match_case = *it;
> >> +
> >> + // strip match case based on outer attributes in match arm
> >> + auto &match_arm = match_case.get_arm ();
> >> + expander.expand_cfg_attrs (match_arm.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ()))
> >> + {
> >> + // strip match case
> >> + it = match_cases.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + for (auto &pattern : match_arm.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + }
> >> +
> >> + /* assuming that guard expression cannot be stripped as
> >> + * strictly speaking you would have to strip the whole guard to
> >> + * make syntactical sense, which you can't do. as such, only
> >> + * strip sub-expressions */
> >> + if (match_arm.has_match_arm_guard ())
> >> + {
> >> + auto &guard_expr = match_arm.get_guard_expr ();
> >> + guard_expr->accept_vis (*this);
> >> + if (guard_expr->is_marked_for_strip ())
> >> + rust_error_at (guard_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> + }
> >> +
> >> + // strip sub-expressions from match cases
> >> + auto &case_expr = match_case.get_expr ();
> >> + case_expr->accept_vis (*this);
> >> + if (case_expr->is_marked_for_strip ())
> >> + rust_error_at (case_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + // increment to next case if haven't continued
> >> + ++it;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::AwaitExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* can't strip awaited expr itself, but can strip sub-expressions
> >> + * - this is because you can't have no expr to await */
> >> + auto &awaited_expr = expr.get_awaited_expr ();
> >> + awaited_expr->accept_vis (*this);
> >> + if (awaited_expr->is_marked_for_strip ())
> >> + rust_error_at (awaited_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::AsyncBlockExpr &expr)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
> >> + {
> >> + expr.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &block_expr = expr.get_block_expr ();
> >> + block_expr->accept_vis (*this);
> >> + if (block_expr->is_marked_for_strip ())
> >> + rust_error_at (block_expr->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::TypeParam ¶m)
> >> +{
> >> + // outer attributes don't actually do anything, so ignore them
> >> +
> >> + if (param.has_type_param_bounds ())
> >> + {
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : param.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> + }
> >> +
> >> + if (param.has_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> + auto &type = param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::LifetimeWhereClauseItem &)
> >> +{
> >> + // shouldn't require
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item)
> >> +{
> >> + // for lifetimes shouldn't require
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = item.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : item.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Method &method)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (method.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (method.get_outer_attrs ()))
> >> + {
> >> + method.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : method.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* assuming you can't strip self param - wouldn't be a method
> >> + * anymore. spec allows outer attrs on self param, but doesn't
> >> + * specify whether cfg is used. */
> >> + expand_self_param (method.get_self_param ());
> >> +
> >> + /* strip method parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_function_params (method.get_function_params ());
> >> +
> >> + if (method.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = method.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + if (method.has_where_clause ())
> >> + expand_where_clause (method.get_where_clause ());
> >> +
> >> + /* body should always exist - if error state, should have returned
> >> + * before now */
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &block_expr = method.get_definition ();
> >> + block_expr->accept_vis (*this);
> >> + if (block_expr->is_marked_for_strip ())
> >> + rust_error_at (block_expr->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Module &module)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (module.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (module.get_outer_attrs ()))
> >> + {
> >> + module.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // A loaded module might have inner attributes
> >> + if (module.get_kind () == AST::Module::ModuleKind::LOADED)
> >> + {
> >> + // strip test based on inner attrs
> >> + expander.expand_cfg_attrs (module.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (module.get_inner_attrs ()))
> >> + {
> >> + module.mark_for_strip ();
> >> + return;
> >> + }
> >> + }
> >> +
> >> + // Parse the module's items if they haven't been expanded and the file
> >> + // should be parsed (i.e isn't hidden behind an untrue or impossible cfg
> >> + // directive)
> >> + if (!module.is_marked_for_strip ()
> >> + && module.get_kind () == AST::Module::ModuleKind::UNLOADED)
> >> + {
> >> + module.load_items ();
> >> + }
> >> +
> >> + // strip items if required
> >> + expand_pointer_allow_strip (module.get_items ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ExternCrate &extern_crate)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (extern_crate.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ()))
> >> + {
> >> + extern_crate.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + if (!extern_crate.references_self ())
> >> + {
> >> + Session &session = Session::get_instance ();
> >> + session.load_extern_crate (extern_crate.get_referenced_crate (),
> >> + extern_crate.get_locus ());
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::UseTreeGlob &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::UseTreeList &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::UseTreeRebind &)
> >> +{
> >> + // shouldn't require?
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::UseDeclaration &use_decl)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (use_decl.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ()))
> >> + {
> >> + use_decl.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Function &function)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (function.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (function.get_outer_attrs ()))
> >> + {
> >> + function.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : function.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* strip function parameters if required - this is specifically
> >> + * allowed by spec */
> >> + expand_function_params (function.get_function_params ());
> >> +
> >> + if (function.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = function.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + if (function.has_where_clause ())
> >> + expand_where_clause (function.get_where_clause ());
> >> +
> >> + /* body should always exist - if error state, should have returned
> >> + * before now */
> >> + // can't strip block itself, but can strip sub-expressions
> >> + auto &block_expr = function.get_definition ();
> >> + block_expr->accept_vis (*this);
> >> + if (block_expr->is_marked_for_strip ())
> >> + rust_error_at (block_expr->get_locus (),
> >> + "cannot strip block expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TypeAlias &type_alias)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (type_alias.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ()))
> >> + {
> >> + type_alias.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : type_alias.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + if (type_alias.has_where_clause ())
> >> + expand_where_clause (type_alias.get_where_clause ());
> >> +
> >> + auto &type = type_alias.get_type_aliased ();
> >> + type->accept_vis (*this);
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructStruct &struct_item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (struct_item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ()))
> >> + {
> >> + struct_item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : struct_item.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + if (struct_item.has_where_clause ())
> >> + expand_where_clause (struct_item.get_where_clause ());
> >> +
> >> + /* strip struct fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_struct_fields (struct_item.get_fields ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleStruct &tuple_struct)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ()))
> >> + {
> >> + tuple_struct.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : tuple_struct.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* strip struct fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_tuple_fields (tuple_struct.get_fields ());
> >> +
> >> + if (tuple_struct.has_where_clause ())
> >> + expand_where_clause (tuple_struct.get_where_clause ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::EnumItem &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::EnumItemTuple &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip item fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_tuple_fields (item.get_tuple_fields ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::EnumItemStruct &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip item fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_struct_fields (item.get_struct_fields ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::EnumItemDiscriminant &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &expr = item.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + rust_error_at (expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Enum &enum_item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (enum_item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ()))
> >> + {
> >> + enum_item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : enum_item.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + if (enum_item.has_where_clause ())
> >> + expand_where_clause (enum_item.get_where_clause ());
> >> +
> >> + /* strip enum fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_pointer_allow_strip (enum_item.get_variants ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Union &union_item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (union_item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ()))
> >> + {
> >> + union_item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : union_item.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + if (union_item.has_where_clause ())
> >> + expand_where_clause (union_item.get_where_clause ());
> >> +
> >> + /* strip union fields if required - this is presumably
> >> + * allowed by spec */
> >> + expand_struct_fields (union_item.get_variants ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ConstantItem &const_item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (const_item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ()))
> >> + {
> >> + const_item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // strip any sub-types
> >> + auto &type = const_item.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &expr = const_item.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + rust_error_at (expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StaticItem &static_item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (static_item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ()))
> >> + {
> >> + static_item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // strip any sub-types
> >> + auto &type = static_item.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &expr = static_item.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + rust_error_at (expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitItemFunc &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expand_trait_function_decl (item.get_trait_function_decl ());
> >> +
> >> + if (item.has_definition ())
> >> + {
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &block = item.get_definition ();
> >> + block->accept_vis (*this);
> >> + if (block->is_marked_for_strip ())
> >> + rust_error_at (block->get_locus (),
> >> + "cannot strip block expression in this "
> >> + "position - outer attributes not allowed");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitItemMethod &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expand_trait_method_decl (item.get_trait_method_decl ());
> >> +
> >> + if (item.has_definition ())
> >> + {
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped. */
> >> + auto &block = item.get_definition ();
> >> + block->accept_vis (*this);
> >> + if (block->is_marked_for_strip ())
> >> + rust_error_at (block->get_locus (),
> >> + "cannot strip block expression in this "
> >> + "position - outer attributes not allowed");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitItemConst &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + // strip any sub-types
> >> + auto &type = item.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped */
> >> + if (item.has_expression ())
> >> + {
> >> + auto &expr = item.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + rust_error_at (expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitItemType &item)
> >> +{
> >> + // initial test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + if (item.has_type_param_bounds ())
> >> + {
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : item.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::Trait &trait)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (trait.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (trait.get_outer_attrs ()))
> >> + {
> >> + trait.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip test based on inner attrs
> >> + expander.expand_cfg_attrs (trait.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (trait.get_inner_attrs ()))
> >> + {
> >> + trait.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : trait.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + if (trait.has_type_param_bounds ())
> >> + {
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : trait.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> + }
> >> +
> >> + if (trait.has_where_clause ())
> >> + expand_where_clause (trait.get_where_clause ());
> >> +
> >> + std::function<std::unique_ptr<AST::TraitItem> (AST::SingleASTNode)> extractor
> >> + = [] (AST::SingleASTNode node) { return node.take_trait_item (); };
> >> +
> >> + expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (),
> >> + extractor);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::InherentImpl &impl)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
> >> + {
> >> + impl.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip test based on inner attrs
> >> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
> >> + {
> >> + impl.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : impl.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = impl.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + if (impl.has_where_clause ())
> >> + expand_where_clause (impl.get_where_clause ());
> >> +
> >> + std::function<std::unique_ptr<AST::InherentImplItem> (AST::SingleASTNode)>
> >> + extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); };
> >> +
> >> + expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (),
> >> + extractor);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitImpl &impl)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
> >> + {
> >> + impl.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip test based on inner attrs
> >> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
> >> + {
> >> + impl.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : impl.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = impl.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + auto &trait_path = impl.get_trait_path ();
> >> + visit (trait_path);
> >> + if (trait_path.is_marked_for_strip ())
> >> + rust_error_at (trait_path.get_locus (),
> >> + "cannot strip typepath in this position");
> >> +
> >> + if (impl.has_where_clause ())
> >> + expand_where_clause (impl.get_where_clause ());
> >> +
> >> + std::function<std::unique_ptr<AST::TraitImplItem> (AST::SingleASTNode)>
> >> + extractor
> >> + = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); };
> >> +
> >> + expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (),
> >> + extractor);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ExternalStaticItem &item)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = item.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (), "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ExternalFunctionItem &item)
> >> +{
> >> + // strip test based on outer attrs
> >> + expander.expand_cfg_attrs (item.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
> >> + {
> >> + item.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // just expand sub-stuff - can't actually strip generic params themselves
> >> + for (auto ¶m : item.get_generic_params ())
> >> + param->accept_vis (*this);
> >> +
> >> + /* strip function parameters if required - this is specifically
> >> + * allowed by spec */
> >> + auto ¶ms = item.get_function_params ();
> >> + for (auto it = params.begin (); it != params.end ();)
> >> + {
> >> + auto ¶m = *it;
> >> +
> >> + auto ¶m_attrs = param.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (param_attrs);
> >> + if (expander.fails_cfg_with_expand (param_attrs))
> >> + {
> >> + it = params.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // increment if nothing else happens
> >> + ++it;
> >> + }
> >> + /* NOTE: these are extern function params, which may have different
> >> + * rules and restrictions to "normal" function params. So expansion
> >> + * handled separately. */
> >> +
> >> + /* TODO: assuming that variadic nature cannot be stripped. If this
> >> + * is not true, then have code here to do so. */
> >> +
> >> + if (item.has_return_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &return_type = item.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (return_type);
> >> +
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + if (item.has_where_clause ())
> >> + expand_where_clause (item.get_where_clause ());
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::ExternBlock &block)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (block.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (block.get_outer_attrs ()))
> >> + {
> >> + block.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip test based on inner attrs
> >> + expander.expand_cfg_attrs (block.get_inner_attrs ());
> >> + if (expander.fails_cfg_with_expand (block.get_inner_attrs ()))
> >> + {
> >> + block.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + std::function<std::unique_ptr<AST::ExternalItem> (AST::SingleASTNode)>
> >> + extractor
> >> + = [] (AST::SingleASTNode node) { return node.take_external_item (); };
> >> +
> >> + expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (),
> >> + extractor);
> >> +}
> >> +
> >> +// I don't think it would be possible to strip macros without expansion
> >> +void
> >> +AttrVisitor::visit (AST::MacroMatchFragment &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MacroMatchRepetition &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MacroMatcher &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MacroRulesDefinition &rules_def)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (rules_def.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ()))
> >> + {
> >> + rules_def.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // I don't think any macro rules can be stripped in any way
> >> +
> >> + auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (),
> >> + rules_def.get_rule_name ());
> >> + expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (),
> >> + rules_def.get_locus ());
> >> + expander.mappings->insert_macro_def (&rules_def);
> >> + rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]",
> >> + path.get ().c_str ());
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::MetaItemPath &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MetaItemSeq &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MetaWord &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MetaNameValueStr &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MetaListPaths &)
> >> +{}
> >> +void
> >> +AttrVisitor::visit (AST::MetaListNameValueStr &)
> >> +{}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::LiteralPattern &)
> >> +{
> >> + // not possible
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::IdentifierPattern &pattern)
> >> +{
> >> + // can only strip sub-patterns of the inner pattern to bind
> >> + if (!pattern.has_pattern_to_bind ())
> >> + return;
> >> +
> >> + auto &sub_pattern = pattern.get_pattern_to_bind ();
> >> + sub_pattern->accept_vis (*this);
> >> + if (sub_pattern->is_marked_for_strip ())
> >> + rust_error_at (sub_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::WildcardPattern &)
> >> +{
> >> + // not possible
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangePatternBoundLiteral &)
> >> +{
> >> + // not possible
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangePatternBoundPath &bound)
> >> +{
> >> + // can expand path, but not strip it directly
> >> + auto &path = bound.get_path ();
> >> + visit (path);
> >> + if (path.is_marked_for_strip ())
> >> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangePatternBoundQualPath &bound)
> >> +{
> >> + // can expand path, but not strip it directly
> >> + auto &path = bound.get_qualified_path ();
> >> + visit (path);
> >> + if (path.is_marked_for_strip ())
> >> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RangePattern &pattern)
> >> +{
> >> + // should have no capability to strip lower or upper bounds, only expand
> >> + pattern.get_lower_bound ()->accept_vis (*this);
> >> + pattern.get_upper_bound ()->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ReferencePattern &pattern)
> >> +{
> >> + auto &sub_pattern = pattern.get_referenced_pattern ();
> >> + sub_pattern->accept_vis (*this);
> >> + if (sub_pattern->is_marked_for_strip ())
> >> + rust_error_at (sub_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructPatternFieldTuplePat &field)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> >> + {
> >> + field.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-patterns (can't strip top-level pattern)
> >> + auto &sub_pattern = field.get_index_pattern ();
> >> + sub_pattern->accept_vis (*this);
> >> + if (sub_pattern->is_marked_for_strip ())
> >> + rust_error_at (sub_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructPatternFieldIdentPat &field)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> >> + {
> >> + field.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // strip sub-patterns (can't strip top-level pattern)
> >> + auto &sub_pattern = field.get_ident_pattern ();
> >> + sub_pattern->accept_vis (*this);
> >> + if (sub_pattern->is_marked_for_strip ())
> >> + rust_error_at (sub_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructPatternFieldIdent &field)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (field.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
> >> + {
> >> + field.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::StructPattern &pattern)
> >> +{
> >> + // expand (but don't strip) path
> >> + auto &path = pattern.get_path ();
> >> + visit (path);
> >> + if (path.is_marked_for_strip ())
> >> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> >> +
> >> + /* TODO: apparently struct pattern fields can have outer attrs. so can they
> >> + * be stripped? */
> >> + if (!pattern.has_struct_pattern_elems ())
> >> + return;
> >> +
> >> + auto &elems = pattern.get_struct_pattern_elems ();
> >> +
> >> + // assuming you can strip struct pattern fields
> >> + expand_pointer_allow_strip (elems.get_struct_pattern_fields ());
> >> +
> >> + // assuming you can strip the ".." part
> >> + if (elems.has_etc ())
> >> + {
> >> + expander.expand_cfg_attrs (elems.get_etc_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ()))
> >> + elems.strip_etc ();
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items)
> >> +{
> >> + // can't strip individual patterns, only sub-patterns
> >> + for (auto &pattern : tuple_items.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> +
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items)
> >> +{
> >> + // can't strip individual patterns, only sub-patterns
> >> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
> >> + {
> >> + lower_pattern->accept_vis (*this);
> >> +
> >> + if (lower_pattern->is_marked_for_strip ())
> >> + rust_error_at (lower_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
> >> + {
> >> + upper_pattern->accept_vis (*this);
> >> +
> >> + if (upper_pattern->is_marked_for_strip ())
> >> + rust_error_at (upper_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleStructPattern &pattern)
> >> +{
> >> + // expand (but don't strip) path
> >> + auto &path = pattern.get_path ();
> >> + visit (path);
> >> + if (path.is_marked_for_strip ())
> >> + rust_error_at (path.get_locus (), "cannot strip path in this position");
> >> +
> >> + if (pattern.has_items ())
> >> + pattern.get_items ()->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items)
> >> +{
> >> + // can't strip individual patterns, only sub-patterns
> >> + for (auto &pattern : tuple_items.get_patterns ())
> >> + {
> >> + pattern->accept_vis (*this);
> >> +
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items)
> >> +{
> >> + // can't strip individual patterns, only sub-patterns
> >> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
> >> + {
> >> + lower_pattern->accept_vis (*this);
> >> +
> >> + if (lower_pattern->is_marked_for_strip ())
> >> + rust_error_at (lower_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
> >> + {
> >> + upper_pattern->accept_vis (*this);
> >> +
> >> + if (upper_pattern->is_marked_for_strip ())
> >> + rust_error_at (upper_pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TuplePattern &pattern)
> >> +{
> >> + if (pattern.has_tuple_pattern_items ())
> >> + pattern.get_items ()->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::GroupedPattern &pattern)
> >> +{
> >> + // can't strip inner pattern, only sub-patterns
> >> + auto &pattern_in_parens = pattern.get_pattern_in_parens ();
> >> +
> >> + pattern_in_parens->accept_vis (*this);
> >> +
> >> + if (pattern_in_parens->is_marked_for_strip ())
> >> + rust_error_at (pattern_in_parens->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::SlicePattern &pattern)
> >> +{
> >> + // can't strip individual patterns, only sub-patterns
> >> + for (auto &item : pattern.get_items ())
> >> + {
> >> + item->accept_vis (*this);
> >> +
> >> + if (item->is_marked_for_strip ())
> >> + rust_error_at (item->get_locus (),
> >> + "cannot strip pattern in this position");
> >> + // TODO: quit stripping now? or keep going?
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::EmptyStmt &)
> >> +{
> >> + // assuming no outer attributes, so nothing can happen
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::LetStmt &stmt)
> >> +{
> >> + // initial strip test based on outer attrs
> >> + expander.expand_cfg_attrs (stmt.get_outer_attrs ());
> >> + if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ()))
> >> + {
> >> + stmt.mark_for_strip ();
> >> + return;
> >> + }
> >> +
> >> + // can't strip pattern, but call for sub-patterns
> >> + auto &pattern = stmt.get_pattern ();
> >> + pattern->accept_vis (*this);
> >> + if (pattern->is_marked_for_strip ())
> >> + rust_error_at (pattern->get_locus (),
> >> + "cannot strip pattern in this position");
> >> +
> >> + // similar for type
> >> + if (stmt.has_type ())
> >> + {
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = stmt.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + /* strip any internal sub-expressions - expression itself isn't
> >> + * allowed to have external attributes in this position so can't be
> >> + * stripped */
> >> + if (stmt.has_init_expr ())
> >> + {
> >> + auto &init_expr = stmt.get_init_expr ();
> >> + init_expr->accept_vis (*this);
> >> +
> >> + if (init_expr->is_marked_for_strip ())
> >> + rust_error_at (init_expr->get_locus (),
> >> + "cannot strip expression in this position - outer "
> >> + "attributes not allowed");
> >> +
> >> + maybe_expand_expr (init_expr);
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt)
> >> +{
> >> + // outer attributes associated with expr, so rely on expr
> >> +
> >> + // guard - should prevent null pointer expr
> >> + if (stmt.is_marked_for_strip ())
> >> + return;
> >> +
> >> + // strip if expr is to be stripped
> >> + auto &expr = stmt.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + {
> >> + stmt.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ExprStmtWithBlock &stmt)
> >> +{
> >> + // outer attributes associated with expr, so rely on expr
> >> +
> >> + // guard - should prevent null pointer expr
> >> + if (stmt.is_marked_for_strip ())
> >> + return;
> >> +
> >> + // strip if expr is to be stripped
> >> + auto &expr = stmt.get_expr ();
> >> + expr->accept_vis (*this);
> >> + if (expr->is_marked_for_strip ())
> >> + {
> >> + stmt.mark_for_strip ();
> >> + return;
> >> + }
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::visit (AST::TraitBound &bound)
> >> +{
> >> + // nothing in for lifetimes to strip
> >> +
> >> + // expand but don't strip type path
> >> + auto &path = bound.get_type_path ();
> >> + visit (path);
> >> + if (path.is_marked_for_strip ())
> >> + rust_error_at (path.get_locus (),
> >> + "cannot strip type path in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ImplTraitType &type)
> >> +{
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : type.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitObjectType &type)
> >> +{
> >> + // don't strip directly, only components of bounds
> >> + for (auto &bound : type.get_type_param_bounds ())
> >> + bound->accept_vis (*this);
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ParenthesisedType &type)
> >> +{
> >> + // expand but don't strip inner type
> >> + auto &inner_type = type.get_type_in_parens ();
> >> + inner_type->accept_vis (*this);
> >> + if (inner_type->is_marked_for_strip ())
> >> + rust_error_at (inner_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ImplTraitTypeOneBound &type)
> >> +{
> >> + // no stripping possible
> >> + visit (type.get_trait_bound ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TraitObjectTypeOneBound &type)
> >> +{
> >> + // no stripping possible
> >> + visit (type.get_trait_bound ());
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::TupleType &type)
> >> +{
> >> + // TODO: assuming that types can't be stripped as types don't have outer
> >> + // attributes
> >> + for (auto &elem_type : type.get_elems ())
> >> + {
> >> + elem_type->accept_vis (*this);
> >> + if (elem_type->is_marked_for_strip ())
> >> + rust_error_at (elem_type->get_locus (),
> >> + "cannot strip type in this position");
> >> + }
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::NeverType &)
> >> +{
> >> + // no stripping possible
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::RawPointerType &type)
> >> +{
> >> + // expand but don't strip type pointed to
> >> + auto &pointed_type = type.get_type_pointed_to ();
> >> + pointed_type->accept_vis (*this);
> >> + if (pointed_type->is_marked_for_strip ())
> >> + rust_error_at (pointed_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ReferenceType &type)
> >> +{
> >> + // expand but don't strip type referenced
> >> + auto &referenced_type = type.get_type_referenced ();
> >> + referenced_type->accept_vis (*this);
> >> + if (referenced_type->is_marked_for_strip ())
> >> + rust_error_at (referenced_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::ArrayType &type)
> >> +{
> >> + // expand but don't strip type referenced
> >> + auto &base_type = type.get_elem_type ();
> >> + base_type->accept_vis (*this);
> >> + if (base_type->is_marked_for_strip ())
> >> + rust_error_at (base_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + // same for expression
> >> + auto &size_expr = type.get_size_expr ();
> >> + size_expr->accept_vis (*this);
> >> + if (size_expr->is_marked_for_strip ())
> >> + rust_error_at (size_expr->get_locus (),
> >> + "cannot strip expression in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::SliceType &type)
> >> +{
> >> + // expand but don't strip elem type
> >> + auto &elem_type = type.get_elem_type ();
> >> + elem_type->accept_vis (*this);
> >> + if (elem_type->is_marked_for_strip ())
> >> + rust_error_at (elem_type->get_locus (),
> >> + "cannot strip type in this position");
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::InferredType &)
> >> +{
> >> + // none possible
> >> +}
> >> +void
> >> +AttrVisitor::visit (AST::BareFunctionType &type)
> >> +{
> >> + // seem to be no generics
> >> +
> >> + // presumably function params can be stripped
> >> + auto ¶ms = type.get_function_params ();
> >> + for (auto it = params.begin (); it != params.end ();)
> >> + {
> >> + auto ¶m = *it;
> >> +
> >> + auto ¶m_attrs = param.get_outer_attrs ();
> >> + expander.expand_cfg_attrs (param_attrs);
> >> + if (expander.fails_cfg_with_expand (param_attrs))
> >> + {
> >> + it = params.erase (it);
> >> + continue;
> >> + }
> >> +
> >> + expander.push_context (MacroExpander::ContextType::TYPE);
> >> +
> >> + auto &type = param.get_type ();
> >> + type->accept_vis (*this);
> >> +
> >> + maybe_expand_type (type);
> >> +
> >> + if (type->is_marked_for_strip ())
> >> + rust_error_at (type->get_locus (),
> >> + "cannot strip type in this position");
> >> +
> >> + expander.pop_context ();
> >> +
> >> + // increment if nothing else happens
> >> + ++it;
> >> + }
> >> +
> >> + /* TODO: assuming that variadic nature cannot be stripped. If this
> >> + * is not true, then have code here to do so. */
> >> +
> >> + if (type.has_return_type ())
> >> + {
> >> + // FIXME: Can we have type expansion in this position?
> >> + // In that case, we need to handle AST::TypeNoBounds on top of just
> >> + // AST::Types
> >> + auto &return_type = type.get_return_type ();
> >> + return_type->accept_vis (*this);
> >> + if (return_type->is_marked_for_strip ())
> >> + rust_error_at (return_type->get_locus (),
> >> + "cannot strip type in this position");
> >> + }
> >> +
> >> + // no where clause, apparently
> >> +}
> >> +void
> >> +AttrVisitor::maybe_expand_expr (std::unique_ptr<AST::Expr> &expr)
> >> +{
> >> + auto final_fragment = expand_macro_fragment_recursive ();
> >> + if (final_fragment.should_expand ())
> >> + expr = final_fragment.take_expression_fragment ();
> >> +}
> >> +
> >> +void
> >> +AttrVisitor::maybe_expand_type (std::unique_ptr<AST::Type> &type)
> >> +{
> >> + auto final_fragment = expand_macro_fragment_recursive ();
> >> + if (final_fragment.should_expand ())
> >> + type = final_fragment.take_type_fragment ();
> >> +}
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-attribute-visitor.h b/gcc/rust/expand/rust-attribute-visitor.h
> >> new file mode 100644
> >> index 00000000000..0f9d1065334
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-attribute-visitor.h
> >> @@ -0,0 +1,316 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#include "rust-ast-visitor.h"
> >> +#include "rust-ast.h"
> >> +#include "rust-macro-expand.h"
> >> +
> >> +namespace Rust {
> >> +// Visitor used to expand attributes.
> >> +class AttrVisitor : public AST::ASTVisitor
> >> +{
> >> +private:
> >> + MacroExpander &expander;
> >> + void maybe_expand_expr (std::unique_ptr<AST::Expr> &expr);
> >> + void maybe_expand_type (std::unique_ptr<AST::Type> &expr);
> >> +
> >> +public:
> >> + AttrVisitor (MacroExpander &expander) : expander (expander) {}
> >> +
> >> + void expand_struct_fields (std::vector<AST::StructField> &fields);
> >> + void expand_tuple_fields (std::vector<AST::TupleField> &fields);
> >> + void expand_function_params (std::vector<AST::FunctionParam> ¶ms);
> >> + void expand_generic_args (AST::GenericArgs &args);
> >> + void expand_qualified_path_type (AST::QualifiedPathType &path_type);
> >> + void expand_closure_params (std::vector<AST::ClosureParam> ¶ms);
> >> + void expand_self_param (AST::SelfParam &self_param);
> >> + void expand_where_clause (AST::WhereClause &where_clause);
> >> + void expand_trait_function_decl (AST::TraitFunctionDecl &decl);
> >> + void expand_trait_method_decl (AST::TraitMethodDecl &decl);
> >> +
> >> + /**
> >> + * Expand The current macro fragment recursively until it could not be
> >> + * expanded further.
> >> + *
> >> + * The return value checking works because correctly
> >> + * expanded fragment can never be an error (if the fragment can not be
> >> + * expanded, a stand-in error fragment will be returned; for fragments that
> >> + * could not be further expanded: the fragment prior to the expansion failure
> >> + * will be returned).
> >> + *
> >> + * @return Either the expanded fragment or an empty errored-out fragment
> >> + * indicating an expansion failure.
> >> + */
> >> + AST::ASTFragment expand_macro_fragment_recursive ()
> >
> > Why is this in the header?
>
> Do you mean the definition or the documentation? If you refer to the
> definition, then that is a mistake and should be in the implementation.
Yep, the definition.
> >> + {
> >> + auto fragment = expander.take_expanded_fragment (*this);
> >> + unsigned int original_depth = expander.expansion_depth;
> >> + auto final_fragment = AST::ASTFragment ({}, true);
> >> +
> >> + while (fragment.should_expand ())
> >> + {
> >> + final_fragment = std::move (fragment);
> >> + expander.expansion_depth++;
> >> + // further expand the previously expanded macro fragment
> >> + auto new_fragment = expander.take_expanded_fragment (*this);
> >> + if (new_fragment.is_error ())
> >> + break;
> >> + fragment = std::move (new_fragment);
> >> + }
> >> + expander.expansion_depth = original_depth;
> >> + return final_fragment;
> >> + }
> >> +
> >> + /**
> >> + * Expand a set of values, erasing them if they are marked for strip, and
> >> + * replacing them with expanded macro nodes if necessary.
> >> + * This function is slightly different from `expand_pointer_allow_strip` as
> >> + * it can only be called in certain expansion contexts - where macro
> >> + * invocations are allowed.
> >> + *
> >> + * @param ctx Context to use for macro expansion
> >> + * @param values Iterable reference over values to replace or erase
> >> + * @param extractor Function to call when replacing values with the content
> >> + * of an expanded AST node
> >> + */
> >> + template <typename T, typename U>
> >> + void expand_macro_children (MacroExpander::ContextType ctx, T &values,
> >> + std::function<U (AST::SingleASTNode)> extractor)
> >> + {
> >> + expander.push_context (ctx);
> >> +
> >> + for (auto it = values.begin (); it != values.end ();)
> >> + {
> >> + auto &value = *it;
> >> +
> >> + // mark for stripping if required
> >> + value->accept_vis (*this);
> >> +
> >> + // recursively expand the children
> >> + auto final_fragment = expand_macro_fragment_recursive ();
> >> +
> >> + if (final_fragment.should_expand ())
> >> + {
> >> + it = values.erase (it);
> >> + for (auto &node : final_fragment.get_nodes ())
> >> + {
> >> + auto new_node = extractor (node);
> >> + if (new_node != nullptr && !new_node->is_marked_for_strip ())
> >> + {
> >> + it = values.insert (it, std::move (new_node));
> >> + it++;
> >> + }
> >> + }
> >> + }
> >> + else if (value->is_marked_for_strip ())
> >> + {
> >> + it = values.erase (it);
> >> + }
> >> + else
> >> + {
> >> + ++it;
> >> + }
> >> + }
> >> +
> >> + expander.pop_context ();
> >> + }
> >> +
> >> + template <typename T> void expand_pointer_allow_strip (T &values)
> >> + {
> >> + for (auto it = values.begin (); it != values.end ();)
> >> + {
> >> + auto &value = *it;
> >> +
> >> + // mark for stripping if required
> >> + value->accept_vis (*this);
> >> + if (value->is_marked_for_strip ())
> >> + {
> >> + it = values.erase (it);
> >> + }
> >> + else
> >> + {
> >> + ++it;
> >> + }
> >> + }
> >> + }
> >> +
> >> + void visit (AST::Token &) override;
> >> + void visit (AST::DelimTokenTree &) override;
> >> + void visit (AST::AttrInputMetaItemContainer &) override;
> >> + void visit (AST::IdentifierExpr &ident_expr) override;
> >> + void visit (AST::Lifetime &) override;
> >> + void visit (AST::LifetimeParam &) override;
> >> + void visit (AST::ConstGenericParam &) override;
> >> +
> >> + void visit (AST::MacroInvocation ¯o_invoc) override;
> >> +
> >> + void visit (AST::PathInExpression &path) override;
> >> + void visit (AST::TypePathSegment &) override;
> >> + void visit (AST::TypePathSegmentGeneric &segment) override;
> >> + void visit (AST::TypePathSegmentFunction &segment) override;
> >> + void visit (AST::TypePath &path) override;
> >> + void visit (AST::QualifiedPathInExpression &path) override;
> >> + void visit (AST::QualifiedPathInType &path) override;
> >> +
> >> + void visit (AST::LiteralExpr &expr) override;
> >> + void visit (AST::AttrInputLiteral &) override;
> >> + void visit (AST::MetaItemLitExpr &) override;
> >> + void visit (AST::MetaItemPathLit &) override;
> >> + void visit (AST::BorrowExpr &expr) override;
> >> + void visit (AST::DereferenceExpr &expr) override;
> >> + void visit (AST::ErrorPropagationExpr &expr) override;
> >> + void visit (AST::NegationExpr &expr) override;
> >> + void visit (AST::ArithmeticOrLogicalExpr &expr) override;
> >> + void visit (AST::ComparisonExpr &expr) override;
> >> + void visit (AST::LazyBooleanExpr &expr) override;
> >> + void visit (AST::TypeCastExpr &expr) override;
> >> + void visit (AST::AssignmentExpr &expr) override;
> >> + void visit (AST::CompoundAssignmentExpr &expr) override;
> >> + void visit (AST::GroupedExpr &expr) override;
> >> + void visit (AST::ArrayElemsValues &elems) override;
> >> + void visit (AST::ArrayElemsCopied &elems) override;
> >> + void visit (AST::ArrayExpr &expr) override;
> >> + void visit (AST::ArrayIndexExpr &expr) override;
> >> + void visit (AST::TupleExpr &expr) override;
> >> + void visit (AST::TupleIndexExpr &expr) override;
> >> + void visit (AST::StructExprStruct &expr) override;
> >> + void visit (AST::StructExprFieldIdentifier &) override;
> >> + void visit (AST::StructExprFieldIdentifierValue &field) override;
> >> +
> >> + void visit (AST::StructExprFieldIndexValue &field) override;
> >> + void visit (AST::StructExprStructFields &expr) override;
> >> + void visit (AST::StructExprStructBase &expr) override;
> >> + void visit (AST::CallExpr &expr) override;
> >> + void visit (AST::MethodCallExpr &expr) override;
> >> + void visit (AST::FieldAccessExpr &expr) override;
> >> + void visit (AST::ClosureExprInner &expr) override;
> >> +
> >> + void visit (AST::BlockExpr &expr) override;
> >> +
> >> + void visit (AST::ClosureExprInnerTyped &expr) override;
> >> + void visit (AST::ContinueExpr &expr) override;
> >> + void visit (AST::BreakExpr &expr) override;
> >> + void visit (AST::RangeFromToExpr &expr) override;
> >> + void visit (AST::RangeFromExpr &expr) override;
> >> + void visit (AST::RangeToExpr &expr) override;
> >> + void visit (AST::RangeFullExpr &) override;
> >> + void visit (AST::RangeFromToInclExpr &expr) override;
> >> + void visit (AST::RangeToInclExpr &expr) override;
> >> + void visit (AST::ReturnExpr &expr) override;
> >> + void visit (AST::UnsafeBlockExpr &expr) override;
> >> + void visit (AST::LoopExpr &expr) override;
> >> + void visit (AST::WhileLoopExpr &expr) override;
> >> + void visit (AST::WhileLetLoopExpr &expr) override;
> >> + void visit (AST::ForLoopExpr &expr) override;
> >> + void visit (AST::IfExpr &expr) override;
> >> + void visit (AST::IfExprConseqElse &expr) override;
> >> + void visit (AST::IfExprConseqIf &expr) override;
> >> + void visit (AST::IfExprConseqIfLet &expr) override;
> >> + void visit (AST::IfLetExpr &expr) override;
> >> + void visit (AST::IfLetExprConseqElse &expr) override;
> >> + void visit (AST::IfLetExprConseqIf &expr) override;
> >> + void visit (AST::IfLetExprConseqIfLet &expr) override;
> >> + void visit (AST::MatchExpr &expr) override;
> >> + void visit (AST::AwaitExpr &expr) override;
> >> + void visit (AST::AsyncBlockExpr &expr) override;
> >> + void visit (AST::TypeParam ¶m) override;
> >> + void visit (AST::LifetimeWhereClauseItem &) override;
> >> + void visit (AST::TypeBoundWhereClauseItem &item) override;
> >> + void visit (AST::Method &method) override;
> >> + void visit (AST::Module &module) override;
> >> + void visit (AST::ExternCrate &crate) override;
> >> + void visit (AST::UseTreeGlob &) override;
> >> + void visit (AST::UseTreeList &) override;
> >> + void visit (AST::UseTreeRebind &) override;
> >> + void visit (AST::UseDeclaration &use_decl) override;
> >> + void visit (AST::Function &function) override;
> >> + void visit (AST::TypeAlias &type_alias) override;
> >> + void visit (AST::StructStruct &struct_item) override;
> >> + void visit (AST::TupleStruct &tuple_struct) override;
> >> + void visit (AST::EnumItem &item) override;
> >> + void visit (AST::EnumItemTuple &item) override;
> >> + void visit (AST::EnumItemStruct &item) override;
> >> + void visit (AST::EnumItemDiscriminant &item) override;
> >> + void visit (AST::Enum &enum_item) override;
> >> + void visit (AST::Union &union_item) override;
> >> + void visit (AST::ConstantItem &const_item) override;
> >> + void visit (AST::StaticItem &static_item) override;
> >> + void visit (AST::TraitItemFunc &item) override;
> >> + void visit (AST::TraitItemMethod &item) override;
> >> + void visit (AST::TraitItemConst &item) override;
> >> + void visit (AST::TraitItemType &item) override;
> >> + void visit (AST::Trait &trait) override;
> >> + void visit (AST::InherentImpl &impl) override;
> >> + void visit (AST::TraitImpl &impl) override;
> >> + void visit (AST::ExternalStaticItem &item) override;
> >> + void visit (AST::ExternalFunctionItem &item) override;
> >> + void visit (AST::ExternBlock &block) override;
> >> +
> >> + // I don't think it would be possible to strip macros without expansion
> >> + void visit (AST::MacroMatchFragment &) override;
> >> + void visit (AST::MacroMatchRepetition &) override;
> >> + void visit (AST::MacroMatcher &) override;
> >> + void visit (AST::MacroRulesDefinition &rules_def) override;
> >> + void visit (AST::MetaItemPath &) override;
> >> + void visit (AST::MetaItemSeq &) override;
> >> + void visit (AST::MetaWord &) override;
> >> + void visit (AST::MetaNameValueStr &) override;
> >> + void visit (AST::MetaListPaths &) override;
> >> + void visit (AST::MetaListNameValueStr &) override;
> >> + void visit (AST::LiteralPattern &) override;
> >> + void visit (AST::IdentifierPattern &pattern) override;
> >> + void visit (AST::WildcardPattern &) override;
> >> + void visit (AST::RangePatternBoundLiteral &) override;
> >> + void visit (AST::RangePatternBoundPath &bound) override;
> >> + void visit (AST::RangePatternBoundQualPath &bound) override;
> >> + void visit (AST::RangePattern &pattern) override;
> >> + void visit (AST::ReferencePattern &pattern) override;
> >> + void visit (AST::StructPatternFieldTuplePat &field) override;
> >> + void visit (AST::StructPatternFieldIdentPat &field) override;
> >> + void visit (AST::StructPatternFieldIdent &field) override;
> >> + void visit (AST::StructPattern &pattern) override;
> >> + void visit (AST::TupleStructItemsNoRange &tuple_items) override;
> >> + void visit (AST::TupleStructItemsRange &tuple_items) override;
> >> + void visit (AST::TupleStructPattern &pattern) override;
> >> + void visit (AST::TuplePatternItemsMultiple &tuple_items) override;
> >> + void visit (AST::TuplePatternItemsRanged &tuple_items) override;
> >> + void visit (AST::TuplePattern &pattern) override;
> >> + void visit (AST::GroupedPattern &pattern) override;
> >> + void visit (AST::SlicePattern &pattern) override;
> >> +
> >> + void visit (AST::EmptyStmt &) override;
> >> + void visit (AST::LetStmt &stmt) override;
> >> + void visit (AST::ExprStmtWithoutBlock &stmt) override;
> >> + void visit (AST::ExprStmtWithBlock &stmt) override;
> >> +
> >> + void visit (AST::TraitBound &bound) override;
> >> + void visit (AST::ImplTraitType &type) override;
> >> + void visit (AST::TraitObjectType &type) override;
> >> + void visit (AST::ParenthesisedType &type) override;
> >> + void visit (AST::ImplTraitTypeOneBound &type) override;
> >> + void visit (AST::TraitObjectTypeOneBound &type) override;
> >> + void visit (AST::TupleType &type) override;
> >> + void visit (AST::NeverType &) override;
> >> + void visit (AST::RawPointerType &type) override;
> >> + void visit (AST::ReferenceType &type) override;
> >> + void visit (AST::ArrayType &type) override;
> >> + void visit (AST::SliceType &type) override;
> >> + void visit (AST::InferredType &) override;
> >> + void visit (AST::BareFunctionType &type) override;
> >> +};
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-macro-builtins.cc b/gcc/rust/expand/rust-macro-builtins.cc
> >> new file mode 100644
> >> index 00000000000..5eace13d197
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-builtins.cc
> >> @@ -0,0 +1,484 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#include "rust-macro-builtins.h"
> >> +#include "rust-diagnostics.h"
> >> +#include "rust-expr.h"
> >> +#include "rust-session-manager.h"
> >> +#include "rust-macro-invoc-lexer.h"
> >> +#include "rust-lex.h"
> >> +#include "rust-parse.h"
> >> +
> >> +namespace Rust {
> >> +namespace {
> >> +std::unique_ptr<AST::Expr>
> >> +make_string (Location locus, std::string value)
> >> +{
> >> + return std::unique_ptr<AST::Expr> (
> >> + new AST::LiteralExpr (value, AST::Literal::STRING,
> >> + PrimitiveCoreType::CORETYPE_STR, {}, locus));
> >> +}
> >> +
> >> +/* Match the end token of a macro given the start delimiter of the macro */
> >> +
> >> +static inline TokenId
> >> +macro_end_token (AST::DelimTokenTree &invoc_token_tree,
> >> + Parser<MacroInvocLexer> &parser)
> >> +{
> >> + auto last_token_id = TokenId::RIGHT_CURLY;
> >> + switch (invoc_token_tree.get_delim_type ())
> >> + {
> >> + case AST::DelimType::PARENS:
> >> + last_token_id = TokenId::RIGHT_PAREN;
> >> + rust_assert (parser.skip_token (LEFT_PAREN));
> >> + break;
> >> +
> >> + case AST::DelimType::CURLY:
> >> + rust_assert (parser.skip_token (LEFT_CURLY));
> >> + break;
> >> +
> >> + case AST::DelimType::SQUARE:
> >> + last_token_id = TokenId::RIGHT_SQUARE;
> >> + rust_assert (parser.skip_token (LEFT_SQUARE));
> >> + break;
> >> + }
> >> +
> >> + return last_token_id;
> >> +}
> >> +
> >> +/* Parse a single string literal from the given delimited token tree,
> >> + and return the LiteralExpr for it. Allow for an optional trailing comma,
> >> + but otherwise enforce that these are the only tokens. */
> >> +
> >> +std::unique_ptr<AST::LiteralExpr>
> >> +parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree,
> >> + Location invoc_locus)
> >> +{
> >> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> >> + Parser<MacroInvocLexer> parser (lex);
> >> +
> >> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> >> +
> >> + std::unique_ptr<AST::LiteralExpr> lit_expr = nullptr;
> >> +
> >> + if (parser.peek_current_token ()->get_id () == STRING_LITERAL)
> >> + {
> >> + lit_expr = parser.parse_literal_expr ();
> >> + parser.maybe_skip_token (COMMA);
> >> + if (parser.peek_current_token ()->get_id () != last_token_id)
> >> + {
> >> + lit_expr = nullptr;
> >> + rust_error_at (invoc_locus, "macro takes 1 argument");
> >> + }
> >> + }
> >> + else if (parser.peek_current_token ()->get_id () == last_token_id)
> >> + rust_error_at (invoc_locus, "macro takes 1 argument");
> >> + else
> >> + rust_error_at (invoc_locus, "argument must be a string literal");
> >> +
> >> + parser.skip_token (last_token_id);
> >> +
> >> + return lit_expr;
> >> +}
> >> +
> >> +/* Treat PATH as a path relative to the source file currently being
> >> + compiled, and return the absolute path for it. */
> >> +
> >> +std::string
> >> +source_relative_path (std::string path, Location locus)
> >> +{
> >> + std::string compile_fname
> >> + = Session::get_instance ().linemap->location_file (locus);
> >> +
> >> + auto dir_separator_pos = compile_fname.rfind (file_separator);
> >> +
> >> + /* If there is no file_separator in the path, use current dir ('.'). */
> >> + std::string dirname;
> >> + if (dir_separator_pos == std::string::npos)
> >> + dirname = std::string (".") + file_separator;
> >> + else
> >> + dirname = compile_fname.substr (0, dir_separator_pos) + file_separator;
> >> +
> >> + return dirname + path;
> >> +}
> >> +
> >> +/* Read the full contents of the file FILENAME and return them in a vector.
> >> + FIXME: platform specific. */
> >> +
> >> +std::vector<uint8_t>
> >> +load_file_bytes (const char *filename)
> >> +{
> >> + RAIIFile file_wrap (filename);
> >> + if (file_wrap.get_raw () == nullptr)
> >> + {
> >> + rust_error_at (Location (), "cannot open filename %s: %m", filename);
> >> + return std::vector<uint8_t> ();
> >> + }
> >> +
> >> + FILE *f = file_wrap.get_raw ();
> >> + fseek (f, 0L, SEEK_END);
> >> + long fsize = ftell (f);
> >> + fseek (f, 0L, SEEK_SET);
> >> +
> >> + std::vector<uint8_t> buf (fsize);
> >> +
> >> + if (fread (&buf[0], fsize, 1, f) != 1)
> >> + {
> >> + rust_error_at (Location (), "error reading file %s: %m", filename);
> >> + return std::vector<uint8_t> ();
> >> + }
> >> +
> >> + return buf;
> >> +}
> >> +} // namespace
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + rust_debug ("assert!() called");
> >> +
> >> + return AST::ASTFragment::create_error ();
> >> +}
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto current_file
> >> + = Session::get_instance ().linemap->location_file (invoc_locus);
> >> + auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file));
> >> +
> >> + return AST::ASTFragment ({file_str});
> >> +}
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto current_column
> >> + = Session::get_instance ().linemap->location_to_column (invoc_locus);
> >> +
> >> + auto column_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> >> + new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT,
> >> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
> >> +
> >> + return AST::ASTFragment ({column_no});
> >> +}
> >> +
> >> +/* Expand builtin macro include_bytes!("filename"), which includes the contents
> >> + of the given file as reference to a byte array. Yields an expression of type
> >> + &'static [u8; N]. */
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + /* Get target filename from the macro invocation, which is treated as a path
> >> + relative to the include!-ing file (currently being compiled). */
> >> + auto lit_expr
> >> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> >> + if (lit_expr == nullptr)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + std::string target_filename
> >> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> >> +
> >> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
> >> +
> >> + /* Is there a more efficient way to do this? */
> >
> > Well, first of all you don't need to read all of the file at once, second you
> > could pre-allocate 'elts' to the correct size and final you could obviously
> > choose a cheaper representation ;)
> >
> >> + std::vector<std::unique_ptr<AST::Expr>> elts;
> >> + for (uint8_t b : bytes)
> >> + {
> >> + elts.emplace_back (
> >> + new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE,
> >> + PrimitiveCoreType::CORETYPE_U8,
> >> + {} /* outer_attrs */, invoc_locus));
> >> + }
> >> +
> >> + auto elems = std::unique_ptr<AST::ArrayElems> (
> >> + new AST::ArrayElemsValues (std::move (elts), invoc_locus));
> >> +
> >> + auto array = std::unique_ptr<AST::Expr> (
> >> + new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus));
> >> +
> >> + auto borrow = std::unique_ptr<AST::Expr> (
> >> + new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus));
> >> +
> >> + auto node = AST::SingleASTNode (std::move (borrow));
> >> + return AST::ASTFragment ({node});
> >> +}
> >> +
> >> +/* Expand builtin macro include_str!("filename"), which includes the contents
> >> + of the given file as a string. The file must be UTF-8 encoded. Yields an
> >> + expression of type &'static str. */
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + /* Get target filename from the macro invocation, which is treated as a path
> >> + relative to the include!-ing file (currently being compiled). */
> >> + auto lit_expr
> >> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> >> + if (lit_expr == nullptr)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + std::string target_filename
> >> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> >> +
> >> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
> >> +
> >> + /* FIXME: Enforce that the file contents are valid UTF-8. */
> >> + std::string str ((const char *) &bytes[0], bytes.size ());
> >> +
> >> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
> >> + return AST::ASTFragment ({node});
> >> +}
> >> +
> >> +/* Expand builtin macro compile_error!("error"), which forces a compile error
> >> + during the compile time. */
> >> +AST::ASTFragment
> >> +MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto lit_expr
> >> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> >> + if (lit_expr == nullptr)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + std::string error_string = lit_expr->as_string ();
> >> + rust_error_at (invoc_locus, "%s", error_string.c_str ());
> >> +
> >> + return AST::ASTFragment::create_error ();
> >> +}
> >> +
> >> +/* Expand builtin macro concat!(), which joins all the literal parameters
> >> + into a string with no delimiter. */
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
> >> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> >> + Parser<MacroInvocLexer> parser (lex);
> >> +
> >> + auto str = std::string ();
> >> + bool has_error = false;
> >> +
> >> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> >> +
> >> + /* NOTE: concat! could accept no argument, so we don't have any checks here */
> >> + while (parser.peek_current_token ()->get_id () != last_token_id)
> >> + {
> >> + auto lit_expr = parser.parse_literal_expr ();
> >> + if (lit_expr)
> >> + {
> >> + str += lit_expr->as_string ();
> >> + }
> >> + else
> >> + {
> >> + auto current_token = parser.peek_current_token ();
> >> + rust_error_at (current_token->get_locus (),
> >> + "argument must be a constant literal");
> >> + has_error = true;
> >> + // Just crash if the current token can't be skipped
> >> + rust_assert (parser.skip_token (current_token->get_id ()));
> >> + }
> >> + parser.maybe_skip_token (COMMA);
> >> + }
> >> +
> >> + parser.skip_token (last_token_id);
> >> +
> >> + if (has_error)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
> >> + return AST::ASTFragment ({node});
> >> +}
> >> +
> >> +/* Expand builtin macro env!(), which inspects an environment variable at
> >> + compile time. */
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
> >> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> >> + Parser<MacroInvocLexer> parser (lex);
> >> +
> >> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
> >> +
> >> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
> >> + {
> >> + if (parser.peek_current_token ()->get_id () == last_token_id)
> >> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
> >> + else
> >> + rust_error_at (parser.peek_current_token ()->get_locus (),
> >> + "argument must be a string literal");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + auto lit_expr = parser.parse_literal_expr ();
> >> + auto comma_skipped = parser.maybe_skip_token (COMMA);
> >> +
> >> + std::unique_ptr<AST::LiteralExpr> error_expr = nullptr;
> >> +
> >> + if (parser.peek_current_token ()->get_id () != last_token_id)
> >> + {
> >> + if (!comma_skipped)
> >> + {
> >> + rust_error_at (parser.peek_current_token ()->get_locus (),
> >> + "expected token: %<,%>");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
> >> + {
> >> + rust_error_at (parser.peek_current_token ()->get_locus (),
> >> + "argument must be a string literal");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + error_expr = parser.parse_literal_expr ();
> >> + parser.maybe_skip_token (COMMA);
> >> + }
> >> +
> >> + if (parser.peek_current_token ()->get_id () != last_token_id)
> >> + {
> >> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + parser.skip_token (last_token_id);
> >> +
> >> + auto env_value = getenv (lit_expr->as_string ().c_str ());
> >> +
> >> + if (env_value == nullptr)
> >> + {
> >> + if (error_expr == nullptr)
> >> + rust_error_at (invoc_locus, "environment variable %qs not defined",
> >> + lit_expr->as_string ().c_str ());
> >> + else
> >> + rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ());
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + auto node = AST::SingleASTNode (make_string (invoc_locus, env_value));
> >> + return AST::ASTFragment ({node});
> >> +}
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + // only parse if not already parsed
> >> + if (!invoc.is_parsed ())
> >> + {
> >> + std::unique_ptr<AST::AttrInputMetaItemContainer> converted_input (
> >> + invoc.get_delim_tok_tree ().parse_to_meta_item ());
> >> +
> >> + if (converted_input == nullptr)
> >> + {
> >> + rust_debug ("DEBUG: failed to parse macro to meta item");
> >> + // TODO: do something now? is this an actual error?
> >> + }
> >> + else
> >> + {
> >> + std::vector<std::unique_ptr<AST::MetaItemInner>> meta_items (
> >> + std::move (converted_input->get_items ()));
> >> + invoc.set_meta_item_output (std::move (meta_items));
> >> + }
> >> + }
> >> +
> >> + /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg
> >> + * attributes */
> >> + if (invoc.get_meta_items ().size () != 1)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + bool result = invoc.get_meta_items ()[0]->check_cfg_predicate (
> >> + Session::get_instance ());
> >> + auto literal_exp = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> >> + new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL,
> >> + PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus)));
> >> +
> >> + return AST::ASTFragment ({literal_exp});
> >> +}
> >> +
> >> +/* Expand builtin macro include!(), which includes a source file at the current
> >> + scope compile time. */
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + /* Get target filename from the macro invocation, which is treated as a path
> >> + relative to the include!-ing file (currently being compiled). */
> >> + auto lit_expr
> >> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
> >> + if (lit_expr == nullptr)
> >> + return AST::ASTFragment::create_error ();
> >> +
> >> + std::string filename
> >> + = source_relative_path (lit_expr->as_string (), invoc_locus);
> >> + auto target_filename
> >> + = Rust::Session::get_instance ().include_extra_file (std::move (filename));
> >> +
> >> + RAIIFile target_file (target_filename);
> >> + Linemap *linemap = Session::get_instance ().linemap;
> >> +
> >> + if (!target_file.ok ())
> >> + {
> >> + rust_error_at (lit_expr->get_locus (),
> >> + "cannot open included file %qs: %m", target_filename);
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + rust_debug ("Attempting to parse included file %s", target_filename);
> >> +
> >> + Lexer lex (target_filename, std::move (target_file), linemap);
> >> + Parser<Lexer> parser (lex);
> >> +
> >> + auto parsed_items = parser.parse_items ();
> >> + bool has_error = !parser.get_errors ().empty ();
> >> +
> >> + for (const auto &error : parser.get_errors ())
> >> + error.emit_error ();
> >> +
> >> + if (has_error)
> >> + {
> >> + // inform the user that the errors above are from a included file
> >> + rust_inform (invoc_locus, "included from here");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + std::vector<AST::SingleASTNode> nodes{};
> >> + for (auto &item : parsed_items)
> >> + {
> >> + AST::SingleASTNode node (std::move (item));
> >> + nodes.push_back (node);
> >> + }
> >> +
> >> + return AST::ASTFragment (nodes);
> >> +}
> >> +
> >> +AST::ASTFragment
> >> +MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc)
> >> +{
> >> + auto current_line
> >> + = Session::get_instance ().linemap->location_to_line (invoc_locus);
> >> +
> >> + auto line_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
> >> + new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT,
> >> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
> >> +
> >> + return AST::ASTFragment ({line_no});
> >> +}
> >> +
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-macro-builtins.h b/gcc/rust/expand/rust-macro-builtins.h
> >> new file mode 100644
> >> index 00000000000..91f3727d450
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-builtins.h
> >> @@ -0,0 +1,107 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#ifndef RUST_MACRO_BUILTINS_H
> >> +#define RUST_MACRO_BUILTINS_H
> >> +
> >> +#include "rust-ast.h"
> >> +#include "rust-location.h"
> >> +
> >> +/**
> >> + * This class provides a list of builtin macros implemented by the compiler.
> >> + * The functions defined are called "builtin transcribers" in that they replace
> >> + * the transcribing part of a macro definition.
> >> + *
> >> + * Like regular macro transcribers, they are responsible for building and
> >> + * returning an AST fragment: basically a vector of AST nodes put together.
> >> + *
> >> + * Unlike regular declarative macros where each match arm has its own associated
> >> + * transcriber, builtin transcribers are responsible for handling all match arms
> >> + * of the macro. This means that you should take extra care when implementing a
> >> + * builtin containing multiple match arms: You will probably need to do some
> >> + * lookahead in order to determine which match arm the user intended to use.
> >> + *
> >> + * An example of this is the `assert!()` macro:
> >> + *
> >> + * ```
> >> + * macro_rules! assert {
> >> + * ($cond:expr $(,)?) => {{ ... }};
> >> + * ($cond : expr, $ ($arg : tt) +) = > {{ ... }};
> >> + * }
> >> + * ```
> >> + *
> >> + * If more tokens exist beyond the optional comma, they need to be handled as
> >> + * a token-tree for a custom panic message.
> >> + *
> >> + * These builtin macros with empty transcribers are defined in the standard
> >> + * library. They are marked with a special attribute, `#[rustc_builtin_macro]`.
> >> + * When this attribute is present on a macro definition, the compiler should
> >> + * look for an associated transcriber in the mappings. Meaning that you must
> >> + * remember to insert your transcriber in the `builtin_macros` map of the
> >> + *`Mappings`.
> >> + *
> >> + * This map is built as a static variable in the `insert_macro_def()` method
> >> + * of the `Mappings` class.
> >> + */
> >> +
> >> +/* If assert is defined as a macro this file will not parse, so undefine this
> >> + before continuing. */
> >> +#ifdef assert
> >> +#undef assert
> >> +#endif
> >> +
> >> +namespace Rust {
> >> +class MacroBuiltin
> >> +{
> >> +public:
> >> + static AST::ASTFragment assert (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment file (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment column (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment include_bytes (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment include_str (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment compile_error (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment concat (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment env (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment cfg (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment include (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +
> >> + static AST::ASTFragment line (Location invoc_locus,
> >> + AST::MacroInvocData &invoc);
> >> +};
> >> +} // namespace Rust
> >> +
> >> +#endif // RUST_MACRO_BUILTINS_H
> >> diff --git a/gcc/rust/expand/rust-macro-expand.cc b/gcc/rust/expand/rust-macro-expand.cc
> >> new file mode 100644
> >> index 00000000000..1d57e394220
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-expand.cc
> >> @@ -0,0 +1,1012 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#include "rust-macro-expand.h"
> >> +#include "rust-macro-substitute-ctx.h"
> >> +#include "rust-ast-full.h"
> >> +#include "rust-ast-visitor.h"
> >> +#include "rust-diagnostics.h"
> >> +#include "rust-parse.h"
> >> +#include "rust-attribute-visitor.h"
> >> +
> >> +namespace Rust {
> >> +AST::ASTFragment
> >> +MacroExpander::expand_decl_macro (Location invoc_locus,
> >> + AST::MacroInvocData &invoc,
> >> + AST::MacroRulesDefinition &rules_def,
> >> + bool semicolon)
> >> +{
> >> + // ensure that both invocation and rules are in a valid state
> >> + rust_assert (!invoc.is_marked_for_strip ());
> >> + rust_assert (!rules_def.is_marked_for_strip ());
> >> + rust_assert (rules_def.get_macro_rules ().size () > 0);
> >> +
> >> + /* probably something here about parsing invoc and rules def token trees to
> >> + * token stream. if not, how would parser handle the captures of exprs and
> >> + * stuff? on the other hand, token trees may be kind of useful in rules def as
> >> + * creating a point where recursion can occur (like having
> >> + * "compare_macro_match" and then it calling itself when it finds delimiters)
> >> + */
> >> +
> >> + /* find matching rule to invoc token tree, based on macro rule's matcher. if
> >> + * none exist, error.
> >> + * - specifically, check each matcher in order. if one fails to match, move
> >> + * onto next. */
> >> + /* TODO: does doing this require parsing expressions and whatever in the
> >> + * invoc? if so, might as well save the results if referenced using $ or
> >> + * whatever. If not, do another pass saving them. Except this is probably
> >> + * useless as different rules could have different starting points for exprs
> >> + * or whatever. Decision trees could avoid this, but they have their own
> >> + * issues. */
> >> + /* TODO: will need to modify the parser so that it can essentially "catch"
> >> + * errors - maybe "try_parse_expr" or whatever methods. */
> >> + // this technically creates a back-tracking parser - this will be the
> >> + // implementation style
> >> +
> >> + /* then, after results are saved, generate the macro output from the
> >> + * transcriber token tree. if i understand this correctly, the macro
> >> + * invocation gets replaced by the transcriber tokens, except with
> >> + * substitutions made (e.g. for $i variables) */
> >> +
> >> + /* TODO: it is probably better to modify AST::Token to store a pointer to a
> >> + * Lexer::Token (rather than being converted) - i.e. not so much have
> >> + * AST::Token as a Token but rather a TokenContainer (as it is another type of
> >> + * TokenTree). This will prevent re-conversion of Tokens between each type
> >> + * all the time, while still allowing the heterogenous storage of token trees.
> >> + */
> >> +
> >> + AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree ();
> >> +
> >> + // find matching arm
> >> + AST::MacroRule *matched_rule = nullptr;
> >> + std::map<std::string, MatchedFragmentContainer> matched_fragments;
> >> + for (auto &rule : rules_def.get_rules ())
> >> + {
> >> + sub_stack.push ();
> >> + bool did_match_rule = try_match_rule (rule, invoc_token_tree);
> >> + matched_fragments = sub_stack.pop ();
> >> +
> >> + if (did_match_rule)
> >> + {
> >> + // // Debugging
> >> + // for (auto &kv : matched_fragments)
> >> + // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (),
> >> + // kv.second.get_fragments ().size (),
> >> + // kv.second.get_kind ()
> >> + // == MatchedFragmentContainer::Kind::Repetition
> >> + // ? "repetition"
> >> + // : "metavar");
> >> +
> >> + matched_rule = &rule;
> >> + break;
> >> + }
> >> + }
> >> +
> >> + if (matched_rule == nullptr)
> >> + {
> >> + RichLocation r (invoc_locus);
> >> + r.add_range (rules_def.get_locus ());
> >> + rust_error_at (r, "Failed to match any rule within macro");
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments,
> >> + semicolon, peek_context ());
> >> +}
> >> +
> >> +void
> >> +MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon)
> >> +{
> >> + if (depth_exceeds_recursion_limit ())
> >> + {
> >> + rust_error_at (invoc.get_locus (), "reached recursion limit");
> >> + return;
> >> + }
> >> +
> >> + AST::MacroInvocData &invoc_data = invoc.get_invoc_data ();
> >> +
> >> + // ??
> >> + // switch on type of macro:
> >> + // - '!' syntax macro (inner switch)
> >> + // - procedural macro - "A token-based function-like macro"
> >> + // - 'macro_rules' (by example/pattern-match) macro? or not? "an
> >> + // AST-based function-like macro"
> >> + // - else is unreachable
> >> + // - attribute syntax macro (inner switch)
> >> + // - procedural macro attribute syntax - "A token-based attribute
> >> + // macro"
> >> + // - legacy macro attribute syntax? - "an AST-based attribute macro"
> >> + // - non-macro attribute: mark known
> >> + // - else is unreachable
> >> + // - derive macro (inner switch)
> >> + // - derive or legacy derive - "token-based" vs "AST-based"
> >> + // - else is unreachable
> >> + // - derive container macro - unreachable
> >> +
> >> + // lookup the rules for this macro
> >> + NodeId resolved_node = UNKNOWN_NODEID;
> >> + NodeId source_node = UNKNOWN_NODEID;
> >> + if (has_semicolon)
> >> + source_node = invoc.get_macro_node_id ();
> >> + else
> >> + source_node = invoc.get_pattern_node_id ();
> >> + auto seg
> >> + = Resolver::CanonicalPath::new_seg (source_node,
> >> + invoc_data.get_path ().as_string ());
> >> +
> >> + bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node);
> >> + if (!found)
> >> + {
> >> + rust_error_at (invoc.get_locus (), "unknown macro: [%s]",
> >> + seg.get ().c_str ());
> >> + return;
> >> + }
> >> +
> >> + // lookup the rules
> >> + AST::MacroRulesDefinition *rules_def = nullptr;
> >> + bool ok = mappings->lookup_macro_def (resolved_node, &rules_def);
> >> + rust_assert (ok);
> >> +
> >> + auto fragment = AST::ASTFragment::create_error ();
> >> +
> >> + if (rules_def->is_builtin ())
> >> + fragment
> >> + = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data);
> >> + else
> >> + fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def,
> >> + has_semicolon);
> >> +
> >> + set_expanded_fragment (std::move (fragment));
> >> +}
> >> +
> >> +/* Determines whether any cfg predicate is false and hence item with attributes
> >> + * should be stripped. Note that attributes must be expanded before calling. */
> >> +bool
> >> +MacroExpander::fails_cfg (const AST::AttrVec &attrs) const
> >> +{
> >> + for (const auto &attr : attrs)
> >> + {
> >> + if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session))
> >> + return true;
> >> + }
> >> + return false;
> >> +}
> >> +
> >> +/* Determines whether any cfg predicate is false and hence item with attributes
> >> + * should be stripped. Will expand attributes as well. */
> >> +bool
> >> +MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const
> >> +{
> >> + // TODO: maybe have something that strips cfg attributes that evaluate true?
> >> + for (auto &attr : attrs)
> >> + {
> >> + if (attr.get_path () == "cfg")
> >> + {
> >> + if (!attr.is_parsed_to_meta_item ())
> >> + attr.parse_attr_to_meta_item ();
> >> +
> >> + // DEBUG
> >> + if (!attr.is_parsed_to_meta_item ())
> >> + rust_debug ("failed to parse attr to meta item, right before "
> >> + "cfg predicate check");
> >> + else
> >> + rust_debug ("attr has been successfully parsed to meta item, "
> >> + "right before cfg predicate check");
> >> +
> >> + if (!attr.check_cfg_predicate (session))
> >> + {
> >> + // DEBUG
> >> + rust_debug (
> >> + "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m",
> >> + attr.as_string ().c_str ());
> >> +
> >> + return true;
> >> + }
> >> + else
> >> + {
> >> + // DEBUG
> >> + rust_debug ("cfg predicate succeeded for attribute: "
> >> + "\033[0;31m'%s'\033[0m",
> >> + attr.as_string ().c_str ());
> >> + }
> >> + }
> >> + }
> >> + return false;
> >> +}
> >> +
> >> +// Expands cfg_attr attributes.
> >> +void
> >> +MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs)
> >> +{
> >> + for (std::size_t i = 0; i < attrs.size (); i++)
> >> + {
> >> + auto &attr = attrs[i];
> >> + if (attr.get_path () == "cfg_attr")
> >> + {
> >> + if (!attr.is_parsed_to_meta_item ())
> >> + attr.parse_attr_to_meta_item ();
> >> +
> >> + if (attr.check_cfg_predicate (session))
> >> + {
> >> + // split off cfg_attr
> >> + AST::AttrVec new_attrs = attr.separate_cfg_attrs ();
> >> +
> >> + // remove attr from vector
> >> + attrs.erase (attrs.begin () + i);
> >> +
> >> + // add new attrs to vector
> >> + attrs.insert (attrs.begin () + i,
> >> + std::make_move_iterator (new_attrs.begin ()),
> >> + std::make_move_iterator (new_attrs.end ()));
> >> + }
> >> +
> >> + /* do something - if feature (first token in tree) is in fact enabled,
> >> + * make tokens listed afterwards into attributes. i.e.: for
> >> + * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add
> >> + * attributes [wow1] and [wow2] to attribute list. This can also be
> >> + * recursive, so check for expanded attributes being recursive and
> >> + * possibly recursively call the expand_attrs? */
> >> + }
> >> + else
> >> + {
> >> + i++;
> >> + }
> >> + }
> >> + attrs.shrink_to_fit ();
> >> +}
> >> +
> >> +void
> >> +MacroExpander::expand_crate ()
> >> +{
> >> + NodeId scope_node_id = crate.get_node_id ();
> >> + resolver->get_macro_scope ().push (scope_node_id);
> >> +
> >> + /* fill macro/decorator map from init list? not sure where init list comes
> >> + * from? */
> >> +
> >> + // TODO: does cfg apply for inner attributes? research.
> >> + // the apparent answer (from playground test) is yes
> >> +
> >> + // expand crate cfg_attr attributes
> >> + expand_cfg_attrs (crate.inner_attrs);
> >> +
> >> + if (fails_cfg_with_expand (crate.inner_attrs))
> >> + {
> >> + // basically, delete whole crate
> >> + crate.strip_crate ();
> >> + // TODO: maybe create warning here? probably not desired behaviour
> >> + }
> >> + // expand module attributes?
> >> +
> >> + push_context (ITEM);
> >> +
> >> + // expand attributes recursively and strip items if required
> >> + AttrVisitor attr_visitor (*this);
> >> + auto &items = crate.items;
> >> + for (auto it = items.begin (); it != items.end ();)
> >> + {
> >> + auto &item = *it;
> >> +
> >> + // mark for stripping if required
> >> + item->accept_vis (attr_visitor);
> >> +
> >> + auto fragment = take_expanded_fragment (attr_visitor);
> >> + if (fragment.should_expand ())
> >> + {
> >> + // Remove the current expanded invocation
> >> + it = items.erase (it);
> >> + for (auto &node : fragment.get_nodes ())
> >> + {
> >> + it = items.insert (it, node.take_item ());
> >> + it++;
> >> + }
> >> + }
> >> + else if (item->is_marked_for_strip ())
> >> + it = items.erase (it);
> >> + else
> >> + it++;
> >> + }
> >> +
> >> + pop_context ();
> >> +
> >> + // TODO: should recursive attribute and macro expansion be done in the same
> >> + // transversal? Or in separate ones like currently?
> >> +
> >> + // expand module tree recursively
> >> +
> >> + // post-process
> >> +
> >> + // extract exported macros?
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::depth_exceeds_recursion_limit () const
> >> +{
> >> + return expansion_depth >= cfg.recursion_limit;
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::try_match_rule (AST::MacroRule &match_rule,
> >> + AST::DelimTokenTree &invoc_token_tree)
> >> +{
> >> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
> >> + Parser<MacroInvocLexer> parser (lex);
> >> +
> >> + AST::MacroMatcher &matcher = match_rule.get_matcher ();
> >> +
> >> + expansion_depth++;
> >> + if (!match_matcher (parser, matcher))
> >> + {
> >> + expansion_depth--;
> >> + return false;
> >> + }
> >> + expansion_depth--;
> >> +
> >> + bool used_all_input_tokens = parser.skip_token (END_OF_FILE);
> >> + return used_all_input_tokens;
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::match_fragment (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchFragment &fragment)
> >> +{
> >> + switch (fragment.get_frag_spec ().get_kind ())
> >> + {
> >> + case AST::MacroFragSpec::EXPR:
> >> + parser.parse_expr ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::BLOCK:
> >> + parser.parse_block_expr ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::IDENT:
> >> + parser.parse_identifier_pattern ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::LITERAL:
> >> + parser.parse_literal_expr ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::ITEM:
> >> + parser.parse_item (false);
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::TY:
> >> + parser.parse_type ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::PAT:
> >> + parser.parse_pattern ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::PATH:
> >> + parser.parse_path_in_expression ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::VIS:
> >> + parser.parse_visibility ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::STMT: {
> >> + auto restrictions = ParseRestrictions ();
> >> + restrictions.consume_semi = false;
> >> + parser.parse_stmt (restrictions);
> >> + break;
> >> + }
> >> +
> >> + case AST::MacroFragSpec::LIFETIME:
> >> + parser.parse_lifetime_params ();
> >> + break;
> >> +
> >> + // is meta attributes?
> >> + case AST::MacroFragSpec::META:
> >> + parser.parse_attribute_body ();
> >> + break;
> >> +
> >> + case AST::MacroFragSpec::TT:
> >> + parser.parse_token_tree ();
> >> + break;
> >> +
> >> + // i guess we just ignore invalid and just error out
> >> + case AST::MacroFragSpec::INVALID:
> >> + return false;
> >> + }
> >> +
> >> + // it matches if the parser did not produce errors trying to parse that type
> >> + // of item
> >> + return !parser.has_errors ();
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::match_matcher (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatcher &matcher)
> >> +{
> >> + if (depth_exceeds_recursion_limit ())
> >> + {
> >> + rust_error_at (matcher.get_match_locus (), "reached recursion limit");
> >> + return false;
> >> + }
> >> +
> >> + auto delimiter = parser.peek_current_token ();
> >> +
> >> + // this is used so we can check that we delimit the stream correctly.
> >> + switch (delimiter->get_id ())
> >> + {
> >> + case LEFT_PAREN: {
> >> + if (!parser.skip_token (LEFT_PAREN))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case LEFT_SQUARE: {
> >> + if (!parser.skip_token (LEFT_SQUARE))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case LEFT_CURLY: {
> >> + if (!parser.skip_token (LEFT_CURLY))
> >> + return false;
> >> + }
> >> + break;
> >> + default:
> >> + gcc_unreachable ();
> >> + }
> >> +
> >> + const MacroInvocLexer &source = parser.get_token_source ();
> >> +
> >> + for (auto &match : matcher.get_matches ())
> >> + {
> >> + size_t offs_begin = source.get_offs ();
> >> +
> >> + switch (match->get_macro_match_type ())
> >> + {
> >> + case AST::MacroMatch::MacroMatchType::Fragment: {
> >> + AST::MacroMatchFragment *fragment
> >> + = static_cast<AST::MacroMatchFragment *> (match.get ());
> >> + if (!match_fragment (parser, *fragment))
> >> + return false;
> >> +
> >> + // matched fragment get the offset in the token stream
> >> + size_t offs_end = source.get_offs ();
> >> + sub_stack.insert_metavar (
> >> + MatchedFragment (fragment->get_ident (), offs_begin, offs_end));
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Tok: {
> >> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
> >> + if (!match_token (parser, *tok))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Repetition: {
> >> + AST::MacroMatchRepetition *rep
> >> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
> >> + if (!match_repetition (parser, *rep))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Matcher: {
> >> + AST::MacroMatcher *m
> >> + = static_cast<AST::MacroMatcher *> (match.get ());
> >> + expansion_depth++;
> >> + if (!match_matcher (parser, *m))
> >> + {
> >> + expansion_depth--;
> >> + return false;
> >> + }
> >> + expansion_depth--;
> >> + }
> >> + break;
> >> + }
> >> + }
> >> +
> >> + switch (delimiter->get_id ())
> >> + {
> >> + case LEFT_PAREN: {
> >> + if (!parser.skip_token (RIGHT_PAREN))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case LEFT_SQUARE: {
> >> + if (!parser.skip_token (RIGHT_SQUARE))
> >> + return false;
> >> + }
> >> + break;
> >> +
> >> + case LEFT_CURLY: {
> >> + if (!parser.skip_token (RIGHT_CURLY))
> >> + return false;
> >> + }
> >> + break;
> >> + default:
> >> + gcc_unreachable ();
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::match_token (Parser<MacroInvocLexer> &parser, AST::Token &token)
> >> +{
> >> + // FIXME this needs to actually match the content and the type
> >> + return parser.skip_token (token.get_id ());
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::match_n_matches (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchRepetition &rep,
> >> + size_t &match_amount, size_t lo_bound,
> >> + size_t hi_bound)
> >> +{
> >> + match_amount = 0;
> >> + auto &matches = rep.get_matches ();
> >> +
> >> + const MacroInvocLexer &source = parser.get_token_source ();
> >> + while (true)
> >> + {
> >> + // If the current token is a closing macro delimiter, break away.
> >> + // TODO: Is this correct?
> >> + auto t_id = parser.peek_current_token ()->get_id ();
> >> + if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY)
> >> + break;
> >> +
> >> + // Skip parsing a separator on the first match, otherwise consume it.
> >> + // If it isn't present, this is an error
> >> + if (rep.has_sep () && match_amount > 0)
> >> + if (!match_token (parser, *rep.get_sep ()))
> >> + break;
> >> +
> >> + bool valid_current_match = false;
> >> + for (auto &match : matches)
> >> + {
> >> + size_t offs_begin = source.get_offs ();
> >> + switch (match->get_macro_match_type ())
> >> + {
> >> + case AST::MacroMatch::MacroMatchType::Fragment: {
> >> + AST::MacroMatchFragment *fragment
> >> + = static_cast<AST::MacroMatchFragment *> (match.get ());
> >> + valid_current_match = match_fragment (parser, *fragment);
> >> +
> >> + // matched fragment get the offset in the token stream
> >> + size_t offs_end = source.get_offs ();
> >> +
> >> + // The main difference with match_matcher happens here: Instead
> >> + // of inserting a new fragment, we append to one. If that
> >> + // fragment does not exist, then the operation is similar to
> >> + // `insert_fragment` with the difference that we are not
> >> + // creating a metavariable, but a repetition of one, which is
> >> + // really different.
> >> + sub_stack.append_fragment (
> >> + MatchedFragment (fragment->get_ident (), offs_begin,
> >> + offs_end));
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Tok: {
> >> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
> >> + valid_current_match = match_token (parser, *tok);
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Repetition: {
> >> + AST::MacroMatchRepetition *rep
> >> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
> >> + valid_current_match = match_repetition (parser, *rep);
> >> + }
> >> + break;
> >> +
> >> + case AST::MacroMatch::MacroMatchType::Matcher: {
> >> + AST::MacroMatcher *m
> >> + = static_cast<AST::MacroMatcher *> (match.get ());
> >> + valid_current_match = match_matcher (parser, *m);
> >> + }
> >> + break;
> >> + }
> >> + }
> >> + // If we've encountered an error once, stop trying to match more
> >> + // repetitions
> >> + if (!valid_current_match)
> >> + break;
> >> +
> >> + match_amount++;
> >> +
> >> + // Break early if we notice there's too many expressions already
> >> + if (hi_bound && match_amount > hi_bound)
> >> + break;
> >> + }
> >> +
> >> + // Check if the amount of matches we got is valid: Is it more than the lower
> >> + // bound and less than the higher bound?
> >> + bool did_meet_lo_bound = match_amount >= lo_bound;
> >> + bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true;
> >> +
> >> + // If the end-result is valid, then we can clear the parse errors: Since
> >> + // repetitions are parsed eagerly, it is okay to fail in some cases
> >> + auto res = did_meet_lo_bound && did_meet_hi_bound;
> >> + if (res)
> >> + parser.clear_errors ();
> >> +
> >> + return res;
> >> +}
> >> +
> >> +bool
> >> +MacroExpander::match_repetition (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchRepetition &rep)
> >> +{
> >> + size_t match_amount = 0;
> >> + bool res = false;
> >> +
> >> + std::string lo_str;
> >> + std::string hi_str;
> >> + switch (rep.get_op ())
> >> + {
> >> + case AST::MacroMatchRepetition::MacroRepOp::ANY:
> >> + lo_str = "0";
> >> + hi_str = "+inf";
> >> + res = match_n_matches (parser, rep, match_amount);
> >> + break;
> >> + case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE:
> >> + lo_str = "1";
> >> + hi_str = "+inf";
> >> + res = match_n_matches (parser, rep, match_amount, 1);
> >> + break;
> >> + case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE:
> >> + lo_str = "0";
> >> + hi_str = "1";
> >> + res = match_n_matches (parser, rep, match_amount, 0, 1);
> >> + break;
> >> + default:
> >> + gcc_unreachable ();
> >> + }
> >> +
> >> + if (!res)
> >> + rust_error_at (rep.get_match_locus (),
> >> + "invalid amount of matches for macro invocation. Expected "
> >> + "between %s and %s, got %lu",
> >> + lo_str.c_str (), hi_str.c_str (),
> >> + (unsigned long) match_amount);
> >> +
> >> + rust_debug_loc (rep.get_match_locus (), "%s matched %lu times",
> >> + res ? "successfully" : "unsuccessfully",
> >> + (unsigned long) match_amount);
> >> +
> >> + // We have to handle zero fragments differently: They will not have been
> >> + // "matched" but they are still valid and should be inserted as a special
> >> + // case. So we go through the stack map, and for every fragment which doesn't
> >> + // exist, insert a zero-matched fragment.
> >> + auto &stack_map = sub_stack.peek ();
> >> + for (auto &match : rep.get_matches ())
> >> + {
> >> + if (match->get_macro_match_type ()
> >> + == AST::MacroMatch::MacroMatchType::Fragment)
> >> + {
> >> + auto fragment = static_cast<AST::MacroMatchFragment *> (match.get ());
> >> + auto it = stack_map.find (fragment->get_ident ());
> >> +
> >> + if (it == stack_map.end ())
> >> + sub_stack.insert_matches (fragment->get_ident (),
> >> + MatchedFragmentContainer::zero ());
> >> + }
> >> + }
> >> +
> >> + return res;
> >> +}
> >> +
> >> +/**
> >> + * Helper function to refactor calling a parsing function 0 or more times
> >> + */
> >> +static AST::ASTFragment
> >> +parse_many (Parser<MacroInvocLexer> &parser, TokenId &delimiter,
> >> + std::function<AST::SingleASTNode ()> parse_fn)
> >> +{
> >> + std::vector<AST::SingleASTNode> nodes;
> >> + while (true)
> >> + {
> >> + if (parser.peek_current_token ()->get_id () == delimiter)
> >> + break;
> >> +
> >> + auto node = parse_fn ();
> >> + nodes.emplace_back (std::move (node));
> >> + }
> >> +
> >> + return AST::ASTFragment (std::move (nodes));
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more items from a macro invocation
> >> + *
> >> + * @param parser Parser to extract items from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> >> +{
> >> + return parse_many (parser, delimiter, [&parser] () {
> >> + auto item = parser.parse_item (true);
> >> + return AST::SingleASTNode (std::move (item));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more external items from a macro invocation
> >> + *
> >> + * @param parser Parser to extract items from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_ext (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> >> +{
> >> + return parse_many (parser, delimiter, [&parser] () {
> >> + auto item = parser.parse_external_item ();
> >> + return AST::SingleASTNode (std::move (item));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more trait items from a macro invocation
> >> + *
> >> + * @param parser Parser to extract items from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_trait_items (Parser<MacroInvocLexer> &parser,
> >> + TokenId &delimiter)
> >> +{
> >> + return parse_many (parser, delimiter, [&parser] () {
> >> + auto item = parser.parse_trait_item ();
> >> + return AST::SingleASTNode (std::move (item));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more impl items from a macro invocation
> >> + *
> >> + * @param parser Parser to extract items from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_impl_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> >> +{
> >> + return parse_many (parser, delimiter, [&parser] () {
> >> + auto item = parser.parse_inherent_impl_item ();
> >> + return AST::SingleASTNode (std::move (item));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more trait impl items from a macro invocation
> >> + *
> >> + * @param parser Parser to extract items from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_trait_impl_items (Parser<MacroInvocLexer> &parser,
> >> + TokenId &delimiter)
> >> +{
> >> + return parse_many (parser, delimiter, [&parser] () {
> >> + auto item = parser.parse_trait_impl_item ();
> >> + return AST::SingleASTNode (std::move (item));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe 0 or more statements from a macro invocation
> >> + *
> >> + * @param parser Parser to extract statements from
> >> + * @param delimiter Id of the token on which parsing should stop
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_many_stmts (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
> >> +{
> >> + auto restrictions = ParseRestrictions ();
> >> + restrictions.consume_semi = false;
> >> +
> >> + // FIXME: This is invalid! It needs to also handle cases where the macro
> >> + // transcriber is an expression, but since the macro call is followed by
> >> + // a semicolon, it's a valid ExprStmt
> >> + return parse_many (parser, delimiter, [&parser, restrictions] () {
> >> + auto stmt = parser.parse_stmt (restrictions);
> >> + return AST::SingleASTNode (std::move (stmt));
> >> + });
> >> +}
> >> +
> >> +/**
> >> + * Transcribe one expression from a macro invocation
> >> + *
> >> + * @param parser Parser to extract statements from
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_expression (Parser<MacroInvocLexer> &parser)
> >> +{
> >> + auto expr = parser.parse_expr ();
> >> +
> >> + return AST::ASTFragment ({std::move (expr)});
> >> +}
> >> +
> >> +/**
> >> + * Transcribe one type from a macro invocation
> >> + *
> >> + * @param parser Parser to extract statements from
> >> + */
> >> +static AST::ASTFragment
> >> +transcribe_type (Parser<MacroInvocLexer> &parser)
> >> +{
> >> + auto type = parser.parse_type ();
> >> +
> >> + return AST::ASTFragment ({std::move (type)});
> >> +}
> >> +
> >> +static AST::ASTFragment
> >> +transcribe_on_delimiter (Parser<MacroInvocLexer> &parser, bool semicolon,
> >> + AST::DelimType delimiter, TokenId last_token_id)
> >> +{
> >> + if (semicolon || delimiter == AST::DelimType::CURLY)
> >> + return transcribe_many_stmts (parser, last_token_id);
> >> + else
> >> + return transcribe_expression (parser);
> >> +} // namespace Rust
> >> +
> >> +static AST::ASTFragment
> >> +transcribe_context (MacroExpander::ContextType ctx,
> >> + Parser<MacroInvocLexer> &parser, bool semicolon,
> >> + AST::DelimType delimiter, TokenId last_token_id)
> >> +{
> >> + // The flow-chart in order to choose a parsing function is as follows:
> >> + //
> >> + // [switch special context]
> >> + // -- Item --> parser.parse_item();
> >> + // -- Trait --> parser.parse_trait_item();
> >> + // -- Impl --> parser.parse_impl_item();
> >> + // -- Extern --> parser.parse_extern_item();
> >> + // -- None --> [has semicolon?]
> >> + // -- Yes --> parser.parse_stmt();
> >> + // -- No --> [switch invocation.delimiter()]
> >> + // -- { } --> parser.parse_stmt();
> >> + // -- _ --> parser.parse_expr(); // once!
> >> +
> >> + // If there is a semicolon OR we are expanding a MacroInvocationSemi, then
> >> + // we can parse multiple items. Otherwise, parse *one* expression
> >> +
> >> + switch (ctx)
> >> + {
> >> + case MacroExpander::ContextType::ITEM:
> >> + return transcribe_many_items (parser, last_token_id);
> >> + break;
> >> + case MacroExpander::ContextType::TRAIT:
> >> + return transcribe_many_trait_items (parser, last_token_id);
> >> + break;
> >> + case MacroExpander::ContextType::IMPL:
> >> + return transcribe_many_impl_items (parser, last_token_id);
> >> + break;
> >> + case MacroExpander::ContextType::TRAIT_IMPL:
> >> + return transcribe_many_trait_impl_items (parser, last_token_id);
> >> + break;
> >> + case MacroExpander::ContextType::EXTERN:
> >> + return transcribe_many_ext (parser, last_token_id);
> >> + break;
> >> + case MacroExpander::ContextType::TYPE:
> >> + return transcribe_type (parser);
> >> + break;
> >> + default:
> >> + return transcribe_on_delimiter (parser, semicolon, delimiter,
> >> + last_token_id);
> >> + }
> >> +}
> >> +
> >> +static std::string
> >> +tokens_to_str (std::vector<std::unique_ptr<AST::Token>> &tokens)
> >> +{
> >> + std::string str;
> >> + if (!tokens.empty ())
> >> + {
> >> + str += tokens[0]->as_string ();
> >> + for (size_t i = 1; i < tokens.size (); i++)
> >> + str += " " + tokens[i]->as_string ();
> >> + }
> >> +
> >> + return str;
> >> +}
> >> +
> >> +AST::ASTFragment
> >> +MacroExpander::transcribe_rule (
> >> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
> >> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
> >> + bool semicolon, ContextType ctx)
> >> +{
> >> + // we can manipulate the token tree to substitute the dollar identifiers so
> >> + // that when we call parse its already substituted for us
> >> + AST::MacroTranscriber &transcriber = match_rule.get_transcriber ();
> >> + AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree ();
> >> +
> >> + auto invoc_stream = invoc_token_tree.to_token_stream ();
> >> + auto macro_rule_tokens = transcribe_tree.to_token_stream ();
> >> +
> >> + auto substitute_context
> >> + = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments);
> >> + std::vector<std::unique_ptr<AST::Token>> substituted_tokens
> >> + = substitute_context.substitute_tokens ();
> >> +
> >> + rust_debug ("substituted tokens: %s",
> >> + tokens_to_str (substituted_tokens).c_str ());
> >> +
> >> + // parse it to an ASTFragment
> >> + MacroInvocLexer lex (std::move (substituted_tokens));
> >> + Parser<MacroInvocLexer> parser (lex);
> >> +
> >> + auto last_token_id = TokenId::RIGHT_CURLY;
> >> +
> >> + // this is used so we can check that we delimit the stream correctly.
> >> + switch (transcribe_tree.get_delim_type ())
> >> + {
> >> + case AST::DelimType::PARENS:
> >> + last_token_id = TokenId::RIGHT_PAREN;
> >> + rust_assert (parser.skip_token (LEFT_PAREN));
> >> + break;
> >> +
> >> + case AST::DelimType::CURLY:
> >> + rust_assert (parser.skip_token (LEFT_CURLY));
> >> + break;
> >> +
> >> + case AST::DelimType::SQUARE:
> >> + last_token_id = TokenId::RIGHT_SQUARE;
> >> + rust_assert (parser.skip_token (LEFT_SQUARE));
> >> + break;
> >> + }
> >> +
> >> + // see https://github.com/Rust-GCC/gccrs/issues/22
> >> + // TL;DR:
> >> + // - Treat all macro invocations with parentheses, (), or square brackets,
> >> + // [], as expressions.
> >> + // - If the macro invocation has curly brackets, {}, it may be parsed as a
> >> + // statement depending on the context.
> >> + // - If the macro invocation has a semicolon at the end, it must be parsed
> >> + // as a statement (either via ExpressionStatement or
> >> + // MacroInvocationWithSemi)
> >> +
> >> + auto fragment
> >> + = transcribe_context (ctx, parser, semicolon,
> >> + invoc_token_tree.get_delim_type (), last_token_id);
> >> +
> >> + // emit any errors
> >> + if (parser.has_errors ())
> >> + {
> >> + for (auto &err : parser.get_errors ())
> >> + rust_error_at (err.locus, "%s", err.message.c_str ());
> >> + return AST::ASTFragment::create_error ();
> >> + }
> >> +
> >> + // are all the tokens used?
> >> + bool did_delimit = parser.skip_token (last_token_id);
> >> +
> >> + bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE);
> >> + if (!reached_end_of_stream)
> >> + {
> >> + const_TokenPtr current_token = parser.peek_current_token ();
> >> + rust_error_at (current_token->get_locus (),
> >> + "tokens here and after are unparsed");
> >> + }
> >> +
> >> + return fragment;
> >> +}
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-macro-expand.h b/gcc/rust/expand/rust-macro-expand.h
> >> new file mode 100644
> >> index 00000000000..94d6702ecb8
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-expand.h
> >> @@ -0,0 +1,366 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#ifndef RUST_MACRO_EXPAND_H
> >> +#define RUST_MACRO_EXPAND_H
> >> +
> >> +#include "rust-buffered-queue.h"
> >> +#include "rust-parse.h"
> >> +#include "rust-token.h"
> >> +#include "rust-ast.h"
> >> +#include "rust-macro.h"
> >> +#include "rust-hir-map.h"
> >> +#include "rust-name-resolver.h"
> >> +#include "rust-macro-invoc-lexer.h"
> >> +
> >> +// Provides objects and method prototypes for macro expansion
> >> +
> >> +namespace Rust {
> >> +// forward decls for AST
> >> +namespace AST {
> >> +class MacroInvocation;
> >> +}
> >> +
> >> +// Object used to store configuration data for macro expansion.
> >> +// NOTE: Keep all these items complying with the latest rustc.
> >> +struct ExpansionCfg
> >> +{
> >> + // features?
> >> + // TODO: Add `features' when we have it.
> >> + unsigned int recursion_limit = 1024;
> >> + bool trace_mac = false; // trace macro
> >> + bool should_test = false; // strip #[test] nodes if false
> >> + bool keep_macs = false; // keep macro definitions
> >> + std::string crate_name = "";
> >> +};
> >> +
> >> +struct MatchedFragment
> >> +{
> >> + std::string fragment_ident;
> >> + size_t token_offset_begin;
> >> + size_t token_offset_end;
> >> +
> >> + MatchedFragment (std::string identifier, size_t token_offset_begin,
> >> + size_t token_offset_end)
> >> + : fragment_ident (identifier), token_offset_begin (token_offset_begin),
> >> + token_offset_end (token_offset_end)
> >> + {}
> >> +
> >> + /**
> >> + * Empty constructor for uninitialized fragments
> >> + */
> >> + MatchedFragment () : MatchedFragment ("", 0, 0) {}
> >> +
> >> + std::string as_string () const
> >> + {
> >> + return fragment_ident + "=" + std::to_string (token_offset_begin) + ":"
> >> + + std::to_string (token_offset_end);
> >> + }
> >> +};
> >> +
> >> +class MatchedFragmentContainer
> >> +{
> >> +public:
> >> + // Does the container refer to a simple metavariable, different from a
> >> + // repetition repeated once
> >> + enum class Kind
> >> + {
> >> + MetaVar,
> >> + Repetition,
> >> + };
> >> +
> >> + MatchedFragmentContainer (std::vector<MatchedFragment> fragments,
> >> + Kind kind = Kind::Repetition)
> >> + : fragments (fragments), kind (kind)
> >> + {}
> >> +
> >> + /**
> >> + * Create a valid fragment matched zero times. This is useful for repetitions
> >> + * which allow the absence of a fragment, such as * and ?
> >> + */
> >> + static MatchedFragmentContainer zero ()
> >> + {
> >> + return MatchedFragmentContainer ({});
> >> + }
> >> +
> >> + /**
> >> + * Create a valid fragment matched one time
> >> + */
> >> + static MatchedFragmentContainer metavar (MatchedFragment fragment)
> >> + {
> >> + return MatchedFragmentContainer ({fragment}, Kind::MetaVar);
> >> + }
> >> +
> >> + /**
> >> + * Add a matched fragment to the container
> >> + */
> >> + void add_fragment (MatchedFragment fragment)
> >> + {
> >> + rust_assert (!is_single_fragment ());
> >> +
> >> + fragments.emplace_back (fragment);
> >> + }
> >> +
> >> + size_t get_match_amount () const { return fragments.size (); }
> >> + const std::vector<MatchedFragment> &get_fragments () const
> >> + {
> >> + return fragments;
> >> + }
> >> + // const std::string &get_fragment_name () const { return fragment_name; }
> >> +
> >> + bool is_single_fragment () const
> >> + {
> >> + return get_match_amount () == 1 && kind == Kind::MetaVar;
> >> + }
> >> +
> >> + const MatchedFragment get_single_fragment () const
> >> + {
> >> + rust_assert (is_single_fragment ());
> >> +
> >> + return fragments[0];
> >> + }
> >> +
> >> + const Kind &get_kind () const { return kind; }
> >> +
> >> +private:
> >> + /**
> >> + * Fragments matched `match_amount` times. This can be an empty vector
> >> + * in case having zero matches is allowed (i.e ? or * operators)
> >> + */
> >> + std::vector<MatchedFragment> fragments;
> >> + Kind kind;
> >> +};
> >> +
> >> +class SubstitutionScope
> >> +{
> >> +public:
> >> + SubstitutionScope () : stack () {}
> >> +
> >> + void push () { stack.push_back ({}); }
> >> +
> >> + std::map<std::string, MatchedFragmentContainer> pop ()
> >> + {
> >> + auto top = stack.back ();
> >> + stack.pop_back ();
> >> + return top;
> >> + }
> >> +
> >> + std::map<std::string, MatchedFragmentContainer> &peek ()
> >> + {
> >> + return stack.back ();
> >> + }
> >> +
> >> + /**
> >> + * Insert a new matched metavar into the current substitution map
> >> + */
> >> + void insert_metavar (MatchedFragment fragment)
> >> + {
> >> + auto ¤t_map = stack.back ();
> >> + auto it = current_map.find (fragment.fragment_ident);
> >> +
> >> + if (it == current_map.end ())
> >> + current_map.insert ({fragment.fragment_ident,
> >> + MatchedFragmentContainer::metavar (fragment)});
> >> + else
> >> + gcc_unreachable ();
> >> + }
> >> +
> >> + /**
> >> + * Append a new matched fragment to a repetition into the current substitution
> >> + * map
> >> + */
> >> + void append_fragment (MatchedFragment fragment)
> >> + {
> >> + auto ¤t_map = stack.back ();
> >> + auto it = current_map.find (fragment.fragment_ident);
> >> +
> >> + if (it == current_map.end ())
> >> + current_map.insert (
> >> + {fragment.fragment_ident, MatchedFragmentContainer ({fragment})});
> >> + else
> >> + it->second.add_fragment (fragment);
> >> + }
> >> +
> >> + void insert_matches (std::string key, MatchedFragmentContainer matches)
> >> + {
> >> + auto ¤t_map = stack.back ();
> >> + auto it = current_map.find (key);
> >> + rust_assert (it == current_map.end ());
> >> +
> >> + current_map.insert ({key, matches});
> >> + }
> >> +
> >> +private:
> >> + std::vector<std::map<std::string, MatchedFragmentContainer>> stack;
> >> +};
> >> +
> >> +// Object used to store shared data (between functions) for macro expansion.
> >> +struct MacroExpander
> >> +{
> >> + enum ContextType
> >> + {
> >> + ITEM,
> >> + BLOCK,
> >> + EXTERN,
> >> + TYPE,
> >> + TRAIT,
> >> + IMPL,
> >> + TRAIT_IMPL,
> >> + };
> >> +
> >> + ExpansionCfg cfg;
> >> + unsigned int expansion_depth = 0;
> >> +
> >> + MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session)
> >> + : cfg (cfg), crate (crate), session (session),
> >> + sub_stack (SubstitutionScope ()),
> >> + expanded_fragment (AST::ASTFragment::create_error ()),
> >> + resolver (Resolver::Resolver::get ()),
> >> + mappings (Analysis::Mappings::get ())
> >> + {}
> >> +
> >> + ~MacroExpander () = default;
> >> +
> >> + // Expands all macros in the crate passed in.
> >> + void expand_crate ();
> >> +
> >> + /* Expands a macro invocation - possibly make both
> >> + * have similar duck-typed interface and use templates?*/
> >> + // should this be public or private?
> >> + void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon);
> >> +
> >> + // Expands a single declarative macro.
> >> + AST::ASTFragment expand_decl_macro (Location locus,
> >> + AST::MacroInvocData &invoc,
> >> + AST::MacroRulesDefinition &rules_def,
> >> + bool semicolon);
> >> +
> >> + void expand_cfg_attrs (AST::AttrVec &attrs);
> >> + bool fails_cfg (const AST::AttrVec &attr) const;
> >> + bool fails_cfg_with_expand (AST::AttrVec &attrs) const;
> >> +
> >> + bool depth_exceeds_recursion_limit () const;
> >> +
> >> + bool try_match_rule (AST::MacroRule &match_rule,
> >> + AST::DelimTokenTree &invoc_token_tree);
> >> +
> >> + AST::ASTFragment transcribe_rule (
> >> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
> >> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
> >> + bool semicolon, ContextType ctx);
> >> +
> >> + bool match_fragment (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchFragment &fragment);
> >> +
> >> + bool match_token (Parser<MacroInvocLexer> &parser, AST::Token &token);
> >> +
> >> + bool match_repetition (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchRepetition &rep);
> >> +
> >> + bool match_matcher (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatcher &matcher);
> >> +
> >> + /**
> >> + * Match any amount of matches
> >> + *
> >> + * @param parser Parser to use for matching
> >> + * @param rep Repetition to try and match
> >> + * @param match_amount Reference in which to store the ammount of succesful
> >> + * and valid matches
> >> + *
> >> + * @param lo_bound Lower bound of the matcher. When specified, the matcher
> >> + * will only succeed if it parses at *least* `lo_bound` fragments. If
> >> + * unspecified, the matcher could succeed when parsing 0 fragments.
> >> + *
> >> + * @param hi_bound Higher bound of the matcher. When specified, the matcher
> >> + * will only succeed if it parses *less than* `hi_bound` fragments. If
> >> + * unspecified, the matcher could succeed when parsing an infinity of
> >> + * fragments.
> >> + *
> >> + * @return true if matching was successful and within the given limits, false
> >> + * otherwise
> >> + */
> >> + bool match_n_matches (Parser<MacroInvocLexer> &parser,
> >> + AST::MacroMatchRepetition &rep, size_t &match_amount,
> >> + size_t lo_bound = 0, size_t hi_bound = 0);
> >> +
> >> + void push_context (ContextType t) { context.push_back (t); }
> >> +
> >> + ContextType pop_context ()
> >> + {
> >> + rust_assert (!context.empty ());
> >> +
> >> + ContextType t = context.back ();
> >> + context.pop_back ();
> >> +
> >> + return t;
> >> + }
> >> +
> >> + ContextType peek_context () { return context.back (); }
> >> +
> >> + void set_expanded_fragment (AST::ASTFragment &&fragment)
> >> + {
> >> + expanded_fragment = std::move (fragment);
> >> + }
> >> +
> >> + AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis)
> >> + {
> >> + AST::ASTFragment old_fragment = std::move (expanded_fragment);
> >> + auto accumulator = std::vector<AST::SingleASTNode> ();
> >> + expanded_fragment = AST::ASTFragment::create_error ();
> >> +
> >> + for (auto &node : old_fragment.get_nodes ())
> >> + {
> >> + expansion_depth++;
> >> + node.accept_vis (vis);
> >> + // we'll decide the next move according to the outcome of the macro
> >> + // expansion
> >> + if (expanded_fragment.is_error ())
> >> + accumulator.push_back (node); // if expansion fails, there might be a
> >> + // non-macro expression we need to keep
> >> + else
> >> + {
> >> + // if expansion succeeded, then we need to merge the fragment with
> >> + // the contents in the accumulator, so that our final expansion
> >> + // result will contain non-macro nodes as it should
> >> + auto new_nodes = expanded_fragment.get_nodes ();
> >> + std::move (new_nodes.begin (), new_nodes.end (),
> >> + std::back_inserter (accumulator));
> >> + expanded_fragment = AST::ASTFragment (accumulator);
> >> + }
> >> + expansion_depth--;
> >> + }
> >> +
> >> + return old_fragment;
> >> + }
> >> +
> >> +private:
> >> + AST::Crate &crate;
> >> + Session &session;
> >> + SubstitutionScope sub_stack;
> >> + std::vector<ContextType> context;
> >> + AST::ASTFragment expanded_fragment;
> >> +
> >> +public:
> >> + Resolver::Resolver *resolver;
> >> + Analysis::Mappings *mappings;
> >> +};
> >> +
> >> +} // namespace Rust
> >> +
> >> +#endif
> >> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.cc b/gcc/rust/expand/rust-macro-invoc-lexer.cc
> >> new file mode 100644
> >> index 00000000000..8a43d29e0d1
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.cc
> >> @@ -0,0 +1,29 @@
> >> +#include "rust-macro-invoc-lexer.h"
> >> +
> >> +namespace Rust {
> >> +
> >> +const_TokenPtr
> >> +MacroInvocLexer::peek_token (int n)
> >> +{
> >> + if ((offs + n) >= token_stream.size ())
> >> + return Token::make (END_OF_FILE, Location ());
> >> +
> >> + return token_stream.at (offs + n)->get_tok_ptr ();
> >> +}
> >> +
> >> +// Advances current token to n + 1 tokens ahead of current position.
> >> +void
> >> +MacroInvocLexer::skip_token (int n)
> >> +{
> >> + offs += (n + 1);
> >> +}
> >> +
> >> +void
> >> +MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)),
> >> + TokenId new_right
> >> + __attribute__ ((unused)))
> >> +{
> >> + // FIXME
> >> + gcc_unreachable ();
> >> +}
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.h b/gcc/rust/expand/rust-macro-invoc-lexer.h
> >> new file mode 100644
> >> index 00000000000..0fd4554d02f
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.h
> >> @@ -0,0 +1,64 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#ifndef RUST_MACRO_INVOC_LEXER_H
> >> +#define RUST_MACRO_INVOC_LEXER_H
> >> +
> >> +#include "rust-ast.h"
> >> +
> >> +namespace Rust {
> >> +class MacroInvocLexer
> >> +{
> >> +public:
> >> + MacroInvocLexer (std::vector<std::unique_ptr<AST::Token>> stream)
> >> + : offs (0), token_stream (std::move (stream))
> >> + {}
> >> +
> >> + // Returns token n tokens ahead of current position.
> >> + const_TokenPtr peek_token (int n);
> >> +
> >> + // Peeks the current token.
> >> + const_TokenPtr peek_token () { return peek_token (0); }
> >> +
> >> + // Advances current token to n + 1 tokens ahead of current position.
> >> + void skip_token (int n);
> >> +
> >> + // Skips the current token.
> >> + void skip_token () { skip_token (0); }
> >> +
> >> + // Splits the current token into two. Intended for use with nested generics
> >> + // closes (i.e. T<U<X>> where >> is wrongly lexed as one token). Note that
> >> + // this will only work with "simple" tokens like punctuation.
> >> + void split_current_token (TokenId new_left, TokenId new_right);
> >> +
> >> + std::string get_filename () const
> >> + {
> >> + // FIXME
> >> + gcc_unreachable ();
> >> + return "FIXME";
> >> + }
> >> +
> >> + size_t get_offs () const { return offs; }
> >> +
> >> +private:
> >> + size_t offs;
> >> + std::vector<std::unique_ptr<AST::Token>> token_stream;
> >> +};
> >> +} // namespace Rust
> >> +
> >> +#endif // RUST_MACRO_INVOC_LEXER_H
> >> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
> >> new file mode 100644
> >> index 00000000000..9592d2d2a9e
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
> >> @@ -0,0 +1,312 @@
> >> +#include "rust-macro-substitute-ctx.h"
> >> +
> >> +namespace Rust {
> >> +
> >> +std::vector<std::unique_ptr<AST::Token>>
> >> +SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
> >> +{
> >> + auto metavar_name = metavar->get_str ();
> >> +
> >> + std::vector<std::unique_ptr<AST::Token>> expanded;
> >> + auto it = fragments.find (metavar_name);
> >> + if (it == fragments.end ())
> >> + {
> >> + // Return a copy of the original token
> >> + expanded.push_back (metavar->clone_token ());
> >> + }
> >> + else
> >> + {
> >> + // If we are expanding a metavar which has a lof of matches, we are
> >> + // currently expanding a repetition metavar - not a simple metavar. We
> >> + // need to error out and inform the user.
> >> + // Associated test case for an example: compile/macro-issue1224.rs
> >> + if (it->second.get_match_amount () != 1)
> >> + {
> >> + rust_error_at (metavar->get_locus (),
> >> + "metavariable is still repeating at this depth");
> >> + rust_inform (
> >> + metavar->get_locus (),
> >> + "you probably forgot the repetition operator: %<%s%s%s%>", "$(",
> >> + metavar->as_string ().c_str (), ")*");
> >> + return expanded;
> >> + }
> >> +
> >> + // We only care about the vector when expanding repetitions.
> >> + // Just access the first element of the vector.
> >> + auto &frag = it->second.get_single_fragment ();
> >> + for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
> >> + offs++)
> >> + {
> >> + auto &tok = input.at (offs);
> >> + expanded.push_back (tok->clone_token ());
> >> + }
> >> + }
> >> +
> >> + return expanded;
> >> +}
> >> +
> >> +bool
> >> +SubstituteCtx::check_repetition_amount (size_t pattern_start,
> >> + size_t pattern_end,
> >> + size_t &expected_repetition_amount)
> >> +{
> >> + bool first_fragment_found = false;
> >> + bool is_valid = true;
> >> +
> >> + for (size_t i = pattern_start; i < pattern_end; i++)
> >> + {
> >> + if (macro.at (i)->get_id () == DOLLAR_SIGN)
> >> + {
> >> + auto &frag_token = macro.at (i + 1);
> >> + if (frag_token->get_id () == IDENTIFIER)
> >> + {
> >> + auto it = fragments.find (frag_token->get_str ());
> >> + if (it == fragments.end ())
> >> + {
> >> + // If the repetition is not anything we know (ie no declared
> >> + // metavars, or metavars which aren't present in the
> >> + // fragment), we can just error out. No need to paste the
> >> + // tokens as if nothing had happened.
> >> + rust_error_at (frag_token->get_locus (),
> >> + "metavar %s used in repetition does not exist",
> >> + frag_token->get_str ().c_str ());
> >> +
> >> + is_valid = false;
> >> + }
> >> +
> >> + auto &fragment = it->second;
> >> +
> >> + size_t repeat_amount = fragment.get_match_amount ();
> >> + if (!first_fragment_found)
> >> + {
> >> + first_fragment_found = true;
> >> + expected_repetition_amount = repeat_amount;
> >> + }
> >> + else
> >> + {
> >> + if (repeat_amount != expected_repetition_amount
> >> + && !fragment.is_single_fragment ())
> >> + {
> >> + rust_error_at (
> >> + frag_token->get_locus (),
> >> + "different amount of matches used in merged "
> >> + "repetitions: expected %lu, got %lu",
> >> + (unsigned long) expected_repetition_amount,
> >> + (unsigned long) repeat_amount);
> >> + is_valid = false;
> >> + }
> >> + }
> >> + }
> >> + }
> >> + }
> >> +
> >> + return is_valid;
> >> +}
> >> +
> >> +std::vector<std::unique_ptr<AST::Token>>
> >> +SubstituteCtx::substitute_repetition (
> >> + size_t pattern_start, size_t pattern_end,
> >> + std::unique_ptr<AST::Token> separator_token)
> >> +{
> >> + rust_assert (pattern_end < macro.size ());
> >> +
> >> + size_t repeat_amount = 0;
> >> + if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount))
> >> + return {};
> >> +
> >> + rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount);
> >> + std::vector<std::unique_ptr<AST::Token>> expanded;
> >> + std::vector<std::unique_ptr<AST::Token>> new_macro;
> >> +
> >> + // We want to generate a "new macro" to substitute with. This new macro
> >> + // should contain only the tokens inside the pattern
> >> + for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++)
> >> + new_macro.emplace_back (macro.at (tok_idx)->clone_token ());
> >> +
> >> + // Then, we want to create a subset of the matches so that
> >> + // `substitute_tokens()` can only see one fragment per metavar. Let's say we
> >> + // have the following user input: (1 145 'h')
> >> + // on the following match arm: ($($lit:literal)*)
> >> + // which causes the following matches: { "lit": [1, 145, 'h'] }
> >> + //
> >> + // The pattern (new_macro) is `$lit:literal`
> >> + // The first time we expand it, we want $lit to have the following token: 1
> >> + // The second time, 145
> >> + // The third and final time, 'h'
> >> + //
> >> + // In order to do so we must create "sub maps", which only contain parts of
> >> + // the original matches
> >> + // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ]
> >> + //
> >> + // and give them to `substitute_tokens` one by one.
> >> +
> >> + for (size_t i = 0; i < repeat_amount; i++)
> >> + {
> >> + std::map<std::string, MatchedFragmentContainer> sub_map;
> >> + for (auto &kv_match : fragments)
> >> + {
> >> + MatchedFragment sub_fragment;
> >> +
> >> + // FIXME: Hack: If a fragment is not repeated, how does it fit in the
> >> + // submap? Do we really want to expand it? Is this normal behavior?
> >> + if (kv_match.second.is_single_fragment ())
> >> + sub_fragment = kv_match.second.get_single_fragment ();
> >> + else
> >> + sub_fragment = kv_match.second.get_fragments ()[i];
> >> +
> >> + sub_map.insert (
> >> + {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)});
> >> + }
> >> +
> >> + auto substitute_context = SubstituteCtx (input, new_macro, sub_map);
> >> + auto new_tokens = substitute_context.substitute_tokens ();
> >> +
> >> + // Skip the first repetition, but add the separator to the expanded
> >> + // tokens if it is present
> >> + if (i != 0 && separator_token)
> >> + expanded.emplace_back (separator_token->clone_token ());
> >> +
> >> + for (auto &new_token : new_tokens)
> >> + expanded.emplace_back (new_token->clone_token ());
> >> + }
> >> +
> >> + // FIXME: We also need to make sure that all subsequent fragments
> >> + // contain the same amount of repetitions as the first one
> >> +
> >> + return expanded;
> >> +}
> >> +
> >> +static bool
> >> +is_rep_op (std::unique_ptr<AST::Token> &tok)
> >> +{
> >> + auto id = tok->get_id ();
> >> + return id == QUESTION_MARK || id == ASTERISK || id == PLUS;
> >> +}
> >> +
> >> +std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
> >> +SubstituteCtx::substitute_token (size_t token_idx)
> >> +{
> >> + auto &token = macro.at (token_idx);
> >> + switch (token->get_id ())
> >> + {
> >> + case IDENTIFIER:
> >> + rust_debug ("expanding metavar: %s", token->get_str ().c_str ());
> >> + return {substitute_metavar (token), 1};
> >> + case LEFT_PAREN: {
> >> + // We need to parse up until the closing delimiter and expand this
> >> + // fragment->n times.
> >> + rust_debug ("expanding repetition");
> >> +
> >> + // We're in a context where macro repetitions have already been
> >> + // parsed and validated: This means that
> >> + // 1/ There will be no delimiters as that is an error
> >> + // 2/ There are no fragment specifiers anymore, which prevents us
> >> + // from reusing parser functions.
> >> + //
> >> + // Repetition patterns are also special in that they cannot contain
> >> + // "rogue" delimiters: For example, this is invalid, as they are
> >> + // parsed as MacroMatches and must contain a correct amount of
> >> + // delimiters.
> >> + // `$($e:expr ) )`
> >> + // ^ rogue closing parenthesis
> >> + //
> >> + // With all of that in mind, we can simply skip ahead from one
> >> + // parenthesis to the other to find the pattern to expand. Of course,
> >> + // pairs of delimiters, including parentheses, are allowed.
> >> + // `$($e:expr ( ) )`
> >> + // Parentheses are the sole delimiter for which we need a special
> >> + // behavior since they delimit the repetition pattern
> >> +
> >> + size_t pattern_start = token_idx + 1;
> >> + size_t pattern_end = pattern_start;
> >> + auto parentheses_stack = 0;
> >> + for (size_t idx = pattern_start; idx < macro.size (); idx++)
> >> + {
> >> + if (macro.at (idx)->get_id () == LEFT_PAREN)
> >> + {
> >> + parentheses_stack++;
> >> + }
> >> + else if (macro.at (idx)->get_id () == RIGHT_PAREN)
> >> + {
> >> + if (parentheses_stack == 0)
> >> + {
> >> + pattern_end = idx;
> >> + break;
> >> + }
> >> + parentheses_stack--;
> >> + }
> >> + }
> >> +
> >> + // Unreachable case, but let's make sure we don't ever run into it
> >> + rust_assert (pattern_end != pattern_start);
> >> +
> >> + std::unique_ptr<AST::Token> separator_token = nullptr;
> >> + if (pattern_end + 1 <= macro.size ())
> >> + {
> >> + auto &post_pattern_token = macro.at (pattern_end + 1);
> >> + if (!is_rep_op (post_pattern_token))
> >> + separator_token = post_pattern_token->clone_token ();
> >> + }
> >> +
> >> + // Amount of tokens to skip
> >> + auto to_skip = 0;
> >> + // Parentheses
> >> + to_skip += 2;
> >> + // Repetition operator
> >> + to_skip += 1;
> >> + // Separator
> >> + if (separator_token)
> >> + to_skip += 1;
> >> +
> >> + return {substitute_repetition (pattern_start, pattern_end,
> >> + std::move (separator_token)),
> >> + pattern_end - pattern_start + to_skip};
> >> + }
> >> + // TODO: We need to check if the $ was alone. In that case, do
> >> + // not error out: Simply act as if there was an empty identifier
> >> + // with no associated fragment and paste the dollar sign in the
> >> + // transcription. Unsure how to do that since we always have at
> >> + // least the closing curly brace after an empty $...
> >> + default:
> >> + rust_error_at (token->get_locus (),
> >> + "unexpected token in macro transcribe: expected "
> >> + "%<(%> or identifier after %<$%>, got %<%s%>",
> >> + get_token_description (token->get_id ()));
> >> + }
> >> +
> >> + // FIXME: gcc_unreachable() error case?
> >> + return {std::vector<std::unique_ptr<AST::Token>> (), 0};
> >> +}
> >> +
> >> +std::vector<std::unique_ptr<AST::Token>>
> >> +SubstituteCtx::substitute_tokens ()
> >> +{
> >> + std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
> >> + rust_debug ("expanding tokens");
> >> +
> >> + for (size_t i = 0; i < macro.size (); i++)
> >> + {
> >> + auto &tok = macro.at (i);
> >> + if (tok->get_id () == DOLLAR_SIGN)
> >> + {
> >> + // Aaaaah, if only we had C++17 :)
> >> + // auto [expanded, tok_to_skip] = ...
> >> + auto p = substitute_token (i + 1);
> >> + auto expanded = std::move (p.first);
> >> + auto tok_to_skip = p.second;
> >> +
> >> + i += tok_to_skip;
> >> +
> >> + for (auto &token : expanded)
> >> + replaced_tokens.emplace_back (token->clone_token ());
> >> + }
> >> + else
> >> + {
> >> + replaced_tokens.emplace_back (tok->clone_token ());
> >> + }
> >> + }
> >> +
> >> + return replaced_tokens;
> >> +}
> >> +
> >> +} // namespace Rust
> >> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.h b/gcc/rust/expand/rust-macro-substitute-ctx.h
> >> new file mode 100644
> >> index 00000000000..81dcab7643b
> >> --- /dev/null
> >> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.h
> >> @@ -0,0 +1,93 @@
> >> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
> >> +
> >> +// This file is part of GCC.
> >> +
> >> +// GCC is free software; you can redistribute it and/or modify it under
> >> +// the terms of the GNU General Public License as published by the Free
> >> +// Software Foundation; either version 3, or (at your option) any later
> >> +// version.
> >> +
> >> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> >> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
> >> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> >> +// for more details.
> >> +
> >> +// You should have received a copy of the GNU General Public License
> >> +// along with GCC; see the file COPYING3. If not see
> >> +// <http://www.gnu.org/licenses/>.
> >> +
> >> +#include "rust-ast.h"
> >> +#include "rust-macro-expand.h"
> >> +
> >> +namespace Rust {
> >> +class SubstituteCtx
> >> +{
> >> + std::vector<std::unique_ptr<AST::Token>> &input;
> >> + std::vector<std::unique_ptr<AST::Token>> ¯o;
> >> + std::map<std::string, MatchedFragmentContainer> &fragments;
> >> +
> >> + /**
> >> + * Find the repetition amount to use when expanding a repetition, and
> >> + * check that all fragments used respect that repetition amount
> >> + *
> >> + * @param pattern_start Start of the repetition pattern
> >> + * @param pattern_end End of the repetition pattern
> >> + * @param repeat_amount Reference to fill with the matched repetition amount
> >> + */
> >> + bool check_repetition_amount (size_t pattern_start, size_t pattern_end,
> >> + size_t &repeat_amount);
> >> +
> >> +public:
> >> + SubstituteCtx (std::vector<std::unique_ptr<AST::Token>> &input,
> >> + std::vector<std::unique_ptr<AST::Token>> ¯o,
> >> + std::map<std::string, MatchedFragmentContainer> &fragments)
> >> + : input (input), macro (macro), fragments (fragments)
> >> + {}
> >> +
> >> + /**
> >> + * Substitute a metavariable by its given fragment in a transcribing context,
> >> + * i.e. replacing $var with the associated fragment.
> >> + *
> >> + * @param metavar Metavariable to try and replace
> >> + *
> >> + * @return A token containing the associated fragment expanded into tokens if
> >> + * any, or the cloned token if no fragment was associated
> >> + */
> >> + std::vector<std::unique_ptr<AST::Token>>
> >> + substitute_metavar (std::unique_ptr<AST::Token> &metavar);
> >> +
> >> + /**
> >> + * Substitute a macro repetition by its given fragments
> >> + *
> >> + * @param pattern_start Start index of the pattern tokens
> >> + * @param pattern_end End index of the patterns tokens
> >> + * @param separator Optional separator to include when expanding tokens
> >> + *
> >> + * @return A vector containing the repeated pattern
> >> + */
> >> + std::vector<std::unique_ptr<AST::Token>>
> >> + substitute_repetition (size_t pattern_start, size_t pattern_end,
> >> + std::unique_ptr<AST::Token> separator);
> >> +
> >> + /**
> >> + * Substitute a given token by its appropriate representation
> >> + *
> >> + * @param token_idx Current token to try and substitute
> >> + *
> >> + * @return A token containing the associated fragment expanded into tokens if
> >> + * any, or the cloned token if no fragment was associated, as well as the
> >> + * amount of tokens that should be skipped before the next invocation. Since
> >> + * this function may consume more than just one token, it is important to skip
> >> + * ahead of the input to avoid mis-substitutions
> >> + */
> >> + std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
> >> + substitute_token (size_t token_idx);
> >> +
> >> + /**
> >> + * Substitute all tokens by their appropriate representation
> >> + *
> >> + * @return A vector containing the substituted tokens
> >> + */
> >> + std::vector<std::unique_ptr<AST::Token>> substitute_tokens ();
> >> +};
> >> +} // namespace Rust
> >> --
> >> 2.37.2
> >>
>
>
> Kindly,
>
> Arthur
On 11/16/22 16:49, Richard Biener wrote:
> On Tue, Nov 15, 2022 at 12:32 PM Arthur Cohen <arthur.cohen@embecosm.com> wrote:
>>
>>
>>
>> On 11/10/22 11:49, Richard Biener wrote:
>>> On Wed, Oct 26, 2022 at 10:28 AM <arthur.cohen@embecosm.com> wrote:
>>>>
>>>> From: Arthur Cohen <arthur.cohen@embecosm.com>
>>>>
>>>> The expansion pass is responsible for two actions on our AST:
>>>>
>>>> 1. Expanding macro calls
>>>> 2. Performing conditional compilation
>>>>
>>>> Calls to macros should be checked and expanded into an AST fragment based on
>>>> the context they've been called in. This is similar to token substitution, with
>>>> a lot of intricacies and checks being performed. A single invocation can result
>>>> in an AST fragment containing multiple statements or multiple expressions,
>>>> which need to be handled as well. Furthermore, Rust macros can contain
>>>> repetitions relying on Kleine operators, similar to regular expression
>>>> patterns, that also need to be expanded properly.
>>>>
>>>> Finally, Rust code can be hidden behind `cfg` directives, which allow the user
>>>> to perform conditional compilation. If a `cfg` predicate is not met, the
>>>> expression or statement it refers to should be marked for strip and removed
>>>> from the AST.
>>>
>>> I'm just picking one of the purely rust frontend specific patches as example and
>>> do not plan to look on others - I don't think meaningful review can happen by
>>> somebody not deeply understanding rust or this frontend can be done which means
>>> approval of those parts shouldn't be needed but would be covered (even
>>> the initial
>>> contribution) by your maintainership of the code.
>>>
>>> The commit comment above is quite helpful in getting an idea of what this part
>>> of the rust frontend is doing. I'd appreciate if something along these lines
>>> would be added to each of the individual source files (preferably the
>>> implementation, not the header).
>>
>> Is there any reason as to why the implementation should be prefered over
>> the header?
>
> That's how it's done elsewhere in GCC and it aligns with the function
> documentation presence.
>
>> This is a comment you also make down the line on one of our
>> function's documentation. My view is that the header should be kept as
>> simple and possible, with as little code as possible, and that it is
>> easy to find documentation around function declarations, much easier
>> than to find it in the implementation when code blocks are present all
>> around.
>
> That's true when working with an API but not when working on the implementation.
>
>> To be clear, I'm not opposed to switching the documentation over to our
>> implementation - just looking to understand. We'll make the necessary
>> changes.
>
> I guess consistency is then most important (consistency in rust/). Consistency
> with GCC is nice. I'll note that when the implementation resides in the header
> like for templates or inlines the documentation also resides there. That might
> be an argument for documentation in the header (in one place).
>
Thanks for taking the time to explain. I'll move over the documentation
to the implementation instead of the headers. We'll have this in the v4
which I'm working on at the moment and will send shortly.
>>> Documentation of the sub-directory structure would
>>> also be nice,
>>> we have (had ..) sourcebuild.texi which contains some of that but not really for
>>> specific frontends (sofar).
>>
>> Right. We will work on adding proper documentation ASAP.
>>
>>> Some comments on this particular patch that probably apply to all others as well
>>> inline below.
>>>
>>>> Co-authored-by: Philip Herron <philip.herron@embecosm.com>
>>>> Co-authored-by: The Other <simplytheother@gmail.com>
>>>> ---
>>>> gcc/rust/expand/rust-attribute-visitor.cc | 3445 ++++++++++++++++++
>>>> gcc/rust/expand/rust-attribute-visitor.h | 316 ++
>>>> gcc/rust/expand/rust-macro-builtins.cc | 484 +++
>>>> gcc/rust/expand/rust-macro-builtins.h | 107 +
>>>> gcc/rust/expand/rust-macro-expand.cc | 1012 +++++
>>>> gcc/rust/expand/rust-macro-expand.h | 366 ++
>>>> gcc/rust/expand/rust-macro-invoc-lexer.cc | 29 +
>>>> gcc/rust/expand/rust-macro-invoc-lexer.h | 64 +
>>>> gcc/rust/expand/rust-macro-substitute-ctx.cc | 312 ++
>>>> gcc/rust/expand/rust-macro-substitute-ctx.h | 93 +
>>>> 10 files changed, 6228 insertions(+)
>>>> create mode 100644 gcc/rust/expand/rust-attribute-visitor.cc
>>>> create mode 100644 gcc/rust/expand/rust-attribute-visitor.h
>>>> create mode 100644 gcc/rust/expand/rust-macro-builtins.cc
>>>> create mode 100644 gcc/rust/expand/rust-macro-builtins.h
>>>> create mode 100644 gcc/rust/expand/rust-macro-expand.cc
>>>> create mode 100644 gcc/rust/expand/rust-macro-expand.h
>>>> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.cc
>>>> create mode 100644 gcc/rust/expand/rust-macro-invoc-lexer.h
>>>> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.cc
>>>> create mode 100644 gcc/rust/expand/rust-macro-substitute-ctx.h
>>>>
>>>> diff --git a/gcc/rust/expand/rust-attribute-visitor.cc b/gcc/rust/expand/rust-attribute-visitor.cc
>>>> new file mode 100644
>>>> index 00000000000..8016f9430eb
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-attribute-visitor.cc
>>>> @@ -0,0 +1,3445 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#include "rust-attribute-visitor.h"
>>>> +#include "rust-session-manager.h"
>>>> +
>>>> +namespace Rust {
>>>> +
>>>> +// Visitor used to expand attributes.
>>>> +void
>>>> +AttrVisitor::expand_struct_fields (std::vector<AST::StructField> &fields)
>>>> +{
>>>> + for (auto it = fields.begin (); it != fields.end ();)
>>>> + {
>>>> + auto &field = *it;
>>>> +
>>>> + auto &field_attrs = field.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (field_attrs);
>>>> + if (expander.fails_cfg_with_expand (field_attrs))
>>>> + {
>>>> + it = fields.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // expand sub-types of type, but can't strip type itself
>>>> + auto &type = field.get_field_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // if nothing else happens, increment
>>>> + ++it;
>>>> + }
>>>> +}
>>>> +
>>>
>>> The coding conventions warrant a function level comment for each function.
>>> If there's anything meaningful to say (in this case "// expand tuple fields"
>>> wouldn't be meaningful) please consider adding such to all functions. In
>>> my view every function that doesn't fit in a 80x24 terminal warrants one.
>>>
>>>> +void
>>>> +AttrVisitor::expand_tuple_fields (std::vector<AST::TupleField> &fields)
>>>> +{
>>>> + for (auto it = fields.begin (); it != fields.end ();)
>>>> + {
>>>> + auto &field = *it;
>>>> +
>>>> + auto &field_attrs = field.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (field_attrs);
>>>> + if (expander.fails_cfg_with_expand (field_attrs))
>>>> + {
>>>> + it = fields.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + // expand sub-types of type, but can't strip type itself
>>>> + auto &type = field.get_field_type ();
>>>> + type->accept_vis (*this);
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>
>>> When reading the other patches I wondered whether rust_error_at are
>>> user-facing diagnostics or internal errors?
>>>
>>>> +
>>>> + // if nothing else happens, increment
>>>> + ++it;
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_function_params (std::vector<AST::FunctionParam> ¶ms)
>>>> +{
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + for (auto it = params.begin (); it != params.end ();)
>>>> + {
>>>> + auto ¶m = *it;
>>>> +
>>>> + auto ¶m_attrs = param.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (param_attrs);
>>>> + if (expander.fails_cfg_with_expand (param_attrs))
>>>> + {
>>>> + it = params.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + // TODO: should an unwanted strip lead to break out of loop?
>>>> + auto &pattern = param.get_pattern ();
>>>
>>> 'auto' is used a lot in the frontend, it does make reading code harder because
>>> critical information on context is lost. Here I can at least 2nd guess
>>> 'param' being of type AST::FunctionParam and thus find get_pattern (),
>>> but I will have to find that to be able to find accept_vis () below.
>>>
>>> In my mind 'auto' is useful when it hides information that's not used as in
>>>
>>> for (auto it = ...)
>>> {
>>> AST::FunctionParam ¶m = *it;
>>>
>>> but not so much for being lazy when typing.
>>>
>>> At least with (some of) the uses the identifiers of the methods look
>>> unique enough so grep will be of help, OTOH is_marked_for_strip
>>> is defined a lot.
>>>
>>
>> I think removing all the `auto`s from the gccrs codebase would be a
>> herculean task at this point, as a quick grep for 'auto ' in our
>> frontend reveals around 2700 matches.
>>
>> From the little research I did, it seems the C++ frontend also uses it
>> somewhat, as well as the D frontend (but that frontend is written in D,
>> so that might be moot).
>>
>> Paired with descriptive variable names and short functions, I personally
>> don't have an issue with reviewing code that uses `auto` but I see how
>> that might be the case, especially for someone who is not familiar with
>> our frontend.
>>
>> So I think the question I'm asking is as follows: considering Philip and
>> I will be reviewers of the Rust frontend part specifically, is it okay
>> to not make such a huge change to the frontend which would cost us a lot
>> of time while not necessarily making a difference to us? Again, I
>> understand that this makes a difference to some people, but as you
>> mentioned I'm not sure anyone would be willing to review all of our
>> frontend code, especially as it contains around 160 000 lines of code.
>
> Yes, my remark was quite general about 'auto' (I see uses creeping in
> in GCC elsewhere), sorry with putting this into a rust frontend review mail ;)
>
> It just occured to me that writing 'tree' isn't any bigger than 'auto' but it
> would have helped me a lot identifying pieces that work with GENERIC.
>
>> Furthering my point on the D frontend: it's also no secret that we would
>> like to rewrite some parts of the frontend in Rust in the future, as a
>> functional programming language is well suited to the compilation of
>> Rust. When that is the case, we will use what I would consider
>> "idiomatic Rust", which makes heavy use of the `let` keyword (similar in
>> many ways to `auto`) and type inference. Types are rarely explicitly
>> specified for variables, but are always specified for function arguments
>> or function return types (as we do presently in the C++ implementation
>> of our frontend).
>>
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +
>>>> + auto &type = param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + // increment
>>>> + ++it;
>>>> + }
>>>> +
>>>> + expander.pop_context ();
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_generic_args (AST::GenericArgs &args)
>>>> +{
>>>> + // lifetime args can't be expanded
>>>> + // FIXME: Can we have macro invocations for lifetimes?
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // expand type args - strip sub-types only
>>>> + for (auto &arg : args.get_generic_args ())
>>>> + {
>>>> + switch (arg.get_kind ())
>>>> + {
>>>> + case AST::GenericArg::Kind::Type: {
>>>> + auto &type = arg.get_type ();
>>>> + type->accept_vis (*this);
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> + break;
>>>> + }
>>>> + case AST::GenericArg::Kind::Const: {
>>>> + auto &expr = arg.get_expression ();
>>>> + expr->accept_vis (*this);
>>>> + maybe_expand_expr (expr);
>>>> +
>>>> + if (expr->is_marked_for_strip ())
>>>> + rust_error_at (expr->get_locus (),
>>>> + "cannot strip expression in this position");
>>>> + break;
>>>> + }
>>>> + default:
>>>> + break;
>>>> + // FIXME: Figure out what to do here if there is ambiguity. Since the
>>>> + // resolver comes after the expansion, we need to figure out a way to
>>>> + // strip ambiguous values here
>>>> + // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg`
>>>> + // or something. This would clean up this whole thing
>>>> + }
>>>> + }
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // FIXME: Can we have macro invocations in generic type bindings?
>>>> + // expand binding args - strip sub-types only
>>>> + for (auto &binding : args.get_binding_args ())
>>>> + {
>>>> + auto &type = binding.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type)
>>>> +{
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = path_type.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + if (path_type.has_as_clause ())
>>>> + {
>>>> + auto &type_path = path_type.get_as_type_path ();
>>>> + visit (type_path);
>>>> + if (type_path.is_marked_for_strip ())
>>>> + rust_error_at (type_path.get_locus (),
>>>> + "cannot strip type path in this position");
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::AttrVisitor::expand_closure_params (
>>>> + std::vector<AST::ClosureParam> ¶ms)
>>>> +{
>>>> + for (auto it = params.begin (); it != params.end ();)
>>>> + {
>>>> + auto ¶m = *it;
>>>> +
>>>> + auto ¶m_attrs = param.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (param_attrs);
>>>> + if (expander.fails_cfg_with_expand (param_attrs))
>>>> + {
>>>> + it = params.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + auto &pattern = param.get_pattern ();
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +
>>>> + if (param.has_type_given ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> + auto &type = param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + // increment if found nothing else so far
>>>> + ++it;
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_self_param (AST::SelfParam &self_param)
>>>> +{
>>>> + if (self_param.has_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> + auto &type = self_param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> + /* TODO: maybe check for invariants being violated - e.g. both type and
>>>> + * lifetime? */
>>>
>>> C-style comment where I see C++-style comments everywhere else.
>>>
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_where_clause (AST::WhereClause &where_clause)
>>>> +{
>>>> + // items cannot be stripped conceptually, so just accept visitor
>>>> + for (auto &item : where_clause.get_items ())
>>>> + item->accept_vis (*this);
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl)
>>>> +{
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : decl.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* strip function parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_function_params (decl.get_function_params ());
>>>> +
>>>> + if (decl.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = decl.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + if (decl.has_where_clause ())
>>>> + expand_where_clause (decl.get_where_clause ());
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl)
>>>> +{
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : decl.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* assuming you can't strip self param - wouldn't be a method
>>>> + * anymore. spec allows outer attrs on self param, but doesn't
>>>> + * specify whether cfg is used. */
>>>> + expand_self_param (decl.get_self_param ());
>>>> +
>>>> + /* strip function parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_function_params (decl.get_function_params ());
>>>> +
>>>> + if (decl.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = decl.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + if (decl.has_where_clause ())
>>>> + expand_where_clause (decl.get_where_clause ());
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::Token &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::DelimTokenTree &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::AttrInputMetaItemContainer &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IdentifierExpr &ident_expr)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (ident_expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ()))
>>>> + {
>>>> + ident_expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Lifetime &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::LifetimeParam &)
>>>> +{
>>>> + // supposedly does not require - cfg does nothing
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ConstGenericParam &)
>>>> +{
>>>> + // likewise
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::MacroInvocation ¯o_invoc)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ()))
>>>> + {
>>>> + macro_invoc.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip simple path
>>>> +
>>>> + // I don't think any macro token trees can be stripped in any way
>>>> +
>>>> + // TODO: maybe have cfg! macro stripping behaviour here?
>>>> + expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ());
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::PathInExpression &path)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (path.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
>>>> + {
>>>> + path.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &segment : path.get_segments ())
>>>> + {
>>>> + if (segment.has_generic_args ())
>>>> + expand_generic_args (segment.get_generic_args ());
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypePathSegment &)
>>>> +{
>>>> + // shouldn't require
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypePathSegmentGeneric &segment)
>>>> +{
>>>> + // TODO: strip inside generic args
>>>> +
>>>> + if (!segment.has_generic_args ())
>>>> + return;
>>>> +
>>>> + expand_generic_args (segment.get_generic_args ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypePathSegmentFunction &segment)
>>>> +{
>>>> + auto &type_path_function = segment.get_type_path_function ();
>>>> +
>>>> + for (auto &type : type_path_function.get_params ())
>>>> + {
>>>> + type->accept_vis (*this);
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> + }
>>>> +
>>>> + if (type_path_function.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = type_path_function.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypePath &path)
>>>> +{
>>>> + // this shouldn't strip any segments, but can strip inside them
>>>> + for (auto &segment : path.get_segments ())
>>>> + segment->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::QualifiedPathInExpression &path)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (path.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
>>>> + {
>>>> + path.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expand_qualified_path_type (path.get_qualified_path_type ());
>>>> +
>>>> + for (auto &segment : path.get_segments ())
>>>> + {
>>>> + if (segment.has_generic_args ())
>>>> + expand_generic_args (segment.get_generic_args ());
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::QualifiedPathInType &path)
>>>> +{
>>>> + expand_qualified_path_type (path.get_qualified_path_type ());
>>>> +
>>>> + // this shouldn't strip any segments, but can strip inside them
>>>> + for (auto &segment : path.get_segments ())
>>>> + segment->accept_vis (*this);
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::LiteralExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::AttrInputLiteral &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaItemLitExpr &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaItemPathLit &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::BorrowExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &borrowed_expr = expr.get_borrowed_expr ();
>>>> + borrowed_expr->accept_vis (*this);
>>>> + if (borrowed_expr->is_marked_for_strip ())
>>>> + rust_error_at (borrowed_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::DereferenceExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &dereferenced_expr = expr.get_dereferenced_expr ();
>>>> + dereferenced_expr->accept_vis (*this);
>>>> + if (dereferenced_expr->is_marked_for_strip ())
>>>> + rust_error_at (dereferenced_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ErrorPropagationExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &propagating_expr = expr.get_propagating_expr ();
>>>> + propagating_expr->accept_vis (*this);
>>>> + if (propagating_expr->is_marked_for_strip ())
>>>> + rust_error_at (propagating_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::NegationExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &negated_expr = expr.get_negated_expr ();
>>>> + negated_expr->accept_vis (*this);
>>>> + if (negated_expr->is_marked_for_strip ())
>>>> + rust_error_at (negated_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &l_expr = expr.get_left_expr ();
>>>> + l_expr->accept_vis (*this);
>>>> + maybe_expand_expr (l_expr);
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &r_expr = expr.get_right_expr ();
>>>> + r_expr->accept_vis (*this);
>>>> + maybe_expand_expr (r_expr);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before binary op exprs");
>>>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ComparisonExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &l_expr = expr.get_left_expr ();
>>>> + l_expr->accept_vis (*this);
>>>> + maybe_expand_expr (l_expr);
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &r_expr = expr.get_right_expr ();
>>>> + r_expr->accept_vis (*this);
>>>> + maybe_expand_expr (r_expr);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before binary op exprs");
>>>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>
>>> vertical space between function definitions missing here (and above and below)
>>>
>>>> +void
>>>> +AttrVisitor::visit (AST::LazyBooleanExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &l_expr = expr.get_left_expr ();
>>>> + l_expr->accept_vis (*this);
>>>> + maybe_expand_expr (l_expr);
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &r_expr = expr.get_right_expr ();
>>>> + r_expr->accept_vis (*this);
>>>> + maybe_expand_expr (r_expr);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before binary op exprs");
>>>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypeCastExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * direct descendant expression, can strip ones below that */
>>>> +
>>>> + auto &casted_expr = expr.get_casted_expr ();
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + casted_expr->accept_vis (*this);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (casted_expr->is_marked_for_strip ())
>>>> + rust_error_at (casted_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed before cast exprs");
>>>> +
>>>> + // TODO: strip sub-types of type
>>>> + auto &type = expr.get_type_to_cast_to ();
>>>> + type->accept_vis (*this);
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::AssignmentExpr &expr)
>>>> +{
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &l_expr = expr.get_left_expr ();
>>>> + l_expr->accept_vis (*this);
>>>> + maybe_expand_expr (l_expr);
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &r_expr = expr.get_right_expr ();
>>>> + r_expr->accept_vis (*this);
>>>> + maybe_expand_expr (r_expr);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before binary op exprs");
>>>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::CompoundAssignmentExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &l_expr = expr.get_left_expr ();
>>>> + l_expr->accept_vis (*this);
>>>> + maybe_expand_expr (l_expr);
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &r_expr = expr.get_right_expr ();
>>>> + r_expr->accept_vis (*this);
>>>> + maybe_expand_expr (r_expr);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_left_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_left_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before binary op exprs");
>>>> + if (expr.get_right_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_right_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::GroupedExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says these are inner
>>>> + * attributes, not outer attributes of inner expr */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &inner_expr = expr.get_expr_in_parens ();
>>>> + inner_expr->accept_vis (*this);
>>>> + if (inner_expr->is_marked_for_strip ())
>>>> + rust_error_at (inner_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArrayElemsValues &elems)
>>>> +{
>>>> + /* apparently outer attributes are allowed in "elements of array
>>>> + * expressions" according to spec */
>>>> + expand_pointer_allow_strip (elems.get_values ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArrayElemsCopied &elems)
>>>> +{
>>>> + /* apparently outer attributes are allowed in "elements of array
>>>> + * expressions" according to spec. on the other hand, it would not
>>>> + * make conceptual sense to be able to remove either expression. As
>>>> + * such, not implementing. TODO clear up the ambiguity here */
>>>> +
>>>> + // only intend stripping for internal sub-expressions
>>>> + auto &copied_expr = elems.get_elem_to_copy ();
>>>> + copied_expr->accept_vis (*this);
>>>> + if (copied_expr->is_marked_for_strip ())
>>>> + rust_error_at (copied_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + auto ©_count = elems.get_num_copies ();
>>>> + copy_count->accept_vis (*this);
>>>> + if (copy_count->is_marked_for_strip ())
>>>> + rust_error_at (copy_count->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArrayExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says there are separate
>>>> + * inner attributes, not just outer attributes of inner exprs */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* assuming you can't strip away the ArrayElems type, but can strip
>>>> + * internal expressions and whatever */
>>>> + expr.get_array_elems ()->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArrayIndexExpr &expr)
>>>> +{
>>>> + /* it is unclear whether outer attributes are supposed to be
>>>> + * allowed, but conceptually it wouldn't make much sense, but
>>>> + * having expansion code anyway. TODO */
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &array_expr = expr.get_array_expr ();
>>>> + array_expr->accept_vis (*this);
>>>> + if (array_expr->is_marked_for_strip ())
>>>> + rust_error_at (array_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + auto &index_expr = expr.get_index_expr ();
>>>> + index_expr->accept_vis (*this);
>>>> + if (index_expr->is_marked_for_strip ())
>>>> + rust_error_at (index_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleExpr &expr)
>>>> +{
>>>> + /* according to spec, outer attributes are allowed on "elements of
>>>> + * tuple expressions" */
>>>> +
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says these are inner
>>>> + * attributes, not outer attributes of inner expr */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* apparently outer attributes are allowed in "elements of tuple
>>>> + * expressions" according to spec */
>>>> + expand_pointer_allow_strip (expr.get_tuple_elems ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleIndexExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* wouldn't strip this directly (as outer attrs should be
>>>> + * associated with this level), but any sub-expressions would be
>>>> + * stripped. Thus, no need to erase when strip check called. */
>>>> + auto &tuple_expr = expr.get_tuple_expr ();
>>>> + tuple_expr->accept_vis (*this);
>>>> + if (tuple_expr->is_marked_for_strip ())
>>>> + rust_error_at (tuple_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprStruct &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says these are inner
>>>> + * attributes, not outer attributes of inner expr */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-exprs of path
>>>> + auto &struct_name = expr.get_struct_name ();
>>>> + visit (struct_name);
>>>> + if (struct_name.is_marked_for_strip ())
>>>> + rust_error_at (struct_name.get_locus (),
>>>> + "cannot strip path in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprFieldIdentifier &)
>>>> +{
>>>> + // as no attrs (at moment, at least), no stripping possible
>>>
>>> has
>>>
>>> is there a reason to not put a gcc_unreachable () here?
>>>
>>
>> The visitor goes through all the nodes in a crate, even if there is
>> nothing to do. In the case of the Attribute visitor, there currently
>> isn't anything to do for the following pattern (which is what
>> `StructExpreFieldIdentifier` is the node for)
>>
>> ```rust
>> struct Foo {
>> a: i32,
>> b: i32,
>> }
>>
>> fn init_struct_with_a(a: i32) -> Foo {
>> Foo {
>> a, // <--- here
>> b: some_fn(),
>> }
>> }
>> ```
>>
>> where a struct's field is initialized by a variable of the same name
>> (this is a short-hand for `a: a`).
>>
>> But that does not mean that the node is not visited. What happens is
>> that we visit all the nodes of a struct's initialization, which are of
>> the base class `StructExprField` and then dynamically dispatch to the
>> proper visitor functions, we must handle the case of a
>> `StructExprFieldIdentifier` being visited even if there is nothing to do.
>>
>> If we add a `gcc_unreachable` here, the above test case would crash, as
>> it would try stripping the attributes of both field initalizations. For
>> the first case, there is nothing to do. For the second case, we want to
>> make sure we should not be stripping the sub-expression (the call to
>> `some_fn`)
>
> Ah - I confused 'no stripping possible' with 'no stripping needed' then
> (or rather you did?)
>
When Philip originally wrote this pass, I believe what he meant by "no
stripping possible" was that it wasn't possible for conditional
compilation to be present here - as in the language and parser do not
allow it. This turns out to mean, as you point out, that "no stripping
[is] needed" for that AST node, which is the smae (in my opinion) as "no
stripping possible [here]". So the way I see it, you and Philip are both
right :)
I don't mind changing the comment over to "no stripping needed" in this
file if it makes it clearer for reviewers. It's an easy fix.
>>
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field)
>>>> +{
>>>> + /* as no attrs possible (at moment, at least), only sub-expression
>>>> + * stripping is possible */
>>>> + auto &value = field.get_value ();
>>>> + value->accept_vis (*this);
>>>> + if (value->is_marked_for_strip ())
>>>> + rust_error_at (value->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprFieldIndexValue &field)
>>>> +{
>>>> + /* as no attrs possible (at moment, at least), only sub-expression
>>>> + * stripping is possible */
>>>> + auto &value = field.get_value ();
>>>> + value->accept_vis (*this);
>>>> + if (value->is_marked_for_strip ())
>>>> + rust_error_at (value->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprStructFields &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says these are inner
>>>> + * attributes, not outer attributes of inner expr */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-exprs of path
>>>> + auto &struct_name = expr.get_struct_name ();
>>>> + visit (struct_name);
>>>> + if (struct_name.is_marked_for_strip ())
>>>> + rust_error_at (struct_name.get_locus (),
>>>> + "cannot strip path in this position");
>>>> +
>>>> + /* spec does not specify whether expressions are allowed to be
>>>> + * stripped at top level of struct fields, but I wouldn't think
>>>> + * that they would be, so operating under the assumption that only
>>>> + * sub-expressions can be stripped. */
>>>> + for (auto &field : expr.get_fields ())
>>>> + {
>>>> + field->accept_vis (*this);
>>>> + // shouldn't strip in this
>>>> + }
>>>> +
>>>> + /* struct base presumably can't be stripped, as the '..' is before
>>>> + * the expression. as such, can only strip sub-expressions. */
>>>> + if (expr.has_struct_base ())
>>>> + {
>>>> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
>>>> + base_struct_expr->accept_vis (*this);
>>>> + if (base_struct_expr->is_marked_for_strip ())
>>>> + rust_error_at (base_struct_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructExprStructBase &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says these are inner
>>>> + * attributes, not outer attributes of inner expr */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-exprs of path
>>>> + auto &struct_name = expr.get_struct_name ();
>>>> + visit (struct_name);
>>>> + if (struct_name.is_marked_for_strip ())
>>>> + rust_error_at (struct_name.get_locus (),
>>>> + "cannot strip path in this position");
>>>> +
>>>> + /* struct base presumably can't be stripped, as the '..' is before
>>>> + * the expression. as such, can only strip sub-expressions. */
>>>> + rust_assert (!expr.get_struct_base ().is_invalid ());
>>>> + auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
>>>> + base_struct_expr->accept_vis (*this);
>>>> + if (base_struct_expr->is_marked_for_strip ())
>>>> + rust_error_at (base_struct_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::CallExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* should not be outer attrs on "function" expression - outer attrs
>>>> + * should be associated with call expr as a whole. only sub-expr
>>>> + * expansion is possible. */
>>>> + auto &function = expr.get_function_expr ();
>>>> + function->accept_vis (*this);
>>>> + if (function->is_marked_for_strip ())
>>>> + rust_error_at (function->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + /* spec says outer attributes are specifically allowed for elements
>>>> + * of call expressions, so full stripping possible */
>>>> + // FIXME: Arthur: Figure out how to refactor this - This is similar to
>>>> + // expanding items in the crate or stmts in blocks
>>>> + expand_pointer_allow_strip (expr.get_params ());
>>>> + auto ¶ms = expr.get_params ();
>>>> + for (auto it = params.begin (); it != params.end ();)
>>>> + {
>>>> + auto &stmt = *it;
>>>> +
>>>> + stmt->accept_vis (*this);
>>>> +
>>>> + auto final_fragment = expand_macro_fragment_recursive ();
>>>> + if (final_fragment.should_expand ())
>>>> + {
>>>> + // Remove the current expanded invocation
>>>> + it = params.erase (it);
>>>> + for (auto &node : final_fragment.get_nodes ())
>>>> + {
>>>> + it = params.insert (it, node.take_expr ());
>>>> + it++;
>>>> + }
>>>> + }
>>>> + else if (stmt->is_marked_for_strip ())
>>>> + it = params.erase (it);
>>>> + else
>>>> + it++;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::MethodCallExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* should not be outer attrs on "receiver" expression - outer attrs
>>>> + * should be associated with call expr as a whole. only sub-expr
>>>> + * expansion is possible. */
>>>> + auto &receiver = expr.get_receiver_expr ();
>>>> + receiver->accept_vis (*this);
>>>> + if (receiver->is_marked_for_strip ())
>>>> + rust_error_at (receiver->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + auto &method_name = expr.get_method_name ();
>>>> + if (method_name.has_generic_args ())
>>>> + expand_generic_args (method_name.get_generic_args ());
>>>> +
>>>> + /* spec says outer attributes are specifically allowed for elements
>>>> + * of method call expressions, so full stripping possible */
>>>> + expand_pointer_allow_strip (expr.get_params ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::FieldAccessExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* should not be outer attrs on "receiver" expression - outer attrs
>>>> + * should be associated with field expr as a whole. only sub-expr
>>>> + * expansion is possible. */
>>>> + auto &receiver = expr.get_receiver_expr ();
>>>> + receiver->accept_vis (*this);
>>>> + if (receiver->is_marked_for_strip ())
>>>> + rust_error_at (receiver->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ClosureExprInner &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip closure parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_closure_params (expr.get_params ());
>>>> +
>>>> + // can't strip expression itself, but can strip sub-expressions
>>>> + auto &definition_expr = expr.get_definition_expr ();
>>>> + definition_expr->accept_vis (*this);
>>>> + if (definition_expr->is_marked_for_strip ())
>>>> + rust_error_at (definition_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::BlockExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip test based on inner attrs - spec says there are inner
>>>> + * attributes, not just outer attributes of inner stmts */
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + std::function<std::unique_ptr<AST::Stmt> (AST::SingleASTNode)> extractor
>>>> + = [] (AST::SingleASTNode node) { return node.take_stmt (); };
>>>> +
>>>> + expand_macro_children (MacroExpander::BLOCK, expr.get_statements (),
>>>> + extractor);
>>>> +
>>>> + expander.push_context (MacroExpander::BLOCK);
>>>> +
>>>> + // strip tail expression if exists - can actually fully remove it
>>>> + if (expr.has_tail_expr ())
>>>> + {
>>>> + auto &tail_expr = expr.get_tail_expr ();
>>>> +
>>>> + tail_expr->accept_vis (*this);
>>>> + maybe_expand_expr (tail_expr);
>>>> +
>>>> + if (tail_expr->is_marked_for_strip ())
>>>> + expr.strip_tail_expr ();
>>>> + }
>>>> + expander.pop_context ();
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::ClosureExprInnerTyped &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip closure parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_closure_params (expr.get_params ());
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // can't strip return type, but can strip sub-types
>>>> + auto &type = expr.get_return_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // can't strip expression itself, but can strip sub-expressions
>>>> + auto &definition_block = expr.get_definition_block ();
>>>> + definition_block->accept_vis (*this);
>>>> + if (definition_block->is_marked_for_strip ())
>>>> + rust_error_at (definition_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ContinueExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::BreakExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* spec does not say that you can have outer attributes on
>>>> + * expression, so assuming you can't. stripping for sub-expressions
>>>> + * is the only thing that can be done */
>>>> + if (expr.has_break_expr ())
>>>> + {
>>>> + auto &break_expr = expr.get_break_expr ();
>>>> +
>>>> + break_expr->accept_vis (*this);
>>>> +
>>>> + if (break_expr->is_marked_for_strip ())
>>>> + rust_error_at (break_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeFromToExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + expr.get_from_expr ()->accept_vis (*this);
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + expr.get_to_expr ()->accept_vis (*this);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_from_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_from_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before range exprs");
>>>> + if (expr.get_to_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_to_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeFromExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * direct descendant expression, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + auto &from_expr = expr.get_from_expr ();
>>>> +
>>>> + from_expr->accept_vis (*this);
>>>> +
>>>> + if (from_expr->is_marked_for_strip ())
>>>> + rust_error_at (from_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed before range exprs");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeToExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * direct descendant expression, can strip ones below that */
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &to_expr = expr.get_to_expr ();
>>>> +
>>>> + to_expr->accept_vis (*this);
>>>> +
>>>> + if (to_expr->is_marked_for_strip ())
>>>> + rust_error_at (to_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeFullExpr &)
>>>> +{
>>>> + // outer attributes never allowed before these, so no stripping
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeFromToInclExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * two direct descendant expressions, can strip ones below that */
>>>> +
>>>> + /* should have no possibility for outer attrs as would be parsed
>>>> + * with outer expr */
>>>> + expr.get_from_expr ()->accept_vis (*this);
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + expr.get_to_expr ()->accept_vis (*this);
>>>> +
>>>> + // ensure that they are not marked for strip
>>>> + if (expr.get_from_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_from_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes are never allowed "
>>>> + "before range exprs");
>>>> + if (expr.get_to_expr ()->is_marked_for_strip ())
>>>> + rust_error_at (expr.get_to_expr ()->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangeToInclExpr &expr)
>>>> +{
>>>> + /* outer attributes never allowed before these. while cannot strip
>>>> + * direct descendant expression, can strip ones below that */
>>>> +
>>>> + /* should syntactically not have outer attributes, though this may
>>>> + * not have worked in practice */
>>>> + auto &to_expr = expr.get_to_expr ();
>>>> +
>>>> + to_expr->accept_vis (*this);
>>>> +
>>>> + if (to_expr->is_marked_for_strip ())
>>>> + rust_error_at (to_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ReturnExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* spec does not say that you can have outer attributes on
>>>> + * expression, so assuming you can't. stripping for sub-expressions
>>>> + * is the only thing that can be done */
>>>> + if (expr.has_returned_expr ())
>>>> + {
>>>> + auto &returned_expr = expr.get_returned_expr ();
>>>> +
>>>> + returned_expr->accept_vis (*this);
>>>> +
>>>> + if (returned_expr->is_marked_for_strip ())
>>>> + rust_error_at (returned_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> + }
>>>> + /* TODO: conceptually, you would maybe be able to remove a returned
>>>> + * expr - e.g. if you had conditional compilation returning void or
>>>> + * returning a type. On the other hand, I think that function
>>>> + * return type cannot be conditionally compiled, so I assumed you
>>>> + * can't do this either. */
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::UnsafeBlockExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &block_expr = expr.get_block_expr ();
>>>> + block_expr->accept_vis (*this);
>>>> + if (block_expr->is_marked_for_strip ())
>>>> + rust_error_at (block_expr->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::LoopExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &loop_block = expr.get_loop_block ();
>>>> + loop_block->accept_vis (*this);
>>>> + if (loop_block->is_marked_for_strip ())
>>>> + rust_error_at (loop_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::WhileLoopExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip predicate expr itself, but can strip sub-expressions
>>>> + auto &predicate_expr = expr.get_predicate_expr ();
>>>> + predicate_expr->accept_vis (*this);
>>>> + if (predicate_expr->is_marked_for_strip ())
>>>> + rust_error_at (predicate_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &loop_block = expr.get_loop_block ();
>>>> + loop_block->accept_vis (*this);
>>>> + if (loop_block->is_marked_for_strip ())
>>>> + rust_error_at (loop_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::WhileLetLoopExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &pattern : expr.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>>>> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
>>>> + scrutinee_expr->accept_vis (*this);
>>>> + if (scrutinee_expr->is_marked_for_strip ())
>>>> + rust_error_at (scrutinee_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &loop_block = expr.get_loop_block ();
>>>> + loop_block->accept_vis (*this);
>>>> + if (loop_block->is_marked_for_strip ())
>>>> + rust_error_at (loop_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ForLoopExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-patterns of pattern
>>>> + auto &pattern = expr.get_pattern ();
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +
>>>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>>>> + auto &iterator_expr = expr.get_iterator_expr ();
>>>> + iterator_expr->accept_vis (*this);
>>>> + if (iterator_expr->is_marked_for_strip ())
>>>> + rust_error_at (iterator_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &loop_block = expr.get_loop_block ();
>>>> + loop_block->accept_vis (*this);
>>>> + if (loop_block->is_marked_for_strip ())
>>>> + rust_error_at (loop_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfExpr &expr)
>>>> +{
>>>> + // rust playground test shows that IfExpr does support outer attrs, at least
>>>> + // when used as statement
>>>> +
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip condition expr itself, but can strip sub-expressions
>>>> + auto &condition_expr = expr.get_condition_expr ();
>>>> + condition_expr->accept_vis (*this);
>>>> + maybe_expand_expr (condition_expr);
>>>> + if (condition_expr->is_marked_for_strip ())
>>>> + rust_error_at (condition_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfExprConseqElse &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip condition expr itself, but can strip sub-expressions
>>>> + auto &condition_expr = expr.get_condition_expr ();
>>>> + condition_expr->accept_vis (*this);
>>>> + maybe_expand_expr (condition_expr);
>>>> + if (condition_expr->is_marked_for_strip ())
>>>> + rust_error_at (condition_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip else block itself, but can strip sub-expressions
>>>> + auto &else_block = expr.get_else_block ();
>>>> + else_block->accept_vis (*this);
>>>> + if (else_block->is_marked_for_strip ())
>>>> + rust_error_at (else_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfExprConseqIf &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip condition expr itself, but can strip sub-expressions
>>>> + auto &condition_expr = expr.get_condition_expr ();
>>>> + condition_expr->accept_vis (*this);
>>>> + maybe_expand_expr (condition_expr);
>>>> + if (condition_expr->is_marked_for_strip ())
>>>> + rust_error_at (condition_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if expr itself, but can strip sub-expressions
>>>> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
>>>> + conseq_if_expr->accept_vis (*this);
>>>> + if (conseq_if_expr->is_marked_for_strip ())
>>>> + rust_error_at (conseq_if_expr->get_locus (),
>>>> + "cannot strip consequent if expression in this "
>>>> + "position - outer attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfExprConseqIfLet &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip condition expr itself, but can strip sub-expressions
>>>> + auto &condition_expr = expr.get_condition_expr ();
>>>> + condition_expr->accept_vis (*this);
>>>> + maybe_expand_expr (condition_expr);
>>>> + if (condition_expr->is_marked_for_strip ())
>>>> + rust_error_at (condition_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if let expr itself, but can strip sub-expressions
>>>> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
>>>> + conseq_if_let_expr->accept_vis (*this);
>>>> + if (conseq_if_let_expr->is_marked_for_strip ())
>>>> + rust_error_at (conseq_if_let_expr->get_locus (),
>>>> + "cannot strip consequent if let expression in this "
>>>> + "position - outer attributes not "
>>>> + "allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfLetExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &pattern : expr.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + // can't strip value expr itself, but can strip sub-expressions
>>>> + auto &value_expr = expr.get_value_expr ();
>>>> + value_expr->accept_vis (*this);
>>>> + if (value_expr->is_marked_for_strip ())
>>>> + rust_error_at (value_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfLetExprConseqElse &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &pattern : expr.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + // can't strip value expr itself, but can strip sub-expressions
>>>> + auto &value_expr = expr.get_value_expr ();
>>>> + value_expr->accept_vis (*this);
>>>> + if (value_expr->is_marked_for_strip ())
>>>> + rust_error_at (value_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip else block itself, but can strip sub-expressions
>>>> + auto &else_block = expr.get_else_block ();
>>>> + else_block->accept_vis (*this);
>>>> + if (else_block->is_marked_for_strip ())
>>>> + rust_error_at (else_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfLetExprConseqIf &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &pattern : expr.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + // can't strip value expr itself, but can strip sub-expressions
>>>> + auto &value_expr = expr.get_value_expr ();
>>>> + value_expr->accept_vis (*this);
>>>> + if (value_expr->is_marked_for_strip ())
>>>> + rust_error_at (value_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if expr itself, but can strip sub-expressions
>>>> + auto &conseq_if_expr = expr.get_conseq_if_expr ();
>>>> + conseq_if_expr->accept_vis (*this);
>>>> + if (conseq_if_expr->is_marked_for_strip ())
>>>> + rust_error_at (conseq_if_expr->get_locus (),
>>>> + "cannot strip consequent if expression in this "
>>>> + "position - outer attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + for (auto &pattern : expr.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + // can't strip value expr itself, but can strip sub-expressions
>>>> + auto &value_expr = expr.get_value_expr ();
>>>> + value_expr->accept_vis (*this);
>>>> + if (value_expr->is_marked_for_strip ())
>>>> + rust_error_at (value_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if block itself, but can strip sub-expressions
>>>> + auto &if_block = expr.get_if_block ();
>>>> + if_block->accept_vis (*this);
>>>> + if (if_block->is_marked_for_strip ())
>>>> + rust_error_at (if_block->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // can't strip if let expr itself, but can strip sub-expressions
>>>> + auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
>>>> + conseq_if_let_expr->accept_vis (*this);
>>>> + if (conseq_if_let_expr->is_marked_for_strip ())
>>>> + rust_error_at (conseq_if_let_expr->get_locus (),
>>>> + "cannot strip consequent if let expression in this "
>>>> + "position - outer attributes not "
>>>> + "allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::MatchExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // inner attr strip test
>>>> + expander.expand_cfg_attrs (expr.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip scrutinee expr itself, but can strip sub-expressions
>>>> + auto &scrutinee_expr = expr.get_scrutinee_expr ();
>>>> + scrutinee_expr->accept_vis (*this);
>>>> + if (scrutinee_expr->is_marked_for_strip ())
>>>> + rust_error_at (scrutinee_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // strip match cases
>>>> + auto &match_cases = expr.get_match_cases ();
>>>> + for (auto it = match_cases.begin (); it != match_cases.end ();)
>>>> + {
>>>> + auto &match_case = *it;
>>>> +
>>>> + // strip match case based on outer attributes in match arm
>>>> + auto &match_arm = match_case.get_arm ();
>>>> + expander.expand_cfg_attrs (match_arm.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ()))
>>>> + {
>>>> + // strip match case
>>>> + it = match_cases.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + for (auto &pattern : match_arm.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + }
>>>> +
>>>> + /* assuming that guard expression cannot be stripped as
>>>> + * strictly speaking you would have to strip the whole guard to
>>>> + * make syntactical sense, which you can't do. as such, only
>>>> + * strip sub-expressions */
>>>> + if (match_arm.has_match_arm_guard ())
>>>> + {
>>>> + auto &guard_expr = match_arm.get_guard_expr ();
>>>> + guard_expr->accept_vis (*this);
>>>> + if (guard_expr->is_marked_for_strip ())
>>>> + rust_error_at (guard_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> + }
>>>> +
>>>> + // strip sub-expressions from match cases
>>>> + auto &case_expr = match_case.get_expr ();
>>>> + case_expr->accept_vis (*this);
>>>> + if (case_expr->is_marked_for_strip ())
>>>> + rust_error_at (case_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + // increment to next case if haven't continued
>>>> + ++it;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::AwaitExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* can't strip awaited expr itself, but can strip sub-expressions
>>>> + * - this is because you can't have no expr to await */
>>>> + auto &awaited_expr = expr.get_awaited_expr ();
>>>> + awaited_expr->accept_vis (*this);
>>>> + if (awaited_expr->is_marked_for_strip ())
>>>> + rust_error_at (awaited_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::AsyncBlockExpr &expr)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (expr.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
>>>> + {
>>>> + expr.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &block_expr = expr.get_block_expr ();
>>>> + block_expr->accept_vis (*this);
>>>> + if (block_expr->is_marked_for_strip ())
>>>> + rust_error_at (block_expr->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::TypeParam ¶m)
>>>> +{
>>>> + // outer attributes don't actually do anything, so ignore them
>>>> +
>>>> + if (param.has_type_param_bounds ())
>>>> + {
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : param.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> + }
>>>> +
>>>> + if (param.has_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> + auto &type = param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::LifetimeWhereClauseItem &)
>>>> +{
>>>> + // shouldn't require
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item)
>>>> +{
>>>> + // for lifetimes shouldn't require
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = item.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : item.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Method &method)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (method.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (method.get_outer_attrs ()))
>>>> + {
>>>> + method.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : method.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* assuming you can't strip self param - wouldn't be a method
>>>> + * anymore. spec allows outer attrs on self param, but doesn't
>>>> + * specify whether cfg is used. */
>>>> + expand_self_param (method.get_self_param ());
>>>> +
>>>> + /* strip method parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_function_params (method.get_function_params ());
>>>> +
>>>> + if (method.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = method.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + if (method.has_where_clause ())
>>>> + expand_where_clause (method.get_where_clause ());
>>>> +
>>>> + /* body should always exist - if error state, should have returned
>>>> + * before now */
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &block_expr = method.get_definition ();
>>>> + block_expr->accept_vis (*this);
>>>> + if (block_expr->is_marked_for_strip ())
>>>> + rust_error_at (block_expr->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Module &module)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (module.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (module.get_outer_attrs ()))
>>>> + {
>>>> + module.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // A loaded module might have inner attributes
>>>> + if (module.get_kind () == AST::Module::ModuleKind::LOADED)
>>>> + {
>>>> + // strip test based on inner attrs
>>>> + expander.expand_cfg_attrs (module.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (module.get_inner_attrs ()))
>>>> + {
>>>> + module.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> + }
>>>> +
>>>> + // Parse the module's items if they haven't been expanded and the file
>>>> + // should be parsed (i.e isn't hidden behind an untrue or impossible cfg
>>>> + // directive)
>>>> + if (!module.is_marked_for_strip ()
>>>> + && module.get_kind () == AST::Module::ModuleKind::UNLOADED)
>>>> + {
>>>> + module.load_items ();
>>>> + }
>>>> +
>>>> + // strip items if required
>>>> + expand_pointer_allow_strip (module.get_items ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ExternCrate &extern_crate)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (extern_crate.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ()))
>>>> + {
>>>> + extern_crate.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + if (!extern_crate.references_self ())
>>>> + {
>>>> + Session &session = Session::get_instance ();
>>>> + session.load_extern_crate (extern_crate.get_referenced_crate (),
>>>> + extern_crate.get_locus ());
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::UseTreeGlob &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::UseTreeList &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::UseTreeRebind &)
>>>> +{
>>>> + // shouldn't require?
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::UseDeclaration &use_decl)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (use_decl.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ()))
>>>> + {
>>>> + use_decl.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Function &function)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (function.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (function.get_outer_attrs ()))
>>>> + {
>>>> + function.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : function.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* strip function parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + expand_function_params (function.get_function_params ());
>>>> +
>>>> + if (function.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = function.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + if (function.has_where_clause ())
>>>> + expand_where_clause (function.get_where_clause ());
>>>> +
>>>> + /* body should always exist - if error state, should have returned
>>>> + * before now */
>>>> + // can't strip block itself, but can strip sub-expressions
>>>> + auto &block_expr = function.get_definition ();
>>>> + block_expr->accept_vis (*this);
>>>> + if (block_expr->is_marked_for_strip ())
>>>> + rust_error_at (block_expr->get_locus (),
>>>> + "cannot strip block expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TypeAlias &type_alias)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (type_alias.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ()))
>>>> + {
>>>> + type_alias.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : type_alias.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + if (type_alias.has_where_clause ())
>>>> + expand_where_clause (type_alias.get_where_clause ());
>>>> +
>>>> + auto &type = type_alias.get_type_aliased ();
>>>> + type->accept_vis (*this);
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructStruct &struct_item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (struct_item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ()))
>>>> + {
>>>> + struct_item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : struct_item.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + if (struct_item.has_where_clause ())
>>>> + expand_where_clause (struct_item.get_where_clause ());
>>>> +
>>>> + /* strip struct fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_struct_fields (struct_item.get_fields ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleStruct &tuple_struct)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ()))
>>>> + {
>>>> + tuple_struct.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : tuple_struct.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* strip struct fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_tuple_fields (tuple_struct.get_fields ());
>>>> +
>>>> + if (tuple_struct.has_where_clause ())
>>>> + expand_where_clause (tuple_struct.get_where_clause ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::EnumItem &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::EnumItemTuple &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip item fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_tuple_fields (item.get_tuple_fields ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::EnumItemStruct &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip item fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_struct_fields (item.get_struct_fields ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::EnumItemDiscriminant &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &expr = item.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + rust_error_at (expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Enum &enum_item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (enum_item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ()))
>>>> + {
>>>> + enum_item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : enum_item.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + if (enum_item.has_where_clause ())
>>>> + expand_where_clause (enum_item.get_where_clause ());
>>>> +
>>>> + /* strip enum fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_pointer_allow_strip (enum_item.get_variants ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Union &union_item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (union_item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ()))
>>>> + {
>>>> + union_item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : union_item.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + if (union_item.has_where_clause ())
>>>> + expand_where_clause (union_item.get_where_clause ());
>>>> +
>>>> + /* strip union fields if required - this is presumably
>>>> + * allowed by spec */
>>>> + expand_struct_fields (union_item.get_variants ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ConstantItem &const_item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (const_item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ()))
>>>> + {
>>>> + const_item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // strip any sub-types
>>>> + auto &type = const_item.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &expr = const_item.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + rust_error_at (expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StaticItem &static_item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (static_item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ()))
>>>> + {
>>>> + static_item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // strip any sub-types
>>>> + auto &type = static_item.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &expr = static_item.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + rust_error_at (expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitItemFunc &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expand_trait_function_decl (item.get_trait_function_decl ());
>>>> +
>>>> + if (item.has_definition ())
>>>> + {
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &block = item.get_definition ();
>>>> + block->accept_vis (*this);
>>>> + if (block->is_marked_for_strip ())
>>>> + rust_error_at (block->get_locus (),
>>>> + "cannot strip block expression in this "
>>>> + "position - outer attributes not allowed");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitItemMethod &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expand_trait_method_decl (item.get_trait_method_decl ());
>>>> +
>>>> + if (item.has_definition ())
>>>> + {
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped. */
>>>> + auto &block = item.get_definition ();
>>>> + block->accept_vis (*this);
>>>> + if (block->is_marked_for_strip ())
>>>> + rust_error_at (block->get_locus (),
>>>> + "cannot strip block expression in this "
>>>> + "position - outer attributes not allowed");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitItemConst &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + // strip any sub-types
>>>> + auto &type = item.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped */
>>>> + if (item.has_expression ())
>>>> + {
>>>> + auto &expr = item.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + rust_error_at (expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitItemType &item)
>>>> +{
>>>> + // initial test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + if (item.has_type_param_bounds ())
>>>> + {
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : item.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::Trait &trait)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (trait.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (trait.get_outer_attrs ()))
>>>> + {
>>>> + trait.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip test based on inner attrs
>>>> + expander.expand_cfg_attrs (trait.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (trait.get_inner_attrs ()))
>>>> + {
>>>> + trait.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : trait.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + if (trait.has_type_param_bounds ())
>>>> + {
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : trait.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> + }
>>>> +
>>>> + if (trait.has_where_clause ())
>>>> + expand_where_clause (trait.get_where_clause ());
>>>> +
>>>> + std::function<std::unique_ptr<AST::TraitItem> (AST::SingleASTNode)> extractor
>>>> + = [] (AST::SingleASTNode node) { return node.take_trait_item (); };
>>>> +
>>>> + expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (),
>>>> + extractor);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::InherentImpl &impl)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
>>>> + {
>>>> + impl.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip test based on inner attrs
>>>> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
>>>> + {
>>>> + impl.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : impl.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = impl.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + if (impl.has_where_clause ())
>>>> + expand_where_clause (impl.get_where_clause ());
>>>> +
>>>> + std::function<std::unique_ptr<AST::InherentImplItem> (AST::SingleASTNode)>
>>>> + extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); };
>>>> +
>>>> + expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (),
>>>> + extractor);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitImpl &impl)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (impl.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
>>>> + {
>>>> + impl.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip test based on inner attrs
>>>> + expander.expand_cfg_attrs (impl.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
>>>> + {
>>>> + impl.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : impl.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = impl.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + auto &trait_path = impl.get_trait_path ();
>>>> + visit (trait_path);
>>>> + if (trait_path.is_marked_for_strip ())
>>>> + rust_error_at (trait_path.get_locus (),
>>>> + "cannot strip typepath in this position");
>>>> +
>>>> + if (impl.has_where_clause ())
>>>> + expand_where_clause (impl.get_where_clause ());
>>>> +
>>>> + std::function<std::unique_ptr<AST::TraitImplItem> (AST::SingleASTNode)>
>>>> + extractor
>>>> + = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); };
>>>> +
>>>> + expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (),
>>>> + extractor);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ExternalStaticItem &item)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = item.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (), "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ExternalFunctionItem &item)
>>>> +{
>>>> + // strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (item.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
>>>> + {
>>>> + item.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // just expand sub-stuff - can't actually strip generic params themselves
>>>> + for (auto ¶m : item.get_generic_params ())
>>>> + param->accept_vis (*this);
>>>> +
>>>> + /* strip function parameters if required - this is specifically
>>>> + * allowed by spec */
>>>> + auto ¶ms = item.get_function_params ();
>>>> + for (auto it = params.begin (); it != params.end ();)
>>>> + {
>>>> + auto ¶m = *it;
>>>> +
>>>> + auto ¶m_attrs = param.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (param_attrs);
>>>> + if (expander.fails_cfg_with_expand (param_attrs))
>>>> + {
>>>> + it = params.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // increment if nothing else happens
>>>> + ++it;
>>>> + }
>>>> + /* NOTE: these are extern function params, which may have different
>>>> + * rules and restrictions to "normal" function params. So expansion
>>>> + * handled separately. */
>>>> +
>>>> + /* TODO: assuming that variadic nature cannot be stripped. If this
>>>> + * is not true, then have code here to do so. */
>>>> +
>>>> + if (item.has_return_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &return_type = item.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (return_type);
>>>> +
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + if (item.has_where_clause ())
>>>> + expand_where_clause (item.get_where_clause ());
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::ExternBlock &block)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (block.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (block.get_outer_attrs ()))
>>>> + {
>>>> + block.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip test based on inner attrs
>>>> + expander.expand_cfg_attrs (block.get_inner_attrs ());
>>>> + if (expander.fails_cfg_with_expand (block.get_inner_attrs ()))
>>>> + {
>>>> + block.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + std::function<std::unique_ptr<AST::ExternalItem> (AST::SingleASTNode)>
>>>> + extractor
>>>> + = [] (AST::SingleASTNode node) { return node.take_external_item (); };
>>>> +
>>>> + expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (),
>>>> + extractor);
>>>> +}
>>>> +
>>>> +// I don't think it would be possible to strip macros without expansion
>>>> +void
>>>> +AttrVisitor::visit (AST::MacroMatchFragment &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MacroMatchRepetition &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MacroMatcher &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MacroRulesDefinition &rules_def)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (rules_def.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ()))
>>>> + {
>>>> + rules_def.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // I don't think any macro rules can be stripped in any way
>>>> +
>>>> + auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (),
>>>> + rules_def.get_rule_name ());
>>>> + expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (),
>>>> + rules_def.get_locus ());
>>>> + expander.mappings->insert_macro_def (&rules_def);
>>>> + rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]",
>>>> + path.get ().c_str ());
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaItemPath &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaItemSeq &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaWord &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaNameValueStr &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaListPaths &)
>>>> +{}
>>>> +void
>>>> +AttrVisitor::visit (AST::MetaListNameValueStr &)
>>>> +{}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::LiteralPattern &)
>>>> +{
>>>> + // not possible
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::IdentifierPattern &pattern)
>>>> +{
>>>> + // can only strip sub-patterns of the inner pattern to bind
>>>> + if (!pattern.has_pattern_to_bind ())
>>>> + return;
>>>> +
>>>> + auto &sub_pattern = pattern.get_pattern_to_bind ();
>>>> + sub_pattern->accept_vis (*this);
>>>> + if (sub_pattern->is_marked_for_strip ())
>>>> + rust_error_at (sub_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::WildcardPattern &)
>>>> +{
>>>> + // not possible
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangePatternBoundLiteral &)
>>>> +{
>>>> + // not possible
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangePatternBoundPath &bound)
>>>> +{
>>>> + // can expand path, but not strip it directly
>>>> + auto &path = bound.get_path ();
>>>> + visit (path);
>>>> + if (path.is_marked_for_strip ())
>>>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangePatternBoundQualPath &bound)
>>>> +{
>>>> + // can expand path, but not strip it directly
>>>> + auto &path = bound.get_qualified_path ();
>>>> + visit (path);
>>>> + if (path.is_marked_for_strip ())
>>>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RangePattern &pattern)
>>>> +{
>>>> + // should have no capability to strip lower or upper bounds, only expand
>>>> + pattern.get_lower_bound ()->accept_vis (*this);
>>>> + pattern.get_upper_bound ()->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ReferencePattern &pattern)
>>>> +{
>>>> + auto &sub_pattern = pattern.get_referenced_pattern ();
>>>> + sub_pattern->accept_vis (*this);
>>>> + if (sub_pattern->is_marked_for_strip ())
>>>> + rust_error_at (sub_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructPatternFieldTuplePat &field)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>>>> + {
>>>> + field.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-patterns (can't strip top-level pattern)
>>>> + auto &sub_pattern = field.get_index_pattern ();
>>>> + sub_pattern->accept_vis (*this);
>>>> + if (sub_pattern->is_marked_for_strip ())
>>>> + rust_error_at (sub_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructPatternFieldIdentPat &field)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>>>> + {
>>>> + field.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // strip sub-patterns (can't strip top-level pattern)
>>>> + auto &sub_pattern = field.get_ident_pattern ();
>>>> + sub_pattern->accept_vis (*this);
>>>> + if (sub_pattern->is_marked_for_strip ())
>>>> + rust_error_at (sub_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructPatternFieldIdent &field)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (field.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
>>>> + {
>>>> + field.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::StructPattern &pattern)
>>>> +{
>>>> + // expand (but don't strip) path
>>>> + auto &path = pattern.get_path ();
>>>> + visit (path);
>>>> + if (path.is_marked_for_strip ())
>>>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>>>> +
>>>> + /* TODO: apparently struct pattern fields can have outer attrs. so can they
>>>> + * be stripped? */
>>>> + if (!pattern.has_struct_pattern_elems ())
>>>> + return;
>>>> +
>>>> + auto &elems = pattern.get_struct_pattern_elems ();
>>>> +
>>>> + // assuming you can strip struct pattern fields
>>>> + expand_pointer_allow_strip (elems.get_struct_pattern_fields ());
>>>> +
>>>> + // assuming you can strip the ".." part
>>>> + if (elems.has_etc ())
>>>> + {
>>>> + expander.expand_cfg_attrs (elems.get_etc_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ()))
>>>> + elems.strip_etc ();
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items)
>>>> +{
>>>> + // can't strip individual patterns, only sub-patterns
>>>> + for (auto &pattern : tuple_items.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> +
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items)
>>>> +{
>>>> + // can't strip individual patterns, only sub-patterns
>>>> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
>>>> + {
>>>> + lower_pattern->accept_vis (*this);
>>>> +
>>>> + if (lower_pattern->is_marked_for_strip ())
>>>> + rust_error_at (lower_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
>>>> + {
>>>> + upper_pattern->accept_vis (*this);
>>>> +
>>>> + if (upper_pattern->is_marked_for_strip ())
>>>> + rust_error_at (upper_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleStructPattern &pattern)
>>>> +{
>>>> + // expand (but don't strip) path
>>>> + auto &path = pattern.get_path ();
>>>> + visit (path);
>>>> + if (path.is_marked_for_strip ())
>>>> + rust_error_at (path.get_locus (), "cannot strip path in this position");
>>>> +
>>>> + if (pattern.has_items ())
>>>> + pattern.get_items ()->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items)
>>>> +{
>>>> + // can't strip individual patterns, only sub-patterns
>>>> + for (auto &pattern : tuple_items.get_patterns ())
>>>> + {
>>>> + pattern->accept_vis (*this);
>>>> +
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items)
>>>> +{
>>>> + // can't strip individual patterns, only sub-patterns
>>>> + for (auto &lower_pattern : tuple_items.get_lower_patterns ())
>>>> + {
>>>> + lower_pattern->accept_vis (*this);
>>>> +
>>>> + if (lower_pattern->is_marked_for_strip ())
>>>> + rust_error_at (lower_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> + for (auto &upper_pattern : tuple_items.get_upper_patterns ())
>>>> + {
>>>> + upper_pattern->accept_vis (*this);
>>>> +
>>>> + if (upper_pattern->is_marked_for_strip ())
>>>> + rust_error_at (upper_pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TuplePattern &pattern)
>>>> +{
>>>> + if (pattern.has_tuple_pattern_items ())
>>>> + pattern.get_items ()->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::GroupedPattern &pattern)
>>>> +{
>>>> + // can't strip inner pattern, only sub-patterns
>>>> + auto &pattern_in_parens = pattern.get_pattern_in_parens ();
>>>> +
>>>> + pattern_in_parens->accept_vis (*this);
>>>> +
>>>> + if (pattern_in_parens->is_marked_for_strip ())
>>>> + rust_error_at (pattern_in_parens->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::SlicePattern &pattern)
>>>> +{
>>>> + // can't strip individual patterns, only sub-patterns
>>>> + for (auto &item : pattern.get_items ())
>>>> + {
>>>> + item->accept_vis (*this);
>>>> +
>>>> + if (item->is_marked_for_strip ())
>>>> + rust_error_at (item->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> + // TODO: quit stripping now? or keep going?
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::EmptyStmt &)
>>>> +{
>>>> + // assuming no outer attributes, so nothing can happen
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::LetStmt &stmt)
>>>> +{
>>>> + // initial strip test based on outer attrs
>>>> + expander.expand_cfg_attrs (stmt.get_outer_attrs ());
>>>> + if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ()))
>>>> + {
>>>> + stmt.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +
>>>> + // can't strip pattern, but call for sub-patterns
>>>> + auto &pattern = stmt.get_pattern ();
>>>> + pattern->accept_vis (*this);
>>>> + if (pattern->is_marked_for_strip ())
>>>> + rust_error_at (pattern->get_locus (),
>>>> + "cannot strip pattern in this position");
>>>> +
>>>> + // similar for type
>>>> + if (stmt.has_type ())
>>>> + {
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = stmt.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + /* strip any internal sub-expressions - expression itself isn't
>>>> + * allowed to have external attributes in this position so can't be
>>>> + * stripped */
>>>> + if (stmt.has_init_expr ())
>>>> + {
>>>> + auto &init_expr = stmt.get_init_expr ();
>>>> + init_expr->accept_vis (*this);
>>>> +
>>>> + if (init_expr->is_marked_for_strip ())
>>>> + rust_error_at (init_expr->get_locus (),
>>>> + "cannot strip expression in this position - outer "
>>>> + "attributes not allowed");
>>>> +
>>>> + maybe_expand_expr (init_expr);
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt)
>>>> +{
>>>> + // outer attributes associated with expr, so rely on expr
>>>> +
>>>> + // guard - should prevent null pointer expr
>>>> + if (stmt.is_marked_for_strip ())
>>>> + return;
>>>> +
>>>> + // strip if expr is to be stripped
>>>> + auto &expr = stmt.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + {
>>>> + stmt.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ExprStmtWithBlock &stmt)
>>>> +{
>>>> + // outer attributes associated with expr, so rely on expr
>>>> +
>>>> + // guard - should prevent null pointer expr
>>>> + if (stmt.is_marked_for_strip ())
>>>> + return;
>>>> +
>>>> + // strip if expr is to be stripped
>>>> + auto &expr = stmt.get_expr ();
>>>> + expr->accept_vis (*this);
>>>> + if (expr->is_marked_for_strip ())
>>>> + {
>>>> + stmt.mark_for_strip ();
>>>> + return;
>>>> + }
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitBound &bound)
>>>> +{
>>>> + // nothing in for lifetimes to strip
>>>> +
>>>> + // expand but don't strip type path
>>>> + auto &path = bound.get_type_path ();
>>>> + visit (path);
>>>> + if (path.is_marked_for_strip ())
>>>> + rust_error_at (path.get_locus (),
>>>> + "cannot strip type path in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ImplTraitType &type)
>>>> +{
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : type.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitObjectType &type)
>>>> +{
>>>> + // don't strip directly, only components of bounds
>>>> + for (auto &bound : type.get_type_param_bounds ())
>>>> + bound->accept_vis (*this);
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ParenthesisedType &type)
>>>> +{
>>>> + // expand but don't strip inner type
>>>> + auto &inner_type = type.get_type_in_parens ();
>>>> + inner_type->accept_vis (*this);
>>>> + if (inner_type->is_marked_for_strip ())
>>>> + rust_error_at (inner_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ImplTraitTypeOneBound &type)
>>>> +{
>>>> + // no stripping possible
>>>> + visit (type.get_trait_bound ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TraitObjectTypeOneBound &type)
>>>> +{
>>>> + // no stripping possible
>>>> + visit (type.get_trait_bound ());
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::TupleType &type)
>>>> +{
>>>> + // TODO: assuming that types can't be stripped as types don't have outer
>>>> + // attributes
>>>> + for (auto &elem_type : type.get_elems ())
>>>> + {
>>>> + elem_type->accept_vis (*this);
>>>> + if (elem_type->is_marked_for_strip ())
>>>> + rust_error_at (elem_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> + }
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::NeverType &)
>>>> +{
>>>> + // no stripping possible
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::RawPointerType &type)
>>>> +{
>>>> + // expand but don't strip type pointed to
>>>> + auto &pointed_type = type.get_type_pointed_to ();
>>>> + pointed_type->accept_vis (*this);
>>>> + if (pointed_type->is_marked_for_strip ())
>>>> + rust_error_at (pointed_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ReferenceType &type)
>>>> +{
>>>> + // expand but don't strip type referenced
>>>> + auto &referenced_type = type.get_type_referenced ();
>>>> + referenced_type->accept_vis (*this);
>>>> + if (referenced_type->is_marked_for_strip ())
>>>> + rust_error_at (referenced_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::ArrayType &type)
>>>> +{
>>>> + // expand but don't strip type referenced
>>>> + auto &base_type = type.get_elem_type ();
>>>> + base_type->accept_vis (*this);
>>>> + if (base_type->is_marked_for_strip ())
>>>> + rust_error_at (base_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + // same for expression
>>>> + auto &size_expr = type.get_size_expr ();
>>>> + size_expr->accept_vis (*this);
>>>> + if (size_expr->is_marked_for_strip ())
>>>> + rust_error_at (size_expr->get_locus (),
>>>> + "cannot strip expression in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::SliceType &type)
>>>> +{
>>>> + // expand but don't strip elem type
>>>> + auto &elem_type = type.get_elem_type ();
>>>> + elem_type->accept_vis (*this);
>>>> + if (elem_type->is_marked_for_strip ())
>>>> + rust_error_at (elem_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::InferredType &)
>>>> +{
>>>> + // none possible
>>>> +}
>>>> +void
>>>> +AttrVisitor::visit (AST::BareFunctionType &type)
>>>> +{
>>>> + // seem to be no generics
>>>> +
>>>> + // presumably function params can be stripped
>>>> + auto ¶ms = type.get_function_params ();
>>>> + for (auto it = params.begin (); it != params.end ();)
>>>> + {
>>>> + auto ¶m = *it;
>>>> +
>>>> + auto ¶m_attrs = param.get_outer_attrs ();
>>>> + expander.expand_cfg_attrs (param_attrs);
>>>> + if (expander.fails_cfg_with_expand (param_attrs))
>>>> + {
>>>> + it = params.erase (it);
>>>> + continue;
>>>> + }
>>>> +
>>>> + expander.push_context (MacroExpander::ContextType::TYPE);
>>>> +
>>>> + auto &type = param.get_type ();
>>>> + type->accept_vis (*this);
>>>> +
>>>> + maybe_expand_type (type);
>>>> +
>>>> + if (type->is_marked_for_strip ())
>>>> + rust_error_at (type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> +
>>>> + expander.pop_context ();
>>>> +
>>>> + // increment if nothing else happens
>>>> + ++it;
>>>> + }
>>>> +
>>>> + /* TODO: assuming that variadic nature cannot be stripped. If this
>>>> + * is not true, then have code here to do so. */
>>>> +
>>>> + if (type.has_return_type ())
>>>> + {
>>>> + // FIXME: Can we have type expansion in this position?
>>>> + // In that case, we need to handle AST::TypeNoBounds on top of just
>>>> + // AST::Types
>>>> + auto &return_type = type.get_return_type ();
>>>> + return_type->accept_vis (*this);
>>>> + if (return_type->is_marked_for_strip ())
>>>> + rust_error_at (return_type->get_locus (),
>>>> + "cannot strip type in this position");
>>>> + }
>>>> +
>>>> + // no where clause, apparently
>>>> +}
>>>> +void
>>>> +AttrVisitor::maybe_expand_expr (std::unique_ptr<AST::Expr> &expr)
>>>> +{
>>>> + auto final_fragment = expand_macro_fragment_recursive ();
>>>> + if (final_fragment.should_expand ())
>>>> + expr = final_fragment.take_expression_fragment ();
>>>> +}
>>>> +
>>>> +void
>>>> +AttrVisitor::maybe_expand_type (std::unique_ptr<AST::Type> &type)
>>>> +{
>>>> + auto final_fragment = expand_macro_fragment_recursive ();
>>>> + if (final_fragment.should_expand ())
>>>> + type = final_fragment.take_type_fragment ();
>>>> +}
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-attribute-visitor.h b/gcc/rust/expand/rust-attribute-visitor.h
>>>> new file mode 100644
>>>> index 00000000000..0f9d1065334
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-attribute-visitor.h
>>>> @@ -0,0 +1,316 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#include "rust-ast-visitor.h"
>>>> +#include "rust-ast.h"
>>>> +#include "rust-macro-expand.h"
>>>> +
>>>> +namespace Rust {
>>>> +// Visitor used to expand attributes.
>>>> +class AttrVisitor : public AST::ASTVisitor
>>>> +{
>>>> +private:
>>>> + MacroExpander &expander;
>>>> + void maybe_expand_expr (std::unique_ptr<AST::Expr> &expr);
>>>> + void maybe_expand_type (std::unique_ptr<AST::Type> &expr);
>>>> +
>>>> +public:
>>>> + AttrVisitor (MacroExpander &expander) : expander (expander) {}
>>>> +
>>>> + void expand_struct_fields (std::vector<AST::StructField> &fields);
>>>> + void expand_tuple_fields (std::vector<AST::TupleField> &fields);
>>>> + void expand_function_params (std::vector<AST::FunctionParam> ¶ms);
>>>> + void expand_generic_args (AST::GenericArgs &args);
>>>> + void expand_qualified_path_type (AST::QualifiedPathType &path_type);
>>>> + void expand_closure_params (std::vector<AST::ClosureParam> ¶ms);
>>>> + void expand_self_param (AST::SelfParam &self_param);
>>>> + void expand_where_clause (AST::WhereClause &where_clause);
>>>> + void expand_trait_function_decl (AST::TraitFunctionDecl &decl);
>>>> + void expand_trait_method_decl (AST::TraitMethodDecl &decl);
>>>> +
>>>> + /**
>>>> + * Expand The current macro fragment recursively until it could not be
>>>> + * expanded further.
>>>> + *
>>>> + * The return value checking works because correctly
>>>> + * expanded fragment can never be an error (if the fragment can not be
>>>> + * expanded, a stand-in error fragment will be returned; for fragments that
>>>> + * could not be further expanded: the fragment prior to the expansion failure
>>>> + * will be returned).
>>>> + *
>>>> + * @return Either the expanded fragment or an empty errored-out fragment
>>>> + * indicating an expansion failure.
>>>> + */
>>>> + AST::ASTFragment expand_macro_fragment_recursive ()
>>>
>>> Why is this in the header?
>>
>> Do you mean the definition or the documentation? If you refer to the
>> definition, then that is a mistake and should be in the implementation.
>
> Yep, the definition.
>
>>>> + {
>>>> + auto fragment = expander.take_expanded_fragment (*this);
>>>> + unsigned int original_depth = expander.expansion_depth;
>>>> + auto final_fragment = AST::ASTFragment ({}, true);
>>>> +
>>>> + while (fragment.should_expand ())
>>>> + {
>>>> + final_fragment = std::move (fragment);
>>>> + expander.expansion_depth++;
>>>> + // further expand the previously expanded macro fragment
>>>> + auto new_fragment = expander.take_expanded_fragment (*this);
>>>> + if (new_fragment.is_error ())
>>>> + break;
>>>> + fragment = std::move (new_fragment);
>>>> + }
>>>> + expander.expansion_depth = original_depth;
>>>> + return final_fragment;
>>>> + }
>>>> +
>>>> + /**
>>>> + * Expand a set of values, erasing them if they are marked for strip, and
>>>> + * replacing them with expanded macro nodes if necessary.
>>>> + * This function is slightly different from `expand_pointer_allow_strip` as
>>>> + * it can only be called in certain expansion contexts - where macro
>>>> + * invocations are allowed.
>>>> + *
>>>> + * @param ctx Context to use for macro expansion
>>>> + * @param values Iterable reference over values to replace or erase
>>>> + * @param extractor Function to call when replacing values with the content
>>>> + * of an expanded AST node
>>>> + */
>>>> + template <typename T, typename U>
>>>> + void expand_macro_children (MacroExpander::ContextType ctx, T &values,
>>>> + std::function<U (AST::SingleASTNode)> extractor)
>>>> + {
>>>> + expander.push_context (ctx);
>>>> +
>>>> + for (auto it = values.begin (); it != values.end ();)
>>>> + {
>>>> + auto &value = *it;
>>>> +
>>>> + // mark for stripping if required
>>>> + value->accept_vis (*this);
>>>> +
>>>> + // recursively expand the children
>>>> + auto final_fragment = expand_macro_fragment_recursive ();
>>>> +
>>>> + if (final_fragment.should_expand ())
>>>> + {
>>>> + it = values.erase (it);
>>>> + for (auto &node : final_fragment.get_nodes ())
>>>> + {
>>>> + auto new_node = extractor (node);
>>>> + if (new_node != nullptr && !new_node->is_marked_for_strip ())
>>>> + {
>>>> + it = values.insert (it, std::move (new_node));
>>>> + it++;
>>>> + }
>>>> + }
>>>> + }
>>>> + else if (value->is_marked_for_strip ())
>>>> + {
>>>> + it = values.erase (it);
>>>> + }
>>>> + else
>>>> + {
>>>> + ++it;
>>>> + }
>>>> + }
>>>> +
>>>> + expander.pop_context ();
>>>> + }
>>>> +
>>>> + template <typename T> void expand_pointer_allow_strip (T &values)
>>>> + {
>>>> + for (auto it = values.begin (); it != values.end ();)
>>>> + {
>>>> + auto &value = *it;
>>>> +
>>>> + // mark for stripping if required
>>>> + value->accept_vis (*this);
>>>> + if (value->is_marked_for_strip ())
>>>> + {
>>>> + it = values.erase (it);
>>>> + }
>>>> + else
>>>> + {
>>>> + ++it;
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + void visit (AST::Token &) override;
>>>> + void visit (AST::DelimTokenTree &) override;
>>>> + void visit (AST::AttrInputMetaItemContainer &) override;
>>>> + void visit (AST::IdentifierExpr &ident_expr) override;
>>>> + void visit (AST::Lifetime &) override;
>>>> + void visit (AST::LifetimeParam &) override;
>>>> + void visit (AST::ConstGenericParam &) override;
>>>> +
>>>> + void visit (AST::MacroInvocation ¯o_invoc) override;
>>>> +
>>>> + void visit (AST::PathInExpression &path) override;
>>>> + void visit (AST::TypePathSegment &) override;
>>>> + void visit (AST::TypePathSegmentGeneric &segment) override;
>>>> + void visit (AST::TypePathSegmentFunction &segment) override;
>>>> + void visit (AST::TypePath &path) override;
>>>> + void visit (AST::QualifiedPathInExpression &path) override;
>>>> + void visit (AST::QualifiedPathInType &path) override;
>>>> +
>>>> + void visit (AST::LiteralExpr &expr) override;
>>>> + void visit (AST::AttrInputLiteral &) override;
>>>> + void visit (AST::MetaItemLitExpr &) override;
>>>> + void visit (AST::MetaItemPathLit &) override;
>>>> + void visit (AST::BorrowExpr &expr) override;
>>>> + void visit (AST::DereferenceExpr &expr) override;
>>>> + void visit (AST::ErrorPropagationExpr &expr) override;
>>>> + void visit (AST::NegationExpr &expr) override;
>>>> + void visit (AST::ArithmeticOrLogicalExpr &expr) override;
>>>> + void visit (AST::ComparisonExpr &expr) override;
>>>> + void visit (AST::LazyBooleanExpr &expr) override;
>>>> + void visit (AST::TypeCastExpr &expr) override;
>>>> + void visit (AST::AssignmentExpr &expr) override;
>>>> + void visit (AST::CompoundAssignmentExpr &expr) override;
>>>> + void visit (AST::GroupedExpr &expr) override;
>>>> + void visit (AST::ArrayElemsValues &elems) override;
>>>> + void visit (AST::ArrayElemsCopied &elems) override;
>>>> + void visit (AST::ArrayExpr &expr) override;
>>>> + void visit (AST::ArrayIndexExpr &expr) override;
>>>> + void visit (AST::TupleExpr &expr) override;
>>>> + void visit (AST::TupleIndexExpr &expr) override;
>>>> + void visit (AST::StructExprStruct &expr) override;
>>>> + void visit (AST::StructExprFieldIdentifier &) override;
>>>> + void visit (AST::StructExprFieldIdentifierValue &field) override;
>>>> +
>>>> + void visit (AST::StructExprFieldIndexValue &field) override;
>>>> + void visit (AST::StructExprStructFields &expr) override;
>>>> + void visit (AST::StructExprStructBase &expr) override;
>>>> + void visit (AST::CallExpr &expr) override;
>>>> + void visit (AST::MethodCallExpr &expr) override;
>>>> + void visit (AST::FieldAccessExpr &expr) override;
>>>> + void visit (AST::ClosureExprInner &expr) override;
>>>> +
>>>> + void visit (AST::BlockExpr &expr) override;
>>>> +
>>>> + void visit (AST::ClosureExprInnerTyped &expr) override;
>>>> + void visit (AST::ContinueExpr &expr) override;
>>>> + void visit (AST::BreakExpr &expr) override;
>>>> + void visit (AST::RangeFromToExpr &expr) override;
>>>> + void visit (AST::RangeFromExpr &expr) override;
>>>> + void visit (AST::RangeToExpr &expr) override;
>>>> + void visit (AST::RangeFullExpr &) override;
>>>> + void visit (AST::RangeFromToInclExpr &expr) override;
>>>> + void visit (AST::RangeToInclExpr &expr) override;
>>>> + void visit (AST::ReturnExpr &expr) override;
>>>> + void visit (AST::UnsafeBlockExpr &expr) override;
>>>> + void visit (AST::LoopExpr &expr) override;
>>>> + void visit (AST::WhileLoopExpr &expr) override;
>>>> + void visit (AST::WhileLetLoopExpr &expr) override;
>>>> + void visit (AST::ForLoopExpr &expr) override;
>>>> + void visit (AST::IfExpr &expr) override;
>>>> + void visit (AST::IfExprConseqElse &expr) override;
>>>> + void visit (AST::IfExprConseqIf &expr) override;
>>>> + void visit (AST::IfExprConseqIfLet &expr) override;
>>>> + void visit (AST::IfLetExpr &expr) override;
>>>> + void visit (AST::IfLetExprConseqElse &expr) override;
>>>> + void visit (AST::IfLetExprConseqIf &expr) override;
>>>> + void visit (AST::IfLetExprConseqIfLet &expr) override;
>>>> + void visit (AST::MatchExpr &expr) override;
>>>> + void visit (AST::AwaitExpr &expr) override;
>>>> + void visit (AST::AsyncBlockExpr &expr) override;
>>>> + void visit (AST::TypeParam ¶m) override;
>>>> + void visit (AST::LifetimeWhereClauseItem &) override;
>>>> + void visit (AST::TypeBoundWhereClauseItem &item) override;
>>>> + void visit (AST::Method &method) override;
>>>> + void visit (AST::Module &module) override;
>>>> + void visit (AST::ExternCrate &crate) override;
>>>> + void visit (AST::UseTreeGlob &) override;
>>>> + void visit (AST::UseTreeList &) override;
>>>> + void visit (AST::UseTreeRebind &) override;
>>>> + void visit (AST::UseDeclaration &use_decl) override;
>>>> + void visit (AST::Function &function) override;
>>>> + void visit (AST::TypeAlias &type_alias) override;
>>>> + void visit (AST::StructStruct &struct_item) override;
>>>> + void visit (AST::TupleStruct &tuple_struct) override;
>>>> + void visit (AST::EnumItem &item) override;
>>>> + void visit (AST::EnumItemTuple &item) override;
>>>> + void visit (AST::EnumItemStruct &item) override;
>>>> + void visit (AST::EnumItemDiscriminant &item) override;
>>>> + void visit (AST::Enum &enum_item) override;
>>>> + void visit (AST::Union &union_item) override;
>>>> + void visit (AST::ConstantItem &const_item) override;
>>>> + void visit (AST::StaticItem &static_item) override;
>>>> + void visit (AST::TraitItemFunc &item) override;
>>>> + void visit (AST::TraitItemMethod &item) override;
>>>> + void visit (AST::TraitItemConst &item) override;
>>>> + void visit (AST::TraitItemType &item) override;
>>>> + void visit (AST::Trait &trait) override;
>>>> + void visit (AST::InherentImpl &impl) override;
>>>> + void visit (AST::TraitImpl &impl) override;
>>>> + void visit (AST::ExternalStaticItem &item) override;
>>>> + void visit (AST::ExternalFunctionItem &item) override;
>>>> + void visit (AST::ExternBlock &block) override;
>>>> +
>>>> + // I don't think it would be possible to strip macros without expansion
>>>> + void visit (AST::MacroMatchFragment &) override;
>>>> + void visit (AST::MacroMatchRepetition &) override;
>>>> + void visit (AST::MacroMatcher &) override;
>>>> + void visit (AST::MacroRulesDefinition &rules_def) override;
>>>> + void visit (AST::MetaItemPath &) override;
>>>> + void visit (AST::MetaItemSeq &) override;
>>>> + void visit (AST::MetaWord &) override;
>>>> + void visit (AST::MetaNameValueStr &) override;
>>>> + void visit (AST::MetaListPaths &) override;
>>>> + void visit (AST::MetaListNameValueStr &) override;
>>>> + void visit (AST::LiteralPattern &) override;
>>>> + void visit (AST::IdentifierPattern &pattern) override;
>>>> + void visit (AST::WildcardPattern &) override;
>>>> + void visit (AST::RangePatternBoundLiteral &) override;
>>>> + void visit (AST::RangePatternBoundPath &bound) override;
>>>> + void visit (AST::RangePatternBoundQualPath &bound) override;
>>>> + void visit (AST::RangePattern &pattern) override;
>>>> + void visit (AST::ReferencePattern &pattern) override;
>>>> + void visit (AST::StructPatternFieldTuplePat &field) override;
>>>> + void visit (AST::StructPatternFieldIdentPat &field) override;
>>>> + void visit (AST::StructPatternFieldIdent &field) override;
>>>> + void visit (AST::StructPattern &pattern) override;
>>>> + void visit (AST::TupleStructItemsNoRange &tuple_items) override;
>>>> + void visit (AST::TupleStructItemsRange &tuple_items) override;
>>>> + void visit (AST::TupleStructPattern &pattern) override;
>>>> + void visit (AST::TuplePatternItemsMultiple &tuple_items) override;
>>>> + void visit (AST::TuplePatternItemsRanged &tuple_items) override;
>>>> + void visit (AST::TuplePattern &pattern) override;
>>>> + void visit (AST::GroupedPattern &pattern) override;
>>>> + void visit (AST::SlicePattern &pattern) override;
>>>> +
>>>> + void visit (AST::EmptyStmt &) override;
>>>> + void visit (AST::LetStmt &stmt) override;
>>>> + void visit (AST::ExprStmtWithoutBlock &stmt) override;
>>>> + void visit (AST::ExprStmtWithBlock &stmt) override;
>>>> +
>>>> + void visit (AST::TraitBound &bound) override;
>>>> + void visit (AST::ImplTraitType &type) override;
>>>> + void visit (AST::TraitObjectType &type) override;
>>>> + void visit (AST::ParenthesisedType &type) override;
>>>> + void visit (AST::ImplTraitTypeOneBound &type) override;
>>>> + void visit (AST::TraitObjectTypeOneBound &type) override;
>>>> + void visit (AST::TupleType &type) override;
>>>> + void visit (AST::NeverType &) override;
>>>> + void visit (AST::RawPointerType &type) override;
>>>> + void visit (AST::ReferenceType &type) override;
>>>> + void visit (AST::ArrayType &type) override;
>>>> + void visit (AST::SliceType &type) override;
>>>> + void visit (AST::InferredType &) override;
>>>> + void visit (AST::BareFunctionType &type) override;
>>>> +};
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-macro-builtins.cc b/gcc/rust/expand/rust-macro-builtins.cc
>>>> new file mode 100644
>>>> index 00000000000..5eace13d197
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-builtins.cc
>>>> @@ -0,0 +1,484 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#include "rust-macro-builtins.h"
>>>> +#include "rust-diagnostics.h"
>>>> +#include "rust-expr.h"
>>>> +#include "rust-session-manager.h"
>>>> +#include "rust-macro-invoc-lexer.h"
>>>> +#include "rust-lex.h"
>>>> +#include "rust-parse.h"
>>>> +
>>>> +namespace Rust {
>>>> +namespace {
>>>> +std::unique_ptr<AST::Expr>
>>>> +make_string (Location locus, std::string value)
>>>> +{
>>>> + return std::unique_ptr<AST::Expr> (
>>>> + new AST::LiteralExpr (value, AST::Literal::STRING,
>>>> + PrimitiveCoreType::CORETYPE_STR, {}, locus));
>>>> +}
>>>> +
>>>> +/* Match the end token of a macro given the start delimiter of the macro */
>>>> +
>>>> +static inline TokenId
>>>> +macro_end_token (AST::DelimTokenTree &invoc_token_tree,
>>>> + Parser<MacroInvocLexer> &parser)
>>>> +{
>>>> + auto last_token_id = TokenId::RIGHT_CURLY;
>>>> + switch (invoc_token_tree.get_delim_type ())
>>>> + {
>>>> + case AST::DelimType::PARENS:
>>>> + last_token_id = TokenId::RIGHT_PAREN;
>>>> + rust_assert (parser.skip_token (LEFT_PAREN));
>>>> + break;
>>>> +
>>>> + case AST::DelimType::CURLY:
>>>> + rust_assert (parser.skip_token (LEFT_CURLY));
>>>> + break;
>>>> +
>>>> + case AST::DelimType::SQUARE:
>>>> + last_token_id = TokenId::RIGHT_SQUARE;
>>>> + rust_assert (parser.skip_token (LEFT_SQUARE));
>>>> + break;
>>>> + }
>>>> +
>>>> + return last_token_id;
>>>> +}
>>>> +
>>>> +/* Parse a single string literal from the given delimited token tree,
>>>> + and return the LiteralExpr for it. Allow for an optional trailing comma,
>>>> + but otherwise enforce that these are the only tokens. */
>>>> +
>>>> +std::unique_ptr<AST::LiteralExpr>
>>>> +parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree,
>>>> + Location invoc_locus)
>>>> +{
>>>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>>>> + Parser<MacroInvocLexer> parser (lex);
>>>> +
>>>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>>>> +
>>>> + std::unique_ptr<AST::LiteralExpr> lit_expr = nullptr;
>>>> +
>>>> + if (parser.peek_current_token ()->get_id () == STRING_LITERAL)
>>>> + {
>>>> + lit_expr = parser.parse_literal_expr ();
>>>> + parser.maybe_skip_token (COMMA);
>>>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>>>> + {
>>>> + lit_expr = nullptr;
>>>> + rust_error_at (invoc_locus, "macro takes 1 argument");
>>>> + }
>>>> + }
>>>> + else if (parser.peek_current_token ()->get_id () == last_token_id)
>>>> + rust_error_at (invoc_locus, "macro takes 1 argument");
>>>> + else
>>>> + rust_error_at (invoc_locus, "argument must be a string literal");
>>>> +
>>>> + parser.skip_token (last_token_id);
>>>> +
>>>> + return lit_expr;
>>>> +}
>>>> +
>>>> +/* Treat PATH as a path relative to the source file currently being
>>>> + compiled, and return the absolute path for it. */
>>>> +
>>>> +std::string
>>>> +source_relative_path (std::string path, Location locus)
>>>> +{
>>>> + std::string compile_fname
>>>> + = Session::get_instance ().linemap->location_file (locus);
>>>> +
>>>> + auto dir_separator_pos = compile_fname.rfind (file_separator);
>>>> +
>>>> + /* If there is no file_separator in the path, use current dir ('.'). */
>>>> + std::string dirname;
>>>> + if (dir_separator_pos == std::string::npos)
>>>> + dirname = std::string (".") + file_separator;
>>>> + else
>>>> + dirname = compile_fname.substr (0, dir_separator_pos) + file_separator;
>>>> +
>>>> + return dirname + path;
>>>> +}
>>>> +
>>>> +/* Read the full contents of the file FILENAME and return them in a vector.
>>>> + FIXME: platform specific. */
>>>> +
>>>> +std::vector<uint8_t>
>>>> +load_file_bytes (const char *filename)
>>>> +{
>>>> + RAIIFile file_wrap (filename);
>>>> + if (file_wrap.get_raw () == nullptr)
>>>> + {
>>>> + rust_error_at (Location (), "cannot open filename %s: %m", filename);
>>>> + return std::vector<uint8_t> ();
>>>> + }
>>>> +
>>>> + FILE *f = file_wrap.get_raw ();
>>>> + fseek (f, 0L, SEEK_END);
>>>> + long fsize = ftell (f);
>>>> + fseek (f, 0L, SEEK_SET);
>>>> +
>>>> + std::vector<uint8_t> buf (fsize);
>>>> +
>>>> + if (fread (&buf[0], fsize, 1, f) != 1)
>>>> + {
>>>> + rust_error_at (Location (), "error reading file %s: %m", filename);
>>>> + return std::vector<uint8_t> ();
>>>> + }
>>>> +
>>>> + return buf;
>>>> +}
>>>> +} // namespace
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + rust_debug ("assert!() called");
>>>> +
>>>> + return AST::ASTFragment::create_error ();
>>>> +}
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto current_file
>>>> + = Session::get_instance ().linemap->location_file (invoc_locus);
>>>> + auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file));
>>>> +
>>>> + return AST::ASTFragment ({file_str});
>>>> +}
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto current_column
>>>> + = Session::get_instance ().linemap->location_to_column (invoc_locus);
>>>> +
>>>> + auto column_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>>>> + new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT,
>>>> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
>>>> +
>>>> + return AST::ASTFragment ({column_no});
>>>> +}
>>>> +
>>>> +/* Expand builtin macro include_bytes!("filename"), which includes the contents
>>>> + of the given file as reference to a byte array. Yields an expression of type
>>>> + &'static [u8; N]. */
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + /* Get target filename from the macro invocation, which is treated as a path
>>>> + relative to the include!-ing file (currently being compiled). */
>>>> + auto lit_expr
>>>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>>>> + if (lit_expr == nullptr)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + std::string target_filename
>>>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>>>> +
>>>> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
>>>> +
>>>> + /* Is there a more efficient way to do this? */
>>>
>>> Well, first of all you don't need to read all of the file at once, second you
>>> could pre-allocate 'elts' to the correct size and final you could obviously
>>> choose a cheaper representation ;)
>>>
>>>> + std::vector<std::unique_ptr<AST::Expr>> elts;
>>>> + for (uint8_t b : bytes)
>>>> + {
>>>> + elts.emplace_back (
>>>> + new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE,
>>>> + PrimitiveCoreType::CORETYPE_U8,
>>>> + {} /* outer_attrs */, invoc_locus));
>>>> + }
>>>> +
>>>> + auto elems = std::unique_ptr<AST::ArrayElems> (
>>>> + new AST::ArrayElemsValues (std::move (elts), invoc_locus));
>>>> +
>>>> + auto array = std::unique_ptr<AST::Expr> (
>>>> + new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus));
>>>> +
>>>> + auto borrow = std::unique_ptr<AST::Expr> (
>>>> + new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus));
>>>> +
>>>> + auto node = AST::SingleASTNode (std::move (borrow));
>>>> + return AST::ASTFragment ({node});
>>>> +}
>>>> +
>>>> +/* Expand builtin macro include_str!("filename"), which includes the contents
>>>> + of the given file as a string. The file must be UTF-8 encoded. Yields an
>>>> + expression of type &'static str. */
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + /* Get target filename from the macro invocation, which is treated as a path
>>>> + relative to the include!-ing file (currently being compiled). */
>>>> + auto lit_expr
>>>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>>>> + if (lit_expr == nullptr)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + std::string target_filename
>>>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>>>> +
>>>> + std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
>>>> +
>>>> + /* FIXME: Enforce that the file contents are valid UTF-8. */
>>>> + std::string str ((const char *) &bytes[0], bytes.size ());
>>>> +
>>>> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
>>>> + return AST::ASTFragment ({node});
>>>> +}
>>>> +
>>>> +/* Expand builtin macro compile_error!("error"), which forces a compile error
>>>> + during the compile time. */
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto lit_expr
>>>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>>>> + if (lit_expr == nullptr)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + std::string error_string = lit_expr->as_string ();
>>>> + rust_error_at (invoc_locus, "%s", error_string.c_str ());
>>>> +
>>>> + return AST::ASTFragment::create_error ();
>>>> +}
>>>> +
>>>> +/* Expand builtin macro concat!(), which joins all the literal parameters
>>>> + into a string with no delimiter. */
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
>>>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>>>> + Parser<MacroInvocLexer> parser (lex);
>>>> +
>>>> + auto str = std::string ();
>>>> + bool has_error = false;
>>>> +
>>>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>>>> +
>>>> + /* NOTE: concat! could accept no argument, so we don't have any checks here */
>>>> + while (parser.peek_current_token ()->get_id () != last_token_id)
>>>> + {
>>>> + auto lit_expr = parser.parse_literal_expr ();
>>>> + if (lit_expr)
>>>> + {
>>>> + str += lit_expr->as_string ();
>>>> + }
>>>> + else
>>>> + {
>>>> + auto current_token = parser.peek_current_token ();
>>>> + rust_error_at (current_token->get_locus (),
>>>> + "argument must be a constant literal");
>>>> + has_error = true;
>>>> + // Just crash if the current token can't be skipped
>>>> + rust_assert (parser.skip_token (current_token->get_id ()));
>>>> + }
>>>> + parser.maybe_skip_token (COMMA);
>>>> + }
>>>> +
>>>> + parser.skip_token (last_token_id);
>>>> +
>>>> + if (has_error)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + auto node = AST::SingleASTNode (make_string (invoc_locus, str));
>>>> + return AST::ASTFragment ({node});
>>>> +}
>>>> +
>>>> +/* Expand builtin macro env!(), which inspects an environment variable at
>>>> + compile time. */
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto invoc_token_tree = invoc.get_delim_tok_tree ();
>>>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>>>> + Parser<MacroInvocLexer> parser (lex);
>>>> +
>>>> + auto last_token_id = macro_end_token (invoc_token_tree, parser);
>>>> +
>>>> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
>>>> + {
>>>> + if (parser.peek_current_token ()->get_id () == last_token_id)
>>>> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
>>>> + else
>>>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>>>> + "argument must be a string literal");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + auto lit_expr = parser.parse_literal_expr ();
>>>> + auto comma_skipped = parser.maybe_skip_token (COMMA);
>>>> +
>>>> + std::unique_ptr<AST::LiteralExpr> error_expr = nullptr;
>>>> +
>>>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>>>> + {
>>>> + if (!comma_skipped)
>>>> + {
>>>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>>>> + "expected token: %<,%>");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> + if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
>>>> + {
>>>> + rust_error_at (parser.peek_current_token ()->get_locus (),
>>>> + "argument must be a string literal");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + error_expr = parser.parse_literal_expr ();
>>>> + parser.maybe_skip_token (COMMA);
>>>> + }
>>>> +
>>>> + if (parser.peek_current_token ()->get_id () != last_token_id)
>>>> + {
>>>> + rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + parser.skip_token (last_token_id);
>>>> +
>>>> + auto env_value = getenv (lit_expr->as_string ().c_str ());
>>>> +
>>>> + if (env_value == nullptr)
>>>> + {
>>>> + if (error_expr == nullptr)
>>>> + rust_error_at (invoc_locus, "environment variable %qs not defined",
>>>> + lit_expr->as_string ().c_str ());
>>>> + else
>>>> + rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ());
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + auto node = AST::SingleASTNode (make_string (invoc_locus, env_value));
>>>> + return AST::ASTFragment ({node});
>>>> +}
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + // only parse if not already parsed
>>>> + if (!invoc.is_parsed ())
>>>> + {
>>>> + std::unique_ptr<AST::AttrInputMetaItemContainer> converted_input (
>>>> + invoc.get_delim_tok_tree ().parse_to_meta_item ());
>>>> +
>>>> + if (converted_input == nullptr)
>>>> + {
>>>> + rust_debug ("DEBUG: failed to parse macro to meta item");
>>>> + // TODO: do something now? is this an actual error?
>>>> + }
>>>> + else
>>>> + {
>>>> + std::vector<std::unique_ptr<AST::MetaItemInner>> meta_items (
>>>> + std::move (converted_input->get_items ()));
>>>> + invoc.set_meta_item_output (std::move (meta_items));
>>>> + }
>>>> + }
>>>> +
>>>> + /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg
>>>> + * attributes */
>>>> + if (invoc.get_meta_items ().size () != 1)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + bool result = invoc.get_meta_items ()[0]->check_cfg_predicate (
>>>> + Session::get_instance ());
>>>> + auto literal_exp = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>>>> + new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL,
>>>> + PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus)));
>>>> +
>>>> + return AST::ASTFragment ({literal_exp});
>>>> +}
>>>> +
>>>> +/* Expand builtin macro include!(), which includes a source file at the current
>>>> + scope compile time. */
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + /* Get target filename from the macro invocation, which is treated as a path
>>>> + relative to the include!-ing file (currently being compiled). */
>>>> + auto lit_expr
>>>> + = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
>>>> + if (lit_expr == nullptr)
>>>> + return AST::ASTFragment::create_error ();
>>>> +
>>>> + std::string filename
>>>> + = source_relative_path (lit_expr->as_string (), invoc_locus);
>>>> + auto target_filename
>>>> + = Rust::Session::get_instance ().include_extra_file (std::move (filename));
>>>> +
>>>> + RAIIFile target_file (target_filename);
>>>> + Linemap *linemap = Session::get_instance ().linemap;
>>>> +
>>>> + if (!target_file.ok ())
>>>> + {
>>>> + rust_error_at (lit_expr->get_locus (),
>>>> + "cannot open included file %qs: %m", target_filename);
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + rust_debug ("Attempting to parse included file %s", target_filename);
>>>> +
>>>> + Lexer lex (target_filename, std::move (target_file), linemap);
>>>> + Parser<Lexer> parser (lex);
>>>> +
>>>> + auto parsed_items = parser.parse_items ();
>>>> + bool has_error = !parser.get_errors ().empty ();
>>>> +
>>>> + for (const auto &error : parser.get_errors ())
>>>> + error.emit_error ();
>>>> +
>>>> + if (has_error)
>>>> + {
>>>> + // inform the user that the errors above are from a included file
>>>> + rust_inform (invoc_locus, "included from here");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + std::vector<AST::SingleASTNode> nodes{};
>>>> + for (auto &item : parsed_items)
>>>> + {
>>>> + AST::SingleASTNode node (std::move (item));
>>>> + nodes.push_back (node);
>>>> + }
>>>> +
>>>> + return AST::ASTFragment (nodes);
>>>> +}
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc)
>>>> +{
>>>> + auto current_line
>>>> + = Session::get_instance ().linemap->location_to_line (invoc_locus);
>>>> +
>>>> + auto line_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
>>>> + new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT,
>>>> + PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
>>>> +
>>>> + return AST::ASTFragment ({line_no});
>>>> +}
>>>> +
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-macro-builtins.h b/gcc/rust/expand/rust-macro-builtins.h
>>>> new file mode 100644
>>>> index 00000000000..91f3727d450
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-builtins.h
>>>> @@ -0,0 +1,107 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#ifndef RUST_MACRO_BUILTINS_H
>>>> +#define RUST_MACRO_BUILTINS_H
>>>> +
>>>> +#include "rust-ast.h"
>>>> +#include "rust-location.h"
>>>> +
>>>> +/**
>>>> + * This class provides a list of builtin macros implemented by the compiler.
>>>> + * The functions defined are called "builtin transcribers" in that they replace
>>>> + * the transcribing part of a macro definition.
>>>> + *
>>>> + * Like regular macro transcribers, they are responsible for building and
>>>> + * returning an AST fragment: basically a vector of AST nodes put together.
>>>> + *
>>>> + * Unlike regular declarative macros where each match arm has its own associated
>>>> + * transcriber, builtin transcribers are responsible for handling all match arms
>>>> + * of the macro. This means that you should take extra care when implementing a
>>>> + * builtin containing multiple match arms: You will probably need to do some
>>>> + * lookahead in order to determine which match arm the user intended to use.
>>>> + *
>>>> + * An example of this is the `assert!()` macro:
>>>> + *
>>>> + * ```
>>>> + * macro_rules! assert {
>>>> + * ($cond:expr $(,)?) => {{ ... }};
>>>> + * ($cond : expr, $ ($arg : tt) +) = > {{ ... }};
>>>> + * }
>>>> + * ```
>>>> + *
>>>> + * If more tokens exist beyond the optional comma, they need to be handled as
>>>> + * a token-tree for a custom panic message.
>>>> + *
>>>> + * These builtin macros with empty transcribers are defined in the standard
>>>> + * library. They are marked with a special attribute, `#[rustc_builtin_macro]`.
>>>> + * When this attribute is present on a macro definition, the compiler should
>>>> + * look for an associated transcriber in the mappings. Meaning that you must
>>>> + * remember to insert your transcriber in the `builtin_macros` map of the
>>>> + *`Mappings`.
>>>> + *
>>>> + * This map is built as a static variable in the `insert_macro_def()` method
>>>> + * of the `Mappings` class.
>>>> + */
>>>> +
>>>> +/* If assert is defined as a macro this file will not parse, so undefine this
>>>> + before continuing. */
>>>> +#ifdef assert
>>>> +#undef assert
>>>> +#endif
>>>> +
>>>> +namespace Rust {
>>>> +class MacroBuiltin
>>>> +{
>>>> +public:
>>>> + static AST::ASTFragment assert (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment file (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment column (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment include_bytes (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment include_str (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment compile_error (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment concat (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment env (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment cfg (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment include (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +
>>>> + static AST::ASTFragment line (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc);
>>>> +};
>>>> +} // namespace Rust
>>>> +
>>>> +#endif // RUST_MACRO_BUILTINS_H
>>>> diff --git a/gcc/rust/expand/rust-macro-expand.cc b/gcc/rust/expand/rust-macro-expand.cc
>>>> new file mode 100644
>>>> index 00000000000..1d57e394220
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-expand.cc
>>>> @@ -0,0 +1,1012 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#include "rust-macro-expand.h"
>>>> +#include "rust-macro-substitute-ctx.h"
>>>> +#include "rust-ast-full.h"
>>>> +#include "rust-ast-visitor.h"
>>>> +#include "rust-diagnostics.h"
>>>> +#include "rust-parse.h"
>>>> +#include "rust-attribute-visitor.h"
>>>> +
>>>> +namespace Rust {
>>>> +AST::ASTFragment
>>>> +MacroExpander::expand_decl_macro (Location invoc_locus,
>>>> + AST::MacroInvocData &invoc,
>>>> + AST::MacroRulesDefinition &rules_def,
>>>> + bool semicolon)
>>>> +{
>>>> + // ensure that both invocation and rules are in a valid state
>>>> + rust_assert (!invoc.is_marked_for_strip ());
>>>> + rust_assert (!rules_def.is_marked_for_strip ());
>>>> + rust_assert (rules_def.get_macro_rules ().size () > 0);
>>>> +
>>>> + /* probably something here about parsing invoc and rules def token trees to
>>>> + * token stream. if not, how would parser handle the captures of exprs and
>>>> + * stuff? on the other hand, token trees may be kind of useful in rules def as
>>>> + * creating a point where recursion can occur (like having
>>>> + * "compare_macro_match" and then it calling itself when it finds delimiters)
>>>> + */
>>>> +
>>>> + /* find matching rule to invoc token tree, based on macro rule's matcher. if
>>>> + * none exist, error.
>>>> + * - specifically, check each matcher in order. if one fails to match, move
>>>> + * onto next. */
>>>> + /* TODO: does doing this require parsing expressions and whatever in the
>>>> + * invoc? if so, might as well save the results if referenced using $ or
>>>> + * whatever. If not, do another pass saving them. Except this is probably
>>>> + * useless as different rules could have different starting points for exprs
>>>> + * or whatever. Decision trees could avoid this, but they have their own
>>>> + * issues. */
>>>> + /* TODO: will need to modify the parser so that it can essentially "catch"
>>>> + * errors - maybe "try_parse_expr" or whatever methods. */
>>>> + // this technically creates a back-tracking parser - this will be the
>>>> + // implementation style
>>>> +
>>>> + /* then, after results are saved, generate the macro output from the
>>>> + * transcriber token tree. if i understand this correctly, the macro
>>>> + * invocation gets replaced by the transcriber tokens, except with
>>>> + * substitutions made (e.g. for $i variables) */
>>>> +
>>>> + /* TODO: it is probably better to modify AST::Token to store a pointer to a
>>>> + * Lexer::Token (rather than being converted) - i.e. not so much have
>>>> + * AST::Token as a Token but rather a TokenContainer (as it is another type of
>>>> + * TokenTree). This will prevent re-conversion of Tokens between each type
>>>> + * all the time, while still allowing the heterogenous storage of token trees.
>>>> + */
>>>> +
>>>> + AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree ();
>>>> +
>>>> + // find matching arm
>>>> + AST::MacroRule *matched_rule = nullptr;
>>>> + std::map<std::string, MatchedFragmentContainer> matched_fragments;
>>>> + for (auto &rule : rules_def.get_rules ())
>>>> + {
>>>> + sub_stack.push ();
>>>> + bool did_match_rule = try_match_rule (rule, invoc_token_tree);
>>>> + matched_fragments = sub_stack.pop ();
>>>> +
>>>> + if (did_match_rule)
>>>> + {
>>>> + // // Debugging
>>>> + // for (auto &kv : matched_fragments)
>>>> + // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (),
>>>> + // kv.second.get_fragments ().size (),
>>>> + // kv.second.get_kind ()
>>>> + // == MatchedFragmentContainer::Kind::Repetition
>>>> + // ? "repetition"
>>>> + // : "metavar");
>>>> +
>>>> + matched_rule = &rule;
>>>> + break;
>>>> + }
>>>> + }
>>>> +
>>>> + if (matched_rule == nullptr)
>>>> + {
>>>> + RichLocation r (invoc_locus);
>>>> + r.add_range (rules_def.get_locus ());
>>>> + rust_error_at (r, "Failed to match any rule within macro");
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments,
>>>> + semicolon, peek_context ());
>>>> +}
>>>> +
>>>> +void
>>>> +MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon)
>>>> +{
>>>> + if (depth_exceeds_recursion_limit ())
>>>> + {
>>>> + rust_error_at (invoc.get_locus (), "reached recursion limit");
>>>> + return;
>>>> + }
>>>> +
>>>> + AST::MacroInvocData &invoc_data = invoc.get_invoc_data ();
>>>> +
>>>> + // ??
>>>> + // switch on type of macro:
>>>> + // - '!' syntax macro (inner switch)
>>>> + // - procedural macro - "A token-based function-like macro"
>>>> + // - 'macro_rules' (by example/pattern-match) macro? or not? "an
>>>> + // AST-based function-like macro"
>>>> + // - else is unreachable
>>>> + // - attribute syntax macro (inner switch)
>>>> + // - procedural macro attribute syntax - "A token-based attribute
>>>> + // macro"
>>>> + // - legacy macro attribute syntax? - "an AST-based attribute macro"
>>>> + // - non-macro attribute: mark known
>>>> + // - else is unreachable
>>>> + // - derive macro (inner switch)
>>>> + // - derive or legacy derive - "token-based" vs "AST-based"
>>>> + // - else is unreachable
>>>> + // - derive container macro - unreachable
>>>> +
>>>> + // lookup the rules for this macro
>>>> + NodeId resolved_node = UNKNOWN_NODEID;
>>>> + NodeId source_node = UNKNOWN_NODEID;
>>>> + if (has_semicolon)
>>>> + source_node = invoc.get_macro_node_id ();
>>>> + else
>>>> + source_node = invoc.get_pattern_node_id ();
>>>> + auto seg
>>>> + = Resolver::CanonicalPath::new_seg (source_node,
>>>> + invoc_data.get_path ().as_string ());
>>>> +
>>>> + bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node);
>>>> + if (!found)
>>>> + {
>>>> + rust_error_at (invoc.get_locus (), "unknown macro: [%s]",
>>>> + seg.get ().c_str ());
>>>> + return;
>>>> + }
>>>> +
>>>> + // lookup the rules
>>>> + AST::MacroRulesDefinition *rules_def = nullptr;
>>>> + bool ok = mappings->lookup_macro_def (resolved_node, &rules_def);
>>>> + rust_assert (ok);
>>>> +
>>>> + auto fragment = AST::ASTFragment::create_error ();
>>>> +
>>>> + if (rules_def->is_builtin ())
>>>> + fragment
>>>> + = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data);
>>>> + else
>>>> + fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def,
>>>> + has_semicolon);
>>>> +
>>>> + set_expanded_fragment (std::move (fragment));
>>>> +}
>>>> +
>>>> +/* Determines whether any cfg predicate is false and hence item with attributes
>>>> + * should be stripped. Note that attributes must be expanded before calling. */
>>>> +bool
>>>> +MacroExpander::fails_cfg (const AST::AttrVec &attrs) const
>>>> +{
>>>> + for (const auto &attr : attrs)
>>>> + {
>>>> + if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session))
>>>> + return true;
>>>> + }
>>>> + return false;
>>>> +}
>>>> +
>>>> +/* Determines whether any cfg predicate is false and hence item with attributes
>>>> + * should be stripped. Will expand attributes as well. */
>>>> +bool
>>>> +MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const
>>>> +{
>>>> + // TODO: maybe have something that strips cfg attributes that evaluate true?
>>>> + for (auto &attr : attrs)
>>>> + {
>>>> + if (attr.get_path () == "cfg")
>>>> + {
>>>> + if (!attr.is_parsed_to_meta_item ())
>>>> + attr.parse_attr_to_meta_item ();
>>>> +
>>>> + // DEBUG
>>>> + if (!attr.is_parsed_to_meta_item ())
>>>> + rust_debug ("failed to parse attr to meta item, right before "
>>>> + "cfg predicate check");
>>>> + else
>>>> + rust_debug ("attr has been successfully parsed to meta item, "
>>>> + "right before cfg predicate check");
>>>> +
>>>> + if (!attr.check_cfg_predicate (session))
>>>> + {
>>>> + // DEBUG
>>>> + rust_debug (
>>>> + "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m",
>>>> + attr.as_string ().c_str ());
>>>> +
>>>> + return true;
>>>> + }
>>>> + else
>>>> + {
>>>> + // DEBUG
>>>> + rust_debug ("cfg predicate succeeded for attribute: "
>>>> + "\033[0;31m'%s'\033[0m",
>>>> + attr.as_string ().c_str ());
>>>> + }
>>>> + }
>>>> + }
>>>> + return false;
>>>> +}
>>>> +
>>>> +// Expands cfg_attr attributes.
>>>> +void
>>>> +MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs)
>>>> +{
>>>> + for (std::size_t i = 0; i < attrs.size (); i++)
>>>> + {
>>>> + auto &attr = attrs[i];
>>>> + if (attr.get_path () == "cfg_attr")
>>>> + {
>>>> + if (!attr.is_parsed_to_meta_item ())
>>>> + attr.parse_attr_to_meta_item ();
>>>> +
>>>> + if (attr.check_cfg_predicate (session))
>>>> + {
>>>> + // split off cfg_attr
>>>> + AST::AttrVec new_attrs = attr.separate_cfg_attrs ();
>>>> +
>>>> + // remove attr from vector
>>>> + attrs.erase (attrs.begin () + i);
>>>> +
>>>> + // add new attrs to vector
>>>> + attrs.insert (attrs.begin () + i,
>>>> + std::make_move_iterator (new_attrs.begin ()),
>>>> + std::make_move_iterator (new_attrs.end ()));
>>>> + }
>>>> +
>>>> + /* do something - if feature (first token in tree) is in fact enabled,
>>>> + * make tokens listed afterwards into attributes. i.e.: for
>>>> + * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add
>>>> + * attributes [wow1] and [wow2] to attribute list. This can also be
>>>> + * recursive, so check for expanded attributes being recursive and
>>>> + * possibly recursively call the expand_attrs? */
>>>> + }
>>>> + else
>>>> + {
>>>> + i++;
>>>> + }
>>>> + }
>>>> + attrs.shrink_to_fit ();
>>>> +}
>>>> +
>>>> +void
>>>> +MacroExpander::expand_crate ()
>>>> +{
>>>> + NodeId scope_node_id = crate.get_node_id ();
>>>> + resolver->get_macro_scope ().push (scope_node_id);
>>>> +
>>>> + /* fill macro/decorator map from init list? not sure where init list comes
>>>> + * from? */
>>>> +
>>>> + // TODO: does cfg apply for inner attributes? research.
>>>> + // the apparent answer (from playground test) is yes
>>>> +
>>>> + // expand crate cfg_attr attributes
>>>> + expand_cfg_attrs (crate.inner_attrs);
>>>> +
>>>> + if (fails_cfg_with_expand (crate.inner_attrs))
>>>> + {
>>>> + // basically, delete whole crate
>>>> + crate.strip_crate ();
>>>> + // TODO: maybe create warning here? probably not desired behaviour
>>>> + }
>>>> + // expand module attributes?
>>>> +
>>>> + push_context (ITEM);
>>>> +
>>>> + // expand attributes recursively and strip items if required
>>>> + AttrVisitor attr_visitor (*this);
>>>> + auto &items = crate.items;
>>>> + for (auto it = items.begin (); it != items.end ();)
>>>> + {
>>>> + auto &item = *it;
>>>> +
>>>> + // mark for stripping if required
>>>> + item->accept_vis (attr_visitor);
>>>> +
>>>> + auto fragment = take_expanded_fragment (attr_visitor);
>>>> + if (fragment.should_expand ())
>>>> + {
>>>> + // Remove the current expanded invocation
>>>> + it = items.erase (it);
>>>> + for (auto &node : fragment.get_nodes ())
>>>> + {
>>>> + it = items.insert (it, node.take_item ());
>>>> + it++;
>>>> + }
>>>> + }
>>>> + else if (item->is_marked_for_strip ())
>>>> + it = items.erase (it);
>>>> + else
>>>> + it++;
>>>> + }
>>>> +
>>>> + pop_context ();
>>>> +
>>>> + // TODO: should recursive attribute and macro expansion be done in the same
>>>> + // transversal? Or in separate ones like currently?
>>>> +
>>>> + // expand module tree recursively
>>>> +
>>>> + // post-process
>>>> +
>>>> + // extract exported macros?
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::depth_exceeds_recursion_limit () const
>>>> +{
>>>> + return expansion_depth >= cfg.recursion_limit;
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::try_match_rule (AST::MacroRule &match_rule,
>>>> + AST::DelimTokenTree &invoc_token_tree)
>>>> +{
>>>> + MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
>>>> + Parser<MacroInvocLexer> parser (lex);
>>>> +
>>>> + AST::MacroMatcher &matcher = match_rule.get_matcher ();
>>>> +
>>>> + expansion_depth++;
>>>> + if (!match_matcher (parser, matcher))
>>>> + {
>>>> + expansion_depth--;
>>>> + return false;
>>>> + }
>>>> + expansion_depth--;
>>>> +
>>>> + bool used_all_input_tokens = parser.skip_token (END_OF_FILE);
>>>> + return used_all_input_tokens;
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::match_fragment (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchFragment &fragment)
>>>> +{
>>>> + switch (fragment.get_frag_spec ().get_kind ())
>>>> + {
>>>> + case AST::MacroFragSpec::EXPR:
>>>> + parser.parse_expr ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::BLOCK:
>>>> + parser.parse_block_expr ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::IDENT:
>>>> + parser.parse_identifier_pattern ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::LITERAL:
>>>> + parser.parse_literal_expr ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::ITEM:
>>>> + parser.parse_item (false);
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::TY:
>>>> + parser.parse_type ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::PAT:
>>>> + parser.parse_pattern ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::PATH:
>>>> + parser.parse_path_in_expression ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::VIS:
>>>> + parser.parse_visibility ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::STMT: {
>>>> + auto restrictions = ParseRestrictions ();
>>>> + restrictions.consume_semi = false;
>>>> + parser.parse_stmt (restrictions);
>>>> + break;
>>>> + }
>>>> +
>>>> + case AST::MacroFragSpec::LIFETIME:
>>>> + parser.parse_lifetime_params ();
>>>> + break;
>>>> +
>>>> + // is meta attributes?
>>>> + case AST::MacroFragSpec::META:
>>>> + parser.parse_attribute_body ();
>>>> + break;
>>>> +
>>>> + case AST::MacroFragSpec::TT:
>>>> + parser.parse_token_tree ();
>>>> + break;
>>>> +
>>>> + // i guess we just ignore invalid and just error out
>>>> + case AST::MacroFragSpec::INVALID:
>>>> + return false;
>>>> + }
>>>> +
>>>> + // it matches if the parser did not produce errors trying to parse that type
>>>> + // of item
>>>> + return !parser.has_errors ();
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::match_matcher (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatcher &matcher)
>>>> +{
>>>> + if (depth_exceeds_recursion_limit ())
>>>> + {
>>>> + rust_error_at (matcher.get_match_locus (), "reached recursion limit");
>>>> + return false;
>>>> + }
>>>> +
>>>> + auto delimiter = parser.peek_current_token ();
>>>> +
>>>> + // this is used so we can check that we delimit the stream correctly.
>>>> + switch (delimiter->get_id ())
>>>> + {
>>>> + case LEFT_PAREN: {
>>>> + if (!parser.skip_token (LEFT_PAREN))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case LEFT_SQUARE: {
>>>> + if (!parser.skip_token (LEFT_SQUARE))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case LEFT_CURLY: {
>>>> + if (!parser.skip_token (LEFT_CURLY))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> + default:
>>>> + gcc_unreachable ();
>>>> + }
>>>> +
>>>> + const MacroInvocLexer &source = parser.get_token_source ();
>>>> +
>>>> + for (auto &match : matcher.get_matches ())
>>>> + {
>>>> + size_t offs_begin = source.get_offs ();
>>>> +
>>>> + switch (match->get_macro_match_type ())
>>>> + {
>>>> + case AST::MacroMatch::MacroMatchType::Fragment: {
>>>> + AST::MacroMatchFragment *fragment
>>>> + = static_cast<AST::MacroMatchFragment *> (match.get ());
>>>> + if (!match_fragment (parser, *fragment))
>>>> + return false;
>>>> +
>>>> + // matched fragment get the offset in the token stream
>>>> + size_t offs_end = source.get_offs ();
>>>> + sub_stack.insert_metavar (
>>>> + MatchedFragment (fragment->get_ident (), offs_begin, offs_end));
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Tok: {
>>>> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
>>>> + if (!match_token (parser, *tok))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Repetition: {
>>>> + AST::MacroMatchRepetition *rep
>>>> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
>>>> + if (!match_repetition (parser, *rep))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Matcher: {
>>>> + AST::MacroMatcher *m
>>>> + = static_cast<AST::MacroMatcher *> (match.get ());
>>>> + expansion_depth++;
>>>> + if (!match_matcher (parser, *m))
>>>> + {
>>>> + expansion_depth--;
>>>> + return false;
>>>> + }
>>>> + expansion_depth--;
>>>> + }
>>>> + break;
>>>> + }
>>>> + }
>>>> +
>>>> + switch (delimiter->get_id ())
>>>> + {
>>>> + case LEFT_PAREN: {
>>>> + if (!parser.skip_token (RIGHT_PAREN))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case LEFT_SQUARE: {
>>>> + if (!parser.skip_token (RIGHT_SQUARE))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> +
>>>> + case LEFT_CURLY: {
>>>> + if (!parser.skip_token (RIGHT_CURLY))
>>>> + return false;
>>>> + }
>>>> + break;
>>>> + default:
>>>> + gcc_unreachable ();
>>>> + }
>>>> +
>>>> + return true;
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::match_token (Parser<MacroInvocLexer> &parser, AST::Token &token)
>>>> +{
>>>> + // FIXME this needs to actually match the content and the type
>>>> + return parser.skip_token (token.get_id ());
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::match_n_matches (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchRepetition &rep,
>>>> + size_t &match_amount, size_t lo_bound,
>>>> + size_t hi_bound)
>>>> +{
>>>> + match_amount = 0;
>>>> + auto &matches = rep.get_matches ();
>>>> +
>>>> + const MacroInvocLexer &source = parser.get_token_source ();
>>>> + while (true)
>>>> + {
>>>> + // If the current token is a closing macro delimiter, break away.
>>>> + // TODO: Is this correct?
>>>> + auto t_id = parser.peek_current_token ()->get_id ();
>>>> + if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY)
>>>> + break;
>>>> +
>>>> + // Skip parsing a separator on the first match, otherwise consume it.
>>>> + // If it isn't present, this is an error
>>>> + if (rep.has_sep () && match_amount > 0)
>>>> + if (!match_token (parser, *rep.get_sep ()))
>>>> + break;
>>>> +
>>>> + bool valid_current_match = false;
>>>> + for (auto &match : matches)
>>>> + {
>>>> + size_t offs_begin = source.get_offs ();
>>>> + switch (match->get_macro_match_type ())
>>>> + {
>>>> + case AST::MacroMatch::MacroMatchType::Fragment: {
>>>> + AST::MacroMatchFragment *fragment
>>>> + = static_cast<AST::MacroMatchFragment *> (match.get ());
>>>> + valid_current_match = match_fragment (parser, *fragment);
>>>> +
>>>> + // matched fragment get the offset in the token stream
>>>> + size_t offs_end = source.get_offs ();
>>>> +
>>>> + // The main difference with match_matcher happens here: Instead
>>>> + // of inserting a new fragment, we append to one. If that
>>>> + // fragment does not exist, then the operation is similar to
>>>> + // `insert_fragment` with the difference that we are not
>>>> + // creating a metavariable, but a repetition of one, which is
>>>> + // really different.
>>>> + sub_stack.append_fragment (
>>>> + MatchedFragment (fragment->get_ident (), offs_begin,
>>>> + offs_end));
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Tok: {
>>>> + AST::Token *tok = static_cast<AST::Token *> (match.get ());
>>>> + valid_current_match = match_token (parser, *tok);
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Repetition: {
>>>> + AST::MacroMatchRepetition *rep
>>>> + = static_cast<AST::MacroMatchRepetition *> (match.get ());
>>>> + valid_current_match = match_repetition (parser, *rep);
>>>> + }
>>>> + break;
>>>> +
>>>> + case AST::MacroMatch::MacroMatchType::Matcher: {
>>>> + AST::MacroMatcher *m
>>>> + = static_cast<AST::MacroMatcher *> (match.get ());
>>>> + valid_current_match = match_matcher (parser, *m);
>>>> + }
>>>> + break;
>>>> + }
>>>> + }
>>>> + // If we've encountered an error once, stop trying to match more
>>>> + // repetitions
>>>> + if (!valid_current_match)
>>>> + break;
>>>> +
>>>> + match_amount++;
>>>> +
>>>> + // Break early if we notice there's too many expressions already
>>>> + if (hi_bound && match_amount > hi_bound)
>>>> + break;
>>>> + }
>>>> +
>>>> + // Check if the amount of matches we got is valid: Is it more than the lower
>>>> + // bound and less than the higher bound?
>>>> + bool did_meet_lo_bound = match_amount >= lo_bound;
>>>> + bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true;
>>>> +
>>>> + // If the end-result is valid, then we can clear the parse errors: Since
>>>> + // repetitions are parsed eagerly, it is okay to fail in some cases
>>>> + auto res = did_meet_lo_bound && did_meet_hi_bound;
>>>> + if (res)
>>>> + parser.clear_errors ();
>>>> +
>>>> + return res;
>>>> +}
>>>> +
>>>> +bool
>>>> +MacroExpander::match_repetition (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchRepetition &rep)
>>>> +{
>>>> + size_t match_amount = 0;
>>>> + bool res = false;
>>>> +
>>>> + std::string lo_str;
>>>> + std::string hi_str;
>>>> + switch (rep.get_op ())
>>>> + {
>>>> + case AST::MacroMatchRepetition::MacroRepOp::ANY:
>>>> + lo_str = "0";
>>>> + hi_str = "+inf";
>>>> + res = match_n_matches (parser, rep, match_amount);
>>>> + break;
>>>> + case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE:
>>>> + lo_str = "1";
>>>> + hi_str = "+inf";
>>>> + res = match_n_matches (parser, rep, match_amount, 1);
>>>> + break;
>>>> + case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE:
>>>> + lo_str = "0";
>>>> + hi_str = "1";
>>>> + res = match_n_matches (parser, rep, match_amount, 0, 1);
>>>> + break;
>>>> + default:
>>>> + gcc_unreachable ();
>>>> + }
>>>> +
>>>> + if (!res)
>>>> + rust_error_at (rep.get_match_locus (),
>>>> + "invalid amount of matches for macro invocation. Expected "
>>>> + "between %s and %s, got %lu",
>>>> + lo_str.c_str (), hi_str.c_str (),
>>>> + (unsigned long) match_amount);
>>>> +
>>>> + rust_debug_loc (rep.get_match_locus (), "%s matched %lu times",
>>>> + res ? "successfully" : "unsuccessfully",
>>>> + (unsigned long) match_amount);
>>>> +
>>>> + // We have to handle zero fragments differently: They will not have been
>>>> + // "matched" but they are still valid and should be inserted as a special
>>>> + // case. So we go through the stack map, and for every fragment which doesn't
>>>> + // exist, insert a zero-matched fragment.
>>>> + auto &stack_map = sub_stack.peek ();
>>>> + for (auto &match : rep.get_matches ())
>>>> + {
>>>> + if (match->get_macro_match_type ()
>>>> + == AST::MacroMatch::MacroMatchType::Fragment)
>>>> + {
>>>> + auto fragment = static_cast<AST::MacroMatchFragment *> (match.get ());
>>>> + auto it = stack_map.find (fragment->get_ident ());
>>>> +
>>>> + if (it == stack_map.end ())
>>>> + sub_stack.insert_matches (fragment->get_ident (),
>>>> + MatchedFragmentContainer::zero ());
>>>> + }
>>>> + }
>>>> +
>>>> + return res;
>>>> +}
>>>> +
>>>> +/**
>>>> + * Helper function to refactor calling a parsing function 0 or more times
>>>> + */
>>>> +static AST::ASTFragment
>>>> +parse_many (Parser<MacroInvocLexer> &parser, TokenId &delimiter,
>>>> + std::function<AST::SingleASTNode ()> parse_fn)
>>>> +{
>>>> + std::vector<AST::SingleASTNode> nodes;
>>>> + while (true)
>>>> + {
>>>> + if (parser.peek_current_token ()->get_id () == delimiter)
>>>> + break;
>>>> +
>>>> + auto node = parse_fn ();
>>>> + nodes.emplace_back (std::move (node));
>>>> + }
>>>> +
>>>> + return AST::ASTFragment (std::move (nodes));
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more items from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract items from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>>>> +{
>>>> + return parse_many (parser, delimiter, [&parser] () {
>>>> + auto item = parser.parse_item (true);
>>>> + return AST::SingleASTNode (std::move (item));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more external items from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract items from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_ext (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>>>> +{
>>>> + return parse_many (parser, delimiter, [&parser] () {
>>>> + auto item = parser.parse_external_item ();
>>>> + return AST::SingleASTNode (std::move (item));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more trait items from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract items from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_trait_items (Parser<MacroInvocLexer> &parser,
>>>> + TokenId &delimiter)
>>>> +{
>>>> + return parse_many (parser, delimiter, [&parser] () {
>>>> + auto item = parser.parse_trait_item ();
>>>> + return AST::SingleASTNode (std::move (item));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more impl items from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract items from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_impl_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>>>> +{
>>>> + return parse_many (parser, delimiter, [&parser] () {
>>>> + auto item = parser.parse_inherent_impl_item ();
>>>> + return AST::SingleASTNode (std::move (item));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more trait impl items from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract items from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_trait_impl_items (Parser<MacroInvocLexer> &parser,
>>>> + TokenId &delimiter)
>>>> +{
>>>> + return parse_many (parser, delimiter, [&parser] () {
>>>> + auto item = parser.parse_trait_impl_item ();
>>>> + return AST::SingleASTNode (std::move (item));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe 0 or more statements from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract statements from
>>>> + * @param delimiter Id of the token on which parsing should stop
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_many_stmts (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
>>>> +{
>>>> + auto restrictions = ParseRestrictions ();
>>>> + restrictions.consume_semi = false;
>>>> +
>>>> + // FIXME: This is invalid! It needs to also handle cases where the macro
>>>> + // transcriber is an expression, but since the macro call is followed by
>>>> + // a semicolon, it's a valid ExprStmt
>>>> + return parse_many (parser, delimiter, [&parser, restrictions] () {
>>>> + auto stmt = parser.parse_stmt (restrictions);
>>>> + return AST::SingleASTNode (std::move (stmt));
>>>> + });
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe one expression from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract statements from
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_expression (Parser<MacroInvocLexer> &parser)
>>>> +{
>>>> + auto expr = parser.parse_expr ();
>>>> +
>>>> + return AST::ASTFragment ({std::move (expr)});
>>>> +}
>>>> +
>>>> +/**
>>>> + * Transcribe one type from a macro invocation
>>>> + *
>>>> + * @param parser Parser to extract statements from
>>>> + */
>>>> +static AST::ASTFragment
>>>> +transcribe_type (Parser<MacroInvocLexer> &parser)
>>>> +{
>>>> + auto type = parser.parse_type ();
>>>> +
>>>> + return AST::ASTFragment ({std::move (type)});
>>>> +}
>>>> +
>>>> +static AST::ASTFragment
>>>> +transcribe_on_delimiter (Parser<MacroInvocLexer> &parser, bool semicolon,
>>>> + AST::DelimType delimiter, TokenId last_token_id)
>>>> +{
>>>> + if (semicolon || delimiter == AST::DelimType::CURLY)
>>>> + return transcribe_many_stmts (parser, last_token_id);
>>>> + else
>>>> + return transcribe_expression (parser);
>>>> +} // namespace Rust
>>>> +
>>>> +static AST::ASTFragment
>>>> +transcribe_context (MacroExpander::ContextType ctx,
>>>> + Parser<MacroInvocLexer> &parser, bool semicolon,
>>>> + AST::DelimType delimiter, TokenId last_token_id)
>>>> +{
>>>> + // The flow-chart in order to choose a parsing function is as follows:
>>>> + //
>>>> + // [switch special context]
>>>> + // -- Item --> parser.parse_item();
>>>> + // -- Trait --> parser.parse_trait_item();
>>>> + // -- Impl --> parser.parse_impl_item();
>>>> + // -- Extern --> parser.parse_extern_item();
>>>> + // -- None --> [has semicolon?]
>>>> + // -- Yes --> parser.parse_stmt();
>>>> + // -- No --> [switch invocation.delimiter()]
>>>> + // -- { } --> parser.parse_stmt();
>>>> + // -- _ --> parser.parse_expr(); // once!
>>>> +
>>>> + // If there is a semicolon OR we are expanding a MacroInvocationSemi, then
>>>> + // we can parse multiple items. Otherwise, parse *one* expression
>>>> +
>>>> + switch (ctx)
>>>> + {
>>>> + case MacroExpander::ContextType::ITEM:
>>>> + return transcribe_many_items (parser, last_token_id);
>>>> + break;
>>>> + case MacroExpander::ContextType::TRAIT:
>>>> + return transcribe_many_trait_items (parser, last_token_id);
>>>> + break;
>>>> + case MacroExpander::ContextType::IMPL:
>>>> + return transcribe_many_impl_items (parser, last_token_id);
>>>> + break;
>>>> + case MacroExpander::ContextType::TRAIT_IMPL:
>>>> + return transcribe_many_trait_impl_items (parser, last_token_id);
>>>> + break;
>>>> + case MacroExpander::ContextType::EXTERN:
>>>> + return transcribe_many_ext (parser, last_token_id);
>>>> + break;
>>>> + case MacroExpander::ContextType::TYPE:
>>>> + return transcribe_type (parser);
>>>> + break;
>>>> + default:
>>>> + return transcribe_on_delimiter (parser, semicolon, delimiter,
>>>> + last_token_id);
>>>> + }
>>>> +}
>>>> +
>>>> +static std::string
>>>> +tokens_to_str (std::vector<std::unique_ptr<AST::Token>> &tokens)
>>>> +{
>>>> + std::string str;
>>>> + if (!tokens.empty ())
>>>> + {
>>>> + str += tokens[0]->as_string ();
>>>> + for (size_t i = 1; i < tokens.size (); i++)
>>>> + str += " " + tokens[i]->as_string ();
>>>> + }
>>>> +
>>>> + return str;
>>>> +}
>>>> +
>>>> +AST::ASTFragment
>>>> +MacroExpander::transcribe_rule (
>>>> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
>>>> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
>>>> + bool semicolon, ContextType ctx)
>>>> +{
>>>> + // we can manipulate the token tree to substitute the dollar identifiers so
>>>> + // that when we call parse its already substituted for us
>>>> + AST::MacroTranscriber &transcriber = match_rule.get_transcriber ();
>>>> + AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree ();
>>>> +
>>>> + auto invoc_stream = invoc_token_tree.to_token_stream ();
>>>> + auto macro_rule_tokens = transcribe_tree.to_token_stream ();
>>>> +
>>>> + auto substitute_context
>>>> + = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments);
>>>> + std::vector<std::unique_ptr<AST::Token>> substituted_tokens
>>>> + = substitute_context.substitute_tokens ();
>>>> +
>>>> + rust_debug ("substituted tokens: %s",
>>>> + tokens_to_str (substituted_tokens).c_str ());
>>>> +
>>>> + // parse it to an ASTFragment
>>>> + MacroInvocLexer lex (std::move (substituted_tokens));
>>>> + Parser<MacroInvocLexer> parser (lex);
>>>> +
>>>> + auto last_token_id = TokenId::RIGHT_CURLY;
>>>> +
>>>> + // this is used so we can check that we delimit the stream correctly.
>>>> + switch (transcribe_tree.get_delim_type ())
>>>> + {
>>>> + case AST::DelimType::PARENS:
>>>> + last_token_id = TokenId::RIGHT_PAREN;
>>>> + rust_assert (parser.skip_token (LEFT_PAREN));
>>>> + break;
>>>> +
>>>> + case AST::DelimType::CURLY:
>>>> + rust_assert (parser.skip_token (LEFT_CURLY));
>>>> + break;
>>>> +
>>>> + case AST::DelimType::SQUARE:
>>>> + last_token_id = TokenId::RIGHT_SQUARE;
>>>> + rust_assert (parser.skip_token (LEFT_SQUARE));
>>>> + break;
>>>> + }
>>>> +
>>>> + // see https://github.com/Rust-GCC/gccrs/issues/22
>>>> + // TL;DR:
>>>> + // - Treat all macro invocations with parentheses, (), or square brackets,
>>>> + // [], as expressions.
>>>> + // - If the macro invocation has curly brackets, {}, it may be parsed as a
>>>> + // statement depending on the context.
>>>> + // - If the macro invocation has a semicolon at the end, it must be parsed
>>>> + // as a statement (either via ExpressionStatement or
>>>> + // MacroInvocationWithSemi)
>>>> +
>>>> + auto fragment
>>>> + = transcribe_context (ctx, parser, semicolon,
>>>> + invoc_token_tree.get_delim_type (), last_token_id);
>>>> +
>>>> + // emit any errors
>>>> + if (parser.has_errors ())
>>>> + {
>>>> + for (auto &err : parser.get_errors ())
>>>> + rust_error_at (err.locus, "%s", err.message.c_str ());
>>>> + return AST::ASTFragment::create_error ();
>>>> + }
>>>> +
>>>> + // are all the tokens used?
>>>> + bool did_delimit = parser.skip_token (last_token_id);
>>>> +
>>>> + bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE);
>>>> + if (!reached_end_of_stream)
>>>> + {
>>>> + const_TokenPtr current_token = parser.peek_current_token ();
>>>> + rust_error_at (current_token->get_locus (),
>>>> + "tokens here and after are unparsed");
>>>> + }
>>>> +
>>>> + return fragment;
>>>> +}
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-macro-expand.h b/gcc/rust/expand/rust-macro-expand.h
>>>> new file mode 100644
>>>> index 00000000000..94d6702ecb8
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-expand.h
>>>> @@ -0,0 +1,366 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#ifndef RUST_MACRO_EXPAND_H
>>>> +#define RUST_MACRO_EXPAND_H
>>>> +
>>>> +#include "rust-buffered-queue.h"
>>>> +#include "rust-parse.h"
>>>> +#include "rust-token.h"
>>>> +#include "rust-ast.h"
>>>> +#include "rust-macro.h"
>>>> +#include "rust-hir-map.h"
>>>> +#include "rust-name-resolver.h"
>>>> +#include "rust-macro-invoc-lexer.h"
>>>> +
>>>> +// Provides objects and method prototypes for macro expansion
>>>> +
>>>> +namespace Rust {
>>>> +// forward decls for AST
>>>> +namespace AST {
>>>> +class MacroInvocation;
>>>> +}
>>>> +
>>>> +// Object used to store configuration data for macro expansion.
>>>> +// NOTE: Keep all these items complying with the latest rustc.
>>>> +struct ExpansionCfg
>>>> +{
>>>> + // features?
>>>> + // TODO: Add `features' when we have it.
>>>> + unsigned int recursion_limit = 1024;
>>>> + bool trace_mac = false; // trace macro
>>>> + bool should_test = false; // strip #[test] nodes if false
>>>> + bool keep_macs = false; // keep macro definitions
>>>> + std::string crate_name = "";
>>>> +};
>>>> +
>>>> +struct MatchedFragment
>>>> +{
>>>> + std::string fragment_ident;
>>>> + size_t token_offset_begin;
>>>> + size_t token_offset_end;
>>>> +
>>>> + MatchedFragment (std::string identifier, size_t token_offset_begin,
>>>> + size_t token_offset_end)
>>>> + : fragment_ident (identifier), token_offset_begin (token_offset_begin),
>>>> + token_offset_end (token_offset_end)
>>>> + {}
>>>> +
>>>> + /**
>>>> + * Empty constructor for uninitialized fragments
>>>> + */
>>>> + MatchedFragment () : MatchedFragment ("", 0, 0) {}
>>>> +
>>>> + std::string as_string () const
>>>> + {
>>>> + return fragment_ident + "=" + std::to_string (token_offset_begin) + ":"
>>>> + + std::to_string (token_offset_end);
>>>> + }
>>>> +};
>>>> +
>>>> +class MatchedFragmentContainer
>>>> +{
>>>> +public:
>>>> + // Does the container refer to a simple metavariable, different from a
>>>> + // repetition repeated once
>>>> + enum class Kind
>>>> + {
>>>> + MetaVar,
>>>> + Repetition,
>>>> + };
>>>> +
>>>> + MatchedFragmentContainer (std::vector<MatchedFragment> fragments,
>>>> + Kind kind = Kind::Repetition)
>>>> + : fragments (fragments), kind (kind)
>>>> + {}
>>>> +
>>>> + /**
>>>> + * Create a valid fragment matched zero times. This is useful for repetitions
>>>> + * which allow the absence of a fragment, such as * and ?
>>>> + */
>>>> + static MatchedFragmentContainer zero ()
>>>> + {
>>>> + return MatchedFragmentContainer ({});
>>>> + }
>>>> +
>>>> + /**
>>>> + * Create a valid fragment matched one time
>>>> + */
>>>> + static MatchedFragmentContainer metavar (MatchedFragment fragment)
>>>> + {
>>>> + return MatchedFragmentContainer ({fragment}, Kind::MetaVar);
>>>> + }
>>>> +
>>>> + /**
>>>> + * Add a matched fragment to the container
>>>> + */
>>>> + void add_fragment (MatchedFragment fragment)
>>>> + {
>>>> + rust_assert (!is_single_fragment ());
>>>> +
>>>> + fragments.emplace_back (fragment);
>>>> + }
>>>> +
>>>> + size_t get_match_amount () const { return fragments.size (); }
>>>> + const std::vector<MatchedFragment> &get_fragments () const
>>>> + {
>>>> + return fragments;
>>>> + }
>>>> + // const std::string &get_fragment_name () const { return fragment_name; }
>>>> +
>>>> + bool is_single_fragment () const
>>>> + {
>>>> + return get_match_amount () == 1 && kind == Kind::MetaVar;
>>>> + }
>>>> +
>>>> + const MatchedFragment get_single_fragment () const
>>>> + {
>>>> + rust_assert (is_single_fragment ());
>>>> +
>>>> + return fragments[0];
>>>> + }
>>>> +
>>>> + const Kind &get_kind () const { return kind; }
>>>> +
>>>> +private:
>>>> + /**
>>>> + * Fragments matched `match_amount` times. This can be an empty vector
>>>> + * in case having zero matches is allowed (i.e ? or * operators)
>>>> + */
>>>> + std::vector<MatchedFragment> fragments;
>>>> + Kind kind;
>>>> +};
>>>> +
>>>> +class SubstitutionScope
>>>> +{
>>>> +public:
>>>> + SubstitutionScope () : stack () {}
>>>> +
>>>> + void push () { stack.push_back ({}); }
>>>> +
>>>> + std::map<std::string, MatchedFragmentContainer> pop ()
>>>> + {
>>>> + auto top = stack.back ();
>>>> + stack.pop_back ();
>>>> + return top;
>>>> + }
>>>> +
>>>> + std::map<std::string, MatchedFragmentContainer> &peek ()
>>>> + {
>>>> + return stack.back ();
>>>> + }
>>>> +
>>>> + /**
>>>> + * Insert a new matched metavar into the current substitution map
>>>> + */
>>>> + void insert_metavar (MatchedFragment fragment)
>>>> + {
>>>> + auto ¤t_map = stack.back ();
>>>> + auto it = current_map.find (fragment.fragment_ident);
>>>> +
>>>> + if (it == current_map.end ())
>>>> + current_map.insert ({fragment.fragment_ident,
>>>> + MatchedFragmentContainer::metavar (fragment)});
>>>> + else
>>>> + gcc_unreachable ();
>>>> + }
>>>> +
>>>> + /**
>>>> + * Append a new matched fragment to a repetition into the current substitution
>>>> + * map
>>>> + */
>>>> + void append_fragment (MatchedFragment fragment)
>>>> + {
>>>> + auto ¤t_map = stack.back ();
>>>> + auto it = current_map.find (fragment.fragment_ident);
>>>> +
>>>> + if (it == current_map.end ())
>>>> + current_map.insert (
>>>> + {fragment.fragment_ident, MatchedFragmentContainer ({fragment})});
>>>> + else
>>>> + it->second.add_fragment (fragment);
>>>> + }
>>>> +
>>>> + void insert_matches (std::string key, MatchedFragmentContainer matches)
>>>> + {
>>>> + auto ¤t_map = stack.back ();
>>>> + auto it = current_map.find (key);
>>>> + rust_assert (it == current_map.end ());
>>>> +
>>>> + current_map.insert ({key, matches});
>>>> + }
>>>> +
>>>> +private:
>>>> + std::vector<std::map<std::string, MatchedFragmentContainer>> stack;
>>>> +};
>>>> +
>>>> +// Object used to store shared data (between functions) for macro expansion.
>>>> +struct MacroExpander
>>>> +{
>>>> + enum ContextType
>>>> + {
>>>> + ITEM,
>>>> + BLOCK,
>>>> + EXTERN,
>>>> + TYPE,
>>>> + TRAIT,
>>>> + IMPL,
>>>> + TRAIT_IMPL,
>>>> + };
>>>> +
>>>> + ExpansionCfg cfg;
>>>> + unsigned int expansion_depth = 0;
>>>> +
>>>> + MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session)
>>>> + : cfg (cfg), crate (crate), session (session),
>>>> + sub_stack (SubstitutionScope ()),
>>>> + expanded_fragment (AST::ASTFragment::create_error ()),
>>>> + resolver (Resolver::Resolver::get ()),
>>>> + mappings (Analysis::Mappings::get ())
>>>> + {}
>>>> +
>>>> + ~MacroExpander () = default;
>>>> +
>>>> + // Expands all macros in the crate passed in.
>>>> + void expand_crate ();
>>>> +
>>>> + /* Expands a macro invocation - possibly make both
>>>> + * have similar duck-typed interface and use templates?*/
>>>> + // should this be public or private?
>>>> + void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon);
>>>> +
>>>> + // Expands a single declarative macro.
>>>> + AST::ASTFragment expand_decl_macro (Location locus,
>>>> + AST::MacroInvocData &invoc,
>>>> + AST::MacroRulesDefinition &rules_def,
>>>> + bool semicolon);
>>>> +
>>>> + void expand_cfg_attrs (AST::AttrVec &attrs);
>>>> + bool fails_cfg (const AST::AttrVec &attr) const;
>>>> + bool fails_cfg_with_expand (AST::AttrVec &attrs) const;
>>>> +
>>>> + bool depth_exceeds_recursion_limit () const;
>>>> +
>>>> + bool try_match_rule (AST::MacroRule &match_rule,
>>>> + AST::DelimTokenTree &invoc_token_tree);
>>>> +
>>>> + AST::ASTFragment transcribe_rule (
>>>> + AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
>>>> + std::map<std::string, MatchedFragmentContainer> &matched_fragments,
>>>> + bool semicolon, ContextType ctx);
>>>> +
>>>> + bool match_fragment (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchFragment &fragment);
>>>> +
>>>> + bool match_token (Parser<MacroInvocLexer> &parser, AST::Token &token);
>>>> +
>>>> + bool match_repetition (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchRepetition &rep);
>>>> +
>>>> + bool match_matcher (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatcher &matcher);
>>>> +
>>>> + /**
>>>> + * Match any amount of matches
>>>> + *
>>>> + * @param parser Parser to use for matching
>>>> + * @param rep Repetition to try and match
>>>> + * @param match_amount Reference in which to store the ammount of succesful
>>>> + * and valid matches
>>>> + *
>>>> + * @param lo_bound Lower bound of the matcher. When specified, the matcher
>>>> + * will only succeed if it parses at *least* `lo_bound` fragments. If
>>>> + * unspecified, the matcher could succeed when parsing 0 fragments.
>>>> + *
>>>> + * @param hi_bound Higher bound of the matcher. When specified, the matcher
>>>> + * will only succeed if it parses *less than* `hi_bound` fragments. If
>>>> + * unspecified, the matcher could succeed when parsing an infinity of
>>>> + * fragments.
>>>> + *
>>>> + * @return true if matching was successful and within the given limits, false
>>>> + * otherwise
>>>> + */
>>>> + bool match_n_matches (Parser<MacroInvocLexer> &parser,
>>>> + AST::MacroMatchRepetition &rep, size_t &match_amount,
>>>> + size_t lo_bound = 0, size_t hi_bound = 0);
>>>> +
>>>> + void push_context (ContextType t) { context.push_back (t); }
>>>> +
>>>> + ContextType pop_context ()
>>>> + {
>>>> + rust_assert (!context.empty ());
>>>> +
>>>> + ContextType t = context.back ();
>>>> + context.pop_back ();
>>>> +
>>>> + return t;
>>>> + }
>>>> +
>>>> + ContextType peek_context () { return context.back (); }
>>>> +
>>>> + void set_expanded_fragment (AST::ASTFragment &&fragment)
>>>> + {
>>>> + expanded_fragment = std::move (fragment);
>>>> + }
>>>> +
>>>> + AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis)
>>>> + {
>>>> + AST::ASTFragment old_fragment = std::move (expanded_fragment);
>>>> + auto accumulator = std::vector<AST::SingleASTNode> ();
>>>> + expanded_fragment = AST::ASTFragment::create_error ();
>>>> +
>>>> + for (auto &node : old_fragment.get_nodes ())
>>>> + {
>>>> + expansion_depth++;
>>>> + node.accept_vis (vis);
>>>> + // we'll decide the next move according to the outcome of the macro
>>>> + // expansion
>>>> + if (expanded_fragment.is_error ())
>>>> + accumulator.push_back (node); // if expansion fails, there might be a
>>>> + // non-macro expression we need to keep
>>>> + else
>>>> + {
>>>> + // if expansion succeeded, then we need to merge the fragment with
>>>> + // the contents in the accumulator, so that our final expansion
>>>> + // result will contain non-macro nodes as it should
>>>> + auto new_nodes = expanded_fragment.get_nodes ();
>>>> + std::move (new_nodes.begin (), new_nodes.end (),
>>>> + std::back_inserter (accumulator));
>>>> + expanded_fragment = AST::ASTFragment (accumulator);
>>>> + }
>>>> + expansion_depth--;
>>>> + }
>>>> +
>>>> + return old_fragment;
>>>> + }
>>>> +
>>>> +private:
>>>> + AST::Crate &crate;
>>>> + Session &session;
>>>> + SubstitutionScope sub_stack;
>>>> + std::vector<ContextType> context;
>>>> + AST::ASTFragment expanded_fragment;
>>>> +
>>>> +public:
>>>> + Resolver::Resolver *resolver;
>>>> + Analysis::Mappings *mappings;
>>>> +};
>>>> +
>>>> +} // namespace Rust
>>>> +
>>>> +#endif
>>>> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.cc b/gcc/rust/expand/rust-macro-invoc-lexer.cc
>>>> new file mode 100644
>>>> index 00000000000..8a43d29e0d1
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.cc
>>>> @@ -0,0 +1,29 @@
>>>> +#include "rust-macro-invoc-lexer.h"
>>>> +
>>>> +namespace Rust {
>>>> +
>>>> +const_TokenPtr
>>>> +MacroInvocLexer::peek_token (int n)
>>>> +{
>>>> + if ((offs + n) >= token_stream.size ())
>>>> + return Token::make (END_OF_FILE, Location ());
>>>> +
>>>> + return token_stream.at (offs + n)->get_tok_ptr ();
>>>> +}
>>>> +
>>>> +// Advances current token to n + 1 tokens ahead of current position.
>>>> +void
>>>> +MacroInvocLexer::skip_token (int n)
>>>> +{
>>>> + offs += (n + 1);
>>>> +}
>>>> +
>>>> +void
>>>> +MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)),
>>>> + TokenId new_right
>>>> + __attribute__ ((unused)))
>>>> +{
>>>> + // FIXME
>>>> + gcc_unreachable ();
>>>> +}
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-macro-invoc-lexer.h b/gcc/rust/expand/rust-macro-invoc-lexer.h
>>>> new file mode 100644
>>>> index 00000000000..0fd4554d02f
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-invoc-lexer.h
>>>> @@ -0,0 +1,64 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#ifndef RUST_MACRO_INVOC_LEXER_H
>>>> +#define RUST_MACRO_INVOC_LEXER_H
>>>> +
>>>> +#include "rust-ast.h"
>>>> +
>>>> +namespace Rust {
>>>> +class MacroInvocLexer
>>>> +{
>>>> +public:
>>>> + MacroInvocLexer (std::vector<std::unique_ptr<AST::Token>> stream)
>>>> + : offs (0), token_stream (std::move (stream))
>>>> + {}
>>>> +
>>>> + // Returns token n tokens ahead of current position.
>>>> + const_TokenPtr peek_token (int n);
>>>> +
>>>> + // Peeks the current token.
>>>> + const_TokenPtr peek_token () { return peek_token (0); }
>>>> +
>>>> + // Advances current token to n + 1 tokens ahead of current position.
>>>> + void skip_token (int n);
>>>> +
>>>> + // Skips the current token.
>>>> + void skip_token () { skip_token (0); }
>>>> +
>>>> + // Splits the current token into two. Intended for use with nested generics
>>>> + // closes (i.e. T<U<X>> where >> is wrongly lexed as one token). Note that
>>>> + // this will only work with "simple" tokens like punctuation.
>>>> + void split_current_token (TokenId new_left, TokenId new_right);
>>>> +
>>>> + std::string get_filename () const
>>>> + {
>>>> + // FIXME
>>>> + gcc_unreachable ();
>>>> + return "FIXME";
>>>> + }
>>>> +
>>>> + size_t get_offs () const { return offs; }
>>>> +
>>>> +private:
>>>> + size_t offs;
>>>> + std::vector<std::unique_ptr<AST::Token>> token_stream;
>>>> +};
>>>> +} // namespace Rust
>>>> +
>>>> +#endif // RUST_MACRO_INVOC_LEXER_H
>>>> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
>>>> new file mode 100644
>>>> index 00000000000..9592d2d2a9e
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
>>>> @@ -0,0 +1,312 @@
>>>> +#include "rust-macro-substitute-ctx.h"
>>>> +
>>>> +namespace Rust {
>>>> +
>>>> +std::vector<std::unique_ptr<AST::Token>>
>>>> +SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
>>>> +{
>>>> + auto metavar_name = metavar->get_str ();
>>>> +
>>>> + std::vector<std::unique_ptr<AST::Token>> expanded;
>>>> + auto it = fragments.find (metavar_name);
>>>> + if (it == fragments.end ())
>>>> + {
>>>> + // Return a copy of the original token
>>>> + expanded.push_back (metavar->clone_token ());
>>>> + }
>>>> + else
>>>> + {
>>>> + // If we are expanding a metavar which has a lof of matches, we are
>>>> + // currently expanding a repetition metavar - not a simple metavar. We
>>>> + // need to error out and inform the user.
>>>> + // Associated test case for an example: compile/macro-issue1224.rs
>>>> + if (it->second.get_match_amount () != 1)
>>>> + {
>>>> + rust_error_at (metavar->get_locus (),
>>>> + "metavariable is still repeating at this depth");
>>>> + rust_inform (
>>>> + metavar->get_locus (),
>>>> + "you probably forgot the repetition operator: %<%s%s%s%>", "$(",
>>>> + metavar->as_string ().c_str (), ")*");
>>>> + return expanded;
>>>> + }
>>>> +
>>>> + // We only care about the vector when expanding repetitions.
>>>> + // Just access the first element of the vector.
>>>> + auto &frag = it->second.get_single_fragment ();
>>>> + for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
>>>> + offs++)
>>>> + {
>>>> + auto &tok = input.at (offs);
>>>> + expanded.push_back (tok->clone_token ());
>>>> + }
>>>> + }
>>>> +
>>>> + return expanded;
>>>> +}
>>>> +
>>>> +bool
>>>> +SubstituteCtx::check_repetition_amount (size_t pattern_start,
>>>> + size_t pattern_end,
>>>> + size_t &expected_repetition_amount)
>>>> +{
>>>> + bool first_fragment_found = false;
>>>> + bool is_valid = true;
>>>> +
>>>> + for (size_t i = pattern_start; i < pattern_end; i++)
>>>> + {
>>>> + if (macro.at (i)->get_id () == DOLLAR_SIGN)
>>>> + {
>>>> + auto &frag_token = macro.at (i + 1);
>>>> + if (frag_token->get_id () == IDENTIFIER)
>>>> + {
>>>> + auto it = fragments.find (frag_token->get_str ());
>>>> + if (it == fragments.end ())
>>>> + {
>>>> + // If the repetition is not anything we know (ie no declared
>>>> + // metavars, or metavars which aren't present in the
>>>> + // fragment), we can just error out. No need to paste the
>>>> + // tokens as if nothing had happened.
>>>> + rust_error_at (frag_token->get_locus (),
>>>> + "metavar %s used in repetition does not exist",
>>>> + frag_token->get_str ().c_str ());
>>>> +
>>>> + is_valid = false;
>>>> + }
>>>> +
>>>> + auto &fragment = it->second;
>>>> +
>>>> + size_t repeat_amount = fragment.get_match_amount ();
>>>> + if (!first_fragment_found)
>>>> + {
>>>> + first_fragment_found = true;
>>>> + expected_repetition_amount = repeat_amount;
>>>> + }
>>>> + else
>>>> + {
>>>> + if (repeat_amount != expected_repetition_amount
>>>> + && !fragment.is_single_fragment ())
>>>> + {
>>>> + rust_error_at (
>>>> + frag_token->get_locus (),
>>>> + "different amount of matches used in merged "
>>>> + "repetitions: expected %lu, got %lu",
>>>> + (unsigned long) expected_repetition_amount,
>>>> + (unsigned long) repeat_amount);
>>>> + is_valid = false;
>>>> + }
>>>> + }
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + return is_valid;
>>>> +}
>>>> +
>>>> +std::vector<std::unique_ptr<AST::Token>>
>>>> +SubstituteCtx::substitute_repetition (
>>>> + size_t pattern_start, size_t pattern_end,
>>>> + std::unique_ptr<AST::Token> separator_token)
>>>> +{
>>>> + rust_assert (pattern_end < macro.size ());
>>>> +
>>>> + size_t repeat_amount = 0;
>>>> + if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount))
>>>> + return {};
>>>> +
>>>> + rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount);
>>>> + std::vector<std::unique_ptr<AST::Token>> expanded;
>>>> + std::vector<std::unique_ptr<AST::Token>> new_macro;
>>>> +
>>>> + // We want to generate a "new macro" to substitute with. This new macro
>>>> + // should contain only the tokens inside the pattern
>>>> + for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++)
>>>> + new_macro.emplace_back (macro.at (tok_idx)->clone_token ());
>>>> +
>>>> + // Then, we want to create a subset of the matches so that
>>>> + // `substitute_tokens()` can only see one fragment per metavar. Let's say we
>>>> + // have the following user input: (1 145 'h')
>>>> + // on the following match arm: ($($lit:literal)*)
>>>> + // which causes the following matches: { "lit": [1, 145, 'h'] }
>>>> + //
>>>> + // The pattern (new_macro) is `$lit:literal`
>>>> + // The first time we expand it, we want $lit to have the following token: 1
>>>> + // The second time, 145
>>>> + // The third and final time, 'h'
>>>> + //
>>>> + // In order to do so we must create "sub maps", which only contain parts of
>>>> + // the original matches
>>>> + // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ]
>>>> + //
>>>> + // and give them to `substitute_tokens` one by one.
>>>> +
>>>> + for (size_t i = 0; i < repeat_amount; i++)
>>>> + {
>>>> + std::map<std::string, MatchedFragmentContainer> sub_map;
>>>> + for (auto &kv_match : fragments)
>>>> + {
>>>> + MatchedFragment sub_fragment;
>>>> +
>>>> + // FIXME: Hack: If a fragment is not repeated, how does it fit in the
>>>> + // submap? Do we really want to expand it? Is this normal behavior?
>>>> + if (kv_match.second.is_single_fragment ())
>>>> + sub_fragment = kv_match.second.get_single_fragment ();
>>>> + else
>>>> + sub_fragment = kv_match.second.get_fragments ()[i];
>>>> +
>>>> + sub_map.insert (
>>>> + {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)});
>>>> + }
>>>> +
>>>> + auto substitute_context = SubstituteCtx (input, new_macro, sub_map);
>>>> + auto new_tokens = substitute_context.substitute_tokens ();
>>>> +
>>>> + // Skip the first repetition, but add the separator to the expanded
>>>> + // tokens if it is present
>>>> + if (i != 0 && separator_token)
>>>> + expanded.emplace_back (separator_token->clone_token ());
>>>> +
>>>> + for (auto &new_token : new_tokens)
>>>> + expanded.emplace_back (new_token->clone_token ());
>>>> + }
>>>> +
>>>> + // FIXME: We also need to make sure that all subsequent fragments
>>>> + // contain the same amount of repetitions as the first one
>>>> +
>>>> + return expanded;
>>>> +}
>>>> +
>>>> +static bool
>>>> +is_rep_op (std::unique_ptr<AST::Token> &tok)
>>>> +{
>>>> + auto id = tok->get_id ();
>>>> + return id == QUESTION_MARK || id == ASTERISK || id == PLUS;
>>>> +}
>>>> +
>>>> +std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
>>>> +SubstituteCtx::substitute_token (size_t token_idx)
>>>> +{
>>>> + auto &token = macro.at (token_idx);
>>>> + switch (token->get_id ())
>>>> + {
>>>> + case IDENTIFIER:
>>>> + rust_debug ("expanding metavar: %s", token->get_str ().c_str ());
>>>> + return {substitute_metavar (token), 1};
>>>> + case LEFT_PAREN: {
>>>> + // We need to parse up until the closing delimiter and expand this
>>>> + // fragment->n times.
>>>> + rust_debug ("expanding repetition");
>>>> +
>>>> + // We're in a context where macro repetitions have already been
>>>> + // parsed and validated: This means that
>>>> + // 1/ There will be no delimiters as that is an error
>>>> + // 2/ There are no fragment specifiers anymore, which prevents us
>>>> + // from reusing parser functions.
>>>> + //
>>>> + // Repetition patterns are also special in that they cannot contain
>>>> + // "rogue" delimiters: For example, this is invalid, as they are
>>>> + // parsed as MacroMatches and must contain a correct amount of
>>>> + // delimiters.
>>>> + // `$($e:expr ) )`
>>>> + // ^ rogue closing parenthesis
>>>> + //
>>>> + // With all of that in mind, we can simply skip ahead from one
>>>> + // parenthesis to the other to find the pattern to expand. Of course,
>>>> + // pairs of delimiters, including parentheses, are allowed.
>>>> + // `$($e:expr ( ) )`
>>>> + // Parentheses are the sole delimiter for which we need a special
>>>> + // behavior since they delimit the repetition pattern
>>>> +
>>>> + size_t pattern_start = token_idx + 1;
>>>> + size_t pattern_end = pattern_start;
>>>> + auto parentheses_stack = 0;
>>>> + for (size_t idx = pattern_start; idx < macro.size (); idx++)
>>>> + {
>>>> + if (macro.at (idx)->get_id () == LEFT_PAREN)
>>>> + {
>>>> + parentheses_stack++;
>>>> + }
>>>> + else if (macro.at (idx)->get_id () == RIGHT_PAREN)
>>>> + {
>>>> + if (parentheses_stack == 0)
>>>> + {
>>>> + pattern_end = idx;
>>>> + break;
>>>> + }
>>>> + parentheses_stack--;
>>>> + }
>>>> + }
>>>> +
>>>> + // Unreachable case, but let's make sure we don't ever run into it
>>>> + rust_assert (pattern_end != pattern_start);
>>>> +
>>>> + std::unique_ptr<AST::Token> separator_token = nullptr;
>>>> + if (pattern_end + 1 <= macro.size ())
>>>> + {
>>>> + auto &post_pattern_token = macro.at (pattern_end + 1);
>>>> + if (!is_rep_op (post_pattern_token))
>>>> + separator_token = post_pattern_token->clone_token ();
>>>> + }
>>>> +
>>>> + // Amount of tokens to skip
>>>> + auto to_skip = 0;
>>>> + // Parentheses
>>>> + to_skip += 2;
>>>> + // Repetition operator
>>>> + to_skip += 1;
>>>> + // Separator
>>>> + if (separator_token)
>>>> + to_skip += 1;
>>>> +
>>>> + return {substitute_repetition (pattern_start, pattern_end,
>>>> + std::move (separator_token)),
>>>> + pattern_end - pattern_start + to_skip};
>>>> + }
>>>> + // TODO: We need to check if the $ was alone. In that case, do
>>>> + // not error out: Simply act as if there was an empty identifier
>>>> + // with no associated fragment and paste the dollar sign in the
>>>> + // transcription. Unsure how to do that since we always have at
>>>> + // least the closing curly brace after an empty $...
>>>> + default:
>>>> + rust_error_at (token->get_locus (),
>>>> + "unexpected token in macro transcribe: expected "
>>>> + "%<(%> or identifier after %<$%>, got %<%s%>",
>>>> + get_token_description (token->get_id ()));
>>>> + }
>>>> +
>>>> + // FIXME: gcc_unreachable() error case?
>>>> + return {std::vector<std::unique_ptr<AST::Token>> (), 0};
>>>> +}
>>>> +
>>>> +std::vector<std::unique_ptr<AST::Token>>
>>>> +SubstituteCtx::substitute_tokens ()
>>>> +{
>>>> + std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
>>>> + rust_debug ("expanding tokens");
>>>> +
>>>> + for (size_t i = 0; i < macro.size (); i++)
>>>> + {
>>>> + auto &tok = macro.at (i);
>>>> + if (tok->get_id () == DOLLAR_SIGN)
>>>> + {
>>>> + // Aaaaah, if only we had C++17 :)
>>>> + // auto [expanded, tok_to_skip] = ...
>>>> + auto p = substitute_token (i + 1);
>>>> + auto expanded = std::move (p.first);
>>>> + auto tok_to_skip = p.second;
>>>> +
>>>> + i += tok_to_skip;
>>>> +
>>>> + for (auto &token : expanded)
>>>> + replaced_tokens.emplace_back (token->clone_token ());
>>>> + }
>>>> + else
>>>> + {
>>>> + replaced_tokens.emplace_back (tok->clone_token ());
>>>> + }
>>>> + }
>>>> +
>>>> + return replaced_tokens;
>>>> +}
>>>> +
>>>> +} // namespace Rust
>>>> diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.h b/gcc/rust/expand/rust-macro-substitute-ctx.h
>>>> new file mode 100644
>>>> index 00000000000..81dcab7643b
>>>> --- /dev/null
>>>> +++ b/gcc/rust/expand/rust-macro-substitute-ctx.h
>>>> @@ -0,0 +1,93 @@
>>>> +// Copyright (C) 2020-2022 Free Software Foundation, Inc.
>>>> +
>>>> +// This file is part of GCC.
>>>> +
>>>> +// GCC is free software; you can redistribute it and/or modify it under
>>>> +// the terms of the GNU General Public License as published by the Free
>>>> +// Software Foundation; either version 3, or (at your option) any later
>>>> +// version.
>>>> +
>>>> +// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
>>>> +// WARRANTY; without even the implied warranty of MERCHANTABILITY or
>>>> +// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
>>>> +// for more details.
>>>> +
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with GCC; see the file COPYING3. If not see
>>>> +// <http://www.gnu.org/licenses/>.
>>>> +
>>>> +#include "rust-ast.h"
>>>> +#include "rust-macro-expand.h"
>>>> +
>>>> +namespace Rust {
>>>> +class SubstituteCtx
>>>> +{
>>>> + std::vector<std::unique_ptr<AST::Token>> &input;
>>>> + std::vector<std::unique_ptr<AST::Token>> ¯o;
>>>> + std::map<std::string, MatchedFragmentContainer> &fragments;
>>>> +
>>>> + /**
>>>> + * Find the repetition amount to use when expanding a repetition, and
>>>> + * check that all fragments used respect that repetition amount
>>>> + *
>>>> + * @param pattern_start Start of the repetition pattern
>>>> + * @param pattern_end End of the repetition pattern
>>>> + * @param repeat_amount Reference to fill with the matched repetition amount
>>>> + */
>>>> + bool check_repetition_amount (size_t pattern_start, size_t pattern_end,
>>>> + size_t &repeat_amount);
>>>> +
>>>> +public:
>>>> + SubstituteCtx (std::vector<std::unique_ptr<AST::Token>> &input,
>>>> + std::vector<std::unique_ptr<AST::Token>> ¯o,
>>>> + std::map<std::string, MatchedFragmentContainer> &fragments)
>>>> + : input (input), macro (macro), fragments (fragments)
>>>> + {}
>>>> +
>>>> + /**
>>>> + * Substitute a metavariable by its given fragment in a transcribing context,
>>>> + * i.e. replacing $var with the associated fragment.
>>>> + *
>>>> + * @param metavar Metavariable to try and replace
>>>> + *
>>>> + * @return A token containing the associated fragment expanded into tokens if
>>>> + * any, or the cloned token if no fragment was associated
>>>> + */
>>>> + std::vector<std::unique_ptr<AST::Token>>
>>>> + substitute_metavar (std::unique_ptr<AST::Token> &metavar);
>>>> +
>>>> + /**
>>>> + * Substitute a macro repetition by its given fragments
>>>> + *
>>>> + * @param pattern_start Start index of the pattern tokens
>>>> + * @param pattern_end End index of the patterns tokens
>>>> + * @param separator Optional separator to include when expanding tokens
>>>> + *
>>>> + * @return A vector containing the repeated pattern
>>>> + */
>>>> + std::vector<std::unique_ptr<AST::Token>>
>>>> + substitute_repetition (size_t pattern_start, size_t pattern_end,
>>>> + std::unique_ptr<AST::Token> separator);
>>>> +
>>>> + /**
>>>> + * Substitute a given token by its appropriate representation
>>>> + *
>>>> + * @param token_idx Current token to try and substitute
>>>> + *
>>>> + * @return A token containing the associated fragment expanded into tokens if
>>>> + * any, or the cloned token if no fragment was associated, as well as the
>>>> + * amount of tokens that should be skipped before the next invocation. Since
>>>> + * this function may consume more than just one token, it is important to skip
>>>> + * ahead of the input to avoid mis-substitutions
>>>> + */
>>>> + std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
>>>> + substitute_token (size_t token_idx);
>>>> +
>>>> + /**
>>>> + * Substitute all tokens by their appropriate representation
>>>> + *
>>>> + * @return A vector containing the substituted tokens
>>>> + */
>>>> + std::vector<std::unique_ptr<AST::Token>> substitute_tokens ();
>>>> +};
>>>> +} // namespace Rust
>>>> --
>>>> 2.37.2
>>>>
>>
>>
>> Kindly,
>>
>> Arthur
--
Arthur Cohen <arthur.cohen@embecosm.com>
Toolchain Engineer
Embecosm GmbH
Geschäftsführer: Jeremy Bennett
Niederlassung: Nürnberg
Handelsregister: HR-B 36368
www.embecosm.de
Fürther Str. 27
90429 Nürnberg
Tel.: 091 - 128 707 040
Fax: 091 - 128 707 077
new file mode 100644
@@ -0,0 +1,3445 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#include "rust-attribute-visitor.h"
+#include "rust-session-manager.h"
+
+namespace Rust {
+
+// Visitor used to expand attributes.
+void
+AttrVisitor::expand_struct_fields (std::vector<AST::StructField> &fields)
+{
+ for (auto it = fields.begin (); it != fields.end ();)
+ {
+ auto &field = *it;
+
+ auto &field_attrs = field.get_outer_attrs ();
+ expander.expand_cfg_attrs (field_attrs);
+ if (expander.fails_cfg_with_expand (field_attrs))
+ {
+ it = fields.erase (it);
+ continue;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // expand sub-types of type, but can't strip type itself
+ auto &type = field.get_field_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ // if nothing else happens, increment
+ ++it;
+ }
+}
+
+void
+AttrVisitor::expand_tuple_fields (std::vector<AST::TupleField> &fields)
+{
+ for (auto it = fields.begin (); it != fields.end ();)
+ {
+ auto &field = *it;
+
+ auto &field_attrs = field.get_outer_attrs ();
+ expander.expand_cfg_attrs (field_attrs);
+ if (expander.fails_cfg_with_expand (field_attrs))
+ {
+ it = fields.erase (it);
+ continue;
+ }
+
+ // expand sub-types of type, but can't strip type itself
+ auto &type = field.get_field_type ();
+ type->accept_vis (*this);
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ // if nothing else happens, increment
+ ++it;
+ }
+}
+
+void
+AttrVisitor::expand_function_params (std::vector<AST::FunctionParam> ¶ms)
+{
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ for (auto it = params.begin (); it != params.end ();)
+ {
+ auto ¶m = *it;
+
+ auto ¶m_attrs = param.get_outer_attrs ();
+ expander.expand_cfg_attrs (param_attrs);
+ if (expander.fails_cfg_with_expand (param_attrs))
+ {
+ it = params.erase (it);
+ continue;
+ }
+
+ // TODO: should an unwanted strip lead to break out of loop?
+ auto &pattern = param.get_pattern ();
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+
+ auto &type = param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ // increment
+ ++it;
+ }
+
+ expander.pop_context ();
+}
+
+void
+AttrVisitor::expand_generic_args (AST::GenericArgs &args)
+{
+ // lifetime args can't be expanded
+ // FIXME: Can we have macro invocations for lifetimes?
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // expand type args - strip sub-types only
+ for (auto &arg : args.get_generic_args ())
+ {
+ switch (arg.get_kind ())
+ {
+ case AST::GenericArg::Kind::Type: {
+ auto &type = arg.get_type ();
+ type->accept_vis (*this);
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+ break;
+ }
+ case AST::GenericArg::Kind::Const: {
+ auto &expr = arg.get_expression ();
+ expr->accept_vis (*this);
+ maybe_expand_expr (expr);
+
+ if (expr->is_marked_for_strip ())
+ rust_error_at (expr->get_locus (),
+ "cannot strip expression in this position");
+ break;
+ }
+ default:
+ break;
+ // FIXME: Figure out what to do here if there is ambiguity. Since the
+ // resolver comes after the expansion, we need to figure out a way to
+ // strip ambiguous values here
+ // TODO: Arthur: Probably add a `mark_as_strip` method to `GenericArg`
+ // or something. This would clean up this whole thing
+ }
+ }
+
+ expander.pop_context ();
+
+ // FIXME: Can we have macro invocations in generic type bindings?
+ // expand binding args - strip sub-types only
+ for (auto &binding : args.get_binding_args ())
+ {
+ auto &type = binding.get_type ();
+ type->accept_vis (*this);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+ }
+}
+
+void
+AttrVisitor::expand_qualified_path_type (AST::QualifiedPathType &path_type)
+{
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = path_type.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ expander.pop_context ();
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ if (path_type.has_as_clause ())
+ {
+ auto &type_path = path_type.get_as_type_path ();
+ visit (type_path);
+ if (type_path.is_marked_for_strip ())
+ rust_error_at (type_path.get_locus (),
+ "cannot strip type path in this position");
+ }
+}
+
+void
+AttrVisitor::AttrVisitor::expand_closure_params (
+ std::vector<AST::ClosureParam> ¶ms)
+{
+ for (auto it = params.begin (); it != params.end ();)
+ {
+ auto ¶m = *it;
+
+ auto ¶m_attrs = param.get_outer_attrs ();
+ expander.expand_cfg_attrs (param_attrs);
+ if (expander.fails_cfg_with_expand (param_attrs))
+ {
+ it = params.erase (it);
+ continue;
+ }
+
+ auto &pattern = param.get_pattern ();
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+
+ if (param.has_type_given ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+ auto &type = param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ // increment if found nothing else so far
+ ++it;
+ }
+}
+
+void
+AttrVisitor::expand_self_param (AST::SelfParam &self_param)
+{
+ if (self_param.has_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+ auto &type = self_param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+ /* TODO: maybe check for invariants being violated - e.g. both type and
+ * lifetime? */
+}
+
+void
+AttrVisitor::expand_where_clause (AST::WhereClause &where_clause)
+{
+ // items cannot be stripped conceptually, so just accept visitor
+ for (auto &item : where_clause.get_items ())
+ item->accept_vis (*this);
+}
+
+void
+AttrVisitor::expand_trait_function_decl (AST::TraitFunctionDecl &decl)
+{
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : decl.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* strip function parameters if required - this is specifically
+ * allowed by spec */
+ expand_function_params (decl.get_function_params ());
+
+ if (decl.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = decl.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ if (decl.has_where_clause ())
+ expand_where_clause (decl.get_where_clause ());
+}
+
+void
+AttrVisitor::expand_trait_method_decl (AST::TraitMethodDecl &decl)
+{
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : decl.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* assuming you can't strip self param - wouldn't be a method
+ * anymore. spec allows outer attrs on self param, but doesn't
+ * specify whether cfg is used. */
+ expand_self_param (decl.get_self_param ());
+
+ /* strip function parameters if required - this is specifically
+ * allowed by spec */
+ expand_function_params (decl.get_function_params ());
+
+ if (decl.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = decl.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ if (decl.has_where_clause ())
+ expand_where_clause (decl.get_where_clause ());
+}
+
+void
+AttrVisitor::visit (AST::Token &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::DelimTokenTree &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::AttrInputMetaItemContainer &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::IdentifierExpr &ident_expr)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (ident_expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (ident_expr.get_outer_attrs ()))
+ {
+ ident_expr.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::Lifetime &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::LifetimeParam &)
+{
+ // supposedly does not require - cfg does nothing
+}
+void
+AttrVisitor::visit (AST::ConstGenericParam &)
+{
+ // likewise
+}
+
+void
+AttrVisitor::visit (AST::MacroInvocation ¯o_invoc)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (macro_invoc.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (macro_invoc.get_outer_attrs ()))
+ {
+ macro_invoc.mark_for_strip ();
+ return;
+ }
+
+ // can't strip simple path
+
+ // I don't think any macro token trees can be stripped in any way
+
+ // TODO: maybe have cfg! macro stripping behaviour here?
+ expander.expand_invoc (macro_invoc, macro_invoc.has_semicolon ());
+}
+
+void
+AttrVisitor::visit (AST::PathInExpression &path)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (path.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
+ {
+ path.mark_for_strip ();
+ return;
+ }
+
+ for (auto &segment : path.get_segments ())
+ {
+ if (segment.has_generic_args ())
+ expand_generic_args (segment.get_generic_args ());
+ }
+}
+void
+AttrVisitor::visit (AST::TypePathSegment &)
+{
+ // shouldn't require
+}
+void
+AttrVisitor::visit (AST::TypePathSegmentGeneric &segment)
+{
+ // TODO: strip inside generic args
+
+ if (!segment.has_generic_args ())
+ return;
+
+ expand_generic_args (segment.get_generic_args ());
+}
+void
+AttrVisitor::visit (AST::TypePathSegmentFunction &segment)
+{
+ auto &type_path_function = segment.get_type_path_function ();
+
+ for (auto &type : type_path_function.get_params ())
+ {
+ type->accept_vis (*this);
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+ }
+
+ if (type_path_function.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = type_path_function.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+}
+void
+AttrVisitor::visit (AST::TypePath &path)
+{
+ // this shouldn't strip any segments, but can strip inside them
+ for (auto &segment : path.get_segments ())
+ segment->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::QualifiedPathInExpression &path)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (path.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (path.get_outer_attrs ()))
+ {
+ path.mark_for_strip ();
+ return;
+ }
+
+ expand_qualified_path_type (path.get_qualified_path_type ());
+
+ for (auto &segment : path.get_segments ())
+ {
+ if (segment.has_generic_args ())
+ expand_generic_args (segment.get_generic_args ());
+ }
+}
+void
+AttrVisitor::visit (AST::QualifiedPathInType &path)
+{
+ expand_qualified_path_type (path.get_qualified_path_type ());
+
+ // this shouldn't strip any segments, but can strip inside them
+ for (auto &segment : path.get_segments ())
+ segment->accept_vis (*this);
+}
+
+void
+AttrVisitor::visit (AST::LiteralExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::AttrInputLiteral &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::MetaItemLitExpr &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::MetaItemPathLit &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::BorrowExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &borrowed_expr = expr.get_borrowed_expr ();
+ borrowed_expr->accept_vis (*this);
+ if (borrowed_expr->is_marked_for_strip ())
+ rust_error_at (borrowed_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::DereferenceExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &dereferenced_expr = expr.get_dereferenced_expr ();
+ dereferenced_expr->accept_vis (*this);
+ if (dereferenced_expr->is_marked_for_strip ())
+ rust_error_at (dereferenced_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ErrorPropagationExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &propagating_expr = expr.get_propagating_expr ();
+ propagating_expr->accept_vis (*this);
+ if (propagating_expr->is_marked_for_strip ())
+ rust_error_at (propagating_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::NegationExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &negated_expr = expr.get_negated_expr ();
+ negated_expr->accept_vis (*this);
+ if (negated_expr->is_marked_for_strip ())
+ rust_error_at (negated_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ArithmeticOrLogicalExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &l_expr = expr.get_left_expr ();
+ l_expr->accept_vis (*this);
+ maybe_expand_expr (l_expr);
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &r_expr = expr.get_right_expr ();
+ r_expr->accept_vis (*this);
+ maybe_expand_expr (r_expr);
+
+ // ensure that they are not marked for strip
+ if (expr.get_left_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_left_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before binary op exprs");
+ if (expr.get_right_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_right_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ComparisonExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &l_expr = expr.get_left_expr ();
+ l_expr->accept_vis (*this);
+ maybe_expand_expr (l_expr);
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &r_expr = expr.get_right_expr ();
+ r_expr->accept_vis (*this);
+ maybe_expand_expr (r_expr);
+
+ // ensure that they are not marked for strip
+ if (expr.get_left_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_left_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before binary op exprs");
+ if (expr.get_right_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_right_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::LazyBooleanExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &l_expr = expr.get_left_expr ();
+ l_expr->accept_vis (*this);
+ maybe_expand_expr (l_expr);
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &r_expr = expr.get_right_expr ();
+ r_expr->accept_vis (*this);
+ maybe_expand_expr (r_expr);
+
+ // ensure that they are not marked for strip
+ if (expr.get_left_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_left_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before binary op exprs");
+ if (expr.get_right_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_right_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::TypeCastExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * direct descendant expression, can strip ones below that */
+
+ auto &casted_expr = expr.get_casted_expr ();
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ casted_expr->accept_vis (*this);
+
+ // ensure that they are not marked for strip
+ if (casted_expr->is_marked_for_strip ())
+ rust_error_at (casted_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed before cast exprs");
+
+ // TODO: strip sub-types of type
+ auto &type = expr.get_type_to_cast_to ();
+ type->accept_vis (*this);
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::AssignmentExpr &expr)
+{
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &l_expr = expr.get_left_expr ();
+ l_expr->accept_vis (*this);
+ maybe_expand_expr (l_expr);
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &r_expr = expr.get_right_expr ();
+ r_expr->accept_vis (*this);
+ maybe_expand_expr (r_expr);
+
+ // ensure that they are not marked for strip
+ if (expr.get_left_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_left_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before binary op exprs");
+ if (expr.get_right_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_right_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::CompoundAssignmentExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &l_expr = expr.get_left_expr ();
+ l_expr->accept_vis (*this);
+ maybe_expand_expr (l_expr);
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &r_expr = expr.get_right_expr ();
+ r_expr->accept_vis (*this);
+ maybe_expand_expr (r_expr);
+
+ // ensure that they are not marked for strip
+ if (expr.get_left_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_left_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before binary op exprs");
+ if (expr.get_right_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_right_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::GroupedExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says these are inner
+ * attributes, not outer attributes of inner expr */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &inner_expr = expr.get_expr_in_parens ();
+ inner_expr->accept_vis (*this);
+ if (inner_expr->is_marked_for_strip ())
+ rust_error_at (inner_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ArrayElemsValues &elems)
+{
+ /* apparently outer attributes are allowed in "elements of array
+ * expressions" according to spec */
+ expand_pointer_allow_strip (elems.get_values ());
+}
+void
+AttrVisitor::visit (AST::ArrayElemsCopied &elems)
+{
+ /* apparently outer attributes are allowed in "elements of array
+ * expressions" according to spec. on the other hand, it would not
+ * make conceptual sense to be able to remove either expression. As
+ * such, not implementing. TODO clear up the ambiguity here */
+
+ // only intend stripping for internal sub-expressions
+ auto &copied_expr = elems.get_elem_to_copy ();
+ copied_expr->accept_vis (*this);
+ if (copied_expr->is_marked_for_strip ())
+ rust_error_at (copied_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ auto ©_count = elems.get_num_copies ();
+ copy_count->accept_vis (*this);
+ if (copy_count->is_marked_for_strip ())
+ rust_error_at (copy_count->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ArrayExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says there are separate
+ * inner attributes, not just outer attributes of inner exprs */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* assuming you can't strip away the ArrayElems type, but can strip
+ * internal expressions and whatever */
+ expr.get_array_elems ()->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::ArrayIndexExpr &expr)
+{
+ /* it is unclear whether outer attributes are supposed to be
+ * allowed, but conceptually it wouldn't make much sense, but
+ * having expansion code anyway. TODO */
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &array_expr = expr.get_array_expr ();
+ array_expr->accept_vis (*this);
+ if (array_expr->is_marked_for_strip ())
+ rust_error_at (array_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ auto &index_expr = expr.get_index_expr ();
+ index_expr->accept_vis (*this);
+ if (index_expr->is_marked_for_strip ())
+ rust_error_at (index_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::TupleExpr &expr)
+{
+ /* according to spec, outer attributes are allowed on "elements of
+ * tuple expressions" */
+
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says these are inner
+ * attributes, not outer attributes of inner expr */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* apparently outer attributes are allowed in "elements of tuple
+ * expressions" according to spec */
+ expand_pointer_allow_strip (expr.get_tuple_elems ());
+}
+void
+AttrVisitor::visit (AST::TupleIndexExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* wouldn't strip this directly (as outer attrs should be
+ * associated with this level), but any sub-expressions would be
+ * stripped. Thus, no need to erase when strip check called. */
+ auto &tuple_expr = expr.get_tuple_expr ();
+ tuple_expr->accept_vis (*this);
+ if (tuple_expr->is_marked_for_strip ())
+ rust_error_at (tuple_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::StructExprStruct &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says these are inner
+ * attributes, not outer attributes of inner expr */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-exprs of path
+ auto &struct_name = expr.get_struct_name ();
+ visit (struct_name);
+ if (struct_name.is_marked_for_strip ())
+ rust_error_at (struct_name.get_locus (),
+ "cannot strip path in this position");
+}
+void
+AttrVisitor::visit (AST::StructExprFieldIdentifier &)
+{
+ // as no attrs (at moment, at least), no stripping possible
+}
+void
+AttrVisitor::visit (AST::StructExprFieldIdentifierValue &field)
+{
+ /* as no attrs possible (at moment, at least), only sub-expression
+ * stripping is possible */
+ auto &value = field.get_value ();
+ value->accept_vis (*this);
+ if (value->is_marked_for_strip ())
+ rust_error_at (value->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::StructExprFieldIndexValue &field)
+{
+ /* as no attrs possible (at moment, at least), only sub-expression
+ * stripping is possible */
+ auto &value = field.get_value ();
+ value->accept_vis (*this);
+ if (value->is_marked_for_strip ())
+ rust_error_at (value->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::StructExprStructFields &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says these are inner
+ * attributes, not outer attributes of inner expr */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-exprs of path
+ auto &struct_name = expr.get_struct_name ();
+ visit (struct_name);
+ if (struct_name.is_marked_for_strip ())
+ rust_error_at (struct_name.get_locus (),
+ "cannot strip path in this position");
+
+ /* spec does not specify whether expressions are allowed to be
+ * stripped at top level of struct fields, but I wouldn't think
+ * that they would be, so operating under the assumption that only
+ * sub-expressions can be stripped. */
+ for (auto &field : expr.get_fields ())
+ {
+ field->accept_vis (*this);
+ // shouldn't strip in this
+ }
+
+ /* struct base presumably can't be stripped, as the '..' is before
+ * the expression. as such, can only strip sub-expressions. */
+ if (expr.has_struct_base ())
+ {
+ auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
+ base_struct_expr->accept_vis (*this);
+ if (base_struct_expr->is_marked_for_strip ())
+ rust_error_at (base_struct_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+ }
+}
+void
+AttrVisitor::visit (AST::StructExprStructBase &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says these are inner
+ * attributes, not outer attributes of inner expr */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-exprs of path
+ auto &struct_name = expr.get_struct_name ();
+ visit (struct_name);
+ if (struct_name.is_marked_for_strip ())
+ rust_error_at (struct_name.get_locus (),
+ "cannot strip path in this position");
+
+ /* struct base presumably can't be stripped, as the '..' is before
+ * the expression. as such, can only strip sub-expressions. */
+ rust_assert (!expr.get_struct_base ().is_invalid ());
+ auto &base_struct_expr = expr.get_struct_base ().get_base_struct ();
+ base_struct_expr->accept_vis (*this);
+ if (base_struct_expr->is_marked_for_strip ())
+ rust_error_at (base_struct_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::CallExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* should not be outer attrs on "function" expression - outer attrs
+ * should be associated with call expr as a whole. only sub-expr
+ * expansion is possible. */
+ auto &function = expr.get_function_expr ();
+ function->accept_vis (*this);
+ if (function->is_marked_for_strip ())
+ rust_error_at (function->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ /* spec says outer attributes are specifically allowed for elements
+ * of call expressions, so full stripping possible */
+ // FIXME: Arthur: Figure out how to refactor this - This is similar to
+ // expanding items in the crate or stmts in blocks
+ expand_pointer_allow_strip (expr.get_params ());
+ auto ¶ms = expr.get_params ();
+ for (auto it = params.begin (); it != params.end ();)
+ {
+ auto &stmt = *it;
+
+ stmt->accept_vis (*this);
+
+ auto final_fragment = expand_macro_fragment_recursive ();
+ if (final_fragment.should_expand ())
+ {
+ // Remove the current expanded invocation
+ it = params.erase (it);
+ for (auto &node : final_fragment.get_nodes ())
+ {
+ it = params.insert (it, node.take_expr ());
+ it++;
+ }
+ }
+ else if (stmt->is_marked_for_strip ())
+ it = params.erase (it);
+ else
+ it++;
+ }
+}
+void
+AttrVisitor::visit (AST::MethodCallExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* should not be outer attrs on "receiver" expression - outer attrs
+ * should be associated with call expr as a whole. only sub-expr
+ * expansion is possible. */
+ auto &receiver = expr.get_receiver_expr ();
+ receiver->accept_vis (*this);
+ if (receiver->is_marked_for_strip ())
+ rust_error_at (receiver->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ auto &method_name = expr.get_method_name ();
+ if (method_name.has_generic_args ())
+ expand_generic_args (method_name.get_generic_args ());
+
+ /* spec says outer attributes are specifically allowed for elements
+ * of method call expressions, so full stripping possible */
+ expand_pointer_allow_strip (expr.get_params ());
+}
+void
+AttrVisitor::visit (AST::FieldAccessExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* should not be outer attrs on "receiver" expression - outer attrs
+ * should be associated with field expr as a whole. only sub-expr
+ * expansion is possible. */
+ auto &receiver = expr.get_receiver_expr ();
+ receiver->accept_vis (*this);
+ if (receiver->is_marked_for_strip ())
+ rust_error_at (receiver->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ClosureExprInner &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip closure parameters if required - this is specifically
+ * allowed by spec */
+ expand_closure_params (expr.get_params ());
+
+ // can't strip expression itself, but can strip sub-expressions
+ auto &definition_expr = expr.get_definition_expr ();
+ definition_expr->accept_vis (*this);
+ if (definition_expr->is_marked_for_strip ())
+ rust_error_at (definition_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+
+void
+AttrVisitor::visit (AST::BlockExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip test based on inner attrs - spec says there are inner
+ * attributes, not just outer attributes of inner stmts */
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ std::function<std::unique_ptr<AST::Stmt> (AST::SingleASTNode)> extractor
+ = [] (AST::SingleASTNode node) { return node.take_stmt (); };
+
+ expand_macro_children (MacroExpander::BLOCK, expr.get_statements (),
+ extractor);
+
+ expander.push_context (MacroExpander::BLOCK);
+
+ // strip tail expression if exists - can actually fully remove it
+ if (expr.has_tail_expr ())
+ {
+ auto &tail_expr = expr.get_tail_expr ();
+
+ tail_expr->accept_vis (*this);
+ maybe_expand_expr (tail_expr);
+
+ if (tail_expr->is_marked_for_strip ())
+ expr.strip_tail_expr ();
+ }
+ expander.pop_context ();
+}
+
+void
+AttrVisitor::visit (AST::ClosureExprInnerTyped &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* strip closure parameters if required - this is specifically
+ * allowed by spec */
+ expand_closure_params (expr.get_params ());
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // can't strip return type, but can strip sub-types
+ auto &type = expr.get_return_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ // can't strip expression itself, but can strip sub-expressions
+ auto &definition_block = expr.get_definition_block ();
+ definition_block->accept_vis (*this);
+ if (definition_block->is_marked_for_strip ())
+ rust_error_at (definition_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ContinueExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::BreakExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* spec does not say that you can have outer attributes on
+ * expression, so assuming you can't. stripping for sub-expressions
+ * is the only thing that can be done */
+ if (expr.has_break_expr ())
+ {
+ auto &break_expr = expr.get_break_expr ();
+
+ break_expr->accept_vis (*this);
+
+ if (break_expr->is_marked_for_strip ())
+ rust_error_at (break_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+ }
+}
+void
+AttrVisitor::visit (AST::RangeFromToExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ expr.get_from_expr ()->accept_vis (*this);
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ expr.get_to_expr ()->accept_vis (*this);
+
+ // ensure that they are not marked for strip
+ if (expr.get_from_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_from_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before range exprs");
+ if (expr.get_to_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_to_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::RangeFromExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * direct descendant expression, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ auto &from_expr = expr.get_from_expr ();
+
+ from_expr->accept_vis (*this);
+
+ if (from_expr->is_marked_for_strip ())
+ rust_error_at (from_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed before range exprs");
+}
+void
+AttrVisitor::visit (AST::RangeToExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * direct descendant expression, can strip ones below that */
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &to_expr = expr.get_to_expr ();
+
+ to_expr->accept_vis (*this);
+
+ if (to_expr->is_marked_for_strip ())
+ rust_error_at (to_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::RangeFullExpr &)
+{
+ // outer attributes never allowed before these, so no stripping
+}
+void
+AttrVisitor::visit (AST::RangeFromToInclExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * two direct descendant expressions, can strip ones below that */
+
+ /* should have no possibility for outer attrs as would be parsed
+ * with outer expr */
+ expr.get_from_expr ()->accept_vis (*this);
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ expr.get_to_expr ()->accept_vis (*this);
+
+ // ensure that they are not marked for strip
+ if (expr.get_from_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_from_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes are never allowed "
+ "before range exprs");
+ if (expr.get_to_expr ()->is_marked_for_strip ())
+ rust_error_at (expr.get_to_expr ()->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::RangeToInclExpr &expr)
+{
+ /* outer attributes never allowed before these. while cannot strip
+ * direct descendant expression, can strip ones below that */
+
+ /* should syntactically not have outer attributes, though this may
+ * not have worked in practice */
+ auto &to_expr = expr.get_to_expr ();
+
+ to_expr->accept_vis (*this);
+
+ if (to_expr->is_marked_for_strip ())
+ rust_error_at (to_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ReturnExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* spec does not say that you can have outer attributes on
+ * expression, so assuming you can't. stripping for sub-expressions
+ * is the only thing that can be done */
+ if (expr.has_returned_expr ())
+ {
+ auto &returned_expr = expr.get_returned_expr ();
+
+ returned_expr->accept_vis (*this);
+
+ if (returned_expr->is_marked_for_strip ())
+ rust_error_at (returned_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+ }
+ /* TODO: conceptually, you would maybe be able to remove a returned
+ * expr - e.g. if you had conditional compilation returning void or
+ * returning a type. On the other hand, I think that function
+ * return type cannot be conditionally compiled, so I assumed you
+ * can't do this either. */
+}
+void
+AttrVisitor::visit (AST::UnsafeBlockExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &block_expr = expr.get_block_expr ();
+ block_expr->accept_vis (*this);
+ if (block_expr->is_marked_for_strip ())
+ rust_error_at (block_expr->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::LoopExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &loop_block = expr.get_loop_block ();
+ loop_block->accept_vis (*this);
+ if (loop_block->is_marked_for_strip ())
+ rust_error_at (loop_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::WhileLoopExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip predicate expr itself, but can strip sub-expressions
+ auto &predicate_expr = expr.get_predicate_expr ();
+ predicate_expr->accept_vis (*this);
+ if (predicate_expr->is_marked_for_strip ())
+ rust_error_at (predicate_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &loop_block = expr.get_loop_block ();
+ loop_block->accept_vis (*this);
+ if (loop_block->is_marked_for_strip ())
+ rust_error_at (loop_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::WhileLetLoopExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ for (auto &pattern : expr.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ // can't strip scrutinee expr itself, but can strip sub-expressions
+ auto &scrutinee_expr = expr.get_scrutinee_expr ();
+ scrutinee_expr->accept_vis (*this);
+ if (scrutinee_expr->is_marked_for_strip ())
+ rust_error_at (scrutinee_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &loop_block = expr.get_loop_block ();
+ loop_block->accept_vis (*this);
+ if (loop_block->is_marked_for_strip ())
+ rust_error_at (loop_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::ForLoopExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-patterns of pattern
+ auto &pattern = expr.get_pattern ();
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+
+ // can't strip scrutinee expr itself, but can strip sub-expressions
+ auto &iterator_expr = expr.get_iterator_expr ();
+ iterator_expr->accept_vis (*this);
+ if (iterator_expr->is_marked_for_strip ())
+ rust_error_at (iterator_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &loop_block = expr.get_loop_block ();
+ loop_block->accept_vis (*this);
+ if (loop_block->is_marked_for_strip ())
+ rust_error_at (loop_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfExpr &expr)
+{
+ // rust playground test shows that IfExpr does support outer attrs, at least
+ // when used as statement
+
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip condition expr itself, but can strip sub-expressions
+ auto &condition_expr = expr.get_condition_expr ();
+ condition_expr->accept_vis (*this);
+ maybe_expand_expr (condition_expr);
+ if (condition_expr->is_marked_for_strip ())
+ rust_error_at (condition_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfExprConseqElse &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip condition expr itself, but can strip sub-expressions
+ auto &condition_expr = expr.get_condition_expr ();
+ condition_expr->accept_vis (*this);
+ maybe_expand_expr (condition_expr);
+ if (condition_expr->is_marked_for_strip ())
+ rust_error_at (condition_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip else block itself, but can strip sub-expressions
+ auto &else_block = expr.get_else_block ();
+ else_block->accept_vis (*this);
+ if (else_block->is_marked_for_strip ())
+ rust_error_at (else_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfExprConseqIf &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip condition expr itself, but can strip sub-expressions
+ auto &condition_expr = expr.get_condition_expr ();
+ condition_expr->accept_vis (*this);
+ maybe_expand_expr (condition_expr);
+ if (condition_expr->is_marked_for_strip ())
+ rust_error_at (condition_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if expr itself, but can strip sub-expressions
+ auto &conseq_if_expr = expr.get_conseq_if_expr ();
+ conseq_if_expr->accept_vis (*this);
+ if (conseq_if_expr->is_marked_for_strip ())
+ rust_error_at (conseq_if_expr->get_locus (),
+ "cannot strip consequent if expression in this "
+ "position - outer attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfExprConseqIfLet &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip condition expr itself, but can strip sub-expressions
+ auto &condition_expr = expr.get_condition_expr ();
+ condition_expr->accept_vis (*this);
+ maybe_expand_expr (condition_expr);
+ if (condition_expr->is_marked_for_strip ())
+ rust_error_at (condition_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if let expr itself, but can strip sub-expressions
+ auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
+ conseq_if_let_expr->accept_vis (*this);
+ if (conseq_if_let_expr->is_marked_for_strip ())
+ rust_error_at (conseq_if_let_expr->get_locus (),
+ "cannot strip consequent if let expression in this "
+ "position - outer attributes not "
+ "allowed");
+}
+void
+AttrVisitor::visit (AST::IfLetExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ for (auto &pattern : expr.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ // can't strip value expr itself, but can strip sub-expressions
+ auto &value_expr = expr.get_value_expr ();
+ value_expr->accept_vis (*this);
+ if (value_expr->is_marked_for_strip ())
+ rust_error_at (value_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfLetExprConseqElse &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ for (auto &pattern : expr.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ // can't strip value expr itself, but can strip sub-expressions
+ auto &value_expr = expr.get_value_expr ();
+ value_expr->accept_vis (*this);
+ if (value_expr->is_marked_for_strip ())
+ rust_error_at (value_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip else block itself, but can strip sub-expressions
+ auto &else_block = expr.get_else_block ();
+ else_block->accept_vis (*this);
+ if (else_block->is_marked_for_strip ())
+ rust_error_at (else_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfLetExprConseqIf &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ for (auto &pattern : expr.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ // can't strip value expr itself, but can strip sub-expressions
+ auto &value_expr = expr.get_value_expr ();
+ value_expr->accept_vis (*this);
+ if (value_expr->is_marked_for_strip ())
+ rust_error_at (value_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if expr itself, but can strip sub-expressions
+ auto &conseq_if_expr = expr.get_conseq_if_expr ();
+ conseq_if_expr->accept_vis (*this);
+ if (conseq_if_expr->is_marked_for_strip ())
+ rust_error_at (conseq_if_expr->get_locus (),
+ "cannot strip consequent if expression in this "
+ "position - outer attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::IfLetExprConseqIfLet &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ for (auto &pattern : expr.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ // can't strip value expr itself, but can strip sub-expressions
+ auto &value_expr = expr.get_value_expr ();
+ value_expr->accept_vis (*this);
+ if (value_expr->is_marked_for_strip ())
+ rust_error_at (value_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if block itself, but can strip sub-expressions
+ auto &if_block = expr.get_if_block ();
+ if_block->accept_vis (*this);
+ if (if_block->is_marked_for_strip ())
+ rust_error_at (if_block->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+
+ // can't strip if let expr itself, but can strip sub-expressions
+ auto &conseq_if_let_expr = expr.get_conseq_if_let_expr ();
+ conseq_if_let_expr->accept_vis (*this);
+ if (conseq_if_let_expr->is_marked_for_strip ())
+ rust_error_at (conseq_if_let_expr->get_locus (),
+ "cannot strip consequent if let expression in this "
+ "position - outer attributes not "
+ "allowed");
+}
+void
+AttrVisitor::visit (AST::MatchExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // inner attr strip test
+ expander.expand_cfg_attrs (expr.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_inner_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip scrutinee expr itself, but can strip sub-expressions
+ auto &scrutinee_expr = expr.get_scrutinee_expr ();
+ scrutinee_expr->accept_vis (*this);
+ if (scrutinee_expr->is_marked_for_strip ())
+ rust_error_at (scrutinee_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // strip match cases
+ auto &match_cases = expr.get_match_cases ();
+ for (auto it = match_cases.begin (); it != match_cases.end ();)
+ {
+ auto &match_case = *it;
+
+ // strip match case based on outer attributes in match arm
+ auto &match_arm = match_case.get_arm ();
+ expander.expand_cfg_attrs (match_arm.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (match_arm.get_outer_attrs ()))
+ {
+ // strip match case
+ it = match_cases.erase (it);
+ continue;
+ }
+
+ for (auto &pattern : match_arm.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ }
+
+ /* assuming that guard expression cannot be stripped as
+ * strictly speaking you would have to strip the whole guard to
+ * make syntactical sense, which you can't do. as such, only
+ * strip sub-expressions */
+ if (match_arm.has_match_arm_guard ())
+ {
+ auto &guard_expr = match_arm.get_guard_expr ();
+ guard_expr->accept_vis (*this);
+ if (guard_expr->is_marked_for_strip ())
+ rust_error_at (guard_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+ }
+
+ // strip sub-expressions from match cases
+ auto &case_expr = match_case.get_expr ();
+ case_expr->accept_vis (*this);
+ if (case_expr->is_marked_for_strip ())
+ rust_error_at (case_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ // increment to next case if haven't continued
+ ++it;
+ }
+}
+void
+AttrVisitor::visit (AST::AwaitExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ /* can't strip awaited expr itself, but can strip sub-expressions
+ * - this is because you can't have no expr to await */
+ auto &awaited_expr = expr.get_awaited_expr ();
+ awaited_expr->accept_vis (*this);
+ if (awaited_expr->is_marked_for_strip ())
+ rust_error_at (awaited_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::AsyncBlockExpr &expr)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (expr.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (expr.get_outer_attrs ()))
+ {
+ expr.mark_for_strip ();
+ return;
+ }
+
+ // can't strip block itself, but can strip sub-expressions
+ auto &block_expr = expr.get_block_expr ();
+ block_expr->accept_vis (*this);
+ if (block_expr->is_marked_for_strip ())
+ rust_error_at (block_expr->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+
+void
+AttrVisitor::visit (AST::TypeParam ¶m)
+{
+ // outer attributes don't actually do anything, so ignore them
+
+ if (param.has_type_param_bounds ())
+ {
+ // don't strip directly, only components of bounds
+ for (auto &bound : param.get_type_param_bounds ())
+ bound->accept_vis (*this);
+ }
+
+ if (param.has_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+ auto &type = param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+}
+void
+AttrVisitor::visit (AST::LifetimeWhereClauseItem &)
+{
+ // shouldn't require
+}
+void
+AttrVisitor::visit (AST::TypeBoundWhereClauseItem &item)
+{
+ // for lifetimes shouldn't require
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = item.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ // don't strip directly, only components of bounds
+ for (auto &bound : item.get_type_param_bounds ())
+ bound->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::Method &method)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (method.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (method.get_outer_attrs ()))
+ {
+ method.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : method.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* assuming you can't strip self param - wouldn't be a method
+ * anymore. spec allows outer attrs on self param, but doesn't
+ * specify whether cfg is used. */
+ expand_self_param (method.get_self_param ());
+
+ /* strip method parameters if required - this is specifically
+ * allowed by spec */
+ expand_function_params (method.get_function_params ());
+
+ if (method.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = method.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ if (method.has_where_clause ())
+ expand_where_clause (method.get_where_clause ());
+
+ /* body should always exist - if error state, should have returned
+ * before now */
+ // can't strip block itself, but can strip sub-expressions
+ auto &block_expr = method.get_definition ();
+ block_expr->accept_vis (*this);
+ if (block_expr->is_marked_for_strip ())
+ rust_error_at (block_expr->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::Module &module)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (module.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (module.get_outer_attrs ()))
+ {
+ module.mark_for_strip ();
+ return;
+ }
+
+ // A loaded module might have inner attributes
+ if (module.get_kind () == AST::Module::ModuleKind::LOADED)
+ {
+ // strip test based on inner attrs
+ expander.expand_cfg_attrs (module.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (module.get_inner_attrs ()))
+ {
+ module.mark_for_strip ();
+ return;
+ }
+ }
+
+ // Parse the module's items if they haven't been expanded and the file
+ // should be parsed (i.e isn't hidden behind an untrue or impossible cfg
+ // directive)
+ if (!module.is_marked_for_strip ()
+ && module.get_kind () == AST::Module::ModuleKind::UNLOADED)
+ {
+ module.load_items ();
+ }
+
+ // strip items if required
+ expand_pointer_allow_strip (module.get_items ());
+}
+void
+AttrVisitor::visit (AST::ExternCrate &extern_crate)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (extern_crate.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (extern_crate.get_outer_attrs ()))
+ {
+ extern_crate.mark_for_strip ();
+ return;
+ }
+
+ if (!extern_crate.references_self ())
+ {
+ Session &session = Session::get_instance ();
+ session.load_extern_crate (extern_crate.get_referenced_crate (),
+ extern_crate.get_locus ());
+ }
+}
+void
+AttrVisitor::visit (AST::UseTreeGlob &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::UseTreeList &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::UseTreeRebind &)
+{
+ // shouldn't require?
+}
+void
+AttrVisitor::visit (AST::UseDeclaration &use_decl)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (use_decl.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (use_decl.get_outer_attrs ()))
+ {
+ use_decl.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::Function &function)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (function.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (function.get_outer_attrs ()))
+ {
+ function.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : function.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* strip function parameters if required - this is specifically
+ * allowed by spec */
+ expand_function_params (function.get_function_params ());
+
+ if (function.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = function.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ if (function.has_where_clause ())
+ expand_where_clause (function.get_where_clause ());
+
+ /* body should always exist - if error state, should have returned
+ * before now */
+ // can't strip block itself, but can strip sub-expressions
+ auto &block_expr = function.get_definition ();
+ block_expr->accept_vis (*this);
+ if (block_expr->is_marked_for_strip ())
+ rust_error_at (block_expr->get_locus (),
+ "cannot strip block expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::TypeAlias &type_alias)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (type_alias.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (type_alias.get_outer_attrs ()))
+ {
+ type_alias.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : type_alias.get_generic_params ())
+ param->accept_vis (*this);
+
+ if (type_alias.has_where_clause ())
+ expand_where_clause (type_alias.get_where_clause ());
+
+ auto &type = type_alias.get_type_aliased ();
+ type->accept_vis (*this);
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::StructStruct &struct_item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (struct_item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (struct_item.get_outer_attrs ()))
+ {
+ struct_item.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : struct_item.get_generic_params ())
+ param->accept_vis (*this);
+
+ if (struct_item.has_where_clause ())
+ expand_where_clause (struct_item.get_where_clause ());
+
+ /* strip struct fields if required - this is presumably
+ * allowed by spec */
+ expand_struct_fields (struct_item.get_fields ());
+}
+void
+AttrVisitor::visit (AST::TupleStruct &tuple_struct)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (tuple_struct.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (tuple_struct.get_outer_attrs ()))
+ {
+ tuple_struct.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : tuple_struct.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* strip struct fields if required - this is presumably
+ * allowed by spec */
+ expand_tuple_fields (tuple_struct.get_fields ());
+
+ if (tuple_struct.has_where_clause ())
+ expand_where_clause (tuple_struct.get_where_clause ());
+}
+void
+AttrVisitor::visit (AST::EnumItem &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::EnumItemTuple &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ /* strip item fields if required - this is presumably
+ * allowed by spec */
+ expand_tuple_fields (item.get_tuple_fields ());
+}
+void
+AttrVisitor::visit (AST::EnumItemStruct &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ /* strip item fields if required - this is presumably
+ * allowed by spec */
+ expand_struct_fields (item.get_struct_fields ());
+}
+void
+AttrVisitor::visit (AST::EnumItemDiscriminant &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &expr = item.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ rust_error_at (expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::Enum &enum_item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (enum_item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (enum_item.get_outer_attrs ()))
+ {
+ enum_item.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : enum_item.get_generic_params ())
+ param->accept_vis (*this);
+
+ if (enum_item.has_where_clause ())
+ expand_where_clause (enum_item.get_where_clause ());
+
+ /* strip enum fields if required - this is presumably
+ * allowed by spec */
+ expand_pointer_allow_strip (enum_item.get_variants ());
+}
+void
+AttrVisitor::visit (AST::Union &union_item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (union_item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (union_item.get_outer_attrs ()))
+ {
+ union_item.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : union_item.get_generic_params ())
+ param->accept_vis (*this);
+
+ if (union_item.has_where_clause ())
+ expand_where_clause (union_item.get_where_clause ());
+
+ /* strip union fields if required - this is presumably
+ * allowed by spec */
+ expand_struct_fields (union_item.get_variants ());
+}
+void
+AttrVisitor::visit (AST::ConstantItem &const_item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (const_item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (const_item.get_outer_attrs ()))
+ {
+ const_item.mark_for_strip ();
+ return;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // strip any sub-types
+ auto &type = const_item.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &expr = const_item.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ rust_error_at (expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::StaticItem &static_item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (static_item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (static_item.get_outer_attrs ()))
+ {
+ static_item.mark_for_strip ();
+ return;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // strip any sub-types
+ auto &type = static_item.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &expr = static_item.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ rust_error_at (expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+}
+void
+AttrVisitor::visit (AST::TraitItemFunc &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ expand_trait_function_decl (item.get_trait_function_decl ());
+
+ if (item.has_definition ())
+ {
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &block = item.get_definition ();
+ block->accept_vis (*this);
+ if (block->is_marked_for_strip ())
+ rust_error_at (block->get_locus (),
+ "cannot strip block expression in this "
+ "position - outer attributes not allowed");
+ }
+}
+void
+AttrVisitor::visit (AST::TraitItemMethod &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ expand_trait_method_decl (item.get_trait_method_decl ());
+
+ if (item.has_definition ())
+ {
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped. */
+ auto &block = item.get_definition ();
+ block->accept_vis (*this);
+ if (block->is_marked_for_strip ())
+ rust_error_at (block->get_locus (),
+ "cannot strip block expression in this "
+ "position - outer attributes not allowed");
+ }
+}
+void
+AttrVisitor::visit (AST::TraitItemConst &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ // strip any sub-types
+ auto &type = item.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped */
+ if (item.has_expression ())
+ {
+ auto &expr = item.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ rust_error_at (expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+ }
+}
+void
+AttrVisitor::visit (AST::TraitItemType &item)
+{
+ // initial test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ if (item.has_type_param_bounds ())
+ {
+ // don't strip directly, only components of bounds
+ for (auto &bound : item.get_type_param_bounds ())
+ bound->accept_vis (*this);
+ }
+}
+void
+AttrVisitor::visit (AST::Trait &trait)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (trait.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (trait.get_outer_attrs ()))
+ {
+ trait.mark_for_strip ();
+ return;
+ }
+
+ // strip test based on inner attrs
+ expander.expand_cfg_attrs (trait.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (trait.get_inner_attrs ()))
+ {
+ trait.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : trait.get_generic_params ())
+ param->accept_vis (*this);
+
+ if (trait.has_type_param_bounds ())
+ {
+ // don't strip directly, only components of bounds
+ for (auto &bound : trait.get_type_param_bounds ())
+ bound->accept_vis (*this);
+ }
+
+ if (trait.has_where_clause ())
+ expand_where_clause (trait.get_where_clause ());
+
+ std::function<std::unique_ptr<AST::TraitItem> (AST::SingleASTNode)> extractor
+ = [] (AST::SingleASTNode node) { return node.take_trait_item (); };
+
+ expand_macro_children (MacroExpander::TRAIT, trait.get_trait_items (),
+ extractor);
+}
+void
+AttrVisitor::visit (AST::InherentImpl &impl)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (impl.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
+ {
+ impl.mark_for_strip ();
+ return;
+ }
+
+ // strip test based on inner attrs
+ expander.expand_cfg_attrs (impl.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
+ {
+ impl.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : impl.get_generic_params ())
+ param->accept_vis (*this);
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = impl.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ if (impl.has_where_clause ())
+ expand_where_clause (impl.get_where_clause ());
+
+ std::function<std::unique_ptr<AST::InherentImplItem> (AST::SingleASTNode)>
+ extractor = [] (AST::SingleASTNode node) { return node.take_impl_item (); };
+
+ expand_macro_children (MacroExpander::IMPL, impl.get_impl_items (),
+ extractor);
+}
+void
+AttrVisitor::visit (AST::TraitImpl &impl)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (impl.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (impl.get_outer_attrs ()))
+ {
+ impl.mark_for_strip ();
+ return;
+ }
+
+ // strip test based on inner attrs
+ expander.expand_cfg_attrs (impl.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (impl.get_inner_attrs ()))
+ {
+ impl.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : impl.get_generic_params ())
+ param->accept_vis (*this);
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = impl.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ auto &trait_path = impl.get_trait_path ();
+ visit (trait_path);
+ if (trait_path.is_marked_for_strip ())
+ rust_error_at (trait_path.get_locus (),
+ "cannot strip typepath in this position");
+
+ if (impl.has_where_clause ())
+ expand_where_clause (impl.get_where_clause ());
+
+ std::function<std::unique_ptr<AST::TraitImplItem> (AST::SingleASTNode)>
+ extractor
+ = [] (AST::SingleASTNode node) { return node.take_trait_impl_item (); };
+
+ expand_macro_children (MacroExpander::TRAIT_IMPL, impl.get_impl_items (),
+ extractor);
+}
+void
+AttrVisitor::visit (AST::ExternalStaticItem &item)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = item.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (), "cannot strip type in this position");
+
+ expander.pop_context ();
+}
+void
+AttrVisitor::visit (AST::ExternalFunctionItem &item)
+{
+ // strip test based on outer attrs
+ expander.expand_cfg_attrs (item.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (item.get_outer_attrs ()))
+ {
+ item.mark_for_strip ();
+ return;
+ }
+
+ // just expand sub-stuff - can't actually strip generic params themselves
+ for (auto ¶m : item.get_generic_params ())
+ param->accept_vis (*this);
+
+ /* strip function parameters if required - this is specifically
+ * allowed by spec */
+ auto ¶ms = item.get_function_params ();
+ for (auto it = params.begin (); it != params.end ();)
+ {
+ auto ¶m = *it;
+
+ auto ¶m_attrs = param.get_outer_attrs ();
+ expander.expand_cfg_attrs (param_attrs);
+ if (expander.fails_cfg_with_expand (param_attrs))
+ {
+ it = params.erase (it);
+ continue;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ // increment if nothing else happens
+ ++it;
+ }
+ /* NOTE: these are extern function params, which may have different
+ * rules and restrictions to "normal" function params. So expansion
+ * handled separately. */
+
+ /* TODO: assuming that variadic nature cannot be stripped. If this
+ * is not true, then have code here to do so. */
+
+ if (item.has_return_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &return_type = item.get_return_type ();
+ return_type->accept_vis (*this);
+
+ maybe_expand_type (return_type);
+
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ if (item.has_where_clause ())
+ expand_where_clause (item.get_where_clause ());
+}
+
+void
+AttrVisitor::visit (AST::ExternBlock &block)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (block.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (block.get_outer_attrs ()))
+ {
+ block.mark_for_strip ();
+ return;
+ }
+
+ // strip test based on inner attrs
+ expander.expand_cfg_attrs (block.get_inner_attrs ());
+ if (expander.fails_cfg_with_expand (block.get_inner_attrs ()))
+ {
+ block.mark_for_strip ();
+ return;
+ }
+
+ std::function<std::unique_ptr<AST::ExternalItem> (AST::SingleASTNode)>
+ extractor
+ = [] (AST::SingleASTNode node) { return node.take_external_item (); };
+
+ expand_macro_children (MacroExpander::EXTERN, block.get_extern_items (),
+ extractor);
+}
+
+// I don't think it would be possible to strip macros without expansion
+void
+AttrVisitor::visit (AST::MacroMatchFragment &)
+{}
+void
+AttrVisitor::visit (AST::MacroMatchRepetition &)
+{}
+void
+AttrVisitor::visit (AST::MacroMatcher &)
+{}
+void
+AttrVisitor::visit (AST::MacroRulesDefinition &rules_def)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (rules_def.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (rules_def.get_outer_attrs ()))
+ {
+ rules_def.mark_for_strip ();
+ return;
+ }
+
+ // I don't think any macro rules can be stripped in any way
+
+ auto path = Resolver::CanonicalPath::new_seg (rules_def.get_node_id (),
+ rules_def.get_rule_name ());
+ expander.resolver->get_macro_scope ().insert (path, rules_def.get_node_id (),
+ rules_def.get_locus ());
+ expander.mappings->insert_macro_def (&rules_def);
+ rust_debug_loc (rules_def.get_locus (), "inserting macro def: [%s]",
+ path.get ().c_str ());
+}
+
+void
+AttrVisitor::visit (AST::MetaItemPath &)
+{}
+void
+AttrVisitor::visit (AST::MetaItemSeq &)
+{}
+void
+AttrVisitor::visit (AST::MetaWord &)
+{}
+void
+AttrVisitor::visit (AST::MetaNameValueStr &)
+{}
+void
+AttrVisitor::visit (AST::MetaListPaths &)
+{}
+void
+AttrVisitor::visit (AST::MetaListNameValueStr &)
+{}
+
+void
+AttrVisitor::visit (AST::LiteralPattern &)
+{
+ // not possible
+}
+void
+AttrVisitor::visit (AST::IdentifierPattern &pattern)
+{
+ // can only strip sub-patterns of the inner pattern to bind
+ if (!pattern.has_pattern_to_bind ())
+ return;
+
+ auto &sub_pattern = pattern.get_pattern_to_bind ();
+ sub_pattern->accept_vis (*this);
+ if (sub_pattern->is_marked_for_strip ())
+ rust_error_at (sub_pattern->get_locus (),
+ "cannot strip pattern in this position");
+}
+void
+AttrVisitor::visit (AST::WildcardPattern &)
+{
+ // not possible
+}
+void
+AttrVisitor::visit (AST::RangePatternBoundLiteral &)
+{
+ // not possible
+}
+void
+AttrVisitor::visit (AST::RangePatternBoundPath &bound)
+{
+ // can expand path, but not strip it directly
+ auto &path = bound.get_path ();
+ visit (path);
+ if (path.is_marked_for_strip ())
+ rust_error_at (path.get_locus (), "cannot strip path in this position");
+}
+void
+AttrVisitor::visit (AST::RangePatternBoundQualPath &bound)
+{
+ // can expand path, but not strip it directly
+ auto &path = bound.get_qualified_path ();
+ visit (path);
+ if (path.is_marked_for_strip ())
+ rust_error_at (path.get_locus (), "cannot strip path in this position");
+}
+void
+AttrVisitor::visit (AST::RangePattern &pattern)
+{
+ // should have no capability to strip lower or upper bounds, only expand
+ pattern.get_lower_bound ()->accept_vis (*this);
+ pattern.get_upper_bound ()->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::ReferencePattern &pattern)
+{
+ auto &sub_pattern = pattern.get_referenced_pattern ();
+ sub_pattern->accept_vis (*this);
+ if (sub_pattern->is_marked_for_strip ())
+ rust_error_at (sub_pattern->get_locus (),
+ "cannot strip pattern in this position");
+}
+void
+AttrVisitor::visit (AST::StructPatternFieldTuplePat &field)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (field.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
+ {
+ field.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-patterns (can't strip top-level pattern)
+ auto &sub_pattern = field.get_index_pattern ();
+ sub_pattern->accept_vis (*this);
+ if (sub_pattern->is_marked_for_strip ())
+ rust_error_at (sub_pattern->get_locus (),
+ "cannot strip pattern in this position");
+}
+void
+AttrVisitor::visit (AST::StructPatternFieldIdentPat &field)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (field.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
+ {
+ field.mark_for_strip ();
+ return;
+ }
+
+ // strip sub-patterns (can't strip top-level pattern)
+ auto &sub_pattern = field.get_ident_pattern ();
+ sub_pattern->accept_vis (*this);
+ if (sub_pattern->is_marked_for_strip ())
+ rust_error_at (sub_pattern->get_locus (),
+ "cannot strip pattern in this position");
+}
+void
+AttrVisitor::visit (AST::StructPatternFieldIdent &field)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (field.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (field.get_outer_attrs ()))
+ {
+ field.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::StructPattern &pattern)
+{
+ // expand (but don't strip) path
+ auto &path = pattern.get_path ();
+ visit (path);
+ if (path.is_marked_for_strip ())
+ rust_error_at (path.get_locus (), "cannot strip path in this position");
+
+ /* TODO: apparently struct pattern fields can have outer attrs. so can they
+ * be stripped? */
+ if (!pattern.has_struct_pattern_elems ())
+ return;
+
+ auto &elems = pattern.get_struct_pattern_elems ();
+
+ // assuming you can strip struct pattern fields
+ expand_pointer_allow_strip (elems.get_struct_pattern_fields ());
+
+ // assuming you can strip the ".." part
+ if (elems.has_etc ())
+ {
+ expander.expand_cfg_attrs (elems.get_etc_outer_attrs ());
+ if (expander.fails_cfg_with_expand (elems.get_etc_outer_attrs ()))
+ elems.strip_etc ();
+ }
+}
+void
+AttrVisitor::visit (AST::TupleStructItemsNoRange &tuple_items)
+{
+ // can't strip individual patterns, only sub-patterns
+ for (auto &pattern : tuple_items.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+}
+void
+AttrVisitor::visit (AST::TupleStructItemsRange &tuple_items)
+{
+ // can't strip individual patterns, only sub-patterns
+ for (auto &lower_pattern : tuple_items.get_lower_patterns ())
+ {
+ lower_pattern->accept_vis (*this);
+
+ if (lower_pattern->is_marked_for_strip ())
+ rust_error_at (lower_pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+ for (auto &upper_pattern : tuple_items.get_upper_patterns ())
+ {
+ upper_pattern->accept_vis (*this);
+
+ if (upper_pattern->is_marked_for_strip ())
+ rust_error_at (upper_pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+}
+void
+AttrVisitor::visit (AST::TupleStructPattern &pattern)
+{
+ // expand (but don't strip) path
+ auto &path = pattern.get_path ();
+ visit (path);
+ if (path.is_marked_for_strip ())
+ rust_error_at (path.get_locus (), "cannot strip path in this position");
+
+ if (pattern.has_items ())
+ pattern.get_items ()->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::TuplePatternItemsMultiple &tuple_items)
+{
+ // can't strip individual patterns, only sub-patterns
+ for (auto &pattern : tuple_items.get_patterns ())
+ {
+ pattern->accept_vis (*this);
+
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+}
+void
+AttrVisitor::visit (AST::TuplePatternItemsRanged &tuple_items)
+{
+ // can't strip individual patterns, only sub-patterns
+ for (auto &lower_pattern : tuple_items.get_lower_patterns ())
+ {
+ lower_pattern->accept_vis (*this);
+
+ if (lower_pattern->is_marked_for_strip ())
+ rust_error_at (lower_pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+ for (auto &upper_pattern : tuple_items.get_upper_patterns ())
+ {
+ upper_pattern->accept_vis (*this);
+
+ if (upper_pattern->is_marked_for_strip ())
+ rust_error_at (upper_pattern->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+}
+void
+AttrVisitor::visit (AST::TuplePattern &pattern)
+{
+ if (pattern.has_tuple_pattern_items ())
+ pattern.get_items ()->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::GroupedPattern &pattern)
+{
+ // can't strip inner pattern, only sub-patterns
+ auto &pattern_in_parens = pattern.get_pattern_in_parens ();
+
+ pattern_in_parens->accept_vis (*this);
+
+ if (pattern_in_parens->is_marked_for_strip ())
+ rust_error_at (pattern_in_parens->get_locus (),
+ "cannot strip pattern in this position");
+}
+void
+AttrVisitor::visit (AST::SlicePattern &pattern)
+{
+ // can't strip individual patterns, only sub-patterns
+ for (auto &item : pattern.get_items ())
+ {
+ item->accept_vis (*this);
+
+ if (item->is_marked_for_strip ())
+ rust_error_at (item->get_locus (),
+ "cannot strip pattern in this position");
+ // TODO: quit stripping now? or keep going?
+ }
+}
+
+void
+AttrVisitor::visit (AST::EmptyStmt &)
+{
+ // assuming no outer attributes, so nothing can happen
+}
+void
+AttrVisitor::visit (AST::LetStmt &stmt)
+{
+ // initial strip test based on outer attrs
+ expander.expand_cfg_attrs (stmt.get_outer_attrs ());
+ if (expander.fails_cfg_with_expand (stmt.get_outer_attrs ()))
+ {
+ stmt.mark_for_strip ();
+ return;
+ }
+
+ // can't strip pattern, but call for sub-patterns
+ auto &pattern = stmt.get_pattern ();
+ pattern->accept_vis (*this);
+ if (pattern->is_marked_for_strip ())
+ rust_error_at (pattern->get_locus (),
+ "cannot strip pattern in this position");
+
+ // similar for type
+ if (stmt.has_type ())
+ {
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = stmt.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+ }
+
+ /* strip any internal sub-expressions - expression itself isn't
+ * allowed to have external attributes in this position so can't be
+ * stripped */
+ if (stmt.has_init_expr ())
+ {
+ auto &init_expr = stmt.get_init_expr ();
+ init_expr->accept_vis (*this);
+
+ if (init_expr->is_marked_for_strip ())
+ rust_error_at (init_expr->get_locus (),
+ "cannot strip expression in this position - outer "
+ "attributes not allowed");
+
+ maybe_expand_expr (init_expr);
+ }
+}
+void
+AttrVisitor::visit (AST::ExprStmtWithoutBlock &stmt)
+{
+ // outer attributes associated with expr, so rely on expr
+
+ // guard - should prevent null pointer expr
+ if (stmt.is_marked_for_strip ())
+ return;
+
+ // strip if expr is to be stripped
+ auto &expr = stmt.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ {
+ stmt.mark_for_strip ();
+ return;
+ }
+}
+void
+AttrVisitor::visit (AST::ExprStmtWithBlock &stmt)
+{
+ // outer attributes associated with expr, so rely on expr
+
+ // guard - should prevent null pointer expr
+ if (stmt.is_marked_for_strip ())
+ return;
+
+ // strip if expr is to be stripped
+ auto &expr = stmt.get_expr ();
+ expr->accept_vis (*this);
+ if (expr->is_marked_for_strip ())
+ {
+ stmt.mark_for_strip ();
+ return;
+ }
+}
+
+void
+AttrVisitor::visit (AST::TraitBound &bound)
+{
+ // nothing in for lifetimes to strip
+
+ // expand but don't strip type path
+ auto &path = bound.get_type_path ();
+ visit (path);
+ if (path.is_marked_for_strip ())
+ rust_error_at (path.get_locus (),
+ "cannot strip type path in this position");
+}
+void
+AttrVisitor::visit (AST::ImplTraitType &type)
+{
+ // don't strip directly, only components of bounds
+ for (auto &bound : type.get_type_param_bounds ())
+ bound->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::TraitObjectType &type)
+{
+ // don't strip directly, only components of bounds
+ for (auto &bound : type.get_type_param_bounds ())
+ bound->accept_vis (*this);
+}
+void
+AttrVisitor::visit (AST::ParenthesisedType &type)
+{
+ // expand but don't strip inner type
+ auto &inner_type = type.get_type_in_parens ();
+ inner_type->accept_vis (*this);
+ if (inner_type->is_marked_for_strip ())
+ rust_error_at (inner_type->get_locus (),
+ "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::ImplTraitTypeOneBound &type)
+{
+ // no stripping possible
+ visit (type.get_trait_bound ());
+}
+void
+AttrVisitor::visit (AST::TraitObjectTypeOneBound &type)
+{
+ // no stripping possible
+ visit (type.get_trait_bound ());
+}
+void
+AttrVisitor::visit (AST::TupleType &type)
+{
+ // TODO: assuming that types can't be stripped as types don't have outer
+ // attributes
+ for (auto &elem_type : type.get_elems ())
+ {
+ elem_type->accept_vis (*this);
+ if (elem_type->is_marked_for_strip ())
+ rust_error_at (elem_type->get_locus (),
+ "cannot strip type in this position");
+ }
+}
+void
+AttrVisitor::visit (AST::NeverType &)
+{
+ // no stripping possible
+}
+void
+AttrVisitor::visit (AST::RawPointerType &type)
+{
+ // expand but don't strip type pointed to
+ auto &pointed_type = type.get_type_pointed_to ();
+ pointed_type->accept_vis (*this);
+ if (pointed_type->is_marked_for_strip ())
+ rust_error_at (pointed_type->get_locus (),
+ "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::ReferenceType &type)
+{
+ // expand but don't strip type referenced
+ auto &referenced_type = type.get_type_referenced ();
+ referenced_type->accept_vis (*this);
+ if (referenced_type->is_marked_for_strip ())
+ rust_error_at (referenced_type->get_locus (),
+ "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::ArrayType &type)
+{
+ // expand but don't strip type referenced
+ auto &base_type = type.get_elem_type ();
+ base_type->accept_vis (*this);
+ if (base_type->is_marked_for_strip ())
+ rust_error_at (base_type->get_locus (),
+ "cannot strip type in this position");
+
+ // same for expression
+ auto &size_expr = type.get_size_expr ();
+ size_expr->accept_vis (*this);
+ if (size_expr->is_marked_for_strip ())
+ rust_error_at (size_expr->get_locus (),
+ "cannot strip expression in this position");
+}
+void
+AttrVisitor::visit (AST::SliceType &type)
+{
+ // expand but don't strip elem type
+ auto &elem_type = type.get_elem_type ();
+ elem_type->accept_vis (*this);
+ if (elem_type->is_marked_for_strip ())
+ rust_error_at (elem_type->get_locus (),
+ "cannot strip type in this position");
+}
+void
+AttrVisitor::visit (AST::InferredType &)
+{
+ // none possible
+}
+void
+AttrVisitor::visit (AST::BareFunctionType &type)
+{
+ // seem to be no generics
+
+ // presumably function params can be stripped
+ auto ¶ms = type.get_function_params ();
+ for (auto it = params.begin (); it != params.end ();)
+ {
+ auto ¶m = *it;
+
+ auto ¶m_attrs = param.get_outer_attrs ();
+ expander.expand_cfg_attrs (param_attrs);
+ if (expander.fails_cfg_with_expand (param_attrs))
+ {
+ it = params.erase (it);
+ continue;
+ }
+
+ expander.push_context (MacroExpander::ContextType::TYPE);
+
+ auto &type = param.get_type ();
+ type->accept_vis (*this);
+
+ maybe_expand_type (type);
+
+ if (type->is_marked_for_strip ())
+ rust_error_at (type->get_locus (),
+ "cannot strip type in this position");
+
+ expander.pop_context ();
+
+ // increment if nothing else happens
+ ++it;
+ }
+
+ /* TODO: assuming that variadic nature cannot be stripped. If this
+ * is not true, then have code here to do so. */
+
+ if (type.has_return_type ())
+ {
+ // FIXME: Can we have type expansion in this position?
+ // In that case, we need to handle AST::TypeNoBounds on top of just
+ // AST::Types
+ auto &return_type = type.get_return_type ();
+ return_type->accept_vis (*this);
+ if (return_type->is_marked_for_strip ())
+ rust_error_at (return_type->get_locus (),
+ "cannot strip type in this position");
+ }
+
+ // no where clause, apparently
+}
+void
+AttrVisitor::maybe_expand_expr (std::unique_ptr<AST::Expr> &expr)
+{
+ auto final_fragment = expand_macro_fragment_recursive ();
+ if (final_fragment.should_expand ())
+ expr = final_fragment.take_expression_fragment ();
+}
+
+void
+AttrVisitor::maybe_expand_type (std::unique_ptr<AST::Type> &type)
+{
+ auto final_fragment = expand_macro_fragment_recursive ();
+ if (final_fragment.should_expand ())
+ type = final_fragment.take_type_fragment ();
+}
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,316 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#include "rust-ast-visitor.h"
+#include "rust-ast.h"
+#include "rust-macro-expand.h"
+
+namespace Rust {
+// Visitor used to expand attributes.
+class AttrVisitor : public AST::ASTVisitor
+{
+private:
+ MacroExpander &expander;
+ void maybe_expand_expr (std::unique_ptr<AST::Expr> &expr);
+ void maybe_expand_type (std::unique_ptr<AST::Type> &expr);
+
+public:
+ AttrVisitor (MacroExpander &expander) : expander (expander) {}
+
+ void expand_struct_fields (std::vector<AST::StructField> &fields);
+ void expand_tuple_fields (std::vector<AST::TupleField> &fields);
+ void expand_function_params (std::vector<AST::FunctionParam> ¶ms);
+ void expand_generic_args (AST::GenericArgs &args);
+ void expand_qualified_path_type (AST::QualifiedPathType &path_type);
+ void expand_closure_params (std::vector<AST::ClosureParam> ¶ms);
+ void expand_self_param (AST::SelfParam &self_param);
+ void expand_where_clause (AST::WhereClause &where_clause);
+ void expand_trait_function_decl (AST::TraitFunctionDecl &decl);
+ void expand_trait_method_decl (AST::TraitMethodDecl &decl);
+
+ /**
+ * Expand The current macro fragment recursively until it could not be
+ * expanded further.
+ *
+ * The return value checking works because correctly
+ * expanded fragment can never be an error (if the fragment can not be
+ * expanded, a stand-in error fragment will be returned; for fragments that
+ * could not be further expanded: the fragment prior to the expansion failure
+ * will be returned).
+ *
+ * @return Either the expanded fragment or an empty errored-out fragment
+ * indicating an expansion failure.
+ */
+ AST::ASTFragment expand_macro_fragment_recursive ()
+ {
+ auto fragment = expander.take_expanded_fragment (*this);
+ unsigned int original_depth = expander.expansion_depth;
+ auto final_fragment = AST::ASTFragment ({}, true);
+
+ while (fragment.should_expand ())
+ {
+ final_fragment = std::move (fragment);
+ expander.expansion_depth++;
+ // further expand the previously expanded macro fragment
+ auto new_fragment = expander.take_expanded_fragment (*this);
+ if (new_fragment.is_error ())
+ break;
+ fragment = std::move (new_fragment);
+ }
+ expander.expansion_depth = original_depth;
+ return final_fragment;
+ }
+
+ /**
+ * Expand a set of values, erasing them if they are marked for strip, and
+ * replacing them with expanded macro nodes if necessary.
+ * This function is slightly different from `expand_pointer_allow_strip` as
+ * it can only be called in certain expansion contexts - where macro
+ * invocations are allowed.
+ *
+ * @param ctx Context to use for macro expansion
+ * @param values Iterable reference over values to replace or erase
+ * @param extractor Function to call when replacing values with the content
+ * of an expanded AST node
+ */
+ template <typename T, typename U>
+ void expand_macro_children (MacroExpander::ContextType ctx, T &values,
+ std::function<U (AST::SingleASTNode)> extractor)
+ {
+ expander.push_context (ctx);
+
+ for (auto it = values.begin (); it != values.end ();)
+ {
+ auto &value = *it;
+
+ // mark for stripping if required
+ value->accept_vis (*this);
+
+ // recursively expand the children
+ auto final_fragment = expand_macro_fragment_recursive ();
+
+ if (final_fragment.should_expand ())
+ {
+ it = values.erase (it);
+ for (auto &node : final_fragment.get_nodes ())
+ {
+ auto new_node = extractor (node);
+ if (new_node != nullptr && !new_node->is_marked_for_strip ())
+ {
+ it = values.insert (it, std::move (new_node));
+ it++;
+ }
+ }
+ }
+ else if (value->is_marked_for_strip ())
+ {
+ it = values.erase (it);
+ }
+ else
+ {
+ ++it;
+ }
+ }
+
+ expander.pop_context ();
+ }
+
+ template <typename T> void expand_pointer_allow_strip (T &values)
+ {
+ for (auto it = values.begin (); it != values.end ();)
+ {
+ auto &value = *it;
+
+ // mark for stripping if required
+ value->accept_vis (*this);
+ if (value->is_marked_for_strip ())
+ {
+ it = values.erase (it);
+ }
+ else
+ {
+ ++it;
+ }
+ }
+ }
+
+ void visit (AST::Token &) override;
+ void visit (AST::DelimTokenTree &) override;
+ void visit (AST::AttrInputMetaItemContainer &) override;
+ void visit (AST::IdentifierExpr &ident_expr) override;
+ void visit (AST::Lifetime &) override;
+ void visit (AST::LifetimeParam &) override;
+ void visit (AST::ConstGenericParam &) override;
+
+ void visit (AST::MacroInvocation ¯o_invoc) override;
+
+ void visit (AST::PathInExpression &path) override;
+ void visit (AST::TypePathSegment &) override;
+ void visit (AST::TypePathSegmentGeneric &segment) override;
+ void visit (AST::TypePathSegmentFunction &segment) override;
+ void visit (AST::TypePath &path) override;
+ void visit (AST::QualifiedPathInExpression &path) override;
+ void visit (AST::QualifiedPathInType &path) override;
+
+ void visit (AST::LiteralExpr &expr) override;
+ void visit (AST::AttrInputLiteral &) override;
+ void visit (AST::MetaItemLitExpr &) override;
+ void visit (AST::MetaItemPathLit &) override;
+ void visit (AST::BorrowExpr &expr) override;
+ void visit (AST::DereferenceExpr &expr) override;
+ void visit (AST::ErrorPropagationExpr &expr) override;
+ void visit (AST::NegationExpr &expr) override;
+ void visit (AST::ArithmeticOrLogicalExpr &expr) override;
+ void visit (AST::ComparisonExpr &expr) override;
+ void visit (AST::LazyBooleanExpr &expr) override;
+ void visit (AST::TypeCastExpr &expr) override;
+ void visit (AST::AssignmentExpr &expr) override;
+ void visit (AST::CompoundAssignmentExpr &expr) override;
+ void visit (AST::GroupedExpr &expr) override;
+ void visit (AST::ArrayElemsValues &elems) override;
+ void visit (AST::ArrayElemsCopied &elems) override;
+ void visit (AST::ArrayExpr &expr) override;
+ void visit (AST::ArrayIndexExpr &expr) override;
+ void visit (AST::TupleExpr &expr) override;
+ void visit (AST::TupleIndexExpr &expr) override;
+ void visit (AST::StructExprStruct &expr) override;
+ void visit (AST::StructExprFieldIdentifier &) override;
+ void visit (AST::StructExprFieldIdentifierValue &field) override;
+
+ void visit (AST::StructExprFieldIndexValue &field) override;
+ void visit (AST::StructExprStructFields &expr) override;
+ void visit (AST::StructExprStructBase &expr) override;
+ void visit (AST::CallExpr &expr) override;
+ void visit (AST::MethodCallExpr &expr) override;
+ void visit (AST::FieldAccessExpr &expr) override;
+ void visit (AST::ClosureExprInner &expr) override;
+
+ void visit (AST::BlockExpr &expr) override;
+
+ void visit (AST::ClosureExprInnerTyped &expr) override;
+ void visit (AST::ContinueExpr &expr) override;
+ void visit (AST::BreakExpr &expr) override;
+ void visit (AST::RangeFromToExpr &expr) override;
+ void visit (AST::RangeFromExpr &expr) override;
+ void visit (AST::RangeToExpr &expr) override;
+ void visit (AST::RangeFullExpr &) override;
+ void visit (AST::RangeFromToInclExpr &expr) override;
+ void visit (AST::RangeToInclExpr &expr) override;
+ void visit (AST::ReturnExpr &expr) override;
+ void visit (AST::UnsafeBlockExpr &expr) override;
+ void visit (AST::LoopExpr &expr) override;
+ void visit (AST::WhileLoopExpr &expr) override;
+ void visit (AST::WhileLetLoopExpr &expr) override;
+ void visit (AST::ForLoopExpr &expr) override;
+ void visit (AST::IfExpr &expr) override;
+ void visit (AST::IfExprConseqElse &expr) override;
+ void visit (AST::IfExprConseqIf &expr) override;
+ void visit (AST::IfExprConseqIfLet &expr) override;
+ void visit (AST::IfLetExpr &expr) override;
+ void visit (AST::IfLetExprConseqElse &expr) override;
+ void visit (AST::IfLetExprConseqIf &expr) override;
+ void visit (AST::IfLetExprConseqIfLet &expr) override;
+ void visit (AST::MatchExpr &expr) override;
+ void visit (AST::AwaitExpr &expr) override;
+ void visit (AST::AsyncBlockExpr &expr) override;
+ void visit (AST::TypeParam ¶m) override;
+ void visit (AST::LifetimeWhereClauseItem &) override;
+ void visit (AST::TypeBoundWhereClauseItem &item) override;
+ void visit (AST::Method &method) override;
+ void visit (AST::Module &module) override;
+ void visit (AST::ExternCrate &crate) override;
+ void visit (AST::UseTreeGlob &) override;
+ void visit (AST::UseTreeList &) override;
+ void visit (AST::UseTreeRebind &) override;
+ void visit (AST::UseDeclaration &use_decl) override;
+ void visit (AST::Function &function) override;
+ void visit (AST::TypeAlias &type_alias) override;
+ void visit (AST::StructStruct &struct_item) override;
+ void visit (AST::TupleStruct &tuple_struct) override;
+ void visit (AST::EnumItem &item) override;
+ void visit (AST::EnumItemTuple &item) override;
+ void visit (AST::EnumItemStruct &item) override;
+ void visit (AST::EnumItemDiscriminant &item) override;
+ void visit (AST::Enum &enum_item) override;
+ void visit (AST::Union &union_item) override;
+ void visit (AST::ConstantItem &const_item) override;
+ void visit (AST::StaticItem &static_item) override;
+ void visit (AST::TraitItemFunc &item) override;
+ void visit (AST::TraitItemMethod &item) override;
+ void visit (AST::TraitItemConst &item) override;
+ void visit (AST::TraitItemType &item) override;
+ void visit (AST::Trait &trait) override;
+ void visit (AST::InherentImpl &impl) override;
+ void visit (AST::TraitImpl &impl) override;
+ void visit (AST::ExternalStaticItem &item) override;
+ void visit (AST::ExternalFunctionItem &item) override;
+ void visit (AST::ExternBlock &block) override;
+
+ // I don't think it would be possible to strip macros without expansion
+ void visit (AST::MacroMatchFragment &) override;
+ void visit (AST::MacroMatchRepetition &) override;
+ void visit (AST::MacroMatcher &) override;
+ void visit (AST::MacroRulesDefinition &rules_def) override;
+ void visit (AST::MetaItemPath &) override;
+ void visit (AST::MetaItemSeq &) override;
+ void visit (AST::MetaWord &) override;
+ void visit (AST::MetaNameValueStr &) override;
+ void visit (AST::MetaListPaths &) override;
+ void visit (AST::MetaListNameValueStr &) override;
+ void visit (AST::LiteralPattern &) override;
+ void visit (AST::IdentifierPattern &pattern) override;
+ void visit (AST::WildcardPattern &) override;
+ void visit (AST::RangePatternBoundLiteral &) override;
+ void visit (AST::RangePatternBoundPath &bound) override;
+ void visit (AST::RangePatternBoundQualPath &bound) override;
+ void visit (AST::RangePattern &pattern) override;
+ void visit (AST::ReferencePattern &pattern) override;
+ void visit (AST::StructPatternFieldTuplePat &field) override;
+ void visit (AST::StructPatternFieldIdentPat &field) override;
+ void visit (AST::StructPatternFieldIdent &field) override;
+ void visit (AST::StructPattern &pattern) override;
+ void visit (AST::TupleStructItemsNoRange &tuple_items) override;
+ void visit (AST::TupleStructItemsRange &tuple_items) override;
+ void visit (AST::TupleStructPattern &pattern) override;
+ void visit (AST::TuplePatternItemsMultiple &tuple_items) override;
+ void visit (AST::TuplePatternItemsRanged &tuple_items) override;
+ void visit (AST::TuplePattern &pattern) override;
+ void visit (AST::GroupedPattern &pattern) override;
+ void visit (AST::SlicePattern &pattern) override;
+
+ void visit (AST::EmptyStmt &) override;
+ void visit (AST::LetStmt &stmt) override;
+ void visit (AST::ExprStmtWithoutBlock &stmt) override;
+ void visit (AST::ExprStmtWithBlock &stmt) override;
+
+ void visit (AST::TraitBound &bound) override;
+ void visit (AST::ImplTraitType &type) override;
+ void visit (AST::TraitObjectType &type) override;
+ void visit (AST::ParenthesisedType &type) override;
+ void visit (AST::ImplTraitTypeOneBound &type) override;
+ void visit (AST::TraitObjectTypeOneBound &type) override;
+ void visit (AST::TupleType &type) override;
+ void visit (AST::NeverType &) override;
+ void visit (AST::RawPointerType &type) override;
+ void visit (AST::ReferenceType &type) override;
+ void visit (AST::ArrayType &type) override;
+ void visit (AST::SliceType &type) override;
+ void visit (AST::InferredType &) override;
+ void visit (AST::BareFunctionType &type) override;
+};
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,484 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#include "rust-macro-builtins.h"
+#include "rust-diagnostics.h"
+#include "rust-expr.h"
+#include "rust-session-manager.h"
+#include "rust-macro-invoc-lexer.h"
+#include "rust-lex.h"
+#include "rust-parse.h"
+
+namespace Rust {
+namespace {
+std::unique_ptr<AST::Expr>
+make_string (Location locus, std::string value)
+{
+ return std::unique_ptr<AST::Expr> (
+ new AST::LiteralExpr (value, AST::Literal::STRING,
+ PrimitiveCoreType::CORETYPE_STR, {}, locus));
+}
+
+/* Match the end token of a macro given the start delimiter of the macro */
+
+static inline TokenId
+macro_end_token (AST::DelimTokenTree &invoc_token_tree,
+ Parser<MacroInvocLexer> &parser)
+{
+ auto last_token_id = TokenId::RIGHT_CURLY;
+ switch (invoc_token_tree.get_delim_type ())
+ {
+ case AST::DelimType::PARENS:
+ last_token_id = TokenId::RIGHT_PAREN;
+ rust_assert (parser.skip_token (LEFT_PAREN));
+ break;
+
+ case AST::DelimType::CURLY:
+ rust_assert (parser.skip_token (LEFT_CURLY));
+ break;
+
+ case AST::DelimType::SQUARE:
+ last_token_id = TokenId::RIGHT_SQUARE;
+ rust_assert (parser.skip_token (LEFT_SQUARE));
+ break;
+ }
+
+ return last_token_id;
+}
+
+/* Parse a single string literal from the given delimited token tree,
+ and return the LiteralExpr for it. Allow for an optional trailing comma,
+ but otherwise enforce that these are the only tokens. */
+
+std::unique_ptr<AST::LiteralExpr>
+parse_single_string_literal (AST::DelimTokenTree &invoc_token_tree,
+ Location invoc_locus)
+{
+ MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
+ Parser<MacroInvocLexer> parser (lex);
+
+ auto last_token_id = macro_end_token (invoc_token_tree, parser);
+
+ std::unique_ptr<AST::LiteralExpr> lit_expr = nullptr;
+
+ if (parser.peek_current_token ()->get_id () == STRING_LITERAL)
+ {
+ lit_expr = parser.parse_literal_expr ();
+ parser.maybe_skip_token (COMMA);
+ if (parser.peek_current_token ()->get_id () != last_token_id)
+ {
+ lit_expr = nullptr;
+ rust_error_at (invoc_locus, "macro takes 1 argument");
+ }
+ }
+ else if (parser.peek_current_token ()->get_id () == last_token_id)
+ rust_error_at (invoc_locus, "macro takes 1 argument");
+ else
+ rust_error_at (invoc_locus, "argument must be a string literal");
+
+ parser.skip_token (last_token_id);
+
+ return lit_expr;
+}
+
+/* Treat PATH as a path relative to the source file currently being
+ compiled, and return the absolute path for it. */
+
+std::string
+source_relative_path (std::string path, Location locus)
+{
+ std::string compile_fname
+ = Session::get_instance ().linemap->location_file (locus);
+
+ auto dir_separator_pos = compile_fname.rfind (file_separator);
+
+ /* If there is no file_separator in the path, use current dir ('.'). */
+ std::string dirname;
+ if (dir_separator_pos == std::string::npos)
+ dirname = std::string (".") + file_separator;
+ else
+ dirname = compile_fname.substr (0, dir_separator_pos) + file_separator;
+
+ return dirname + path;
+}
+
+/* Read the full contents of the file FILENAME and return them in a vector.
+ FIXME: platform specific. */
+
+std::vector<uint8_t>
+load_file_bytes (const char *filename)
+{
+ RAIIFile file_wrap (filename);
+ if (file_wrap.get_raw () == nullptr)
+ {
+ rust_error_at (Location (), "cannot open filename %s: %m", filename);
+ return std::vector<uint8_t> ();
+ }
+
+ FILE *f = file_wrap.get_raw ();
+ fseek (f, 0L, SEEK_END);
+ long fsize = ftell (f);
+ fseek (f, 0L, SEEK_SET);
+
+ std::vector<uint8_t> buf (fsize);
+
+ if (fread (&buf[0], fsize, 1, f) != 1)
+ {
+ rust_error_at (Location (), "error reading file %s: %m", filename);
+ return std::vector<uint8_t> ();
+ }
+
+ return buf;
+}
+} // namespace
+
+AST::ASTFragment
+MacroBuiltin::assert (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ rust_debug ("assert!() called");
+
+ return AST::ASTFragment::create_error ();
+}
+
+AST::ASTFragment
+MacroBuiltin::file (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto current_file
+ = Session::get_instance ().linemap->location_file (invoc_locus);
+ auto file_str = AST::SingleASTNode (make_string (invoc_locus, current_file));
+
+ return AST::ASTFragment ({file_str});
+}
+
+AST::ASTFragment
+MacroBuiltin::column (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto current_column
+ = Session::get_instance ().linemap->location_to_column (invoc_locus);
+
+ auto column_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
+ new AST::LiteralExpr (std::to_string (current_column), AST::Literal::INT,
+ PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
+
+ return AST::ASTFragment ({column_no});
+}
+
+/* Expand builtin macro include_bytes!("filename"), which includes the contents
+ of the given file as reference to a byte array. Yields an expression of type
+ &'static [u8; N]. */
+
+AST::ASTFragment
+MacroBuiltin::include_bytes (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ /* Get target filename from the macro invocation, which is treated as a path
+ relative to the include!-ing file (currently being compiled). */
+ auto lit_expr
+ = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
+ if (lit_expr == nullptr)
+ return AST::ASTFragment::create_error ();
+
+ std::string target_filename
+ = source_relative_path (lit_expr->as_string (), invoc_locus);
+
+ std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
+
+ /* Is there a more efficient way to do this? */
+ std::vector<std::unique_ptr<AST::Expr>> elts;
+ for (uint8_t b : bytes)
+ {
+ elts.emplace_back (
+ new AST::LiteralExpr (std::string (1, (char) b), AST::Literal::BYTE,
+ PrimitiveCoreType::CORETYPE_U8,
+ {} /* outer_attrs */, invoc_locus));
+ }
+
+ auto elems = std::unique_ptr<AST::ArrayElems> (
+ new AST::ArrayElemsValues (std::move (elts), invoc_locus));
+
+ auto array = std::unique_ptr<AST::Expr> (
+ new AST::ArrayExpr (std::move (elems), {}, {}, invoc_locus));
+
+ auto borrow = std::unique_ptr<AST::Expr> (
+ new AST::BorrowExpr (std::move (array), false, false, {}, invoc_locus));
+
+ auto node = AST::SingleASTNode (std::move (borrow));
+ return AST::ASTFragment ({node});
+}
+
+/* Expand builtin macro include_str!("filename"), which includes the contents
+ of the given file as a string. The file must be UTF-8 encoded. Yields an
+ expression of type &'static str. */
+
+AST::ASTFragment
+MacroBuiltin::include_str (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ /* Get target filename from the macro invocation, which is treated as a path
+ relative to the include!-ing file (currently being compiled). */
+ auto lit_expr
+ = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
+ if (lit_expr == nullptr)
+ return AST::ASTFragment::create_error ();
+
+ std::string target_filename
+ = source_relative_path (lit_expr->as_string (), invoc_locus);
+
+ std::vector<uint8_t> bytes = load_file_bytes (target_filename.c_str ());
+
+ /* FIXME: Enforce that the file contents are valid UTF-8. */
+ std::string str ((const char *) &bytes[0], bytes.size ());
+
+ auto node = AST::SingleASTNode (make_string (invoc_locus, str));
+ return AST::ASTFragment ({node});
+}
+
+/* Expand builtin macro compile_error!("error"), which forces a compile error
+ during the compile time. */
+AST::ASTFragment
+MacroBuiltin::compile_error (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto lit_expr
+ = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
+ if (lit_expr == nullptr)
+ return AST::ASTFragment::create_error ();
+
+ std::string error_string = lit_expr->as_string ();
+ rust_error_at (invoc_locus, "%s", error_string.c_str ());
+
+ return AST::ASTFragment::create_error ();
+}
+
+/* Expand builtin macro concat!(), which joins all the literal parameters
+ into a string with no delimiter. */
+
+AST::ASTFragment
+MacroBuiltin::concat (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto invoc_token_tree = invoc.get_delim_tok_tree ();
+ MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
+ Parser<MacroInvocLexer> parser (lex);
+
+ auto str = std::string ();
+ bool has_error = false;
+
+ auto last_token_id = macro_end_token (invoc_token_tree, parser);
+
+ /* NOTE: concat! could accept no argument, so we don't have any checks here */
+ while (parser.peek_current_token ()->get_id () != last_token_id)
+ {
+ auto lit_expr = parser.parse_literal_expr ();
+ if (lit_expr)
+ {
+ str += lit_expr->as_string ();
+ }
+ else
+ {
+ auto current_token = parser.peek_current_token ();
+ rust_error_at (current_token->get_locus (),
+ "argument must be a constant literal");
+ has_error = true;
+ // Just crash if the current token can't be skipped
+ rust_assert (parser.skip_token (current_token->get_id ()));
+ }
+ parser.maybe_skip_token (COMMA);
+ }
+
+ parser.skip_token (last_token_id);
+
+ if (has_error)
+ return AST::ASTFragment::create_error ();
+
+ auto node = AST::SingleASTNode (make_string (invoc_locus, str));
+ return AST::ASTFragment ({node});
+}
+
+/* Expand builtin macro env!(), which inspects an environment variable at
+ compile time. */
+
+AST::ASTFragment
+MacroBuiltin::env (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto invoc_token_tree = invoc.get_delim_tok_tree ();
+ MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
+ Parser<MacroInvocLexer> parser (lex);
+
+ auto last_token_id = macro_end_token (invoc_token_tree, parser);
+
+ if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
+ {
+ if (parser.peek_current_token ()->get_id () == last_token_id)
+ rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
+ else
+ rust_error_at (parser.peek_current_token ()->get_locus (),
+ "argument must be a string literal");
+ return AST::ASTFragment::create_error ();
+ }
+
+ auto lit_expr = parser.parse_literal_expr ();
+ auto comma_skipped = parser.maybe_skip_token (COMMA);
+
+ std::unique_ptr<AST::LiteralExpr> error_expr = nullptr;
+
+ if (parser.peek_current_token ()->get_id () != last_token_id)
+ {
+ if (!comma_skipped)
+ {
+ rust_error_at (parser.peek_current_token ()->get_locus (),
+ "expected token: %<,%>");
+ return AST::ASTFragment::create_error ();
+ }
+ if (parser.peek_current_token ()->get_id () != STRING_LITERAL)
+ {
+ rust_error_at (parser.peek_current_token ()->get_locus (),
+ "argument must be a string literal");
+ return AST::ASTFragment::create_error ();
+ }
+
+ error_expr = parser.parse_literal_expr ();
+ parser.maybe_skip_token (COMMA);
+ }
+
+ if (parser.peek_current_token ()->get_id () != last_token_id)
+ {
+ rust_error_at (invoc_locus, "env! takes 1 or 2 arguments");
+ return AST::ASTFragment::create_error ();
+ }
+
+ parser.skip_token (last_token_id);
+
+ auto env_value = getenv (lit_expr->as_string ().c_str ());
+
+ if (env_value == nullptr)
+ {
+ if (error_expr == nullptr)
+ rust_error_at (invoc_locus, "environment variable %qs not defined",
+ lit_expr->as_string ().c_str ());
+ else
+ rust_error_at (invoc_locus, "%s", error_expr->as_string ().c_str ());
+ return AST::ASTFragment::create_error ();
+ }
+
+ auto node = AST::SingleASTNode (make_string (invoc_locus, env_value));
+ return AST::ASTFragment ({node});
+}
+
+AST::ASTFragment
+MacroBuiltin::cfg (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ // only parse if not already parsed
+ if (!invoc.is_parsed ())
+ {
+ std::unique_ptr<AST::AttrInputMetaItemContainer> converted_input (
+ invoc.get_delim_tok_tree ().parse_to_meta_item ());
+
+ if (converted_input == nullptr)
+ {
+ rust_debug ("DEBUG: failed to parse macro to meta item");
+ // TODO: do something now? is this an actual error?
+ }
+ else
+ {
+ std::vector<std::unique_ptr<AST::MetaItemInner>> meta_items (
+ std::move (converted_input->get_items ()));
+ invoc.set_meta_item_output (std::move (meta_items));
+ }
+ }
+
+ /* TODO: assuming that cfg! macros can only have one meta item inner, like cfg
+ * attributes */
+ if (invoc.get_meta_items ().size () != 1)
+ return AST::ASTFragment::create_error ();
+
+ bool result = invoc.get_meta_items ()[0]->check_cfg_predicate (
+ Session::get_instance ());
+ auto literal_exp = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
+ new AST::LiteralExpr (result ? "true" : "false", AST::Literal::BOOL,
+ PrimitiveCoreType::CORETYPE_BOOL, {}, invoc_locus)));
+
+ return AST::ASTFragment ({literal_exp});
+}
+
+/* Expand builtin macro include!(), which includes a source file at the current
+ scope compile time. */
+
+AST::ASTFragment
+MacroBuiltin::include (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ /* Get target filename from the macro invocation, which is treated as a path
+ relative to the include!-ing file (currently being compiled). */
+ auto lit_expr
+ = parse_single_string_literal (invoc.get_delim_tok_tree (), invoc_locus);
+ if (lit_expr == nullptr)
+ return AST::ASTFragment::create_error ();
+
+ std::string filename
+ = source_relative_path (lit_expr->as_string (), invoc_locus);
+ auto target_filename
+ = Rust::Session::get_instance ().include_extra_file (std::move (filename));
+
+ RAIIFile target_file (target_filename);
+ Linemap *linemap = Session::get_instance ().linemap;
+
+ if (!target_file.ok ())
+ {
+ rust_error_at (lit_expr->get_locus (),
+ "cannot open included file %qs: %m", target_filename);
+ return AST::ASTFragment::create_error ();
+ }
+
+ rust_debug ("Attempting to parse included file %s", target_filename);
+
+ Lexer lex (target_filename, std::move (target_file), linemap);
+ Parser<Lexer> parser (lex);
+
+ auto parsed_items = parser.parse_items ();
+ bool has_error = !parser.get_errors ().empty ();
+
+ for (const auto &error : parser.get_errors ())
+ error.emit_error ();
+
+ if (has_error)
+ {
+ // inform the user that the errors above are from a included file
+ rust_inform (invoc_locus, "included from here");
+ return AST::ASTFragment::create_error ();
+ }
+
+ std::vector<AST::SingleASTNode> nodes{};
+ for (auto &item : parsed_items)
+ {
+ AST::SingleASTNode node (std::move (item));
+ nodes.push_back (node);
+ }
+
+ return AST::ASTFragment (nodes);
+}
+
+AST::ASTFragment
+MacroBuiltin::line (Location invoc_locus, AST::MacroInvocData &invoc)
+{
+ auto current_line
+ = Session::get_instance ().linemap->location_to_line (invoc_locus);
+
+ auto line_no = AST::SingleASTNode (std::unique_ptr<AST::Expr> (
+ new AST::LiteralExpr (std::to_string (current_line), AST::Literal::INT,
+ PrimitiveCoreType::CORETYPE_U32, {}, invoc_locus)));
+
+ return AST::ASTFragment ({line_no});
+}
+
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,107 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef RUST_MACRO_BUILTINS_H
+#define RUST_MACRO_BUILTINS_H
+
+#include "rust-ast.h"
+#include "rust-location.h"
+
+/**
+ * This class provides a list of builtin macros implemented by the compiler.
+ * The functions defined are called "builtin transcribers" in that they replace
+ * the transcribing part of a macro definition.
+ *
+ * Like regular macro transcribers, they are responsible for building and
+ * returning an AST fragment: basically a vector of AST nodes put together.
+ *
+ * Unlike regular declarative macros where each match arm has its own associated
+ * transcriber, builtin transcribers are responsible for handling all match arms
+ * of the macro. This means that you should take extra care when implementing a
+ * builtin containing multiple match arms: You will probably need to do some
+ * lookahead in order to determine which match arm the user intended to use.
+ *
+ * An example of this is the `assert!()` macro:
+ *
+ * ```
+ * macro_rules! assert {
+ * ($cond:expr $(,)?) => {{ ... }};
+ * ($cond : expr, $ ($arg : tt) +) = > {{ ... }};
+ * }
+ * ```
+ *
+ * If more tokens exist beyond the optional comma, they need to be handled as
+ * a token-tree for a custom panic message.
+ *
+ * These builtin macros with empty transcribers are defined in the standard
+ * library. They are marked with a special attribute, `#[rustc_builtin_macro]`.
+ * When this attribute is present on a macro definition, the compiler should
+ * look for an associated transcriber in the mappings. Meaning that you must
+ * remember to insert your transcriber in the `builtin_macros` map of the
+ *`Mappings`.
+ *
+ * This map is built as a static variable in the `insert_macro_def()` method
+ * of the `Mappings` class.
+ */
+
+/* If assert is defined as a macro this file will not parse, so undefine this
+ before continuing. */
+#ifdef assert
+#undef assert
+#endif
+
+namespace Rust {
+class MacroBuiltin
+{
+public:
+ static AST::ASTFragment assert (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment file (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment column (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment include_bytes (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment include_str (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment compile_error (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment concat (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment env (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment cfg (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment include (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+
+ static AST::ASTFragment line (Location invoc_locus,
+ AST::MacroInvocData &invoc);
+};
+} // namespace Rust
+
+#endif // RUST_MACRO_BUILTINS_H
new file mode 100644
@@ -0,0 +1,1012 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#include "rust-macro-expand.h"
+#include "rust-macro-substitute-ctx.h"
+#include "rust-ast-full.h"
+#include "rust-ast-visitor.h"
+#include "rust-diagnostics.h"
+#include "rust-parse.h"
+#include "rust-attribute-visitor.h"
+
+namespace Rust {
+AST::ASTFragment
+MacroExpander::expand_decl_macro (Location invoc_locus,
+ AST::MacroInvocData &invoc,
+ AST::MacroRulesDefinition &rules_def,
+ bool semicolon)
+{
+ // ensure that both invocation and rules are in a valid state
+ rust_assert (!invoc.is_marked_for_strip ());
+ rust_assert (!rules_def.is_marked_for_strip ());
+ rust_assert (rules_def.get_macro_rules ().size () > 0);
+
+ /* probably something here about parsing invoc and rules def token trees to
+ * token stream. if not, how would parser handle the captures of exprs and
+ * stuff? on the other hand, token trees may be kind of useful in rules def as
+ * creating a point where recursion can occur (like having
+ * "compare_macro_match" and then it calling itself when it finds delimiters)
+ */
+
+ /* find matching rule to invoc token tree, based on macro rule's matcher. if
+ * none exist, error.
+ * - specifically, check each matcher in order. if one fails to match, move
+ * onto next. */
+ /* TODO: does doing this require parsing expressions and whatever in the
+ * invoc? if so, might as well save the results if referenced using $ or
+ * whatever. If not, do another pass saving them. Except this is probably
+ * useless as different rules could have different starting points for exprs
+ * or whatever. Decision trees could avoid this, but they have their own
+ * issues. */
+ /* TODO: will need to modify the parser so that it can essentially "catch"
+ * errors - maybe "try_parse_expr" or whatever methods. */
+ // this technically creates a back-tracking parser - this will be the
+ // implementation style
+
+ /* then, after results are saved, generate the macro output from the
+ * transcriber token tree. if i understand this correctly, the macro
+ * invocation gets replaced by the transcriber tokens, except with
+ * substitutions made (e.g. for $i variables) */
+
+ /* TODO: it is probably better to modify AST::Token to store a pointer to a
+ * Lexer::Token (rather than being converted) - i.e. not so much have
+ * AST::Token as a Token but rather a TokenContainer (as it is another type of
+ * TokenTree). This will prevent re-conversion of Tokens between each type
+ * all the time, while still allowing the heterogenous storage of token trees.
+ */
+
+ AST::DelimTokenTree &invoc_token_tree = invoc.get_delim_tok_tree ();
+
+ // find matching arm
+ AST::MacroRule *matched_rule = nullptr;
+ std::map<std::string, MatchedFragmentContainer> matched_fragments;
+ for (auto &rule : rules_def.get_rules ())
+ {
+ sub_stack.push ();
+ bool did_match_rule = try_match_rule (rule, invoc_token_tree);
+ matched_fragments = sub_stack.pop ();
+
+ if (did_match_rule)
+ {
+ // // Debugging
+ // for (auto &kv : matched_fragments)
+ // rust_debug ("[fragment]: %s (%ld - %s)", kv.first.c_str (),
+ // kv.second.get_fragments ().size (),
+ // kv.second.get_kind ()
+ // == MatchedFragmentContainer::Kind::Repetition
+ // ? "repetition"
+ // : "metavar");
+
+ matched_rule = &rule;
+ break;
+ }
+ }
+
+ if (matched_rule == nullptr)
+ {
+ RichLocation r (invoc_locus);
+ r.add_range (rules_def.get_locus ());
+ rust_error_at (r, "Failed to match any rule within macro");
+ return AST::ASTFragment::create_error ();
+ }
+
+ return transcribe_rule (*matched_rule, invoc_token_tree, matched_fragments,
+ semicolon, peek_context ());
+}
+
+void
+MacroExpander::expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon)
+{
+ if (depth_exceeds_recursion_limit ())
+ {
+ rust_error_at (invoc.get_locus (), "reached recursion limit");
+ return;
+ }
+
+ AST::MacroInvocData &invoc_data = invoc.get_invoc_data ();
+
+ // ??
+ // switch on type of macro:
+ // - '!' syntax macro (inner switch)
+ // - procedural macro - "A token-based function-like macro"
+ // - 'macro_rules' (by example/pattern-match) macro? or not? "an
+ // AST-based function-like macro"
+ // - else is unreachable
+ // - attribute syntax macro (inner switch)
+ // - procedural macro attribute syntax - "A token-based attribute
+ // macro"
+ // - legacy macro attribute syntax? - "an AST-based attribute macro"
+ // - non-macro attribute: mark known
+ // - else is unreachable
+ // - derive macro (inner switch)
+ // - derive or legacy derive - "token-based" vs "AST-based"
+ // - else is unreachable
+ // - derive container macro - unreachable
+
+ // lookup the rules for this macro
+ NodeId resolved_node = UNKNOWN_NODEID;
+ NodeId source_node = UNKNOWN_NODEID;
+ if (has_semicolon)
+ source_node = invoc.get_macro_node_id ();
+ else
+ source_node = invoc.get_pattern_node_id ();
+ auto seg
+ = Resolver::CanonicalPath::new_seg (source_node,
+ invoc_data.get_path ().as_string ());
+
+ bool found = resolver->get_macro_scope ().lookup (seg, &resolved_node);
+ if (!found)
+ {
+ rust_error_at (invoc.get_locus (), "unknown macro: [%s]",
+ seg.get ().c_str ());
+ return;
+ }
+
+ // lookup the rules
+ AST::MacroRulesDefinition *rules_def = nullptr;
+ bool ok = mappings->lookup_macro_def (resolved_node, &rules_def);
+ rust_assert (ok);
+
+ auto fragment = AST::ASTFragment::create_error ();
+
+ if (rules_def->is_builtin ())
+ fragment
+ = rules_def->get_builtin_transcriber () (invoc.get_locus (), invoc_data);
+ else
+ fragment = expand_decl_macro (invoc.get_locus (), invoc_data, *rules_def,
+ has_semicolon);
+
+ set_expanded_fragment (std::move (fragment));
+}
+
+/* Determines whether any cfg predicate is false and hence item with attributes
+ * should be stripped. Note that attributes must be expanded before calling. */
+bool
+MacroExpander::fails_cfg (const AST::AttrVec &attrs) const
+{
+ for (const auto &attr : attrs)
+ {
+ if (attr.get_path () == "cfg" && !attr.check_cfg_predicate (session))
+ return true;
+ }
+ return false;
+}
+
+/* Determines whether any cfg predicate is false and hence item with attributes
+ * should be stripped. Will expand attributes as well. */
+bool
+MacroExpander::fails_cfg_with_expand (AST::AttrVec &attrs) const
+{
+ // TODO: maybe have something that strips cfg attributes that evaluate true?
+ for (auto &attr : attrs)
+ {
+ if (attr.get_path () == "cfg")
+ {
+ if (!attr.is_parsed_to_meta_item ())
+ attr.parse_attr_to_meta_item ();
+
+ // DEBUG
+ if (!attr.is_parsed_to_meta_item ())
+ rust_debug ("failed to parse attr to meta item, right before "
+ "cfg predicate check");
+ else
+ rust_debug ("attr has been successfully parsed to meta item, "
+ "right before cfg predicate check");
+
+ if (!attr.check_cfg_predicate (session))
+ {
+ // DEBUG
+ rust_debug (
+ "cfg predicate failed for attribute: \033[0;31m'%s'\033[0m",
+ attr.as_string ().c_str ());
+
+ return true;
+ }
+ else
+ {
+ // DEBUG
+ rust_debug ("cfg predicate succeeded for attribute: "
+ "\033[0;31m'%s'\033[0m",
+ attr.as_string ().c_str ());
+ }
+ }
+ }
+ return false;
+}
+
+// Expands cfg_attr attributes.
+void
+MacroExpander::expand_cfg_attrs (AST::AttrVec &attrs)
+{
+ for (std::size_t i = 0; i < attrs.size (); i++)
+ {
+ auto &attr = attrs[i];
+ if (attr.get_path () == "cfg_attr")
+ {
+ if (!attr.is_parsed_to_meta_item ())
+ attr.parse_attr_to_meta_item ();
+
+ if (attr.check_cfg_predicate (session))
+ {
+ // split off cfg_attr
+ AST::AttrVec new_attrs = attr.separate_cfg_attrs ();
+
+ // remove attr from vector
+ attrs.erase (attrs.begin () + i);
+
+ // add new attrs to vector
+ attrs.insert (attrs.begin () + i,
+ std::make_move_iterator (new_attrs.begin ()),
+ std::make_move_iterator (new_attrs.end ()));
+ }
+
+ /* do something - if feature (first token in tree) is in fact enabled,
+ * make tokens listed afterwards into attributes. i.e.: for
+ * [cfg_attr(feature = "wow", wow1, wow2)], if "wow" is true, then add
+ * attributes [wow1] and [wow2] to attribute list. This can also be
+ * recursive, so check for expanded attributes being recursive and
+ * possibly recursively call the expand_attrs? */
+ }
+ else
+ {
+ i++;
+ }
+ }
+ attrs.shrink_to_fit ();
+}
+
+void
+MacroExpander::expand_crate ()
+{
+ NodeId scope_node_id = crate.get_node_id ();
+ resolver->get_macro_scope ().push (scope_node_id);
+
+ /* fill macro/decorator map from init list? not sure where init list comes
+ * from? */
+
+ // TODO: does cfg apply for inner attributes? research.
+ // the apparent answer (from playground test) is yes
+
+ // expand crate cfg_attr attributes
+ expand_cfg_attrs (crate.inner_attrs);
+
+ if (fails_cfg_with_expand (crate.inner_attrs))
+ {
+ // basically, delete whole crate
+ crate.strip_crate ();
+ // TODO: maybe create warning here? probably not desired behaviour
+ }
+ // expand module attributes?
+
+ push_context (ITEM);
+
+ // expand attributes recursively and strip items if required
+ AttrVisitor attr_visitor (*this);
+ auto &items = crate.items;
+ for (auto it = items.begin (); it != items.end ();)
+ {
+ auto &item = *it;
+
+ // mark for stripping if required
+ item->accept_vis (attr_visitor);
+
+ auto fragment = take_expanded_fragment (attr_visitor);
+ if (fragment.should_expand ())
+ {
+ // Remove the current expanded invocation
+ it = items.erase (it);
+ for (auto &node : fragment.get_nodes ())
+ {
+ it = items.insert (it, node.take_item ());
+ it++;
+ }
+ }
+ else if (item->is_marked_for_strip ())
+ it = items.erase (it);
+ else
+ it++;
+ }
+
+ pop_context ();
+
+ // TODO: should recursive attribute and macro expansion be done in the same
+ // transversal? Or in separate ones like currently?
+
+ // expand module tree recursively
+
+ // post-process
+
+ // extract exported macros?
+}
+
+bool
+MacroExpander::depth_exceeds_recursion_limit () const
+{
+ return expansion_depth >= cfg.recursion_limit;
+}
+
+bool
+MacroExpander::try_match_rule (AST::MacroRule &match_rule,
+ AST::DelimTokenTree &invoc_token_tree)
+{
+ MacroInvocLexer lex (invoc_token_tree.to_token_stream ());
+ Parser<MacroInvocLexer> parser (lex);
+
+ AST::MacroMatcher &matcher = match_rule.get_matcher ();
+
+ expansion_depth++;
+ if (!match_matcher (parser, matcher))
+ {
+ expansion_depth--;
+ return false;
+ }
+ expansion_depth--;
+
+ bool used_all_input_tokens = parser.skip_token (END_OF_FILE);
+ return used_all_input_tokens;
+}
+
+bool
+MacroExpander::match_fragment (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchFragment &fragment)
+{
+ switch (fragment.get_frag_spec ().get_kind ())
+ {
+ case AST::MacroFragSpec::EXPR:
+ parser.parse_expr ();
+ break;
+
+ case AST::MacroFragSpec::BLOCK:
+ parser.parse_block_expr ();
+ break;
+
+ case AST::MacroFragSpec::IDENT:
+ parser.parse_identifier_pattern ();
+ break;
+
+ case AST::MacroFragSpec::LITERAL:
+ parser.parse_literal_expr ();
+ break;
+
+ case AST::MacroFragSpec::ITEM:
+ parser.parse_item (false);
+ break;
+
+ case AST::MacroFragSpec::TY:
+ parser.parse_type ();
+ break;
+
+ case AST::MacroFragSpec::PAT:
+ parser.parse_pattern ();
+ break;
+
+ case AST::MacroFragSpec::PATH:
+ parser.parse_path_in_expression ();
+ break;
+
+ case AST::MacroFragSpec::VIS:
+ parser.parse_visibility ();
+ break;
+
+ case AST::MacroFragSpec::STMT: {
+ auto restrictions = ParseRestrictions ();
+ restrictions.consume_semi = false;
+ parser.parse_stmt (restrictions);
+ break;
+ }
+
+ case AST::MacroFragSpec::LIFETIME:
+ parser.parse_lifetime_params ();
+ break;
+
+ // is meta attributes?
+ case AST::MacroFragSpec::META:
+ parser.parse_attribute_body ();
+ break;
+
+ case AST::MacroFragSpec::TT:
+ parser.parse_token_tree ();
+ break;
+
+ // i guess we just ignore invalid and just error out
+ case AST::MacroFragSpec::INVALID:
+ return false;
+ }
+
+ // it matches if the parser did not produce errors trying to parse that type
+ // of item
+ return !parser.has_errors ();
+}
+
+bool
+MacroExpander::match_matcher (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatcher &matcher)
+{
+ if (depth_exceeds_recursion_limit ())
+ {
+ rust_error_at (matcher.get_match_locus (), "reached recursion limit");
+ return false;
+ }
+
+ auto delimiter = parser.peek_current_token ();
+
+ // this is used so we can check that we delimit the stream correctly.
+ switch (delimiter->get_id ())
+ {
+ case LEFT_PAREN: {
+ if (!parser.skip_token (LEFT_PAREN))
+ return false;
+ }
+ break;
+
+ case LEFT_SQUARE: {
+ if (!parser.skip_token (LEFT_SQUARE))
+ return false;
+ }
+ break;
+
+ case LEFT_CURLY: {
+ if (!parser.skip_token (LEFT_CURLY))
+ return false;
+ }
+ break;
+ default:
+ gcc_unreachable ();
+ }
+
+ const MacroInvocLexer &source = parser.get_token_source ();
+
+ for (auto &match : matcher.get_matches ())
+ {
+ size_t offs_begin = source.get_offs ();
+
+ switch (match->get_macro_match_type ())
+ {
+ case AST::MacroMatch::MacroMatchType::Fragment: {
+ AST::MacroMatchFragment *fragment
+ = static_cast<AST::MacroMatchFragment *> (match.get ());
+ if (!match_fragment (parser, *fragment))
+ return false;
+
+ // matched fragment get the offset in the token stream
+ size_t offs_end = source.get_offs ();
+ sub_stack.insert_metavar (
+ MatchedFragment (fragment->get_ident (), offs_begin, offs_end));
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Tok: {
+ AST::Token *tok = static_cast<AST::Token *> (match.get ());
+ if (!match_token (parser, *tok))
+ return false;
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Repetition: {
+ AST::MacroMatchRepetition *rep
+ = static_cast<AST::MacroMatchRepetition *> (match.get ());
+ if (!match_repetition (parser, *rep))
+ return false;
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Matcher: {
+ AST::MacroMatcher *m
+ = static_cast<AST::MacroMatcher *> (match.get ());
+ expansion_depth++;
+ if (!match_matcher (parser, *m))
+ {
+ expansion_depth--;
+ return false;
+ }
+ expansion_depth--;
+ }
+ break;
+ }
+ }
+
+ switch (delimiter->get_id ())
+ {
+ case LEFT_PAREN: {
+ if (!parser.skip_token (RIGHT_PAREN))
+ return false;
+ }
+ break;
+
+ case LEFT_SQUARE: {
+ if (!parser.skip_token (RIGHT_SQUARE))
+ return false;
+ }
+ break;
+
+ case LEFT_CURLY: {
+ if (!parser.skip_token (RIGHT_CURLY))
+ return false;
+ }
+ break;
+ default:
+ gcc_unreachable ();
+ }
+
+ return true;
+}
+
+bool
+MacroExpander::match_token (Parser<MacroInvocLexer> &parser, AST::Token &token)
+{
+ // FIXME this needs to actually match the content and the type
+ return parser.skip_token (token.get_id ());
+}
+
+bool
+MacroExpander::match_n_matches (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchRepetition &rep,
+ size_t &match_amount, size_t lo_bound,
+ size_t hi_bound)
+{
+ match_amount = 0;
+ auto &matches = rep.get_matches ();
+
+ const MacroInvocLexer &source = parser.get_token_source ();
+ while (true)
+ {
+ // If the current token is a closing macro delimiter, break away.
+ // TODO: Is this correct?
+ auto t_id = parser.peek_current_token ()->get_id ();
+ if (t_id == RIGHT_PAREN || t_id == RIGHT_SQUARE || t_id == RIGHT_CURLY)
+ break;
+
+ // Skip parsing a separator on the first match, otherwise consume it.
+ // If it isn't present, this is an error
+ if (rep.has_sep () && match_amount > 0)
+ if (!match_token (parser, *rep.get_sep ()))
+ break;
+
+ bool valid_current_match = false;
+ for (auto &match : matches)
+ {
+ size_t offs_begin = source.get_offs ();
+ switch (match->get_macro_match_type ())
+ {
+ case AST::MacroMatch::MacroMatchType::Fragment: {
+ AST::MacroMatchFragment *fragment
+ = static_cast<AST::MacroMatchFragment *> (match.get ());
+ valid_current_match = match_fragment (parser, *fragment);
+
+ // matched fragment get the offset in the token stream
+ size_t offs_end = source.get_offs ();
+
+ // The main difference with match_matcher happens here: Instead
+ // of inserting a new fragment, we append to one. If that
+ // fragment does not exist, then the operation is similar to
+ // `insert_fragment` with the difference that we are not
+ // creating a metavariable, but a repetition of one, which is
+ // really different.
+ sub_stack.append_fragment (
+ MatchedFragment (fragment->get_ident (), offs_begin,
+ offs_end));
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Tok: {
+ AST::Token *tok = static_cast<AST::Token *> (match.get ());
+ valid_current_match = match_token (parser, *tok);
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Repetition: {
+ AST::MacroMatchRepetition *rep
+ = static_cast<AST::MacroMatchRepetition *> (match.get ());
+ valid_current_match = match_repetition (parser, *rep);
+ }
+ break;
+
+ case AST::MacroMatch::MacroMatchType::Matcher: {
+ AST::MacroMatcher *m
+ = static_cast<AST::MacroMatcher *> (match.get ());
+ valid_current_match = match_matcher (parser, *m);
+ }
+ break;
+ }
+ }
+ // If we've encountered an error once, stop trying to match more
+ // repetitions
+ if (!valid_current_match)
+ break;
+
+ match_amount++;
+
+ // Break early if we notice there's too many expressions already
+ if (hi_bound && match_amount > hi_bound)
+ break;
+ }
+
+ // Check if the amount of matches we got is valid: Is it more than the lower
+ // bound and less than the higher bound?
+ bool did_meet_lo_bound = match_amount >= lo_bound;
+ bool did_meet_hi_bound = hi_bound ? match_amount <= hi_bound : true;
+
+ // If the end-result is valid, then we can clear the parse errors: Since
+ // repetitions are parsed eagerly, it is okay to fail in some cases
+ auto res = did_meet_lo_bound && did_meet_hi_bound;
+ if (res)
+ parser.clear_errors ();
+
+ return res;
+}
+
+bool
+MacroExpander::match_repetition (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchRepetition &rep)
+{
+ size_t match_amount = 0;
+ bool res = false;
+
+ std::string lo_str;
+ std::string hi_str;
+ switch (rep.get_op ())
+ {
+ case AST::MacroMatchRepetition::MacroRepOp::ANY:
+ lo_str = "0";
+ hi_str = "+inf";
+ res = match_n_matches (parser, rep, match_amount);
+ break;
+ case AST::MacroMatchRepetition::MacroRepOp::ONE_OR_MORE:
+ lo_str = "1";
+ hi_str = "+inf";
+ res = match_n_matches (parser, rep, match_amount, 1);
+ break;
+ case AST::MacroMatchRepetition::MacroRepOp::ZERO_OR_ONE:
+ lo_str = "0";
+ hi_str = "1";
+ res = match_n_matches (parser, rep, match_amount, 0, 1);
+ break;
+ default:
+ gcc_unreachable ();
+ }
+
+ if (!res)
+ rust_error_at (rep.get_match_locus (),
+ "invalid amount of matches for macro invocation. Expected "
+ "between %s and %s, got %lu",
+ lo_str.c_str (), hi_str.c_str (),
+ (unsigned long) match_amount);
+
+ rust_debug_loc (rep.get_match_locus (), "%s matched %lu times",
+ res ? "successfully" : "unsuccessfully",
+ (unsigned long) match_amount);
+
+ // We have to handle zero fragments differently: They will not have been
+ // "matched" but they are still valid and should be inserted as a special
+ // case. So we go through the stack map, and for every fragment which doesn't
+ // exist, insert a zero-matched fragment.
+ auto &stack_map = sub_stack.peek ();
+ for (auto &match : rep.get_matches ())
+ {
+ if (match->get_macro_match_type ()
+ == AST::MacroMatch::MacroMatchType::Fragment)
+ {
+ auto fragment = static_cast<AST::MacroMatchFragment *> (match.get ());
+ auto it = stack_map.find (fragment->get_ident ());
+
+ if (it == stack_map.end ())
+ sub_stack.insert_matches (fragment->get_ident (),
+ MatchedFragmentContainer::zero ());
+ }
+ }
+
+ return res;
+}
+
+/**
+ * Helper function to refactor calling a parsing function 0 or more times
+ */
+static AST::ASTFragment
+parse_many (Parser<MacroInvocLexer> &parser, TokenId &delimiter,
+ std::function<AST::SingleASTNode ()> parse_fn)
+{
+ std::vector<AST::SingleASTNode> nodes;
+ while (true)
+ {
+ if (parser.peek_current_token ()->get_id () == delimiter)
+ break;
+
+ auto node = parse_fn ();
+ nodes.emplace_back (std::move (node));
+ }
+
+ return AST::ASTFragment (std::move (nodes));
+}
+
+/**
+ * Transcribe 0 or more items from a macro invocation
+ *
+ * @param parser Parser to extract items from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
+{
+ return parse_many (parser, delimiter, [&parser] () {
+ auto item = parser.parse_item (true);
+ return AST::SingleASTNode (std::move (item));
+ });
+}
+
+/**
+ * Transcribe 0 or more external items from a macro invocation
+ *
+ * @param parser Parser to extract items from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_ext (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
+{
+ return parse_many (parser, delimiter, [&parser] () {
+ auto item = parser.parse_external_item ();
+ return AST::SingleASTNode (std::move (item));
+ });
+}
+
+/**
+ * Transcribe 0 or more trait items from a macro invocation
+ *
+ * @param parser Parser to extract items from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_trait_items (Parser<MacroInvocLexer> &parser,
+ TokenId &delimiter)
+{
+ return parse_many (parser, delimiter, [&parser] () {
+ auto item = parser.parse_trait_item ();
+ return AST::SingleASTNode (std::move (item));
+ });
+}
+
+/**
+ * Transcribe 0 or more impl items from a macro invocation
+ *
+ * @param parser Parser to extract items from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_impl_items (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
+{
+ return parse_many (parser, delimiter, [&parser] () {
+ auto item = parser.parse_inherent_impl_item ();
+ return AST::SingleASTNode (std::move (item));
+ });
+}
+
+/**
+ * Transcribe 0 or more trait impl items from a macro invocation
+ *
+ * @param parser Parser to extract items from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_trait_impl_items (Parser<MacroInvocLexer> &parser,
+ TokenId &delimiter)
+{
+ return parse_many (parser, delimiter, [&parser] () {
+ auto item = parser.parse_trait_impl_item ();
+ return AST::SingleASTNode (std::move (item));
+ });
+}
+
+/**
+ * Transcribe 0 or more statements from a macro invocation
+ *
+ * @param parser Parser to extract statements from
+ * @param delimiter Id of the token on which parsing should stop
+ */
+static AST::ASTFragment
+transcribe_many_stmts (Parser<MacroInvocLexer> &parser, TokenId &delimiter)
+{
+ auto restrictions = ParseRestrictions ();
+ restrictions.consume_semi = false;
+
+ // FIXME: This is invalid! It needs to also handle cases where the macro
+ // transcriber is an expression, but since the macro call is followed by
+ // a semicolon, it's a valid ExprStmt
+ return parse_many (parser, delimiter, [&parser, restrictions] () {
+ auto stmt = parser.parse_stmt (restrictions);
+ return AST::SingleASTNode (std::move (stmt));
+ });
+}
+
+/**
+ * Transcribe one expression from a macro invocation
+ *
+ * @param parser Parser to extract statements from
+ */
+static AST::ASTFragment
+transcribe_expression (Parser<MacroInvocLexer> &parser)
+{
+ auto expr = parser.parse_expr ();
+
+ return AST::ASTFragment ({std::move (expr)});
+}
+
+/**
+ * Transcribe one type from a macro invocation
+ *
+ * @param parser Parser to extract statements from
+ */
+static AST::ASTFragment
+transcribe_type (Parser<MacroInvocLexer> &parser)
+{
+ auto type = parser.parse_type ();
+
+ return AST::ASTFragment ({std::move (type)});
+}
+
+static AST::ASTFragment
+transcribe_on_delimiter (Parser<MacroInvocLexer> &parser, bool semicolon,
+ AST::DelimType delimiter, TokenId last_token_id)
+{
+ if (semicolon || delimiter == AST::DelimType::CURLY)
+ return transcribe_many_stmts (parser, last_token_id);
+ else
+ return transcribe_expression (parser);
+} // namespace Rust
+
+static AST::ASTFragment
+transcribe_context (MacroExpander::ContextType ctx,
+ Parser<MacroInvocLexer> &parser, bool semicolon,
+ AST::DelimType delimiter, TokenId last_token_id)
+{
+ // The flow-chart in order to choose a parsing function is as follows:
+ //
+ // [switch special context]
+ // -- Item --> parser.parse_item();
+ // -- Trait --> parser.parse_trait_item();
+ // -- Impl --> parser.parse_impl_item();
+ // -- Extern --> parser.parse_extern_item();
+ // -- None --> [has semicolon?]
+ // -- Yes --> parser.parse_stmt();
+ // -- No --> [switch invocation.delimiter()]
+ // -- { } --> parser.parse_stmt();
+ // -- _ --> parser.parse_expr(); // once!
+
+ // If there is a semicolon OR we are expanding a MacroInvocationSemi, then
+ // we can parse multiple items. Otherwise, parse *one* expression
+
+ switch (ctx)
+ {
+ case MacroExpander::ContextType::ITEM:
+ return transcribe_many_items (parser, last_token_id);
+ break;
+ case MacroExpander::ContextType::TRAIT:
+ return transcribe_many_trait_items (parser, last_token_id);
+ break;
+ case MacroExpander::ContextType::IMPL:
+ return transcribe_many_impl_items (parser, last_token_id);
+ break;
+ case MacroExpander::ContextType::TRAIT_IMPL:
+ return transcribe_many_trait_impl_items (parser, last_token_id);
+ break;
+ case MacroExpander::ContextType::EXTERN:
+ return transcribe_many_ext (parser, last_token_id);
+ break;
+ case MacroExpander::ContextType::TYPE:
+ return transcribe_type (parser);
+ break;
+ default:
+ return transcribe_on_delimiter (parser, semicolon, delimiter,
+ last_token_id);
+ }
+}
+
+static std::string
+tokens_to_str (std::vector<std::unique_ptr<AST::Token>> &tokens)
+{
+ std::string str;
+ if (!tokens.empty ())
+ {
+ str += tokens[0]->as_string ();
+ for (size_t i = 1; i < tokens.size (); i++)
+ str += " " + tokens[i]->as_string ();
+ }
+
+ return str;
+}
+
+AST::ASTFragment
+MacroExpander::transcribe_rule (
+ AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
+ std::map<std::string, MatchedFragmentContainer> &matched_fragments,
+ bool semicolon, ContextType ctx)
+{
+ // we can manipulate the token tree to substitute the dollar identifiers so
+ // that when we call parse its already substituted for us
+ AST::MacroTranscriber &transcriber = match_rule.get_transcriber ();
+ AST::DelimTokenTree &transcribe_tree = transcriber.get_token_tree ();
+
+ auto invoc_stream = invoc_token_tree.to_token_stream ();
+ auto macro_rule_tokens = transcribe_tree.to_token_stream ();
+
+ auto substitute_context
+ = SubstituteCtx (invoc_stream, macro_rule_tokens, matched_fragments);
+ std::vector<std::unique_ptr<AST::Token>> substituted_tokens
+ = substitute_context.substitute_tokens ();
+
+ rust_debug ("substituted tokens: %s",
+ tokens_to_str (substituted_tokens).c_str ());
+
+ // parse it to an ASTFragment
+ MacroInvocLexer lex (std::move (substituted_tokens));
+ Parser<MacroInvocLexer> parser (lex);
+
+ auto last_token_id = TokenId::RIGHT_CURLY;
+
+ // this is used so we can check that we delimit the stream correctly.
+ switch (transcribe_tree.get_delim_type ())
+ {
+ case AST::DelimType::PARENS:
+ last_token_id = TokenId::RIGHT_PAREN;
+ rust_assert (parser.skip_token (LEFT_PAREN));
+ break;
+
+ case AST::DelimType::CURLY:
+ rust_assert (parser.skip_token (LEFT_CURLY));
+ break;
+
+ case AST::DelimType::SQUARE:
+ last_token_id = TokenId::RIGHT_SQUARE;
+ rust_assert (parser.skip_token (LEFT_SQUARE));
+ break;
+ }
+
+ // see https://github.com/Rust-GCC/gccrs/issues/22
+ // TL;DR:
+ // - Treat all macro invocations with parentheses, (), or square brackets,
+ // [], as expressions.
+ // - If the macro invocation has curly brackets, {}, it may be parsed as a
+ // statement depending on the context.
+ // - If the macro invocation has a semicolon at the end, it must be parsed
+ // as a statement (either via ExpressionStatement or
+ // MacroInvocationWithSemi)
+
+ auto fragment
+ = transcribe_context (ctx, parser, semicolon,
+ invoc_token_tree.get_delim_type (), last_token_id);
+
+ // emit any errors
+ if (parser.has_errors ())
+ {
+ for (auto &err : parser.get_errors ())
+ rust_error_at (err.locus, "%s", err.message.c_str ());
+ return AST::ASTFragment::create_error ();
+ }
+
+ // are all the tokens used?
+ bool did_delimit = parser.skip_token (last_token_id);
+
+ bool reached_end_of_stream = did_delimit && parser.skip_token (END_OF_FILE);
+ if (!reached_end_of_stream)
+ {
+ const_TokenPtr current_token = parser.peek_current_token ();
+ rust_error_at (current_token->get_locus (),
+ "tokens here and after are unparsed");
+ }
+
+ return fragment;
+}
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,366 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef RUST_MACRO_EXPAND_H
+#define RUST_MACRO_EXPAND_H
+
+#include "rust-buffered-queue.h"
+#include "rust-parse.h"
+#include "rust-token.h"
+#include "rust-ast.h"
+#include "rust-macro.h"
+#include "rust-hir-map.h"
+#include "rust-name-resolver.h"
+#include "rust-macro-invoc-lexer.h"
+
+// Provides objects and method prototypes for macro expansion
+
+namespace Rust {
+// forward decls for AST
+namespace AST {
+class MacroInvocation;
+}
+
+// Object used to store configuration data for macro expansion.
+// NOTE: Keep all these items complying with the latest rustc.
+struct ExpansionCfg
+{
+ // features?
+ // TODO: Add `features' when we have it.
+ unsigned int recursion_limit = 1024;
+ bool trace_mac = false; // trace macro
+ bool should_test = false; // strip #[test] nodes if false
+ bool keep_macs = false; // keep macro definitions
+ std::string crate_name = "";
+};
+
+struct MatchedFragment
+{
+ std::string fragment_ident;
+ size_t token_offset_begin;
+ size_t token_offset_end;
+
+ MatchedFragment (std::string identifier, size_t token_offset_begin,
+ size_t token_offset_end)
+ : fragment_ident (identifier), token_offset_begin (token_offset_begin),
+ token_offset_end (token_offset_end)
+ {}
+
+ /**
+ * Empty constructor for uninitialized fragments
+ */
+ MatchedFragment () : MatchedFragment ("", 0, 0) {}
+
+ std::string as_string () const
+ {
+ return fragment_ident + "=" + std::to_string (token_offset_begin) + ":"
+ + std::to_string (token_offset_end);
+ }
+};
+
+class MatchedFragmentContainer
+{
+public:
+ // Does the container refer to a simple metavariable, different from a
+ // repetition repeated once
+ enum class Kind
+ {
+ MetaVar,
+ Repetition,
+ };
+
+ MatchedFragmentContainer (std::vector<MatchedFragment> fragments,
+ Kind kind = Kind::Repetition)
+ : fragments (fragments), kind (kind)
+ {}
+
+ /**
+ * Create a valid fragment matched zero times. This is useful for repetitions
+ * which allow the absence of a fragment, such as * and ?
+ */
+ static MatchedFragmentContainer zero ()
+ {
+ return MatchedFragmentContainer ({});
+ }
+
+ /**
+ * Create a valid fragment matched one time
+ */
+ static MatchedFragmentContainer metavar (MatchedFragment fragment)
+ {
+ return MatchedFragmentContainer ({fragment}, Kind::MetaVar);
+ }
+
+ /**
+ * Add a matched fragment to the container
+ */
+ void add_fragment (MatchedFragment fragment)
+ {
+ rust_assert (!is_single_fragment ());
+
+ fragments.emplace_back (fragment);
+ }
+
+ size_t get_match_amount () const { return fragments.size (); }
+ const std::vector<MatchedFragment> &get_fragments () const
+ {
+ return fragments;
+ }
+ // const std::string &get_fragment_name () const { return fragment_name; }
+
+ bool is_single_fragment () const
+ {
+ return get_match_amount () == 1 && kind == Kind::MetaVar;
+ }
+
+ const MatchedFragment get_single_fragment () const
+ {
+ rust_assert (is_single_fragment ());
+
+ return fragments[0];
+ }
+
+ const Kind &get_kind () const { return kind; }
+
+private:
+ /**
+ * Fragments matched `match_amount` times. This can be an empty vector
+ * in case having zero matches is allowed (i.e ? or * operators)
+ */
+ std::vector<MatchedFragment> fragments;
+ Kind kind;
+};
+
+class SubstitutionScope
+{
+public:
+ SubstitutionScope () : stack () {}
+
+ void push () { stack.push_back ({}); }
+
+ std::map<std::string, MatchedFragmentContainer> pop ()
+ {
+ auto top = stack.back ();
+ stack.pop_back ();
+ return top;
+ }
+
+ std::map<std::string, MatchedFragmentContainer> &peek ()
+ {
+ return stack.back ();
+ }
+
+ /**
+ * Insert a new matched metavar into the current substitution map
+ */
+ void insert_metavar (MatchedFragment fragment)
+ {
+ auto ¤t_map = stack.back ();
+ auto it = current_map.find (fragment.fragment_ident);
+
+ if (it == current_map.end ())
+ current_map.insert ({fragment.fragment_ident,
+ MatchedFragmentContainer::metavar (fragment)});
+ else
+ gcc_unreachable ();
+ }
+
+ /**
+ * Append a new matched fragment to a repetition into the current substitution
+ * map
+ */
+ void append_fragment (MatchedFragment fragment)
+ {
+ auto ¤t_map = stack.back ();
+ auto it = current_map.find (fragment.fragment_ident);
+
+ if (it == current_map.end ())
+ current_map.insert (
+ {fragment.fragment_ident, MatchedFragmentContainer ({fragment})});
+ else
+ it->second.add_fragment (fragment);
+ }
+
+ void insert_matches (std::string key, MatchedFragmentContainer matches)
+ {
+ auto ¤t_map = stack.back ();
+ auto it = current_map.find (key);
+ rust_assert (it == current_map.end ());
+
+ current_map.insert ({key, matches});
+ }
+
+private:
+ std::vector<std::map<std::string, MatchedFragmentContainer>> stack;
+};
+
+// Object used to store shared data (between functions) for macro expansion.
+struct MacroExpander
+{
+ enum ContextType
+ {
+ ITEM,
+ BLOCK,
+ EXTERN,
+ TYPE,
+ TRAIT,
+ IMPL,
+ TRAIT_IMPL,
+ };
+
+ ExpansionCfg cfg;
+ unsigned int expansion_depth = 0;
+
+ MacroExpander (AST::Crate &crate, ExpansionCfg cfg, Session &session)
+ : cfg (cfg), crate (crate), session (session),
+ sub_stack (SubstitutionScope ()),
+ expanded_fragment (AST::ASTFragment::create_error ()),
+ resolver (Resolver::Resolver::get ()),
+ mappings (Analysis::Mappings::get ())
+ {}
+
+ ~MacroExpander () = default;
+
+ // Expands all macros in the crate passed in.
+ void expand_crate ();
+
+ /* Expands a macro invocation - possibly make both
+ * have similar duck-typed interface and use templates?*/
+ // should this be public or private?
+ void expand_invoc (AST::MacroInvocation &invoc, bool has_semicolon);
+
+ // Expands a single declarative macro.
+ AST::ASTFragment expand_decl_macro (Location locus,
+ AST::MacroInvocData &invoc,
+ AST::MacroRulesDefinition &rules_def,
+ bool semicolon);
+
+ void expand_cfg_attrs (AST::AttrVec &attrs);
+ bool fails_cfg (const AST::AttrVec &attr) const;
+ bool fails_cfg_with_expand (AST::AttrVec &attrs) const;
+
+ bool depth_exceeds_recursion_limit () const;
+
+ bool try_match_rule (AST::MacroRule &match_rule,
+ AST::DelimTokenTree &invoc_token_tree);
+
+ AST::ASTFragment transcribe_rule (
+ AST::MacroRule &match_rule, AST::DelimTokenTree &invoc_token_tree,
+ std::map<std::string, MatchedFragmentContainer> &matched_fragments,
+ bool semicolon, ContextType ctx);
+
+ bool match_fragment (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchFragment &fragment);
+
+ bool match_token (Parser<MacroInvocLexer> &parser, AST::Token &token);
+
+ bool match_repetition (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchRepetition &rep);
+
+ bool match_matcher (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatcher &matcher);
+
+ /**
+ * Match any amount of matches
+ *
+ * @param parser Parser to use for matching
+ * @param rep Repetition to try and match
+ * @param match_amount Reference in which to store the ammount of succesful
+ * and valid matches
+ *
+ * @param lo_bound Lower bound of the matcher. When specified, the matcher
+ * will only succeed if it parses at *least* `lo_bound` fragments. If
+ * unspecified, the matcher could succeed when parsing 0 fragments.
+ *
+ * @param hi_bound Higher bound of the matcher. When specified, the matcher
+ * will only succeed if it parses *less than* `hi_bound` fragments. If
+ * unspecified, the matcher could succeed when parsing an infinity of
+ * fragments.
+ *
+ * @return true if matching was successful and within the given limits, false
+ * otherwise
+ */
+ bool match_n_matches (Parser<MacroInvocLexer> &parser,
+ AST::MacroMatchRepetition &rep, size_t &match_amount,
+ size_t lo_bound = 0, size_t hi_bound = 0);
+
+ void push_context (ContextType t) { context.push_back (t); }
+
+ ContextType pop_context ()
+ {
+ rust_assert (!context.empty ());
+
+ ContextType t = context.back ();
+ context.pop_back ();
+
+ return t;
+ }
+
+ ContextType peek_context () { return context.back (); }
+
+ void set_expanded_fragment (AST::ASTFragment &&fragment)
+ {
+ expanded_fragment = std::move (fragment);
+ }
+
+ AST::ASTFragment take_expanded_fragment (AST::ASTVisitor &vis)
+ {
+ AST::ASTFragment old_fragment = std::move (expanded_fragment);
+ auto accumulator = std::vector<AST::SingleASTNode> ();
+ expanded_fragment = AST::ASTFragment::create_error ();
+
+ for (auto &node : old_fragment.get_nodes ())
+ {
+ expansion_depth++;
+ node.accept_vis (vis);
+ // we'll decide the next move according to the outcome of the macro
+ // expansion
+ if (expanded_fragment.is_error ())
+ accumulator.push_back (node); // if expansion fails, there might be a
+ // non-macro expression we need to keep
+ else
+ {
+ // if expansion succeeded, then we need to merge the fragment with
+ // the contents in the accumulator, so that our final expansion
+ // result will contain non-macro nodes as it should
+ auto new_nodes = expanded_fragment.get_nodes ();
+ std::move (new_nodes.begin (), new_nodes.end (),
+ std::back_inserter (accumulator));
+ expanded_fragment = AST::ASTFragment (accumulator);
+ }
+ expansion_depth--;
+ }
+
+ return old_fragment;
+ }
+
+private:
+ AST::Crate &crate;
+ Session &session;
+ SubstitutionScope sub_stack;
+ std::vector<ContextType> context;
+ AST::ASTFragment expanded_fragment;
+
+public:
+ Resolver::Resolver *resolver;
+ Analysis::Mappings *mappings;
+};
+
+} // namespace Rust
+
+#endif
new file mode 100644
@@ -0,0 +1,29 @@
+#include "rust-macro-invoc-lexer.h"
+
+namespace Rust {
+
+const_TokenPtr
+MacroInvocLexer::peek_token (int n)
+{
+ if ((offs + n) >= token_stream.size ())
+ return Token::make (END_OF_FILE, Location ());
+
+ return token_stream.at (offs + n)->get_tok_ptr ();
+}
+
+// Advances current token to n + 1 tokens ahead of current position.
+void
+MacroInvocLexer::skip_token (int n)
+{
+ offs += (n + 1);
+}
+
+void
+MacroInvocLexer::split_current_token (TokenId new_left __attribute__ ((unused)),
+ TokenId new_right
+ __attribute__ ((unused)))
+{
+ // FIXME
+ gcc_unreachable ();
+}
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,64 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#ifndef RUST_MACRO_INVOC_LEXER_H
+#define RUST_MACRO_INVOC_LEXER_H
+
+#include "rust-ast.h"
+
+namespace Rust {
+class MacroInvocLexer
+{
+public:
+ MacroInvocLexer (std::vector<std::unique_ptr<AST::Token>> stream)
+ : offs (0), token_stream (std::move (stream))
+ {}
+
+ // Returns token n tokens ahead of current position.
+ const_TokenPtr peek_token (int n);
+
+ // Peeks the current token.
+ const_TokenPtr peek_token () { return peek_token (0); }
+
+ // Advances current token to n + 1 tokens ahead of current position.
+ void skip_token (int n);
+
+ // Skips the current token.
+ void skip_token () { skip_token (0); }
+
+ // Splits the current token into two. Intended for use with nested generics
+ // closes (i.e. T<U<X>> where >> is wrongly lexed as one token). Note that
+ // this will only work with "simple" tokens like punctuation.
+ void split_current_token (TokenId new_left, TokenId new_right);
+
+ std::string get_filename () const
+ {
+ // FIXME
+ gcc_unreachable ();
+ return "FIXME";
+ }
+
+ size_t get_offs () const { return offs; }
+
+private:
+ size_t offs;
+ std::vector<std::unique_ptr<AST::Token>> token_stream;
+};
+} // namespace Rust
+
+#endif // RUST_MACRO_INVOC_LEXER_H
new file mode 100644
@@ -0,0 +1,312 @@
+#include "rust-macro-substitute-ctx.h"
+
+namespace Rust {
+
+std::vector<std::unique_ptr<AST::Token>>
+SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
+{
+ auto metavar_name = metavar->get_str ();
+
+ std::vector<std::unique_ptr<AST::Token>> expanded;
+ auto it = fragments.find (metavar_name);
+ if (it == fragments.end ())
+ {
+ // Return a copy of the original token
+ expanded.push_back (metavar->clone_token ());
+ }
+ else
+ {
+ // If we are expanding a metavar which has a lof of matches, we are
+ // currently expanding a repetition metavar - not a simple metavar. We
+ // need to error out and inform the user.
+ // Associated test case for an example: compile/macro-issue1224.rs
+ if (it->second.get_match_amount () != 1)
+ {
+ rust_error_at (metavar->get_locus (),
+ "metavariable is still repeating at this depth");
+ rust_inform (
+ metavar->get_locus (),
+ "you probably forgot the repetition operator: %<%s%s%s%>", "$(",
+ metavar->as_string ().c_str (), ")*");
+ return expanded;
+ }
+
+ // We only care about the vector when expanding repetitions.
+ // Just access the first element of the vector.
+ auto &frag = it->second.get_single_fragment ();
+ for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
+ offs++)
+ {
+ auto &tok = input.at (offs);
+ expanded.push_back (tok->clone_token ());
+ }
+ }
+
+ return expanded;
+}
+
+bool
+SubstituteCtx::check_repetition_amount (size_t pattern_start,
+ size_t pattern_end,
+ size_t &expected_repetition_amount)
+{
+ bool first_fragment_found = false;
+ bool is_valid = true;
+
+ for (size_t i = pattern_start; i < pattern_end; i++)
+ {
+ if (macro.at (i)->get_id () == DOLLAR_SIGN)
+ {
+ auto &frag_token = macro.at (i + 1);
+ if (frag_token->get_id () == IDENTIFIER)
+ {
+ auto it = fragments.find (frag_token->get_str ());
+ if (it == fragments.end ())
+ {
+ // If the repetition is not anything we know (ie no declared
+ // metavars, or metavars which aren't present in the
+ // fragment), we can just error out. No need to paste the
+ // tokens as if nothing had happened.
+ rust_error_at (frag_token->get_locus (),
+ "metavar %s used in repetition does not exist",
+ frag_token->get_str ().c_str ());
+
+ is_valid = false;
+ }
+
+ auto &fragment = it->second;
+
+ size_t repeat_amount = fragment.get_match_amount ();
+ if (!first_fragment_found)
+ {
+ first_fragment_found = true;
+ expected_repetition_amount = repeat_amount;
+ }
+ else
+ {
+ if (repeat_amount != expected_repetition_amount
+ && !fragment.is_single_fragment ())
+ {
+ rust_error_at (
+ frag_token->get_locus (),
+ "different amount of matches used in merged "
+ "repetitions: expected %lu, got %lu",
+ (unsigned long) expected_repetition_amount,
+ (unsigned long) repeat_amount);
+ is_valid = false;
+ }
+ }
+ }
+ }
+ }
+
+ return is_valid;
+}
+
+std::vector<std::unique_ptr<AST::Token>>
+SubstituteCtx::substitute_repetition (
+ size_t pattern_start, size_t pattern_end,
+ std::unique_ptr<AST::Token> separator_token)
+{
+ rust_assert (pattern_end < macro.size ());
+
+ size_t repeat_amount = 0;
+ if (!check_repetition_amount (pattern_start, pattern_end, repeat_amount))
+ return {};
+
+ rust_debug ("repetition amount to use: %lu", (unsigned long) repeat_amount);
+ std::vector<std::unique_ptr<AST::Token>> expanded;
+ std::vector<std::unique_ptr<AST::Token>> new_macro;
+
+ // We want to generate a "new macro" to substitute with. This new macro
+ // should contain only the tokens inside the pattern
+ for (size_t tok_idx = pattern_start; tok_idx < pattern_end; tok_idx++)
+ new_macro.emplace_back (macro.at (tok_idx)->clone_token ());
+
+ // Then, we want to create a subset of the matches so that
+ // `substitute_tokens()` can only see one fragment per metavar. Let's say we
+ // have the following user input: (1 145 'h')
+ // on the following match arm: ($($lit:literal)*)
+ // which causes the following matches: { "lit": [1, 145, 'h'] }
+ //
+ // The pattern (new_macro) is `$lit:literal`
+ // The first time we expand it, we want $lit to have the following token: 1
+ // The second time, 145
+ // The third and final time, 'h'
+ //
+ // In order to do so we must create "sub maps", which only contain parts of
+ // the original matches
+ // sub-maps: [ { "lit": 1 }, { "lit": 145 }, { "lit": 'h' } ]
+ //
+ // and give them to `substitute_tokens` one by one.
+
+ for (size_t i = 0; i < repeat_amount; i++)
+ {
+ std::map<std::string, MatchedFragmentContainer> sub_map;
+ for (auto &kv_match : fragments)
+ {
+ MatchedFragment sub_fragment;
+
+ // FIXME: Hack: If a fragment is not repeated, how does it fit in the
+ // submap? Do we really want to expand it? Is this normal behavior?
+ if (kv_match.second.is_single_fragment ())
+ sub_fragment = kv_match.second.get_single_fragment ();
+ else
+ sub_fragment = kv_match.second.get_fragments ()[i];
+
+ sub_map.insert (
+ {kv_match.first, MatchedFragmentContainer::metavar (sub_fragment)});
+ }
+
+ auto substitute_context = SubstituteCtx (input, new_macro, sub_map);
+ auto new_tokens = substitute_context.substitute_tokens ();
+
+ // Skip the first repetition, but add the separator to the expanded
+ // tokens if it is present
+ if (i != 0 && separator_token)
+ expanded.emplace_back (separator_token->clone_token ());
+
+ for (auto &new_token : new_tokens)
+ expanded.emplace_back (new_token->clone_token ());
+ }
+
+ // FIXME: We also need to make sure that all subsequent fragments
+ // contain the same amount of repetitions as the first one
+
+ return expanded;
+}
+
+static bool
+is_rep_op (std::unique_ptr<AST::Token> &tok)
+{
+ auto id = tok->get_id ();
+ return id == QUESTION_MARK || id == ASTERISK || id == PLUS;
+}
+
+std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
+SubstituteCtx::substitute_token (size_t token_idx)
+{
+ auto &token = macro.at (token_idx);
+ switch (token->get_id ())
+ {
+ case IDENTIFIER:
+ rust_debug ("expanding metavar: %s", token->get_str ().c_str ());
+ return {substitute_metavar (token), 1};
+ case LEFT_PAREN: {
+ // We need to parse up until the closing delimiter and expand this
+ // fragment->n times.
+ rust_debug ("expanding repetition");
+
+ // We're in a context where macro repetitions have already been
+ // parsed and validated: This means that
+ // 1/ There will be no delimiters as that is an error
+ // 2/ There are no fragment specifiers anymore, which prevents us
+ // from reusing parser functions.
+ //
+ // Repetition patterns are also special in that they cannot contain
+ // "rogue" delimiters: For example, this is invalid, as they are
+ // parsed as MacroMatches and must contain a correct amount of
+ // delimiters.
+ // `$($e:expr ) )`
+ // ^ rogue closing parenthesis
+ //
+ // With all of that in mind, we can simply skip ahead from one
+ // parenthesis to the other to find the pattern to expand. Of course,
+ // pairs of delimiters, including parentheses, are allowed.
+ // `$($e:expr ( ) )`
+ // Parentheses are the sole delimiter for which we need a special
+ // behavior since they delimit the repetition pattern
+
+ size_t pattern_start = token_idx + 1;
+ size_t pattern_end = pattern_start;
+ auto parentheses_stack = 0;
+ for (size_t idx = pattern_start; idx < macro.size (); idx++)
+ {
+ if (macro.at (idx)->get_id () == LEFT_PAREN)
+ {
+ parentheses_stack++;
+ }
+ else if (macro.at (idx)->get_id () == RIGHT_PAREN)
+ {
+ if (parentheses_stack == 0)
+ {
+ pattern_end = idx;
+ break;
+ }
+ parentheses_stack--;
+ }
+ }
+
+ // Unreachable case, but let's make sure we don't ever run into it
+ rust_assert (pattern_end != pattern_start);
+
+ std::unique_ptr<AST::Token> separator_token = nullptr;
+ if (pattern_end + 1 <= macro.size ())
+ {
+ auto &post_pattern_token = macro.at (pattern_end + 1);
+ if (!is_rep_op (post_pattern_token))
+ separator_token = post_pattern_token->clone_token ();
+ }
+
+ // Amount of tokens to skip
+ auto to_skip = 0;
+ // Parentheses
+ to_skip += 2;
+ // Repetition operator
+ to_skip += 1;
+ // Separator
+ if (separator_token)
+ to_skip += 1;
+
+ return {substitute_repetition (pattern_start, pattern_end,
+ std::move (separator_token)),
+ pattern_end - pattern_start + to_skip};
+ }
+ // TODO: We need to check if the $ was alone. In that case, do
+ // not error out: Simply act as if there was an empty identifier
+ // with no associated fragment and paste the dollar sign in the
+ // transcription. Unsure how to do that since we always have at
+ // least the closing curly brace after an empty $...
+ default:
+ rust_error_at (token->get_locus (),
+ "unexpected token in macro transcribe: expected "
+ "%<(%> or identifier after %<$%>, got %<%s%>",
+ get_token_description (token->get_id ()));
+ }
+
+ // FIXME: gcc_unreachable() error case?
+ return {std::vector<std::unique_ptr<AST::Token>> (), 0};
+}
+
+std::vector<std::unique_ptr<AST::Token>>
+SubstituteCtx::substitute_tokens ()
+{
+ std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
+ rust_debug ("expanding tokens");
+
+ for (size_t i = 0; i < macro.size (); i++)
+ {
+ auto &tok = macro.at (i);
+ if (tok->get_id () == DOLLAR_SIGN)
+ {
+ // Aaaaah, if only we had C++17 :)
+ // auto [expanded, tok_to_skip] = ...
+ auto p = substitute_token (i + 1);
+ auto expanded = std::move (p.first);
+ auto tok_to_skip = p.second;
+
+ i += tok_to_skip;
+
+ for (auto &token : expanded)
+ replaced_tokens.emplace_back (token->clone_token ());
+ }
+ else
+ {
+ replaced_tokens.emplace_back (tok->clone_token ());
+ }
+ }
+
+ return replaced_tokens;
+}
+
+} // namespace Rust
new file mode 100644
@@ -0,0 +1,93 @@
+// Copyright (C) 2020-2022 Free Software Foundation, Inc.
+
+// This file is part of GCC.
+
+// GCC is free software; you can redistribute it and/or modify it under
+// the terms of the GNU General Public License as published by the Free
+// Software Foundation; either version 3, or (at your option) any later
+// version.
+
+// GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+// WARRANTY; without even the implied warranty of MERCHANTABILITY or
+// FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+// for more details.
+
+// You should have received a copy of the GNU General Public License
+// along with GCC; see the file COPYING3. If not see
+// <http://www.gnu.org/licenses/>.
+
+#include "rust-ast.h"
+#include "rust-macro-expand.h"
+
+namespace Rust {
+class SubstituteCtx
+{
+ std::vector<std::unique_ptr<AST::Token>> &input;
+ std::vector<std::unique_ptr<AST::Token>> ¯o;
+ std::map<std::string, MatchedFragmentContainer> &fragments;
+
+ /**
+ * Find the repetition amount to use when expanding a repetition, and
+ * check that all fragments used respect that repetition amount
+ *
+ * @param pattern_start Start of the repetition pattern
+ * @param pattern_end End of the repetition pattern
+ * @param repeat_amount Reference to fill with the matched repetition amount
+ */
+ bool check_repetition_amount (size_t pattern_start, size_t pattern_end,
+ size_t &repeat_amount);
+
+public:
+ SubstituteCtx (std::vector<std::unique_ptr<AST::Token>> &input,
+ std::vector<std::unique_ptr<AST::Token>> ¯o,
+ std::map<std::string, MatchedFragmentContainer> &fragments)
+ : input (input), macro (macro), fragments (fragments)
+ {}
+
+ /**
+ * Substitute a metavariable by its given fragment in a transcribing context,
+ * i.e. replacing $var with the associated fragment.
+ *
+ * @param metavar Metavariable to try and replace
+ *
+ * @return A token containing the associated fragment expanded into tokens if
+ * any, or the cloned token if no fragment was associated
+ */
+ std::vector<std::unique_ptr<AST::Token>>
+ substitute_metavar (std::unique_ptr<AST::Token> &metavar);
+
+ /**
+ * Substitute a macro repetition by its given fragments
+ *
+ * @param pattern_start Start index of the pattern tokens
+ * @param pattern_end End index of the patterns tokens
+ * @param separator Optional separator to include when expanding tokens
+ *
+ * @return A vector containing the repeated pattern
+ */
+ std::vector<std::unique_ptr<AST::Token>>
+ substitute_repetition (size_t pattern_start, size_t pattern_end,
+ std::unique_ptr<AST::Token> separator);
+
+ /**
+ * Substitute a given token by its appropriate representation
+ *
+ * @param token_idx Current token to try and substitute
+ *
+ * @return A token containing the associated fragment expanded into tokens if
+ * any, or the cloned token if no fragment was associated, as well as the
+ * amount of tokens that should be skipped before the next invocation. Since
+ * this function may consume more than just one token, it is important to skip
+ * ahead of the input to avoid mis-substitutions
+ */
+ std::pair<std::vector<std::unique_ptr<AST::Token>>, size_t>
+ substitute_token (size_t token_idx);
+
+ /**
+ * Substitute all tokens by their appropriate representation
+ *
+ * @return A vector containing the substituted tokens
+ */
+ std::vector<std::unique_ptr<AST::Token>> substitute_tokens ();
+};
+} // namespace Rust