Enum rustc_ast::tokenstream::TokenTree

pub enum TokenTree {
    Token(Token, Spacing),
    Delimited(DelimSpan, DelimSpacing, Delimiter, TokenStream),
}

When the main Rust parser encounters a syntax-extension invocation, it parses the arguments to the invocation as a token tree. This is a very loose structure, such that all sorts of different AST fragments can be passed to syntax extensions using a uniform type.

If the syntax extension is an MBE macro, it will attempt to match its LHS token tree against the provided token tree, and if it finds a match, will transcribe the RHS token tree, splicing in any captured macro_parser::matched_nonterminals into the SubstNts it finds.

The RHS of an MBE macro is the only place SubstNts are substituted. Nothing special happens to misnamed or misplaced SubstNts.

Variants

Token(Token, Spacing)

A single token. Should never be OpenDelim or CloseDelim, because delimiters are implicitly represented by Delimited.

Delimited(DelimSpan, DelimSpacing, Delimiter, TokenStream)

A delimited sequence of token trees.

Implementations

impl TokenTree

pub fn eq_unspanned(&self, other: &TokenTree) -> bool

Checks if this TokenTree is equal to the other, regardless of span/spacing information.

pub fn span(&self) -> Span

Retrieves the TokenTree’s span.

pub fn token_alone(kind: TokenKind, span: Span) -> TokenTree

Create a TokenTree::Token with alone spacing.

pub fn token_joint(kind: TokenKind, span: Span) -> TokenTree

Create a TokenTree::Token with joint spacing.

pub fn token_joint_hidden(kind: TokenKind, span: Span) -> TokenTree

Create a TokenTree::Token with joint-hidden spacing.

pub fn uninterpolate(&self) -> Cow<'_, TokenTree>

Trait Implementations

impl Clone for TokenTree

fn clone(&self) -> TokenTree

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenTree

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<__D: SpanDecoder> Decodable<__D> for TokenTree

fn decode(__decoder: &mut __D) -> Self

impl<__E: SpanEncoder> Encodable<__E> for TokenTree

fn encode(&self, __encoder: &mut __E)

impl FromIterator<TokenTree> for TokenStream

fn from_iter<I: IntoIterator<Item = TokenTree>>(iter: I) -> Self

Creates a value from an iterator.

impl<__CTX> HashStable<__CTX> for TokenTree

where __CTX: HashStableContext,

fn hash_stable(&self, __hcx: &mut __CTX, __hasher: &mut StableHasher)

impl PartialEq for TokenTree

fn eq(&self, other: &TokenTree) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for TokenTree

Layout

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain repr(...) attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 32 bytes

Size for each variant:

• Token: 31 bytes
• Delimited: 31 bytes