Struct rustc_ast::tokenstream::TokenStream

pub struct TokenStream(pub Lrc<Vec<TreeAndJoint>>);

A TokenStream is an abstract sequence of tokens, organized into TokenTrees.

The goal is for procedural macros to work with TokenStreams and TokenTrees instead of a representation of the abstract syntax tree. Today's TokenTrees can still contain AST via token::Interpolated for back-compat.

Implementations

impl TokenStream

pub fn add_comma(&self) -> Option<(TokenStream, Span)>

Given a TokenStream with a Stream of only two arguments, return a new TokenStream separating the two arguments with a comma for diagnostic suggestions.

impl TokenStream

pub fn new(streams: Vec<TreeAndJoint>) -> TokenStream

pub fn is_empty(&self) -> bool

pub fn len(&self) -> usize

pub fn span(&self) -> Option<Span>

pub fn from_streams(streams: SmallVec<[TokenStream; 2]>) -> TokenStream

pub fn trees(&self) -> Cursorⓘ

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn into_trees(self) -> Cursorⓘ

Important traits for Cursor

impl Iterator for Cursor type Item = TokenTree;

pub fn eq_unspanned(&self, other: &TokenStream) -> bool

Compares two TokenStreams, checking equality without regarding span information.

pub fn map_enumerated<F: FnMut(usize, TokenTree) -> TokenTree>(
    self,
    f: F
) -> TokenStream

pub fn map<F: FnMut(TokenTree) -> TokenTree>(self, f: F) -> TokenStream

Trait Implementations

impl Clone for TokenStream

fn clone(&self) -> TokenStream

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenStream

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Decodable for TokenStream

fn decode<__D: Decoder>(d: &mut __D) -> Result<TokenStream, __D::Error>

impl Default for TokenStream

fn default() -> TokenStream

Returns the "default value" for a type.

impl Encodable for TokenStream

fn encode<__S: Encoder>(&self, s: &mut __S) -> Result<(), __S::Error>

impl Eq for TokenStream

impl From<TokenTree> for TokenStream

fn from(tree: TokenTree) -> TokenStream

Performs the conversion.

impl FromIterator<TokenTree> for TokenStream

fn from_iter<I: IntoIterator<Item = TokenTree>>(iter: I) -> Self

Creates a value from an iterator.

impl<CTX> HashStable<CTX> for TokenStream where
    CTX: HashStableContext, 

fn hash_stable(&self, hcx: &mut CTX, hasher: &mut StableHasher)

impl PartialEq<TokenStream> for TokenStream

fn eq(&self, other: &TokenStream) -> bool

This method tests for self and other values to be equal, and is used by ==.

#[must_use]fn ne(&self, other: &Rhs) -> bool

This method tests for !=.