Struct rustc_ast::tokenstream::TokenStream
source · pub struct TokenStream(pub(crate) Lrc<Vec<TokenTree>>);
Expand description
A TokenStream
is an abstract sequence of tokens, organized into TokenTree
s.
The goal is for procedural macros to work with TokenStream
s and TokenTree
s
instead of a representation of the abstract syntax tree.
Today’s TokenTree
s can still contain AST via token::Interpolated
for
backwards compatibility.
Tuple Fields§
§0: Lrc<Vec<TokenTree>>
Implementations§
source§impl TokenStream
impl TokenStream
sourcepub fn add_comma(&self) -> Option<(TokenStream, Span)>
pub fn add_comma(&self) -> Option<(TokenStream, Span)>
Given a TokenStream
with a Stream
of only two arguments, return a new TokenStream
separating the two arguments with a comma for diagnostic suggestions.
source§impl TokenStream
impl TokenStream
pub fn new(tts: Vec<TokenTree>) -> TokenStream
pub fn is_empty(&self) -> bool
pub fn len(&self) -> usize
pub fn trees(&self) -> RefTokenTreeCursor<'_> ⓘ
pub fn into_trees(self) -> TokenTreeCursor
sourcepub fn eq_unspanned(&self, other: &TokenStream) -> bool
pub fn eq_unspanned(&self, other: &TokenStream) -> bool
Compares two TokenStream
s, checking equality without regarding span information.
sourcepub fn token_alone(kind: TokenKind, span: Span) -> TokenStream
pub fn token_alone(kind: TokenKind, span: Span) -> TokenStream
Create a token stream containing a single token with alone spacing. The
spacing used for the final token in a constructed stream doesn’t matter
because it’s never used. In practice we arbitrarily use
Spacing::Alone
.
pub fn from_ast(node: &(impl HasAttrs + HasTokens + Debug)) -> TokenStream
pub fn from_nonterminal_ast(nt: &Nonterminal) -> TokenStream
fn flatten_token(token: &Token, spacing: Spacing) -> TokenTree
fn flatten_token_tree(tree: &TokenTree) -> TokenTree
pub fn flattened(&self) -> TokenStream
fn try_glue_to_last(vec: &mut Vec<TokenTree>, tt: &TokenTree) -> bool
sourcepub fn push_tree(&mut self, tt: TokenTree)
pub fn push_tree(&mut self, tt: TokenTree)
Push tt
onto the end of the stream, possibly gluing it to the last
token. Uses make_mut
to maximize efficiency.
sourcepub fn push_stream(&mut self, stream: TokenStream)
pub fn push_stream(&mut self, stream: TokenStream)
Push stream
onto the end of the stream, possibly gluing the first
token tree to the last token. (No other token trees will be glued.)
Uses make_mut
to maximize efficiency.
pub fn chunks(&self, chunk_size: usize) -> Chunks<'_, TokenTree>
sourcepub fn desugar_doc_comments(&mut self)
pub fn desugar_doc_comments(&mut self)
Desugar doc comments like /// foo
in the stream into #[doc = r"foo"]
. Modifies the TokenStream
via Lrc::make_mut
, but as little
as possible.
Trait Implementations§
source§impl Clone for TokenStream
impl Clone for TokenStream
source§fn clone(&self) -> TokenStream
fn clone(&self) -> TokenStream
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moresource§impl Debug for TokenStream
impl Debug for TokenStream
source§impl<__D: SpanDecoder> Decodable<__D> for TokenStream
impl<__D: SpanDecoder> Decodable<__D> for TokenStream
source§impl Default for TokenStream
impl Default for TokenStream
source§fn default() -> TokenStream
fn default() -> TokenStream
source§impl<__E: SpanEncoder> Encodable<__E> for TokenStream
impl<__E: SpanEncoder> Encodable<__E> for TokenStream
source§impl FromIterator<TokenTree> for TokenStream
impl FromIterator<TokenTree> for TokenStream
source§impl<CTX> HashStable<CTX> for TokenStreamwhere
CTX: HashStableContext,
impl<CTX> HashStable<CTX> for TokenStreamwhere
CTX: HashStableContext,
fn hash_stable(&self, hcx: &mut CTX, hasher: &mut StableHasher)
source§impl PartialEq for TokenStream
impl PartialEq for TokenStream
source§fn eq(&self, other: &TokenStream) -> bool
fn eq(&self, other: &TokenStream) -> bool
self
and other
values to be equal, and is used
by ==
.impl Eq for TokenStream
Auto Trait Implementations§
impl Freeze for TokenStream
impl !RefUnwindSafe for TokenStream
impl !Send for TokenStream
impl !Sync for TokenStream
impl Unpin for TokenStream
impl !UnwindSafe for TokenStream
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
source§default unsafe fn clone_to_uninit(&self, dst: *mut T)
default unsafe fn clone_to_uninit(&self, dst: *mut T)
clone_to_uninit
)source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§impl<T> WithSubscriber for T
impl<T> WithSubscriber for T
source§fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
source§fn with_current_subscriber(self) -> WithDispatch<Self>
fn with_current_subscriber(self) -> WithDispatch<Self>
impl<'a, T> Captures<'a> for Twhere
T: ?Sized,
Layout§
Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain repr(...)
attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.
Size: 8 bytes