rustc_ast::token

Enum TokenKind

source
pub enum TokenKind {
Show 39 variants Eq, Lt, Le, EqEq, Ne, Ge, Gt, AndAnd, OrOr, Not, Tilde, BinOp(BinOpToken), BinOpEq(BinOpToken), At, Dot, DotDot, DotDotDot, DotDotEq, Comma, Semi, Colon, PathSep, RArrow, LArrow, FatArrow, Pound, Dollar, Question, SingleQuote, OpenDelim(Delimiter), CloseDelim(Delimiter), Literal(Lit), Ident(Symbol, IdentIsRaw), NtIdent(Ident, IdentIsRaw), Lifetime(Symbol, IdentIsRaw), NtLifetime(Ident, IdentIsRaw), Interpolated(Lrc<Nonterminal>), DocComment(CommentKind, AttrStyle, Symbol), Eof,
}

Variants§

§

Eq

=

§

Lt

<

§

Le

<=

§

EqEq

==

§

Ne

!=

§

Ge

>=

§

Gt

>

§

AndAnd

&&

§

OrOr

||

§

Not

!

§

Tilde

~

§

BinOp(BinOpToken)

§

BinOpEq(BinOpToken)

§

At

@

§

Dot

.

§

DotDot

..

§

DotDotDot

...

§

DotDotEq

..=

§

Comma

,

§

Semi

;

§

Colon

:

§

PathSep

::

§

RArrow

->

§

LArrow

<-

§

FatArrow

=>

§

Pound

#

§

Dollar

$

§

Question

?

§

SingleQuote

Used by proc macros for representing lifetimes, not generated by lexer right now.

§

OpenDelim(Delimiter)

An opening delimiter (e.g., {).

§

CloseDelim(Delimiter)

A closing delimiter (e.g., }).

§

Literal(Lit)

§

Ident(Symbol, IdentIsRaw)

Identifier token. Do not forget about NtIdent when you want to match on identifiers. It’s recommended to use Token::(ident,uninterpolate,uninterpolated_span) to treat regular and interpolated identifiers in the same way.

§

NtIdent(Ident, IdentIsRaw)

This identifier (and its span) is the identifier passed to the declarative macro. The span in the surrounding Token is the span of the ident metavariable in the macro’s RHS.

§

Lifetime(Symbol, IdentIsRaw)

Lifetime identifier token. Do not forget about NtLifetime when you want to match on lifetime identifiers. It’s recommended to use Token::(lifetime,uninterpolate,uninterpolated_span) to treat regular and interpolated lifetime identifiers in the same way.

§

NtLifetime(Ident, IdentIsRaw)

This identifier (and its span) is the lifetime passed to the declarative macro. The span in the surrounding Token is the span of the lifetime metavariable in the macro’s RHS.

§

Interpolated(Lrc<Nonterminal>)

An embedded AST node, as produced by a macro. This only exists for historical reasons. We’d like to get rid of it, for multiple reasons.

  • It’s conceptually very strange. Saying a token can contain an AST node is like saying, in natural language, that a word can contain a sentence.
  • It requires special handling in a bunch of places in the parser.
  • It prevents Token from implementing Copy. It adds complexity and likely slows things down. Please don’t add new occurrences of this token kind!

The span in the surrounding Token is that of the metavariable in the macro’s RHS. The span within the Nonterminal is that of the fragment passed to the macro at the call site.

§

DocComment(CommentKind, AttrStyle, Symbol)

A doc comment token. Symbol is the doc comment’s data excluding its “quotes” (///, /**, etc) similarly to symbols in string literal tokens.

§

Eof

End Of File

Implementations§

source§

impl TokenKind

source

pub fn lit(kind: LitKind, symbol: Symbol, suffix: Option<Symbol>) -> TokenKind

source

pub fn break_two_token_op(&self, n: u32) -> Option<(TokenKind, TokenKind)>

An approximation to proc-macro-style single-character operators used by rustc parser. If the operator token can be broken into two tokens, the first of which has n (1 or 2) chars, then this function performs that operation, otherwise it returns None.

source

pub fn similar_tokens(&self) -> Option<Vec<TokenKind>>

Returns tokens that are likely to be typed accidentally instead of the current token. Enables better error recovery when the wrong token is found.

source

pub fn should_end_const_arg(&self) -> bool

Trait Implementations§

source§

impl Clone for TokenKind

source§

fn clone(&self) -> Self

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl Debug for TokenKind

source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
source§

impl<__D: SpanDecoder> Decodable<__D> for TokenKind

source§

fn decode(__decoder: &mut __D) -> Self

source§

impl<__E: SpanEncoder> Encodable<__E> for TokenKind

source§

fn encode(&self, __encoder: &mut __E)

source§

impl<__CTX> HashStable<__CTX> for TokenKind
where __CTX: HashStableContext,

source§

fn hash_stable(&self, __hcx: &mut __CTX, __hasher: &mut StableHasher)

source§

impl PartialEq<TokenKind> for Token

source§

fn eq(&self, rhs: &TokenKind) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl PartialEq for TokenKind

source§

fn eq(&self, other: &TokenKind) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 · source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
source§

impl StructuralPartialEq for TokenKind

Auto Trait Implementations§

Blanket Implementations§

source§

impl<T> Aligned for T

source§

const ALIGN: Alignment = _

Alignment of Self.
source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> CloneToUninit for T
where T: Clone,

source§

unsafe fn clone_to_uninit(&self, dst: *mut T)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dst. Read more
source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T> Instrument for T

source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoEither for T

source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
source§

impl<T> Same for T

source§

type Output = T

Should always be Self
source§

impl<T> ToOwned for T
where T: Clone,

source§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

source§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<T> WithSubscriber for T

source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more
source§

impl<'a, T> Captures<'a> for T
where T: ?Sized,

Layout§

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain repr(...) attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 16 bytes

Size for each variant:

  • Eq: 0 bytes
  • Lt: 0 bytes
  • Le: 0 bytes
  • EqEq: 0 bytes
  • Ne: 0 bytes
  • Ge: 0 bytes
  • Gt: 0 bytes
  • AndAnd: 0 bytes
  • OrOr: 0 bytes
  • Not: 0 bytes
  • Tilde: 0 bytes
  • BinOp: 1 byte
  • BinOpEq: 1 byte
  • At: 0 bytes
  • Dot: 0 bytes
  • DotDot: 0 bytes
  • DotDotDot: 0 bytes
  • DotDotEq: 0 bytes
  • Comma: 0 bytes
  • Semi: 0 bytes
  • Colon: 0 bytes
  • PathSep: 0 bytes
  • RArrow: 0 bytes
  • LArrow: 0 bytes
  • FatArrow: 0 bytes
  • Pound: 0 bytes
  • Dollar: 0 bytes
  • Question: 0 bytes
  • SingleQuote: 0 bytes
  • OpenDelim: 1 byte
  • CloseDelim: 1 byte
  • Literal: 15 bytes
  • Ident: 7 bytes
  • NtIdent: 15 bytes
  • Lifetime: 7 bytes
  • NtLifetime: 15 bytes
  • Interpolated: 15 bytes
  • DocComment: 7 bytes
  • Eof: 0 bytes