Struct Classifier

struct Classifier<'src> {
    tokens: PeekIter<'src>,
    in_attribute: bool,
    in_macro: bool,
    in_macro_nonterminal: bool,
    byte_pos: u32,
    file_span: Span,
    src: &'src str,
    decorations: Option<Decorations>,
}

Processes program tokens, classifying strings of text by highlighting category (Class).

Fields

tokens: PeekIter<'src>

in_attribute: bool

in_macro: bool

in_macro_nonterminal: bool

byte_pos: u32

file_span: Span

src: &'src str

decorations: Option<Decorations>

Implementations

impl<'src> Classifier<'src>

fn new( src: &str, file_span: Span, decoration_info: Option<DecorationInfo>, ) -> Classifier<'_>

Takes as argument the source code to HTML-ify, the rust edition to use and the source code file span which will be used later on by the span_correspondence_map.

fn new_span(&self, lo: u32, text: &str) -> Span

Convenient wrapper to create a Span from a position in the file.

fn get_full_ident_path(&mut self) -> Vec<(TokenKind, usize, usize)>

Concatenate colons and idents as one when possible.

fn next(&mut self) -> Option<(TokenKind, &'src str, u32)>

Wraps the tokens iteration to ensure that the byte_pos is always correct.

It returns the token’s kind, the token as a string and its byte position in the source string.

fn highlight(self, sink: &mut dyn FnMut(Highlight<'src>))

Exhausts the Classifier writing the output into sink.

The general structure for this method is to iterate over each token, possibly giving it an HTML span with a class specifying what flavor of token is used.

fn advance( &mut self, token: TokenKind, text: &'src str, sink: &mut dyn FnMut(Highlight<'src>), before: u32, )

Single step of highlighting. This will classify token, but maybe also a couple of following ones as well.

before is the position of the given token in the source string and is used as “lo” byte in case we want to try to generate a link for this token using the span_correspondence_map.

fn peek(&mut self) -> Option<TokenKind>

fn check_if_is_union_keyword(&mut self) -> bool

Layout

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain repr(...) attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 160 bytes