Function rustc_lexer::tokenize

source ·
pub fn tokenize(input: &str) -> impl Iterator<Item = Token> + '_
Expand description

Creates an iterator that produces tokens from the input string.