clippy_utils

Function tokenize_with_text

source 
pub fn tokenize_with_text(
    s: &str,
) -> impl Iterator<Item = (TokenKind, &str, InnerSpan)>
Expand description

Tokenizes the input while keeping the text associated with each token.