Struct Lexer 

pub(super) struct Lexer<'input, const VERSION: u8, const OWNED: bool> {
    input: &'input [u8],
    depth: u8,
    byte_pos: u32,
}

An iterator over the lexed tokens.

Fields

input: &'input [u8]

depth: u8

byte_pos: u32

Implementations

impl<'input, const VERSION: u8, const OWNED: bool> Lexer<'input, VERSION, OWNED>

pub(super) const fn new(input: &'input str) -> Self

Parse the string into a series of [Token]s.

VERSION controls the version of the format description that is being parsed.

• When VERSION is 1, [[ is the only escape sequence, resulting in a literal [. For the start of a nested format description, a single [ is used and is never part of the escape sequence. For example, [optional [[day]]] will lex successfully, ultimately resulting in a component named optional with the nested component day.
• When VERSION is 2 or 3, all escape sequences begin with \. The only characters that may currently follow are \, [, and ], all of which result in the literal character. All other characters result in a lex error.

fn advance(&mut self, bytes: u32)

Advance the input by the given number of bytes.

const fn context(&self) -> Context

Whether the lexer is currently parsing a component or a literal.

fn consume_whitespace(&mut self) -> Option<Spanned<&'input str>>

Consume the next token if it is a component item that is whitespace.

fn consume_component_part(&mut self) -> Option<Spanned<&'input str>>

Consume the next token if it is a component item that is not whitespace.

fn consume_closing_bracket(&mut self) -> Option<Location>

Consume the next token if it is a closing bracket.

fn consume_component_name( &mut self, opening_bracket: Location, ) -> Result<Spanned<&'input str>, Error>

Consume the next token if it is a component name. The caller is expected to be inside a component header.

fn consume_modifier(&mut self) -> Result<NextModifier<'input>, Error>

fn consume_component( &mut self, opening_bracket: Location, ) -> Result<<() as ParseTarget<'input, VERSION, OWNED>>::ItemWithLiteralLifetime, Error>

where (): ParseTarget<'input, VERSION, OWNED>,

Parse a component.

fn consume_nested( &mut self, last_location: Location, ) -> Result<NestedFormatDescription<'input, <() as ParseTarget<'input, VERSION, OWNED>>::ItemWithLiteralLifetime>, Error>

where (): ParseTarget<'input, VERSION, OWNED>,

Parse a nested format description. The location provided is the most recent one consumed.

fn modifier_from_token( &self, token: Spanned<&'input str>, ) -> Result<Modifier<'input>, Error>

fn is_nested_description_start(&self) -> bool

Check whether the next tokens start a nested format description. Does not consume any input.

Note that this call is strictly an optimization, as checking the error path on parse_nested is sufficient for knowing if a nested format description is present. This method avoids the overhead of constructing an error only to throw it away.

fn consume_literal(&mut self) -> &'input str

fn consume_backslash_escape_sequence( &mut self, location: Location, ) -> Result<&'input str, Error>

impl<'input, const VERSION: u8, const OWNED: bool> Lexer<'input, VERSION, OWNED>

fn parse_next_item( &mut self, ) -> Result<<() as ParseTarget<'input, VERSION, OWNED>>::ItemWithLiteralLifetime, Error>

where (): ParseTarget<'input, VERSION, OWNED>,