Trait _

pub(crate) trait _: Offset<Self::Checkpoint> + Debug {
    type Token: Debug;
    type Slice: Debug;
    type IterOffsets: Iterator<Item = (usize, Self::Token)>;
    type Checkpoint: Offset + Clone + Debug;

Show 15 methods    // Required methods
    fn iter_offsets(&self) -> Self::IterOffsets;
    fn eof_offset(&self) -> usize;
    fn next_token(&mut self) -> Option<Self::Token>;
    fn peek_token(&self) -> Option<Self::Token>;
    fn offset_for<P>(&self, predicate: P) -> Option<usize>
       where P: Fn(Self::Token) -> bool;
    fn offset_at(&self, tokens: usize) -> Result<usize, Needed>;
    fn next_slice(&mut self, offset: usize) -> Self::Slice;
    fn peek_slice(&self, offset: usize) -> Self::Slice;
    fn checkpoint(&self) -> Self::Checkpoint;
    fn reset(&mut self, checkpoint: &Self::Checkpoint);
    fn trace(&self, f: &mut Formatter<'_>) -> Result<(), Error>;

    // Provided methods
    unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice { ... }
    unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice { ... }
    fn finish(&mut self) -> Self::Slice { ... }
    fn peek_finish(&self) -> Self::Slice
       where Self: Clone { ... }
}

Expand description

Core definition for parser input state

Required Associated Types§

type Token: Debug

The smallest unit being parsed

Example: u8 for &[u8] or char for &str

type Slice: Debug

Sequence of Tokens

Example: &[u8] for LocatingSlice<&[u8]> or &str for LocatingSlice<&str>

type IterOffsets: Iterator<Item = (usize, Self::Token)>

Iterate with the offset from the current location

type Checkpoint: Offset + Clone + Debug

A parse location within the stream

Required Methods§

fn iter_offsets(&self) -> Self::IterOffsets

Iterate with the offset from the current location

fn eof_offset(&self) -> usize

Returns the offset to the end of the input

fn next_token(&mut self) -> Option<Self::Token>

Split off the next token from the input

fn peek_token(&self) -> Option<Self::Token>

Split off the next token from the input

fn offset_for(&self, predicate: P) -> Option<usize>
where P: Fn(Self::Token) -> bool,

Finds the offset of the next matching token

fn offset_at(&self, tokens: usize) -> Result<usize, Needed>

Get the offset for the number of tokens into the stream

This means “0 tokens” will return 0 offset

fn next_slice(&mut self, offset: usize) -> Self::Slice

Split off a slice of tokens from the input

Note: For inputs with variable width tokens, like &str’s char, offset might not correspond with the number of tokens. To get a valid offset, use:

§Panic

This will panic if

Indexes must be within bounds of the original input;
Indexes must uphold invariants of the stream, like for str they must lie on UTF-8 sequence boundaries.

fn peek_slice(&self, offset: usize) -> Self::Slice

Split off a slice of tokens from the input

fn checkpoint(&self) -> Self::Checkpoint

Save the current parse location within the stream

fn reset(&mut self, checkpoint: &Self::Checkpoint)

Revert the stream to a prior Self::Checkpoint

§Panic

May panic if an invalid Self::Checkpoint is provided

fn trace(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Write out a single-line summary of the current parse location

Provided Methods§

unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice

Split off a slice of tokens from the input

Note: For inputs with variable width tokens, like &str’s char, offset might not correspond with the number of tokens. To get a valid offset, use:

§Safety

Callers of this function are responsible that these preconditions are satisfied:

Indexes must be within bounds of the original input;
Indexes must uphold invariants of the stream, like for str they must lie on UTF-8 sequence boundaries.

unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice

Split off a slice of tokens from the input

§Safety

Callers of this function are responsible that these preconditions are satisfied:

Indexes must be within bounds of the original input;
Indexes must uphold invariants of the stream, like for str they must lie on UTF-8 sequence boundaries.