Struct CollationElements 

pub(crate) struct CollationElements<'data, I, T>
where
    I: Iterator<Item = (char, u32)> + WithTrie<'data, T, u32>,
    T: AbstractCodePointTrie<'data, u32>,
{

    iter: I,
    pending: SmallVec<[CollationElement; 6]>,
    pending_pos: usize,
    prefix: [char; 2],
    upcoming: SmallVec<[CharacterAndClassAndTrieValue; 10]>,
    root: &'data CollationData<'data>,
    tailoring: &'data CollationData<'data>,
    jamo: &'data [<u32 as AsULE>::ULE; 256],
    diacritics: &'data ZeroSlice<u16>,
    scalars16: &'data ZeroSlice<u16>,
    scalars32: &'data ZeroSlice<char>,
    numeric_primary: Option<u8>,
    lithuanian_dot_above: bool,
    upcoming_normalized: bool,
    iter_exhausted: bool,
    initialized: bool,
    _phantom: PhantomData<T>,
}

Iterator that transforms an iterator over char into an iterator over CollationElement with a tailoring. Not a real Rust iterator: Instead of None uses NO_CE to indicate end of iteration to optimize comparison.

It is extremely important for performance that SmallVecs not be moved. To facilitate move-avoidance, this struct has the following life cycle where new returns the struct in a state that is not yet valid for a next call until init is called:

1. new.
2. Some number of calls to iter_next_before_init and prepend_upcoming_before_init.
3. init.
4. Some number of calls to next.

Fields

iter: I

See components/normalizer/trie-value-format.md for the trie wrapped in iter.

pending: SmallVec<[CollationElement; 6]>

Already computed but not yet returned CollationElements.

pending_pos: usize

The index of the next item to be returned from pending. The purpose of this index is to avoid moving the rest of the items.

prefix: [char; 2]

The characters most previously seen (or never-matching placeholders) CLDR, as of 40, has two kinds of prefixes: Prefixes that contain a single starter Prefixes that contain a starter followed by either U+3099 or U+309A Last-pushed is at index 0 and previously-pushed at index 1

upcoming: SmallVec<[CharacterAndClassAndTrieValue; 10]>

upcoming holds the characters that have already been read from iter but haven’t yet been mapped to CollationElements.

Typically, upcoming holds one character and corresponds semantically to pending_unnormalized_starter in icu::normalizer::Decomposition. This is why there isn’t a move avoidance optimization similar to pending_pos above for this buffer. A complex decomposition, a Hangul syllable followed by a non-starter, or lookahead can cause pending to hold more than one char.

Invariant: upcoming is allowed to become empty only after iter has been exhausted.

Invariant: (Checked by debug_assert!) At the start of next() call, if upcoming isn’t empty (with iter having been exhausted), the first char in upcoming must have its decomposition start with a starter.

TODO: Reverse the order, since now insert(0, x) and remove(0) are used more often than push() and pop().

root: &'data CollationData<'data>

The root collation data.

tailoring: &'data CollationData<'data>

Tailoring if applicable.

jamo: &'data [<u32 as AsULE>::ULE; 256]

The CollationElement32 mapping for the Hangul Jamo block.

Note: in ICU4C the jamo table contains only modern jamo. Here, the jamo table contains the whole Unicode block.

diacritics: &'data ZeroSlice<u16>

The CollationElement32 mapping for the Combining Diacritical Marks block.

scalars16: &'data ZeroSlice<u16>

NFD complex decompositions on the BMP

scalars32: &'data ZeroSlice<char>

NFD complex decompositions on supplementary planes

numeric_primary: Option<u8>

If numeric mode is enabled, the 8 high bits of the numeric primary. None if disabled.

lithuanian_dot_above: bool

Whether the Lithuanian combining dot above handling is enabled.

upcoming_normalized: bool

Whether upcoming (except the last item) has been normalized already

iter_exhausted: bool

Whether iter has been exhausted

initialized: bool

Whether init has been called

_phantom: PhantomData<T>

Implementations

impl<'data, I, T> CollationElements<'data, I, T>

where I: Iterator<Item = (char, u32)> + WithTrie<'data, T, u32> + 'data, T: AbstractCodePointTrie<'data, u32> + 'data,

pub fn new( delegate: I, root: &'data CollationData<'_>, tailoring: &'data CollationData<'_>, jamo: &'data [<u32 as AsULE>::ULE; 256], diacritics: &'data ZeroSlice<u16>, tables: &'data DecompositionTables<'_>, numeric_primary: Option<u8>, lithuanian_dot_above: bool, ) -> Self

pub fn iter_next_before_init(&mut self) -> Option<CharacterAndClassAndTrieValue>

pub fn prepend_upcoming_before_init(&mut self, c: CharacterAndClassAndTrieValue)

pub fn init(&mut self)

fn iter_next(&mut self) -> Option<CharacterAndClassAndTrieValue>

fn next_internal(&mut self) -> Option<CharacterAndClassAndTrieValue>

fn maybe_gather_combining(&mut self)

fn ensure_upcoming_normalized(&mut self)

Ensures that upcoming is normalized to NFD, except:

1. When the last item is a starter, it isn’t necessarily normalized.
2. Hangul syllable are unnormalized.

fn push_decomposed_combining(&mut self, c: CharacterAndClassAndTrieValue)

fn push_decomposed_starter(&mut self, c: CharacterAndClassAndTrieValue) -> usize

fn push_decomposed_and_gather_combining( &mut self, c: CharacterAndClassAndTrieValue, )

fn look_ahead(&mut self, pos: usize) -> Option<CharacterAndClassAndTrieValue>

fn is_next_decomposition_starts_with_starter(&self) -> bool

fn prepend_and_sort_non_starter_prefix_of_suffix( &mut self, c: CharacterAndClassAndTrieValue, )

fn prefix_push(&mut self, c: char)

fn mark_prefix_unmatchable(&mut self)

Micro optimization for doing a simpler write when we know the most recent character was a non-starter that is not a kana voicing mark.

pub fn next(&mut self) -> CollationElement

fn collect_combining( &mut self, combining_characters: &mut SmallVec<[CharacterAndClass; 7]>, )