Struct CodePointTrie 

pub struct CodePointTrie<'trie, T: TrieValue> {
    pub(crate) header: CodePointTrieHeader,
    pub(crate) index: ZeroVec<'trie, u16>,
    pub(crate) data: ZeroVec<'trie, T>,
    pub(crate) error_value: T,
}

This struct represents a de-serialized CodePointTrie that was exported from ICU binary data.

For more information:

• ICU Site design doc
• ICU User Guide section on Properties lookup

Fields

header: CodePointTrieHeader

Safety Invariant

The value of header.trie_type must not change after construction.

index: ZeroVec<'trie, u16>

Safety Invariant

If header.trie_type == TrieType::Fast, index.len() must be greater than FAST_TYPE_FAST_INDEXING_MAX. Otherwise, index.len() must be greater than SMALL_TYPE_FAST_INDEXING_MAX. Furthermore, this field must not change after construction. (Strictly: It must not become shorter than the length requirement stated above and the values within the prefix up to the length requirement must not change.)

data: ZeroVec<'trie, T>

Safety Invariant

If header.trie_type == TrieType::Fast, data.len() must be greater than FAST_TYPE_DATA_MASK plus the largest value in index[0..FAST_TYPE_FAST_INDEXING_MAX + 1]. Otherwise, data.len() must be greater than FAST_TYPE_DATA_MASK plus the largest value in index[0..SMALL_TYPE_FAST_INDEXING_MAX + 1]. Furthermore, this field must not change after construction. (Strictly: The stated length requirement must continue to hold.)

error_value: T

Implementations

impl<'trie, T: TrieValue> CodePointTrie<'trie, T>

pub fn try_new( header: CodePointTrieHeader, index: ZeroVec<'trie, u16>, data: ZeroVec<'trie, T>, ) -> Result<CodePointTrie<'trie, T>, Error>

Returns a new CodePointTrie backed by borrowed data for the index array and data array, whose data values have width W.

pub(crate) fn validate_fields( header: &CodePointTrieHeader, index: &ZeroSlice<u16>, data: &ZeroSlice<T>, ) -> Result<T, Error>

Checks the invariant on the fields that fast-path access relies on for safety in order to omit slice bound checks and upon success returns the error_value for the trie.

Safety Usable Invariant

Iff this function returns Ok(T), the arguments satisfy the invariants for corresponding fields of CodePointTrie. (Other than proving that nothing else changes the fields subsequently.)

pub const fn to_typed( self, ) -> Typed<FastCodePointTrie<'trie, T>, SmallCodePointTrie<'trie, T>>

Turns this trie into a version whose trie type is encoded in the Rust type.

pub fn as_typed_ref( &self, ) -> Typed<&FastCodePointTrie<'trie, T>, &SmallCodePointTrie<'trie, T>>

Obtains a reference to this trie as a Rust type that encodes the trie type in the Rust type.

fn trie_error_val_index(&self) -> u32

Returns the position in the data array containing the trie’s stored error value.

fn internal_small_index(&self, code_point: u32) -> u32

fn small_index(&self, code_point: u32) -> u32

Returns the position in the data array for the given code point, where this code point is at or above the fast limit associated for the trie_type. We will refer to that limit as “fastMax” here.

A lookup of the value in the code point trie for a code point in the code point space range [fastMax, high_start) will be a 4-step lookup: 3 lookups in the index array and one lookup in the data array. Lookups for code points in the range [high_start, CODE_POINT_MAX] are short-circuited to be a single lookup, see CodePointTrieHeader::high_start.

fn fast_index(&self, code_point: u32) -> u32

Returns the position in the data array for the given code point, where this code point is below the fast limit associated for the trie type. We will refer to that limit as “fastMax” here.

A lookup of the value in the code point trie for a code point in the code point space range [0, fastMax) will be a 2-step lookup: 1 lookup in the index array and one lookup in the data array. By design, for trie type T, there is an element allocated in the index array for each block of code points in [0, fastMax), which in turn guarantees that those code points are represented and only need 1 lookup.

fn get32_by_fast_index(&self, code_point: u32) -> Option<T>

Returns the value that is associated with code_point in this CodePointTrie if code_point uses fast-path lookup or None if code_point should use small-path lookup or is above the supported range.

unsafe fn get32_assuming_fast_index(&self, code_point: u32) -> T

Performs the actual fast-mode lookup

Safety

If self.header.trie_type == TrieType::Small, code_point must be at most SMALL_TYPE_FAST_INDEXING_MAX. If self.header.trie_type == TrieType::Fast, code_point must be at most FAST_TYPE_FAST_INDEXING_MAX.

unsafe fn get_bit_prefix_suffix_assuming_fast_index( &self, bit_prefix: usize, bit_suffix: usize, ) -> T

fn get32_by_small_index_cold(&self, code_point: u32) -> T

Coldness wrapper for get32_by_small_index to also allow calls without the effects of #[cold].

fn get32_by_small_index(&self, code_point: u32) -> T

Returns the value that is associated with code_point in this CodePointTrie assuming that the small index path should be used.

Intended Precondition

code_point must be at most CODE_POINT_MAX AND greter than FAST_TYPE_FAST_INDEXING_MAX if the trie type is fast or greater than SMALL_TYPE_FAST_INDEXING_MAX if the trie type is small. This is checked when debug assertions are enabled. If this precondition is violated, the behavior of this method is memory-safe, but the returned value may be bogus (not necessarily the designated error value).

pub fn get32(&self, code_point: u32) -> T

Returns the value that is associated with code_point in this CodePointTrie.

Examples

use icu::collections::codepointtrie::planes;
let trie = planes::get_planes_trie();

assert_eq!(0, trie.get32(0x41)); // 'A' as u32
assert_eq!(0, trie.get32(0x13E0)); // 'Ꮰ' as u32
assert_eq!(1, trie.get32(0x10044)); // '𐁄' as u32

pub fn get(&self, c: char) -> T

Returns the value that is associated with char in this CodePointTrie.

Examples

use icu::collections::codepointtrie::planes;
let trie = planes::get_planes_trie();

assert_eq!(0, trie.get('A')); // 'A' as u32
assert_eq!(0, trie.get('Ꮰ')); // 'Ꮰ' as u32
assert_eq!(1, trie.get('𐁄')); // '𐁄' as u32

pub fn get16(&self, bmp: u16) -> T

Returns the value that is associated with bmp in this CodePointTrie.

pub fn get8(&self, latin1: u8) -> T

Returns the value that is associated with latin1 in this CodePointTrie.

pub unsafe fn get7(&self, ascii: u8) -> T

Returns the value that is associated with ascii in this CodePointTrie.

Safety

ascii must be less than 128.

pub unsafe fn get_utf8_two_byte(&self, high_five: u32, low_six: u32) -> T

Returns the value that is associated with a two-byte UTF-8 sequence in this CodePointTrie.

high_five is the low five bits of the lead byte of a two-byte UTF-8 sequence. low_six is the low six bits of the trail byte of a two-byte UTF-8 sequence.

Safety

high_five must not have bit positions other than the lowest 5 set to 1. low_six must not have bit positions other than the lowest 6 set to 1.

Panics

With debug assertions enabled, panics if the above safety invariants are violated or high_five represents non-shortest form.

pub unsafe fn get_utf8_three_byte(&self, high_ten: u32, low_six: u32) -> T

Returns the value that is associated with a three-byte UTF-8 or WTF-8 sequence in this CodePointTrie.

high_ten is the low four bits of the lead byte of three-byte UTF-8 or WTF-8 sequence shifted left by 6 followed by the low six bits of the first trail byte. low_six is the low six bits of the last trail byte of a three-byte UTF-8 or WTF-8 sequence.

Sequences representing surrogates (WTF-8) are allowed.

Safety

low_six must not have bit positions other than the lowest 6 set to 1.

Intended Invariant

high_ten must not have bit positions other than the lowest 10 set to 1.

Panics

With debug assertions enabled, panics if the above safety invariant is violated or high_ten is out of range for three-byte WTF-8 (or UTF-8) sequence.

pub fn get32_supplementary(&self, supplementary: u32) -> T

Lookup trie value by non-Basic Multilingual Plane Scalar Value.

The return value may be bogus (not necessarily error_value) is the argument is actually in the Basic Multilingual Plane or above the Unicode Scalar Value range (panics instead with debug assertions enabled).

pub fn get32_ule(&self, code_point: u32) -> Option<&T::ULE>

Returns a reference to the ULE of the value that is associated with code_point in this CodePointTrie.

Examples

use icu::collections::codepointtrie::planes;
let trie = planes::get_planes_trie();

assert_eq!(Some(&0), trie.get32_ule(0x41)); // 'A' as u32
assert_eq!(Some(&0), trie.get32_ule(0x13E0)); // 'Ꮰ' as u32
assert_eq!(Some(&1), trie.get32_ule(0x10044)); // '𐁄' as u32

pub fn try_into_converted<P>(self) -> Result<CodePointTrie<'trie, P>, UleError>

where P: TrieValue,

Converts the CodePointTrie into one that returns another type of the same size.

Borrowed data remains borrowed, and owned data remains owned.

If the old and new types are not the same size, use CodePointTrie::try_alloc_map_value.

Panics

Panics if T and P are different sizes.

More specifically, panics if ZeroVec::try_into_converted() panics when converting ZeroVec<T> into ZeroVec<P>, which happens if T::ULE and P::ULE differ in size.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::collections::codepointtrie::planes;
use icu::collections::codepointtrie::CodePointTrie;

let planes_trie_u8: CodePointTrie<u8> = planes::get_planes_trie();
let planes_trie_i8: CodePointTrie<i8> =
    planes_trie_u8.try_into_converted().expect("infallible");

assert_eq!(planes_trie_i8.get32(0x30000), 3);

pub fn try_alloc_map_value<P, E>( &self, f: impl FnMut(T) -> Result<P, E>, ) -> Result<CodePointTrie<'trie, P>, E>

where P: TrieValue,

Maps the CodePointTrie into one that returns a different type.

This function returns owned data.

If the old and new types are the same size, use the more efficient CodePointTrie::try_into_converted.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::collections::codepointtrie::planes;
use icu::collections::codepointtrie::CodePointTrie;

let planes_trie_u8: CodePointTrie<u8> = planes::get_planes_trie();
let planes_trie_u16: CodePointTrie<u16> = planes_trie_u8
    .try_alloc_map_value(TryFrom::try_from)
    .expect("infallible");

assert_eq!(planes_trie_u16.get32(0x30000), 3);

pub fn get_range(&self, start: u32) -> Option<CodePointMapRange<T>>

Returns a CodePointMapRange struct which represents a range of code points associated with the same trie value. The returned range will be the longest stretch of consecutive code points starting at start that share this value.

This method is designed to use the internal details of the structure of CodePointTrie to be optimally efficient. This will outperform a naive approach that just uses CodePointTrie::get().

This method provides lower-level functionality that can be used in the implementation of other methods that are more convenient to the user. To obtain an optimal partition of the code point space for this trie resulting in the fewest number of ranges, see CodePointTrie::iter_ranges().

Examples

use icu::collections::codepointtrie::planes;

let trie = planes::get_planes_trie();

const CODE_POINT_MAX: u32 = 0x10ffff;
let start = 0x1_0000;
let exp_end = 0x1_ffff;

let start_val = trie.get32(start);
assert_eq!(trie.get32(exp_end), start_val);
assert_ne!(trie.get32(exp_end + 1), start_val);

use icu::collections::codepointtrie::CodePointMapRange;

let cpm_range: CodePointMapRange<u8> = trie.get_range(start).unwrap();

assert_eq!(cpm_range.range.start(), &start);
assert_eq!(cpm_range.range.end(), &exp_end);
assert_eq!(cpm_range.value, start_val);

// `start` can be any code point, whether or not it lies on the boundary
// of a maximally large range that still contains `start`

let submaximal_1_start = start + 0x1234;
let submaximal_1 = trie.get_range(submaximal_1_start).unwrap();
assert_eq!(submaximal_1.range.start(), &0x1_1234);
assert_eq!(submaximal_1.range.end(), &0x1_ffff);
assert_eq!(submaximal_1.value, start_val);

let submaximal_2_start = start + 0xffff;
let submaximal_2 = trie.get_range(submaximal_2_start).unwrap();
assert_eq!(submaximal_2.range.start(), &0x1_ffff);
assert_eq!(submaximal_2.range.end(), &0x1_ffff);
assert_eq!(submaximal_2.value, start_val);

pub fn iter_ranges(&self) -> CodePointMapRangeIterator<'_, T>

Yields an Iterator returning ranges of consecutive code points that share the same value in the CodePointTrie, as given by CodePointTrie::get_range().

Examples

use core::ops::RangeInclusive;
use icu::collections::codepointtrie::planes;
use icu::collections::codepointtrie::CodePointMapRange;

let planes_trie = planes::get_planes_trie();

let mut ranges = planes_trie.iter_ranges();

for plane in 0..=16 {
    let exp_start = plane * 0x1_0000;
    let exp_end = exp_start + 0xffff;
    assert_eq!(
        ranges.next(),
        Some(CodePointMapRange {
            range: exp_start..=exp_end,
            value: plane as u8
        })
    );
}

// Hitting the end of the iterator returns `None`, as will subsequent
// calls to .next().
assert_eq!(ranges.next(), None);
assert_eq!(ranges.next(), None);

pub fn iter_ranges_for_value( &self, value: T, ) -> impl Iterator<Item = RangeInclusive<u32>> + '_

Yields an Iterator returning the ranges of the code points whose values match value in the CodePointTrie.

Examples

use icu::collections::codepointtrie::planes;

let trie = planes::get_planes_trie();

let plane_val = 2;
let mut sip_range_iter = trie.iter_ranges_for_value(plane_val as u8);

let start = plane_val * 0x1_0000;
let end = start + 0xffff;

let sip_range = sip_range_iter.next()
    .expect("Plane 2 (SIP) should exist in planes data");
assert_eq!(start..=end, sip_range);

assert!(sip_range_iter.next().is_none());

pub fn iter_ranges_mapped<'a, U: Eq + 'a>( &'a self, map: impl FnMut(T) -> U + Copy + 'a, ) -> impl Iterator<Item = CodePointMapRange<U>> + 'a

Yields an Iterator returning the ranges of the code points after passing the value through a mapping function.

This is preferable to calling .get_ranges().map() since it will coalesce adjacent ranges into one.

Examples

use icu::collections::codepointtrie::planes;

let trie = planes::get_planes_trie();

let plane_val = 2;
let mut sip_range_iter = trie.iter_ranges_mapped(|value| value != plane_val as u8).filter(|range| range.value);

let end = plane_val * 0x1_0000 - 1;

let sip_range = sip_range_iter.next()
    .expect("Complemented planes data should have at least one entry");
assert_eq!(0..=end, sip_range.range);

pub fn get_set_for_value(&self, value: T) -> CodePointInversionList<'static>

Returns a CodePointInversionList for the code points that have the given TrieValue in the trie.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::collections::codepointtrie::planes;

let trie = planes::get_planes_trie();

let plane_val = 2;
let sip = trie.get_set_for_value(plane_val as u8);

let start = plane_val * 0x1_0000;
let end = start + 0xffff;

assert!(!sip.contains32(start - 1));
assert!(sip.contains32(start));
assert!(sip.contains32(end));
assert!(!sip.contains32(end + 1));

pub fn error_value(&self) -> T

Returns the value used as an error value for this trie

impl<T: TrieValue + Into<u32>> CodePointTrie<'_, T>

pub fn get32_u32(&self, code_point: u32) -> u32

Returns the value that is associated with code_point for this CodePointTrie as a u32.

Examples

use icu::collections::codepointtrie::planes;
let trie = planes::get_planes_trie();

let cp = '𑖎' as u32;
assert_eq!(cp, 0x1158E);

let plane_num: u8 = trie.get32(cp);
assert_eq!(trie.get32_u32(cp), plane_num as u32);

Trait Implementations

impl<'trie, T: TrieValue> AbstractCodePointTrie<'trie, T> for CodePointTrie<'trie, T>

unsafe fn ascii(&self, ascii: u8) -> T

Look up trie value by an ASCII character.

unsafe fn utf8_two_byte(&self, high_five: u32, low_six: u32) -> T

Look up trie value by a two-byte UTF-8 sequence.

unsafe fn utf8_three_byte(&self, high_ten: u32, low_six: u32) -> T

Look up trie value by a three-byte UTF-8 or WTF-8 sequence.

fn latin1(&self, latin1: u8) -> T

Look up trie value by a Latin1 character.

fn bmp(&self, bmp: u16) -> T

Look up trie value by a Basic Multilingual Plane character.

fn supplementary(&self, supplementary: u32) -> T

Look up trie value by a non-Basic Multilingual Plane character.

fn scalar(&self, scalar: char) -> T

Look up trie value by a Unicode Scalar Value.

fn code_point(&self, code_point: u32) -> T

Look up trie value by Unicode Code Point.

impl<T: TrieValue> Clone for CodePointTrie<'_, T>

where <T as AsULE>::ULE: Clone,

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'trie, T: Debug + TrieValue> Debug for CodePointTrie<'trie, T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'trie, T: PartialEq + TrieValue> PartialEq for CodePointTrie<'trie, T>

fn eq(&self, other: &CodePointTrie<'trie, T>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'trie, T: TrieValue> TryFrom<&'trie CodePointTrie<'trie, T>> for &'trie FastCodePointTrie<'trie, T>

type Error = TypedCodePointTrieError

The type returned in the event of a conversion error.

fn try_from( reference: &'trie CodePointTrie<'trie, T>, ) -> Result<&'trie FastCodePointTrie<'trie, T>, TypedCodePointTrieError>

Performs the conversion.

impl<'trie, T: TrieValue> TryFrom<&'trie CodePointTrie<'trie, T>> for &'trie SmallCodePointTrie<'trie, T>

type Error = TypedCodePointTrieError

The type returned in the event of a conversion error.

fn try_from( reference: &'trie CodePointTrie<'trie, T>, ) -> Result<&'trie SmallCodePointTrie<'trie, T>, TypedCodePointTrieError>

Performs the conversion.

impl<'trie, T: TrieValue> TryFrom<CodePointTrie<'trie, T>> for FastCodePointTrie<'trie, T>

type Error = TypedCodePointTrieError

The type returned in the event of a conversion error.

fn try_from( value: CodePointTrie<'trie, T>, ) -> Result<FastCodePointTrie<'trie, T>, TypedCodePointTrieError>

Performs the conversion.

impl<'trie, T: TrieValue> TryFrom<CodePointTrie<'trie, T>> for SmallCodePointTrie<'trie, T>

type Error = TypedCodePointTrieError

The type returned in the event of a conversion error.

fn try_from( value: CodePointTrie<'trie, T>, ) -> Result<SmallCodePointTrie<'trie, T>, TypedCodePointTrieError>

Performs the conversion.

impl<'yoke, T> Yokeable<'yoke> for CodePointTrie<'static, T>

where T: 'static + TrieValue,

type Output = CodePointTrie<'yoke, T>

This type MUST be Self with the 'static replaced with 'a, i.e. Self<'a>

fn transform(&'yoke self) -> &'yoke Self::Output

This method must cast self between &'a Self<'static> and &'a Self<'a>.

fn transform_owned(self) -> Self::Output

This method must cast self between Self<'static> and Self<'a>.

unsafe fn make(from: Self::Output) -> Self

This method can be used to cast away Self<'a>’s lifetime.

fn transform_mut<F>(&'yoke mut self, f: F)

where F: 'static + for<'_yoke> FnOnce(&'_yoke mut Self::Output),

This method must cast self between &'a mut Self<'static> and &'a mut Self<'a>, and pass it to f.

impl<'zf, 'zf_inner, T: TrieValue> ZeroFrom<'zf, CodePointTrie<'zf_inner, T>> for CodePointTrie<'zf, T>

where ZeroVec<'zf, T>: ZeroFrom<'zf, ZeroVec<'zf_inner, T>>,

fn zero_from(this: &'zf CodePointTrie<'zf_inner, T>) -> Self

Clone the other C into a struct that may retain references into C.

impl<'trie, T: Eq + TrieValue> Eq for CodePointTrie<'trie, T>

impl<'trie, T: TrieValue> Seal for CodePointTrie<'trie, T>

impl<'trie, T: TrieValue> StructuralPartialEq for CodePointTrie<'trie, T>