Module elements 

This module holds the 64-bit CollationElement struct used for the actual comparison, the 32-bit CollationElement32 struct that’s used for storage. (Strictly speaking, the storage is RawBytesULE<4>.) And the CollationElements iterator adapter that turns an iterator over char into an iterator over CollationElement. (To match the structure of ICU4C, this isn’t a real Rust Iterator. Instead of signaling end by returning None, it signals end by returning NO_CE.)

This module also declares various constants that are also used by the comparison module.

Structs

CharacterAndClass 🔒

Pack a char and a CanonicalCombiningClass in 32 bits (the former in the lower 24 bits and the latter in the high 8 bits). The latter can be initialized to 0xFF upon creation, in which case it can be actually set later by calling set_ccc_from_trie_if_not_already_set. This is a micro optimization to avoid the Canonical Combining Class trie lookup when there is only one combining character in a sequence. This type is intentionally non-Copy to get compiler help in making sure that the class is set on the instance on which it is intended to be set and not on a temporary copy.

CharacterAndClassAndTrieValue 🔒

This struct makes the handling of the upcoming buffer easily so that trie lookups are done at most once. However, when upcoming[0] is an undecomposed starter, we don’t need the ccc yet, and when lookahead has already done the trie lookups, we don’t need trie_value, as it is implied by ccc.

CollationElement 🔒

A collation element is a 64-bit value.

CollationElement32 🔒

A compressed form of a collation element as stored in the collation data.

CollationElements 🔒

Iterator that transforms an iterator over char into an iterator over CollationElement with a tailoring. Not a real Rust iterator: Instead of None uses NO_CE to indicate end of iteration to optimize comparison.

NonPrimary 🔒

The purpose of grouping the non-primary bits into a struct is to allow for a future optimization that specializes code over whether storage for primary weights is needed or not. (I.e. whether to specialize on CollationElement or NonPrimary.)

Enums

Tag 🔒

Special-CE32 tags, from bits 3..0 of a special 32-bit CE. Bits 31..8 are available for tag-specific data. Bits 5..4: Reserved. May be used in the future to indicate lccc!=0 and tccc!=0.

Constants

BACKWARD_COMBINING_MARKER 🔒

Marker that the first character of the decomposition can combine backwards.

CASE_MASK 🔒

CE_BUFFER_SIZE 🔒

The number of full 64-bit collation units that get buffered in the primary comparison loop so that they can be examined by the subsequent comparison stregths.

COMBINING_CHARACTER_BUFFER_SIZE 🔒

The contiguous sequence of combining characters.

COMBINING_DIACRITICS_BASE 🔒

COMMON_SECONDARY_CE 🔒

COMMON_SEC_AND_TER_CE 🔒

COMMON_TERTIARY_CE 🔒

CONTRACT_HAS_STARTER 🔒

Set if at least one contraction suffix contains a starter

CONTRACT_NEXT_CCC 🔒

Set if the first character of every contraction suffix has lccc!=0.

CONTRACT_TRAILING_CCC 🔒

Set if any contraction suffix ends with lccc!=0.

DIGIT_BUFFER_SIZE 🔒

The sequence of digits in the numeric mode.

EMPTY_CHAR 🔒

EMPTY_U16 🔒

FALLBACK_CE32 🔒

FDFA_MARKER 🔒

Marker value for U+FDFA in NFKD. (Unified with HANGUL_SYLLABLE_MARKER, but they differ by NON_ROUND_TRIP_MARKER.)

FFFD_CE 🔒

FFFD_CE32 🔒

FFFD_CE32_VALUE 🔒

FFFD_CE_VALUE 🔒

FFFD_PRIMARY 🔒

HANGUL_L_BASE 🔒

HANGUL_N_COUNT 🔒

HANGUL_SYLLABLE_MARKER 🔒

Marker value for Hangul syllables. (Unified with FDFA_MARKER, but they differ by NON_ROUND_TRIP_MARKER.)

HANGUL_S_BASE 🔒

HANGUL_S_COUNT 🔒

HANGUL_T_BASE 🔒

HANGUL_T_COUNT 🔒

HANGUL_V_BASE 🔒

HIGH_ZEROS_MASK 🔒

Mask for the bits have to be zero for this to be a BMP singleton decomposition, or value baked into the surrogate range.

IDENTICAL_PREFIX_HANGUL_MARKER_CE32 🔒

Used only as a placeholder on the indentical prefix path. The requirement is that this CE32 fails the quick mapping to a primary, which is does, because the tag byte is higher than LONG_PRIMARY_CE32_LOW_BYTE.

JAMO_COUNT 🔒

LONG_PRIMARY_CE32_LOW_BYTE 🔒

LOW_ZEROS_MASK 🔒

Mask for the bits have to be zero for this to be a complex decomposition.

NON_ROUND_TRIP_MARKER 🔒

Marker that the decomposition does not round trip via NFC.

NO_CE 🔒

NO_CE_PRIMARY 🔒

NO_CE_QUATERNARY 🔒

NO_CE_SECONDARY 🔒

NO_CE_TERTIARY 🔒

NO_CE_VALUE 🔒

OPTIMIZED_DIACRITICS_LIMIT 🔒

OPTIMIZED_DIACRITICS_MAX_COUNT 🔒

PENDING_CE_BUFFER_SIZE 🔒

The number of extra full 64-bit collation units that have already been computed as part of an operation that yields multiple collation units at a time.

PENDING_REMOVALS_SIZE 🔒

The number of combining characters that a contraction has matched.

QUATERNARY_MASK 🔒

SINGLE_REPLACEMENT_CHARACTER_CHAR 🔒

SINGLE_REPLACEMENT_CHARACTER_U16 🔒

SPECIAL_CE32_LOW_BYTE 🔒

TERTIARY_MASK 🔒

UNASSIGNED_IMPLICIT_BYTE 🔒

UPCOMING_CHARACTER_BUFFER_SIZE 🔒

Either the identical prefix or the lookahead plus the next upcoming character.

Functions

ccc_from_trie_value 🔒

Extracts a canonical combining class (possibly zero) from a trie value.

char_from_u16 🔒

Convert a u16 obtained from data provider data to char.

char_from_u32 🔒

Convert a u32 obtained from data provider data to char.

decomposition_starts_with_non_starter 🔒

Checks if a trie value signifies a character whose decomposition starts with a non-starter.

in_inclusive_range 🔒

trie_value_has_ccc 🔒

Checks if a trie value carries a (non-zero) canonical combining class.

trie_value_indicates_special_non_starter_decomposition 🔒

Checks if the trie signifies a special non-starter decomposition.

unwrap_or_gigo 🔒

If opt is Some, unwrap it. If None, panic if debug assertions are enabled and return default if debug assertions are not enabled.