Module icu_collator::elements

This module holds the 64-bit CollationElement struct used for the actual comparison, the 32-bit CollationElement32 struct that’s used for storage. (Strictly speaking, the storage is RawBytesULE<4>.) And the CollationElements iterator adapter that turns an iterator over char into an iterator over CollationElement. (To match the structure of ICU4C, this isn’t a real Rust Iterator. Instead of signaling end by returning None, it signals end by returning NO_CE.)

This module also declares various constants that are also used by the comparison module.

Structs

• CharacterAndClass 🔒
  Pack a char and a CanonicalCombiningClass in 32 bits (the former in the lower 24 bits and the latter in the high 8 bits). The latter can be initialized to 0xFF upon creation, in which case it can be actually set later by calling set_ccc_from_trie_if_not_already_set. This is a micro optimization to avoid the Canonical Combining Class trie lookup when there is only one combining character in a sequence. This type is intentionally non-Copy to get compiler help in making sure that the class is set on the instance on which it is intended to be set and not on a temporary copy.
• CharacterAndClassAndTrieValue 🔒
  This struct makes the handling of the upcoming buffer easily so that trie lookups are done at most once. However, when upcoming[0] is an undecomposed starter, we don’t need the ccc yet, and when lookahead has already done the trie lookups, we don’t need trie_value, as it is implied by ccc.
• CollationElement 🔒
  A collation element is a 64-bit value.
• CollationElement32 🔒
  A compressed form of a collation element as stored in the collation data.
• CollationElements 🔒
  Iterator that transforms an iterator over char into an iterator over CollationElement with a tailoring. Not a real Rust iterator: Instead of None uses NO_CE to indicate end of iteration to optimize comparison.
• NonPrimary 🔒
  The purpose of grouping the non-primary bits into a struct is to allow for a future optimization that specializes code over whether storage for primary weights is needed or not. (I.e. whether to specialize on CollationElement or NonPrimary.)

Enums

• Tag 🔒
  Special-CE32 tags, from bits 3..0 of a special 32-bit CE. Bits 31..8 are available for tag-specific data. Bits 5..4: Reserved. May be used in the future to indicate lccc!=0 and tccc!=0.

Constants

• BACKWARD_COMBINING_STARTER_MARKER 🔒
  Marker for starters that decompose to themselves but may combine backwards under canonical composition. (Main trie only; not used in the supplementary trie.)
• CASE_MASK 🔒
• COMBINING_DIACRITICS_BASE 🔒
• COMMON_SECONDARY_CE 🔒
• COMMON_SEC_AND_TER_CE 🔒
• COMMON_TERTIARY_CE 🔒
• CONTRACT_HAS_STARTER 🔒
  Set if at least one contraction suffix contains a starter
• CONTRACT_NEXT_CCC 🔒
  Set if there is no match for the single (no-suffix) character itself. This is only possible if there is a prefix. In this case, discontiguous contraction matching cannot add combining marks starting from an empty suffix. The default CE32 is used anyway if there is no suffix match. Set if the first character of every contraction suffix has lccc!=0.
• CONTRACT_TRAILING_CCC 🔒
  Set if any contraction suffix ends with lccc!=0.
• EMPTY_CHAR 🔒
• EMPTY_U16 🔒
• FALLBACK_CE32 🔒
• FDFA_MARKER 🔒
  Marker value for U+FDFA in NFKD
• FFFD_CE 🔒
• FFFD_CE32 🔒
• FFFD_CE32_VALUE 🔒
• FFFD_CE_VALUE 🔒
• FFFD_PRIMARY 🔒
• HANGUL_L_BASE 🔒
• HANGUL_N_COUNT 🔒
• HANGUL_S_BASE 🔒
• HANGUL_S_COUNT 🔒
• HANGUL_T_BASE 🔒
• HANGUL_T_COUNT 🔒
• HANGUL_V_BASE 🔒
• JAMO_COUNT 🔒
• LONG_PRIMARY_CE32_LOW_BYTE 🔒
• NON_ROUND_TRIP_MARKER 🔒
  Marker that a complex decomposition isn’t round-trippable under re-composition.
• NO_CE 🔒
• NO_CE_PRIMARY 🔒
• NO_CE_SECONDARY 🔒
• NO_CE_TERTIARY 🔒
• NO_CE_VALUE 🔒
• OPTIMIZED_DIACRITICS_LIMIT 🔒
• OPTIMIZED_DIACRITICS_MAX_COUNT 🔒
• QUATERNARY_MASK 🔒
• SINGLE_REPLACEMENT_CHARACTER_CHAR 🔒
• SINGLE_REPLACEMENT_CHARACTER_U16 🔒
• SPECIAL_CE32_LOW_BYTE 🔒
• SPECIAL_NON_STARTER_DECOMPOSITION_MARKER 🔒
• SPECIAL_NON_STARTER_DECOMPOSITION_MARKER_U16 🔒
  u16 version of the previous marker value.
• TERTIARY_MASK 🔒
• UNASSIGNED_IMPLICIT_BYTE 🔒

Functions

• ccc_from_trie_value 🔒
  Extracts a canonical combining class (possibly zero) from a trie value.
• char_from_u16 🔒
  Convert a u16 obtained from data provider data to char.
• char_from_u32 🔒
  Convert a u32 obtained from data provider data to char.
• decomposition_starts_with_non_starter 🔒
  Checks if a trie value signifies a character whose decomposition starts with a non-starter.
• in_inclusive_range 🔒
• trie_value_has_ccc 🔒
  Checks if a trie value carries a (non-zero) canonical combining class.
• trie_value_indicates_special_non_starter_decomposition 🔒
  Checks if the trie signifies a special non-starter decomposition.
• unwrap_or_gigo 🔒
  If opt is Some, unwrap it. If None, panic if debug assertions are enabled and return default if debug assertions are not enabled.