Struct icu_normalizer::ComposingNormalizer

pub struct ComposingNormalizer {
    pub(crate) decomposing_normalizer: DecomposingNormalizer,
    pub(crate) canonical_compositions: DataPayload<CanonicalCompositionsV1Marker>,
}

A normalizer for performing composing normalization.

Fields

decomposing_normalizer: DecomposingNormalizer

canonical_compositions: DataPayload<CanonicalCompositionsV1Marker>

Implementations

impl ComposingNormalizer

pub const fn new_nfc() -> Self

NFC constructor using compiled data.

✨ Enabled with the compiled_data Cargo feature.

📚 Help choosing a constructor

pub fn try_new_nfc_with_any_provider( provider: &(impl AnyProvider + ?Sized), ) -> Result<Self, NormalizerError>

A version of Self::new_nfc that uses custom data provided by an AnyProvider.

📚 Help choosing a constructor

pub fn try_new_nfc_unstable<D>(provider: &D) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + DataProvider<CanonicalCompositionsV1Marker> + ?Sized,

A version of Self::new_nfc that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub const fn new_nfkc() -> Self

NFKC constructor using compiled data.

✨ Enabled with the compiled_data Cargo feature.

📚 Help choosing a constructor

pub fn try_new_nfkc_with_any_provider( provider: &(impl AnyProvider + ?Sized), ) -> Result<Self, NormalizerError>

A version of Self::new_nfkc that uses custom data provided by an AnyProvider.

📚 Help choosing a constructor

pub fn try_new_nfkc_unstable<D>(provider: &D) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<CompatibilityDecompositionSupplementV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + DataProvider<CompatibilityDecompositionTablesV1Marker> + DataProvider<CanonicalCompositionsV1Marker> + ?Sized,

A version of Self::new_nfkc that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub(crate) const fn new_uts46() -> Self

This is a special building block normalization for IDNA that implements parts of the Map step and the following Normalize step.

Warning: In this normalization, U+0345 COMBINING GREEK YPOGEGRAMMENI exhibits a behavior that no character in Unicode exhibits in NFD, NFKD, NFC, or NFKC: Case folding turns U+0345 from a reordered character into a non-reordered character before reordering happens. Therefore, the output of this normalization may differ for different inputs that are canonically equivalents with each other if they differ by how U+0345 is ordered relative to other reorderable characters.

pub(crate) fn try_new_uts46_unstable<D>( provider: &D, ) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<Uts46DecompositionSupplementV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + DataProvider<CompatibilityDecompositionTablesV1Marker> + DataProvider<CanonicalCompositionsV1Marker> + ?Sized,

A version of Self::new_uts46 that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub fn normalize_iter<I: Iterator<Item = char>>( &self, iter: I, ) -> Composition<'_, I>

Wraps a delegate iterator into a composing iterator adapter by using the data already held by this normalizer.

pub(crate) fn normalize_iter_private<I: Iterator<Item = char>>( &self, iter: I, ignorable_behavior: IgnorableBehavior, ) -> Composition<'_, I>

pub fn normalize(&self, text: &str) -> String

Normalize a string slice into a String.

pub fn is_normalized(&self, text: &str) -> bool

Check whether a string slice is normalized.

pub fn normalize_utf16(&self, text: &[u16]) -> Vec<u16>

Normalize a slice of potentially-invalid UTF-16 into a Vec.

Unpaired surrogates are mapped to the REPLACEMENT CHARACTER before normalizing.

pub fn is_normalized_utf16(&self, text: &[u16]) -> bool

Checks whether a slice of potentially-invalid UTF-16 is normalized.

Unpaired surrogates are treated as the REPLACEMENT CHARACTER.

pub fn normalize_utf8(&self, text: &[u8]) -> String

Normalize a slice of potentially-invalid UTF-8 into a String.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard.

pub fn is_normalized_utf8(&self, text: &[u8]) -> bool

Check if a slice of potentially-invalid UTF-8 is normalized.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard before checking.

pub fn normalize_to<W: Write + ?Sized>( &self, text: &str, sink: &mut W, ) -> Result

Normalize a string slice into a Write sink.

pub fn normalize_utf8_to<W: Write + ?Sized>( &self, text: &[u8], sink: &mut W, ) -> Result

Normalize a slice of potentially-invalid UTF-8 into a Write sink.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard.

pub fn normalize_utf16_to<W: Write16 + ?Sized>( &self, text: &[u16], sink: &mut W, ) -> Result

Normalize a slice of potentially-invalid UTF-16 into a Write16 sink.

Unpaired surrogates are mapped to the REPLACEMENT CHARACTER before normalizing.

Trait Implementations

impl Debug for ComposingNormalizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.