Struct icu_normalizer::DecomposingNormalizer

pub struct DecomposingNormalizer {
    pub(crate) decompositions: DataPayload<CanonicalDecompositionDataV1Marker>,
    pub(crate) supplementary_decompositions: Option<SupplementPayloadHolder>,
    pub(crate) tables: DataPayload<CanonicalDecompositionTablesV1Marker>,
    pub(crate) supplementary_tables: Option<DataPayload<CompatibilityDecompositionTablesV1Marker>>,
    pub(crate) decomposition_passthrough_bound: u8,
    pub(crate) composition_passthrough_bound: u16,
}

A normalizer for performing decomposing normalization.

Fields

decompositions: DataPayload<CanonicalDecompositionDataV1Marker>

supplementary_decompositions: Option<SupplementPayloadHolder>

tables: DataPayload<CanonicalDecompositionTablesV1Marker>

supplementary_tables: Option<DataPayload<CompatibilityDecompositionTablesV1Marker>>

decomposition_passthrough_bound: u8

composition_passthrough_bound: u16

Implementations

impl DecomposingNormalizer

pub const fn new_nfd() -> Self

NFD constructor using compiled data.

✨ Enabled with the compiled_data Cargo feature.

📚 Help choosing a constructor

pub fn try_new_nfd_with_any_provider( provider: &(impl AnyProvider + ?Sized), ) -> Result<Self, NormalizerError>

A version of Self::new_nfd that uses custom data provided by an AnyProvider.

📚 Help choosing a constructor

pub fn try_new_nfd_unstable<D>(provider: &D) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + ?Sized,

A version of Self::new_nfd that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub const fn new_nfkd() -> Self

NFKD constructor using compiled data.

✨ Enabled with the compiled_data Cargo feature.

📚 Help choosing a constructor

pub fn try_new_nfkd_with_any_provider( provider: &(impl AnyProvider + ?Sized), ) -> Result<Self, NormalizerError>

A version of Self::new_nfkd that uses custom data provided by an AnyProvider.

📚 Help choosing a constructor

pub fn try_new_nfkd_unstable<D>(provider: &D) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<CompatibilityDecompositionSupplementV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + DataProvider<CompatibilityDecompositionTablesV1Marker> + ?Sized,

A version of Self::new_nfkd that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub fn normalize_iter<I: Iterator<Item = char>>( &self, iter: I, ) -> Decomposition<'_, I>

Wraps a delegate iterator into a decomposing iterator adapter by using the data already held by this normalizer.

pub fn normalize(&self, text: &str) -> String

Normalize a string slice into a String.

pub fn is_normalized(&self, text: &str) -> bool

Check whether a string slice is normalized.

pub fn normalize_utf16(&self, text: &[u16]) -> Vec<u16>

Normalize a slice of potentially-invalid UTF-16 into a Vec.

Unpaired surrogates are mapped to the REPLACEMENT CHARACTER before normalizing.

pub fn is_normalized_utf16(&self, text: &[u16]) -> bool

Checks whether a slice of potentially-invalid UTF-16 is normalized.

Unpaired surrogates are treated as the REPLACEMENT CHARACTER.

pub fn normalize_utf8(&self, text: &[u8]) -> String

Normalize a slice of potentially-invalid UTF-8 into a String.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard.

pub fn is_normalized_utf8(&self, text: &[u8]) -> bool

Check if a slice of potentially-invalid UTF-8 is normalized.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard before checking.

pub fn normalize_to<W: Write + ?Sized>( &self, text: &str, sink: &mut W, ) -> Result

Normalize a string slice into a Write sink.

pub fn normalize_utf8_to<W: Write + ?Sized>( &self, text: &[u8], sink: &mut W, ) -> Result

Normalize a slice of potentially-invalid UTF-8 into a Write sink.

Ill-formed byte sequences are mapped to the REPLACEMENT CHARACTER according to the WHATWG Encoding Standard.

pub fn normalize_utf16_to<W: Write16 + ?Sized>( &self, text: &[u16], sink: &mut W, ) -> Result

Normalize a slice of potentially-invalid UTF-16 into a Write16 sink.

Unpaired surrogates are mapped to the REPLACEMENT CHARACTER before normalizing.

Trait Implementations

impl Debug for DecomposingNormalizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.