Struct icu_normalizer::uts46::Uts46Mapper

pub struct Uts46Mapper {
    normalizer: ComposingNormalizer,
}

A mapper that knows how to performs the subsets of UTS 46 processing documented on the methods.

Fields

normalizer: ComposingNormalizer

Implementations

impl Uts46Mapper

pub const fn new() -> Self

Construct with compiled data.

pub fn try_new<D>(provider: &D) -> Result<Self, NormalizerError>

where D: DataProvider<CanonicalDecompositionDataV1Marker> + DataProvider<Uts46DecompositionSupplementV1Marker> + DataProvider<CanonicalDecompositionTablesV1Marker> + DataProvider<CompatibilityDecompositionTablesV1Marker> + DataProvider<CanonicalCompositionsV1Marker> + ?Sized,

Construct with provider. A version of Self::new that uses custom data provided by a DataProvider.

📚 Help choosing a constructor

⚠️ The bounds on provider may change over time, including in SemVer minor releases.

pub fn map_normalize<'delegate, I: Iterator<Item = char> + 'delegate>( &'delegate self, iter: I, ) -> impl Iterator<Item = char> + 'delegate

Returns an iterator adaptor that turns an Iterator over char into an iterator yielding a char sequence that gets the following operations from the “Map” and “Normalize” steps of the “Processing” section of UTS 46 lazily applied to it:

1. The ignored characters are ignored.
2. The mapped characters are mapped.
3. The disallowed characters are replaced with U+FFFD, which itself is a disallowed character.
4. The deviation characters are treated as mapped or valid as appropriate.
5. The disallowed_STD3_valid characters are treated as allowed.
6. The disallowed_STD3_mapped characters are treated as mapped.
7. The result is normalized to NFC.

Notably:

• The STD3 or WHATWG ASCII deny list should be implemented as a post-processing step.
• Transitional processing is not performed. Transitional mapping would be a pre-processing step, but transitional processing is deprecated, and none of Firefox, Safari, or Chrome use it.

pub fn normalize_validate<'delegate, I: Iterator<Item = char> + 'delegate>( &'delegate self, iter: I, ) -> impl Iterator<Item = char> + 'delegate

Returns an iterator adaptor that turns an Iterator over char into an iterator yielding a char sequence that gets the following operations from the NFC check and statucs steps of the “Validity Criteria” section of UTS 46 lazily applied to it:

1. The ignored characters are treated as disallowed.
2. The mapped characters are mapped.
3. The disallowed characters are replaced with U+FFFD, which itself is a disallowed character.
4. The deviation characters are treated as mapped or valid as appropriate.
5. The disallowed_STD3_valid characters are treated as allowed.
6. The disallowed_STD3_mapped characters are treated as mapped.
7. The result is normalized to NFC.

Notably:

• The STD3 or WHATWG ASCII deny list should be implemented as a post-processing step.
• Transitional processing is not performed. Transitional mapping would be a pre-processing step, but transitional processing is deprecated, and none of Firefox, Safari, or Chrome use it.
• The output needs to be compared with input to see if anything changed. This check catches failures to adhere to the normalization and status requirements. In particular, this comparison results in mapped characters resulting in error like “Validity Criteria” requires.

Trait Implementations

impl Debug for Uts46Mapper

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Uts46Mapper

fn default() -> Self

Returns the “default value” for a type.