icu_segmenter::complex

Function complex_language_segment_utf16

Source
pub(crate) fn complex_language_segment_utf16(
    payloads: &ComplexPayloads,
    input: &[u16],
) -> Vec<usize>
Expand description

Return UTF-16 segment offset array using dictionary or lstm segmenter.