Function icu_segmenter::complex::complex_language_segment_str

source ยท
pub(crate) fn complex_language_segment_str(
    payloads: &ComplexPayloads,
    input: &str,
) -> Vec<usize>
Expand description

Return UTF-8 segment offset array using dictionary or lstm segmenter.