Struct LanguageIdentifier 

pub struct LanguageIdentifier {
    pub language: Language,
    pub script: Option<Script>,
    pub region: Option<Region>,
    pub variants: Variants,
}

A core struct representing a Unicode BCP47 Language Identifier.

Ordering

This type deliberately does not implement Ord or PartialOrd because there are multiple possible orderings. Depending on your use case, two orderings are available:

1. A string ordering, suitable for stable serialization: LanguageIdentifier::strict_cmp
2. A struct ordering, suitable for use with a BTreeSet: LanguageIdentifier::total_cmp

See issue: https://github.com/unicode-org/icu4x/issues/1215

Parsing

Unicode recognizes three levels of standard conformance for any language identifier:

• well-formed - syntactically correct
• valid - well-formed and only uses registered language, region, script and variant subtags…
• canonical - valid and no deprecated codes or structure.

At the moment parsing normalizes a well-formed language identifier converting _ separators to - and adjusting casing to conform to the Unicode standard.

Any syntactically invalid subtags will cause the parsing to fail with an error.

This operation normalizes syntax to be well-formed. No legacy subtag replacements is performed. For validation and canonicalization, see LocaleCanonicalizer.

Serde

This type implements serde::Serialize and serde::Deserialize if the "serde" Cargo feature is enabled on the crate.

The value will be serialized as a string and parsed when deserialized. For tips on efficient storage and retrieval of locales, see crate::zerovec.

Examples

Simple example:

use icu::locale::{
    langid,
    subtags::{language, region},
};

let li = langid!("en-US");

assert_eq!(li.language, language!("en"));
assert_eq!(li.script, None);
assert_eq!(li.region, Some(region!("US")));
assert_eq!(li.variants.len(), 0);

More complex example:

use icu::locale::{
    langid,
    subtags::{language, region, script, variant},
};

let li = langid!("eN-latn-Us-Valencia");

assert_eq!(li.language, language!("en"));
assert_eq!(li.script, Some(script!("Latn")));
assert_eq!(li.region, Some(region!("US")));
assert_eq!(li.variants.first(), Some(&variant!("valencia")));

Fields

language: Language

Language subtag of the language identifier.

script: Option<Script>

Script subtag of the language identifier.

region: Option<Region>

Region subtag of the language identifier.

variants: Variants

Variant subtags of the language identifier.

Implementations

impl LanguageIdentifier

pub const UNKNOWN: Self

The unknown language identifier “und”.

pub fn try_from_str(s: &str) -> Result<Self, ParseError>

A constructor which takes a utf8 slice, parses it and produces a well-formed LanguageIdentifier.

✨ Enabled with the alloc Cargo feature.

Note: Support for the legacy _ separator has been dropped since 2.0.0. Users of ICU4X need to convert the _ to - before calling the function.

Examples

use icu::locale::LanguageIdentifier;

LanguageIdentifier::try_from_str("en-US").expect("Parsing failed");

pub fn try_from_utf8(code_units: &[u8]) -> Result<Self, ParseError>

See Self::try_from_str

✨ Enabled with the alloc Cargo feature.

pub fn try_from_locale_bytes(v: &[u8]) -> Result<Self, ParseError>

A constructor which takes a utf8 slice which may contain extension keys, parses it and produces a well-formed LanguageIdentifier.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::locale::{LanguageIdentifier, langid};

let li = LanguageIdentifier::try_from_locale_bytes(b"en-US-x-posix")
    .expect("Parsing failed.");

assert_eq!(li, langid!("en-US"));

This method should be used for input that may be a locale identifier. All extensions will be lost.

pub const fn is_unknown(&self) -> bool

Whether this LanguageIdentifier equals LanguageIdentifier::UNKNOWN.

pub fn normalize_utf8(input: &[u8]) -> Result<Cow<'_, str>, ParseError>

Normalize the language identifier (operating on UTF-8 formatted byte slices)

This operation will normalize casing.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::locale::LanguageIdentifier;

assert_eq!(
    LanguageIdentifier::normalize_utf8(b"pL-latn-pl").as_deref(),
    Ok("pl-Latn-PL")
);

pub fn normalize(input: &str) -> Result<Cow<'_, str>, ParseError>

Normalize the language identifier (operating on strings)

This operation will normalize casing.

✨ Enabled with the alloc Cargo feature.

Examples

use icu::locale::LanguageIdentifier;

assert_eq!(
    LanguageIdentifier::normalize("pL-latn-pl").as_deref(),
    Ok("pl-Latn-PL")
);

pub fn strict_cmp(&self, other: &[u8]) -> Ordering

Compare this LanguageIdentifier with BCP-47 bytes.

The return value is equivalent to what would happen if you first converted this LanguageIdentifier to a BCP-47 string and then performed a byte comparison.

This function is case-sensitive and results in a total order, so it is appropriate for binary search. The only argument producing Ordering::Equal is self.to_string().

Examples

Sorting a list of langids with this method requires converting one of them to a string:

use icu::locale::LanguageIdentifier;
use std::cmp::Ordering;
use writeable::Writeable;

// Random input order:
let bcp47_strings: &[&str] = &[
    "ar-Latn",
    "zh-Hant-TW",
    "zh-TW",
    "und-fonipa",
    "zh-Hant",
    "ar-SA",
];

let mut langids = bcp47_strings
    .iter()
    .map(|s| s.parse().unwrap())
    .collect::<Vec<LanguageIdentifier>>();
langids.sort_by(|a, b| {
    let b = b.write_to_string();
    a.strict_cmp(b.as_bytes())
});
let strict_cmp_strings = langids
    .iter()
    .map(|l| l.to_string())
    .collect::<Vec<String>>();

// Output ordering, sorted alphabetically
let expected_ordering: &[&str] = &[
    "ar-Latn",
    "ar-SA",
    "und-fonipa",
    "zh-Hant",
    "zh-Hant-TW",
    "zh-TW",
];

assert_eq!(expected_ordering, strict_cmp_strings);

pub(crate) fn as_tuple( &self, ) -> (Language, Option<Script>, Option<Region>, &Variants)

pub fn total_cmp(&self, other: &Self) -> Ordering

Compare this LanguageIdentifier with another LanguageIdentifier field-by-field. The result is a total ordering sufficient for use in a BTreeSet.

Unlike LanguageIdentifier::strict_cmp, the ordering may or may not be equivalent to string ordering, and it may or may not be stable across ICU4X releases.

Examples

This method returns a nonsensical ordering derived from the fields of the struct:

use icu::locale::LanguageIdentifier;
use std::cmp::Ordering;

// Input strings, sorted alphabetically
let bcp47_strings: &[&str] = &[
    "ar-Latn",
    "ar-SA",
    "und-fonipa",
    "zh-Hant",
    "zh-Hant-TW",
    "zh-TW",
];
assert!(bcp47_strings.windows(2).all(|w| w[0] < w[1]));

let mut langids = bcp47_strings
    .iter()
    .map(|s| s.parse().unwrap())
    .collect::<Vec<LanguageIdentifier>>();
langids.sort_by(LanguageIdentifier::total_cmp);
let total_cmp_strings = langids
    .iter()
    .map(|l| l.to_string())
    .collect::<Vec<String>>();

// Output ordering, sorted arbitrarily
let expected_ordering: &[&str] = &[
    "ar-SA",
    "ar-Latn",
    "und-fonipa",
    "zh-TW",
    "zh-Hant",
    "zh-Hant-TW",
];

assert_eq!(expected_ordering, total_cmp_strings);

Use a wrapper to add a LanguageIdentifier to a BTreeSet:

use icu::locale::LanguageIdentifier;
use std::cmp::Ordering;
use std::collections::BTreeSet;

#[derive(PartialEq, Eq)]
struct LanguageIdentifierTotalOrd(LanguageIdentifier);

impl Ord for LanguageIdentifierTotalOrd {
    fn cmp(&self, other: &Self) -> Ordering {
        self.0.total_cmp(&other.0)
    }
}

impl PartialOrd for LanguageIdentifierTotalOrd {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        Some(self.cmp(other))
    }
}

let _: BTreeSet<LanguageIdentifierTotalOrd> = unimplemented!();

pub fn normalizing_eq(&self, other: &str) -> bool

Compare this LanguageIdentifier with a potentially unnormalized BCP-47 string.

The return value is equivalent to what would happen if you first parsed the BCP-47 string to a LanguageIdentifier and then performed a structural comparison.

Examples

use icu::locale::LanguageIdentifier;

let bcp47_strings: &[&str] = &[
    "pl-LaTn-pL",
    "uNd",
    "UnD-adlm",
    "uNd-GB",
    "UND-FONIPA",
    "ZH",
];

for a in bcp47_strings {
    assert!(a.parse::<LanguageIdentifier>().unwrap().normalizing_eq(a));
}

pub(crate) fn for_each_subtag_str<E, F>(&self, f: &mut F) -> Result<(), E>

where F: FnMut(&str) -> Result<(), E>,

pub(crate) fn for_each_subtag_str_lowercased<E, F>( &self, f: &mut F, ) -> Result<(), E>

where F: FnMut(&str) -> Result<(), E>,

Executes f on each subtag string of this LanguageIdentifier, with every string in lowercase ascii form.

The default normalization of language identifiers uses titlecase scripts and uppercase regions. However, this differs from RFC6497 (BCP 47 Extension T), which specifies:

The canonical form for all subtags in the extension is lowercase, with the fields ordered by the separators, alphabetically.

Hence, this method is used inside Transform Extensions to be able to get the correct normalization of the language identifier.

As an example, the canonical form of locale EN-LATN-CA-T-EN-LATN-CA is en-Latn-CA-t-en-latn-ca, with the script and region parts lowercased inside T extensions, but titlecased and uppercased outside T extensions respectively.

pub(crate) fn write_lowercased_to<W: Write + ?Sized>( &self, sink: &mut W, ) -> Result

Writes this LanguageIdentifier to a sink, replacing uppercase ascii chars with lowercase ascii chars.

The default normalization of language identifiers uses titlecase scripts and uppercase regions. However, this differs from RFC6497 (BCP 47 Extension T), which specifies:

The canonical form for all subtags in the extension is lowercase, with the fields ordered by the separators, alphabetically.

Hence, this method is used inside Transform Extensions to be able to get the correct normalization of the language identifier.

As an example, the canonical form of locale EN-LATN-CA-T-EN-LATN-CA is en-Latn-CA-t-en-latn-ca, with the script and region parts lowercased inside T extensions, but titlecased and uppercased outside T extensions respectively.

impl LanguageIdentifier

pub fn to_string(&self) -> String

Converts the given value to a String.

Under the hood, this uses an efficient [Writeable] implementation.

If you don’t need an allocated String, but e.g. need to write this to some sink, it is more efficient to use [Writeable] directly.

Trait Implementations

impl AsRef<LanguageIdentifier> for LanguageIdentifier

fn as_ref(&self) -> &LanguageIdentifier

Converts this type into a shared reference of the (usually inferred) input type.

impl AsRef<LanguageIdentifier> for Locale

fn as_ref(&self) -> &LanguageIdentifier

Converts this type into a shared reference of the (usually inferred) input type.

impl Clone for LanguageIdentifier

fn clone(&self) -> LanguageIdentifier

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LanguageIdentifier

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for LanguageIdentifier

This trait is implemented for compatibility with fmt!. To create a string, [Writeable::write_to_string] is usually more efficient.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&LanguageIdentifier> for (Language, Option<Script>, Option<Region>)

Convert from a LanguageIdentifier to an LSR tuple.

Examples

use icu::locale::{
    langid,
    subtags::{language, region, script},
};

let lid = langid!("en-Latn-US");
let (lang, script, region) = (&lid).into();

assert_eq!(lang, language!("en"));
assert_eq!(script, Some(script!("Latn")));
assert_eq!(region, Some(region!("US")));

fn from(langid: &LanguageIdentifier) -> Self

Converts to this type from the input type.

impl From<&LanguageIdentifier> for DataLocale

fn from(langid: &LanguageIdentifier) -> Self

Converts to this type from the input type.

impl From<&LanguageIdentifier> for LocalePreferences

fn from(lid: &LanguageIdentifier) -> Self

Converts to this type from the input type.

impl From<(Language, Option<Script>, Option<Region>)> for LanguageIdentifier

Convert from an LSR tuple to a LanguageIdentifier.

Examples

use icu::locale::{
    LanguageIdentifier, langid,
    subtags::{language, region, script},
};

let lang = language!("en");
let script = script!("Latn");
let region = region!("US");
assert_eq!(
    LanguageIdentifier::from((lang, Some(script), Some(region))),
    langid!("en-Latn-US")
);

fn from(lsr: (Language, Option<Script>, Option<Region>)) -> Self

Converts to this type from the input type.

impl From<Language> for LanguageIdentifier

Examples

use icu::locale::{LanguageIdentifier, langid, subtags::language};

assert_eq!(LanguageIdentifier::from(language!("en")), langid!("en"));

fn from(language: Language) -> Self

Converts to this type from the input type.

impl From<LanguageIdentifier> for DataLocale

fn from(langid: LanguageIdentifier) -> Self

Converts to this type from the input type.

impl From<LanguageIdentifier> for Locale

fn from(id: LanguageIdentifier) -> Self

Converts to this type from the input type.

impl From<Locale> for LanguageIdentifier

fn from(loc: Locale) -> Self

Converts to this type from the input type.

impl From<Option<Region>> for LanguageIdentifier

Examples

use icu::locale::{LanguageIdentifier, langid, subtags::region};

assert_eq!(
    LanguageIdentifier::from(Some(region!("US"))),
    langid!("und-US")
);

fn from(region: Option<Region>) -> Self

Converts to this type from the input type.

impl From<Option<Script>> for LanguageIdentifier

Examples

use icu::locale::{LanguageIdentifier, langid, subtags::script};

assert_eq!(
    LanguageIdentifier::from(Some(script!("latn"))),
    langid!("und-Latn")
);

fn from(script: Option<Script>) -> Self

Converts to this type from the input type.

impl FromStr for LanguageIdentifier

Available on crate feature alloc only.

✨ Enabled with the alloc Cargo feature.

type Err = ParseError

The associated error which can be returned from parsing.

fn from_str(s: &str) -> Result<Self, Self::Err>

Parses a string s to return a value of this type.

impl Hash for LanguageIdentifier

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for LanguageIdentifier

fn eq(&self, other: &LanguageIdentifier) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Writeable for LanguageIdentifier

fn write_to<W: Write + ?Sized>(&self, sink: &mut W) -> Result

Writes a string to the given sink. Errors from the sink are bubbled up. The default implementation delegates to write_to_parts, and discards any Part annotations.

fn writeable_length_hint(&self) -> LengthHint

Returns a hint for the number of UTF-8 bytes that will be written to the sink.

fn writeable_borrow(&self) -> Option<&str>

Returns a &str that matches the output of write_to, if possible.

fn write_to_parts<S>(&self, sink: &mut S) -> Result<(), Error>

where S: PartsWrite + ?Sized,

Write bytes and Part annotations to the given sink. Errors from the sink are bubbled up. The default implementation delegates to write_to, and doesn’t produce any Part annotations.

fn write_to_string(&self) -> Cow<'_, str>

Creates a new string with the data from this Writeable.

impl Eq for LanguageIdentifier

impl StructuralPartialEq for LanguageIdentifier