The Intl.Segmenter object is now part of Baseline

Rachel Andrew

You can now use Intl.Segmenter for locale-sensitive text segmentation to split a string into words, sentences, or graphemes.

Many non-Latin languages, such as Chinese and Japanese, don't use spaces to separate words. Therefore, using the JavaScript split() method on whitespace to split text into words, will return incorrect results.

When creating a new Intl.Segmenter object with the Intl.segmenter() constructor, pass in a locale and options including granularity, which can have values of "grapheme", "word", or "sentence". The following example creates a new Intl.Segmenter object for Japanese, splitting on words.

const segmenter = new Intl.Segmenter('ja-JP', { granularity: 'word' });

Calling the segment() method on an Intl.Segmenter object with a string of text returns an iterable:

const segments = segmenter.segment(str);
console.table(Array.from(segments));

Read Using the Intl.Segmenter API on the Polypane blog for an excellent tutorial on how to use this feature.

International Text Segmentation with Intl.Segmenter in JavaScript has more examples, including how to use Intl.Segmenter with emoji.