Skip to main content

Accurate Word Counter for non-Latin characters in Javascript regex

There is a problem that involves Javascript and regular expressions. The JS implementation of regexp does not support Unicode properly, for example /\b\S+\b/g regular expression will not count words with Unicode characters of many national alphabets and scripts, such as Cyrillic, Greek and Hindi. Unfortunately \S is restricted to Latin-only characters of English alphabet.

To solve this problem we must explicitly include all Unicode characters. My solution is to use /([\u0080-\uFFFF\w]\u0027?)+/g regular expression instead. It covers the wide range of Unicode characters (from 0080 to FFFF) that includes all national alphabets + apostrophe symbol (0027). This regex has been tested with the following sample text and it counts all 55 words accurately, ignoring all special characters and punctuation, I used https://regexr.com to test it with this sample text that includes words from several alphabets.

One Jack's two "three" 'happy' Кав′ярня два три нав'язливо ένα δύο τρία âêîôû,O’Brien/ëïü,är en häst बर्ताव करना चाहिए।.!@#$%^&*()-+=<>?~`|\{}[],.?
Jess' pencils are sharp.
Direct speech in British English - 'That,' he said, 'is nonsense.'
Direct speech in American English - "What time will he arrive?" she asked.
What does 'integrated circuit' mean?
Rock 'n' Roll
view raw sample_unicode_text.txt hosted with ❤ by GitHub

Labels

Labels:

Popular posts from this blog

Integrating TestRail and Gitlab CI/CD

Perhaps you are using Gitlab CI/CD at your project. Every project requires some test cases for regression testing, and Gitlab actually provides this feature, but it’s only available in Ultimate version that is more expensive. TestRail is another popular platform for managing your test suite that provides way more extensive capabilities and options than Gitlab’s own test case management feature. So the chances are that you are still willing to use TestRail for your acceptance and regression testing efforts. Why not combining the best of two worlds — the flexibility of Gitlab CI/CD and rich test case management capabilities of TestRail? In the following example I’ll demonstrate how this goal could be achieved with ease. Let’s assume that we need to create a new Milestone in TestRail that contains two test runs — the one with Acceptance tests, and another one with Regression tests. The step in your .gitlab-ci.yml Gitlab configuration file would look like this: This step is reading C...
Read more

Using querySelector in Selenium WebDriver

Sometimes it's very helpful if we are able to click some element directly, by calling JavaScript click event. For example, this can be the case when the element is not visible, or not clickable, thus we cannot use native WebDriver methods. WebDriver driver = new FirefoxDriver(); JavascriptExecutor js = (JavascriptExecutor)driver; js.executeScript("document.querySelector('.pane .object_options .dropdown-toggle span').click();");
Read more

Switching between keyboard layouts in Openbox (Arch Linux)

Switching between two (or more) keyboard layouts in Openbox DE is a task that's quite easy to accomplish, although it might not be so obvious as in other desktop environments. This solution was tested on Arch Linux. You just need to edit this file (assuming you want to switch between English and Ukrainian Phonetic layouts with Alt-Shift): /etc/X11/xorg.conf.d/01-keyboard-layout.conf Section "InputClass" Identifier "keyboard-layout" Driver "evdev" MatchIsKeyboard "yes" Option "XkbLayout" "us,ua(phonetic)" Option "XkbModel" "pc105" Option "XkbOptions" "grp:alt_shift_toggle" EndSection If you have Nvidia card, don't forget to edit /etc/X11/xorg.conf.d/20-nvidia.conf and change Driver from "kbd" to "evdev" in InputDevice section: Section "InputDevice" Identifier "Keyboard0" Driver "evdev" EndSection Y...
Read more