Skip to content

Use Intl.Locale internally#605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zbraniecki wants to merge 1 commit into
base:main
Choose a base branch
from
Open

Use Intl.Locale internally #605

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 64 additions & 35 deletions fluent-langneg/src/locale.ts
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,32 @@
/* eslint no-magic-numbers: 0 */
function convertMasks(locale: string): string{
let result;
if (locale[0] === "*"){
result = "und" + locale.substr(1);
} else{
result = locale;
};
return result.replace(/\-\*/g, "");
}

const languageCodeRe = "([a-z]{2,3}|\\*)"
const scriptCodeRe = "(?:-([a-z]{4}|\\*))"
const regionCodeRe = "(?:-([a-z]{2}|\\*))"
const variantCodeRe = "(?:-(([0-9][a-z0-9]{3}|[a-z0-9]{5,8})|\\*))"
function getVisibleLangTagLength(language: any, script: any, region: any){
Copy link
Member

@eemelieemeliFeb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should really be typed as string | undefined; there should be no need for any here.

let result = 0;
result += language ? language.length : "und".length;
result += script ? script.length + 1 : 0;
result += region ? region.length + 1 : 0;
return result;
}

/**
* Regular expression splitting locale id into four pieces:
*
* Example: `en-Latn-US-macos`
*
* language: en
* script: Latn
* region: US
* variant: macos
*
* It can also accept a range `*` character on any position.
*/
const localeRe = new RegExp(
`^${languageCodeRe}${scriptCodeRe}?${regionCodeRe}?${variantCodeRe}?$`,
"i"
);
function getExtensionStart(locale: string): number | undefined{
let pos = locale.search(/-[a-zA-Z]-/);
if (pos === -1){
return undefined;
}
return pos;
}

export class Locale{
isWellFormed: boolean;
language?: string;
language: string;
script?: string;
region?: string;
variant?: string;
Expand All@@ -39,29 +41,55 @@ export class Locale{
* properly parsed as `en-*-US-*`.
*/
constructor(locale: string){
const result = localeRe.exec(locale.replace(/_/g, "-"));
if (!result){
let result;
let normalized = convertMasks(locale.replace(/_/g, "-"));
try{
result = new Intl.Locale(normalized);
} catch (e){
this.isWellFormed = false;
this.language = "und"
return;
}

let [, language, script, region, variant] = result;
this.language = result.language || "und"
Copy link
Member

@eemelieemeliFeb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How could result.language ever be empty?

this.script = result.script;
this.region = result.region;

if (language){
this.language = language.toLowerCase();
let visiblelangTagLength = getVisibleLangTagLength(this.language, this.script, this.region);

if (normalized.length > visiblelangTagLength){
let extStart = getExtensionStart(locale);
this.variant = locale.substring(visiblelangTagLength + 1, extStart);
}

this.isWellFormed = true;
}

static fromComponents({language, script, region, variant}:{
Copy link
Member

@eemelieemeliFeb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could/should this also use Intl.Locale? The original included some normalizations that are included in Intl.Locale processing, but they're dropped in these changes. Code here could construct an options bag to pass as the second arg of the Intl constructor, and then read the language/script/region from the result.

language?: string,
script?: string,
region?: string,
variant?: string,
}): Locale{
let result = new Locale("und");
if (language && language !== "*"){
result.language = language;
}
if (script){
this.script = script[0].toUpperCase() + script.slice(1);
if (script && script !== "*"){
result.script = script;
}
if (region){
this.region = region.toUpperCase();
if (region && region !== "*"){
result.region = region;
}
this.variant = variant;
this.isWellFormed = true;
if (variant && variant !== "*"){
result.variant = variant;
}
return result;
}

isEqual(other: Locale): boolean{
return (
this.isWellFormed === other.isWellFormed &&
this.language === other.language &&
this.script === other.script &&
this.region === other.region &&
Expand All@@ -71,9 +99,10 @@ export class Locale{

matches(other: Locale, thisRange = false, otherRange = false): boolean{
return (
this.isWellFormed && other.isWellFormed &&
(this.language === other.language ||
(thisRange && this.language === undefined) ||
(otherRange && other.language === undefined)) &&
(thisRange && this.language === "und") ||
(otherRange && other.language === "und")) &&
(this.script === other.script ||
(thisRange && this.script === undefined) ||
(otherRange && other.script === undefined)) &&
Expand Down
7 changes: 7 additions & 0 deletions fluent-langneg/test/langneg_test.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -130,6 +130,13 @@ const data ={
"matching",
["fr", "en"],
],
[
["es-419"],
["es", "en"],
undefined,
"matching",
["es"],
],
],
},
lookup:{
Expand Down
13 changes: 12 additions & 1 deletion fluent-langneg/test/locale_test.js
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,7 +3,8 @@ import{Locale } from "../esm/locale.js"

function isLocaleEqual(str, ref){
const locale = new Locale(str);
return locale.isEqual(ref);
const other = Locale.fromComponents(ref);
return locale.isEqual(other);
}

suite("Parses simple locales", () =>{
Expand DownExpand Up@@ -108,6 +109,16 @@ suite("Parses simple locales", () =>{
})
);
});

test("skipping extensions", () =>{
assert.ok(
isLocaleEqual("en-US-macos-linux-u-hc-h12",{
language: "en",
region: "US",
variant: "macos-linux",
})
);
});
});

suite("Parses locale ranges", () =>{
Expand Down
1 change: 1 addition & 0 deletions fluent-langneg/tsconfig.json
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
{
"extends": "../tsconfig.json",
"compilerOptions":{
"target": "es2020",
Copy link
Member

@eemelieemeliFeb 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this enough? I was under the impression that TS only added the Intl.Locale types in es2021.

"outDir": "./esm"
},
"include": ["./src/**/*.ts"]
Expand Down