Compiling CLDR data

[clazette]

The examples that I have seen, here and in the globalization repo, leave me wondering how to load CLDR data. This data, as you know, comes from multiple JSON files yet the examples show loading a single, manually created file, like this example.

Are there examples other than the extremely simple use case that doesn't really articulate how someone would use this in the real world? Why not have an example that illustrates how one would accomplish compiling actual data from the CLDR source? What good is manually creating a single file? Who would do that and why?

Updated 28 Sep
Just through a lot of trial and error I have determined that globalize-compiler just utilizes the CLDR data installed as a dependency, via npm install, to ensure the necessary information is added to the "compiled" js file. Hopefully, some of this information is useful for others attempting to use this library.

I don't see a lot of activity in this repo and I can see hours of frustration headed my way if there isn't any support for this. Will someone be able to assist?

Thanks in advance.

[clazette]

Here is where things begin to breakdown relative to the documentation and what's really going on. I have executed the globalize-compile statement with a couple results.

1. If I use a wildcard for SRC_FILES:
globalize-compiler -l en -m Scripts\i18n\globalize-1.1.1\cldr\main\en-US\messages.json -o i18n-en-US-compiled.js D:\Demo\Globalization\Globalization\Scripts\*.js

I receive a No such file or directory error.

2. If I use a specific js file, that contains calls to the Globalize formatting functions e.g. formatDate, formatNumber, etc.

globalize-compiler -l en -m Scripts\i18n\globalize-1.1.1\cldr\main\en-US\messages.json -o i18n-en-US-compiled.js D:\Demo\Globalization\Globalization\Scripts\i18n.js

I receive a No formatters or parsers has been provided error.

Where are the formatters and parsers? I haven't created anything custom. I have a stock install of Globalize. So, what exactly is Compiler looking for? How will compiler pull in the CLDR data from the multiple JSON files that contain said data since, from what I can tell, only one file can be specified for the CLDR_FILE parameter?

Update 28 Sep
I've made some progress relative to issue 2. In my code, I am instantiating an instance of Globalize e.g. var globalize = Globalize(selectedLocale);. This causes calls to formatters to be globalize.formatDate(new Date(dateTime), { datetime: "medium" }) and globalize-compiler appears to look for "Globalize". Therefore, globalize isn't recognized and the **No formatters or parsers has been provided"" error is received. Unless I am missing something, this is going to mean that all calls to Globalize are going to require passing the locale. Well, no, that doesn't work either. A statement like 'Globalize(selectedLocale).formatMessage("hello");' isn't detected by compiler either which is odd given that the plain javascript example for globalize uses the approach of instantiating a Globalize object with a specific locale 'en = Globalize( "en" );'. So, the example you've provided for globalize won't be picked up by compiler. Is this an accurate statement? What is it that you're expecting in the code in order for the CLI call to work?

More research clarifies why instance calls to formatters doesn't work. The usage portion of the documentation for the CLI makes no mention of this constraint, however, the API section does.

So, what is the proper approach to using Globalize in code so that formatter/parser use is identified by the CLI and the locale can be set?

Extracting constrains

Global methods and its aliases are extracted, for example:

Globalize.numberFormatter( ... );
Globalize.formatNumber( ... );
Instance methods and its aliases are NOT extracted, for example:

// Currently, NOT extracted:
var globalize = new Globalize( locale );
globalize.numberFormatter( ... );
globalize.formatNumber( ... );

Even a literal for locale e.g. Globalize("en").formatDate(new Date()); doesn't result in the formatter's usage being identified by the CLI which wouldn't work anyway but I figured what the heck at this point. I'm just trying to understand how this is supposed to work. These approaches for using Globalize are right out of the globalize documentation yet the globalize-compiler doesn't identify their usage.

Is there anyone available to assist? Can you update the documentation? Are these potential bugs? Am I missing something?

[clazette]

Can someone explain what it is, exactly, that we're accomplishing my using globalize-compiler? Thus far, I have had to fake compiler into recognizing the formatters that I am using because it doesn't recognize instance methods or even Global methods that utilize the constructor that takes the locale string as a parameter. So, to "solve" that issue, I am, basically, hacking my code by placing Globalize statements that mirror what my instance method calls are doing, sans the locale, in order to get globalize-compile to actually generate a file. Then, when I attempt to use the compiled file, I, now, receive the following error - E_MISSING_CLDR: Missing required CLDR content supplemental/likelySubtags. Isn't the whole point of this package to prevent me from having to include all of the CLDR JSON content? Isn't that what the compiled file is supposed to provide? Again, what am I missing about the purpose of compiler and its proper use?

[clazette]

Use globalize-compiler (a) to extract your Globalize formatters and parsers statically from your source code and (b) to compile + to bundle them into a JavaScript file.

So this has nothing to do with CLDR data. It's just the for the formatters and parsers. So, it has nothing to do with extracting locale specific CLDR data based on the formatting that a particular app is using. It's all about optimizing the process of formatting and parsing. It's intended to address the (a) of the process outlined below.

Ok, this is beginning to make sense. So, the big issue at this point is getting the CLI to find my formatter and parser usage. That part of the process isn't clear to me yet.

Performance

When formatting or parsing, there's actually a two-step process: (a) the formatter (or parser) creation and (b) its execution, where creation takes an order of magnitude more time (more expensive) than execution. In the creation phase, Globalize traverses the CLDR tree, processes data (e.g., expands date patterns, parses plural rules, etc), and returns a function that actually executes the formatting or parsing.

[clazette]

More research has indicated that CLDR data doesn't have to be loaded after using globalize-compiler either. Comparing the Development and Production html files located here has been very helpful.

[rxaviers]

how someone would use this in the real world

That very much depends on the stack you use. For example,Twitter mobile site https://mobile.twitter.com/ uses Globalize in a stack similar to this example https://github.com/jquery/globalize/tree/master/examples/app-npm-webpack.

[clazette]

It has taken many days and a whole lot of trial and error but I, basically, have things working now...even with globalize-compiler. I think the main struggle was just understanding how everything came together. The compiler example in the globalize repo helped a lot. I still have some nuances to address but I am making progress.

That said, more prompt responses from someone relative to support could have saved me days I suspect and probably would have helped many others...who knows.

[rxaviers]

Firstly, thanks for your interest in the library. Allow me a couple of days and I will give you a better response.

[kborchers]

@clazette I just want to make sure you realize that, for the most part, everyone working on and maintaining this project does so as a volunteer and is not paid to do this work. People respond as soon as they can and it can be quite discouraging to a maintainer that is doing everything they can to support a project on their own free time when they are being criticized for not giving someone more of their free time sooner.

What would be very much appreciated is that, since it seems you have been able to wrap your head around the project and the issues you faced, any help you could provide through things like sending pull requests to improve documentation, examples, etc based on your experiences could improve that experience for everyone going forward.

Thank you for your patience and we hope you will help us out going forward!

[rxaviers]

About SRC_FILES ... D:\Demo\Globalization\Globalization\Scripts*.js ... I receive a No such file or directory error: It seems like the shell you're using on Windows doesn't expand the * for you... Perhaps a quick fix is using bash in Windows. I'm open for suggestions.

About Globalize instances, e.g., en = new Globalize("en"): yeap, the current compiler doesn't support compiling instances, only the static method, e.g., Globalize.<fn>.

Unless I am missing something, this is going to mean that all calls to Globalize are going to require passing the locale

Nope, check https://github.com/jquery/globalize/tree/master/examples/globalize-compiler, specifically app.js and globalizejs/globalize#637.

A statement like 'Globalize(selectedLocale).formatMessage("hello");' isn't detected by compiler

Correct. This is a static compiler and by that time a variable value is undetermined. You should use the compiler -l parameter to set the default locale for you.

Even a literal for locale e.g. Globalize("en").formatDate(new Date()); doesn't result in the formatter's usage being identified by the CLI which wouldn't work anyway

Sorry, I didn't understan d what you said here. In case this is still a question that you want to make, could you please detail your steps and what problem do you see?

Ok, this is beginning to make sense. So, the big issue at this point is getting the CLI to find my formatter and parser usage. That part of the process isn't clear to me yet.

Ok. Please, let me know if this is still an issue... I assume that after going through the example above, this is responded.

I'd be happy to add some of my findings to the documentation but a pull request places me in some compare changes state and I've not used GitHub as a contributor so I don't know how to proceed.

Excellent and don't worry about tools and processes for now because just sending an improved content already helps. Also, you can click in any documentation file (e.g., https://github.com/jquery-support/globalize-compiler/blob/master/README.md) and click "Edit" and save it.

Thanks in advance for contributing back to the project. That's the spirit.

[rxaviers]

Emphasizing, we should use StackOverflow for Q/A. Let's use this issue for spotting issues with code or documentation that needs fix and to provide action items to address them.

[klihelp]

Hello @rxaviers

Since there is official support from CLDR for fragment packages, would it be possible to utilize them? I am not planning to have the full cldr-data as a dependency.
https://github.com/unicode-cldr/cldr-json#package-organization