feat: add `translations:` CLI commands and update runtime i18n integration by brian-smith-tcril · Pull Request #205 · openedx/frontend-base

brian-smith-tcril · 2026-04-02T08:34:34Z

Part of i18n pipeline #176

Summary

Adds two new CLI commands for managing translations, plus the runtime wiring to feed pulled translations into the app.

CLI commands:

translations:prepare — generates src/i18n/messages.ts and per-package index.ts files from the locale JSON files in src/i18n/messages/ and src/i18n/site-messages/.
translations:pull — resolves atlasTranslations from the site's own package.json and atlasTranslations.dependencies transitively from each dependency's package.json, calls atlas pull with the resulting FROM:TO mappings, then runs translations:prepare.

Runtime integration:

Adds a site.i18n webpack alias (mirrors site.config) pointing at src/i18n/ in the site. A getResolvedSiteI18nPath utility handles resolution, with SITE_I18N_PATH env var override support.
shell/site.tsx now imports messages from site.i18n instead of the shell-internal placeholder.
Removes App.messages from the App type and addAppMessages() from the runtime — all translations now flow through initialize({ messages }) from the site-level pull.
Adds SiteMessages type (LocalizedMessages[]) to types.ts and declares site.i18n in frontend-base.d.ts.
Deletes shell/i18n/ placeholder directory (was for the old make pull_translations system).
Adds a NormalModuleReplacementPlugin to webpack configs that substitutes an empty module for the ./messages import in src/i18n/ when messages.ts hasn't been generated yet, so the app builds cleanly on a fresh clone before translations:pull has been run.

Structure

All logic lives in tools/cli/utils/translations/:

tools/cli/commands/translations.ts         — runPrepare, runPull (thin runners, argv parsing only)
tools/cli/utils/translations/index.ts      — re-exports prepare and pull
tools/cli/utils/translations/prepare.ts    — prepare + helpers (directory walking, index writing)
tools/cli/utils/translations/pull.ts       — pull + helpers (dependency resolution, atlas invocation)
tools/cli/utils/translations/messagesObject.ts — pure data/rendering for locale index files

Directory layout (after pull)

src/i18n/
├── index.ts              ← checked in, re-exports from messages.ts
├── messages.d.ts         ← checked in, TypeScript type stub (satisfies type checker before translations:pull has been run)
├── messages.ts           ← gitignored, generated by translations:prepare
├── messages/             ← gitignored, populated by atlas pull
│   └── @openedx/
│       ├── frontend-app-authn/
│       ├── frontend-base/
│       └── paragon/
└── site-messages/        ← checked in, operator-managed local translations
    ├── README.md
    └── *.json            ← operator's own translations; merged in last (override behavior)

Gitignore entries for src/i18n/:

src/i18n/messages.ts
src/i18n/messages/
src/i18n/site-messages/index.ts

CLI usage

# Generate src/i18n/messages.ts (and per-package index.ts files)
openedx translations:prepare

# Pull translations from atlas and then prepare
openedx translations:pull

# Pull without regenerating (useful for debugging)
openedx translations:pull --no-prepare

# Pass atlas flags (e.g. to target a specific repo/revision)
openedx translations:pull --atlas-options="--repository=https://github.com/example/translations --revision=main"

Makefile parity

The old pull_translations Makefile target used ATLAS_OPTIONS, ATLAS_EXTRA_SOURCES, and ATLAS_EXTRA_INTL_IMPORTS. This PR covers:

ATLAS_OPTIONS → --atlas-options="..." CLI arg, passed through as a raw string
ATLAS_EXTRA_SOURCES → add entries to atlasTranslations.dependencies in package.json
ATLAS_EXTRA_INTL_IMPORTS → not applicable; translations:prepare derives its inputs from the messages directory directly

For backwards compatibility with how ATLAS_OPTIONS is passed via Makefile, sites should have a pull_translations target like:

pull_translations:
	openedx translations:pull --atlas-options="$(ATLAS_OPTIONS)"

Also included

lint:fix and lint:fix:tools npm scripts
Upgrade tools/tsconfig.json from @tsconfig/node20 to @tsconfig/node24 (project runs on Node 24)
Upgrade @types/node from ^18 to ^24
Exclude test files from tools/tsconfig.build.json to prevent build errors from Node 24-only test APIs
Add "types": ["jest"] to tools/tsconfig.json to fix VS Code not recognizing Jest globals in test files

Implementation log

Full design context and implementation decisions: https://gist.github.com/brian-smith-tcril/548f661196820ebe89a0a6ba9a29cf61

Before moving out of draft

Add atlasTranslations to frontend-base/package.json with the correct atlas path
Create a testing branch of frontend-template-site pointing at branches of each dependency that have atlasTranslations, and verify end-to-end pull and prepare behavior
- Add atlasTranslations to frontend-app-authn/package.json
- Add atlasTranslations to frontend-app-learner-dashboard/package.json
- Add atlasTranslations to paragon/package.json
Add frontend-base translations to openedx/openedx-translations

🤖 Generated with Claude Code

brian-smith-tcril

Did a quick self-review to add context/spark conversation.

brian-smith-tcril · 2026-04-08T14:29:34Z

shell/site.tsx

 import { addAppConfigs } from '../runtime/config';
-import { addAppMessages } from '../runtime/i18n';
-import messages from './i18n';
+import messages from 'site.i18n';


see ‎tools/webpack/utils/getResolvedSiteI18nPath.ts

brian-smith-tcril · 2026-04-08T14:30:57Z

test-site/src/i18n/README.md

@@ -1,3 +0,0 @@
-# Test i18n directories
-
-These test files are used by the `src/i18n/scripts/intl-imports.test.js` file.


intl-imports no longer exists. It has been fully replaced by the openedx translations:pull and openedx translations:prepare commands.

brian-smith-tcril · 2026-04-08T14:36:10Z

tools/cli/commands/translations.ts

+import { prepare, pull } from '../utils/translations';
+
+export function runPrepare(): void {
+  prepare({ siteRoot: process.cwd() });
+}
+
+export function runPull(): void {
+  pull({
+    siteRoot: process.cwd(),
+    execSync: (cmd) => child_process.execSync(cmd, { stdio: 'inherit' }),
+    shouldPrepare: !process.argv.includes('--no-prepare'),
+    atlasOptions: process.argv.find(a => a.startsWith('--atlas-options='))?.slice('--atlas-options='.length),
+  });
+}


The pull/runPull, prepare/runPrepare split is mostly to make testing easier, but it has an added benefit of separation of CLI concerns (pull doesn't need to parse args or know about child process stuff).

brian-smith-tcril · 2026-04-08T14:43:45Z

tools/cli/README.md

-All micro-frontends have a `Makefile` with a line that loads the scripts from the above path:
-
-```
-intl_imports = ./node_modules/.bin/intl-imports.js
-transifex_utils = ./node_modules/.bin/transifex-utils.js
-```
-
-So if you delete either of the files or the `cli` directory, you'll break all micro-frontend builds. Happy coding!


Assuming the migration docs are followed, intl_imports and transifex_utils won't be used in frontend-app Makefiles anymore.

brian-smith-tcril · 2026-04-08T14:46:41Z

tools/webpack/common-config/site/getI18nMessagesFallbackPlugin/index.ts

+// When messages.ts hasn't been generated yet (i.e. translations:pull hasn't been run),
+// replace the ./messages import in src/i18n/index.ts with an empty fallback so webpack
+// doesn't error on the missing file.
+export default function getI18nMessagesFallbackPlugin(): webpack.NormalModuleReplacementPlugin {


Context on this one (from a claude plan doc):

src/i18n/messages.ts is gitignored and generated by prepare/translations:pull. src/i18n/index.ts is committed boilerplate that site operators can customize — by default it re-exports from ./messages. prepare's blast radius is intentionally limited to messages.ts; it never touches index.ts.

The current state in the repo has messages.ts force-added to git (via git add -f). The intent was to make it always exist on clone without tracking changes — but force-adding actually does track the file, overriding .gitignore. This means prepare regenerating it after translations:pull shows it as modified in git status, which is undesirable.

The correct fix: remove messages.ts from git tracking (git rm --cached), leave it gitignored as intended, and solve the "file doesn't exist on fresh clone" problem at the webpack level instead. On a fresh clone (before running translations:pull), webpack fails to resolve the ./messages import in index.ts and shows a "compiled with problems" overlay.

Approaches considered and rejected

Commit a stub messages.ts (not gitignored) — after translations:pull + prepare, messages.ts shows as modified in git status. Undesirable since translations should be invisible to git.

Run prepare before dev/build — rejected because dev/build should have nothing to do with translations; developers shouldn't need to think about translations just to run the app.

postinstall script — messy, especially with workspaces; adds fragility to npm install.

Webpack stub creation (webpack config writes a stub messages.ts if missing) — "doing something silly to make sure the file exists."

try/catch require('./messages') in index.ts — webpack statically analyzes all require() calls regardless of try/catch, still produces a "Module not found" warning and shows a "compiled with problems" overlay in the dev server. Also requires a declare const require and loses the clean static import in index.ts, which site operators should be able to read and customize easily.

Alias site.i18n to an empty module when messages.ts is missing — rejected because site.i18n must always resolve to the app's src/i18n/index.ts. Site operators are expected to customize index.ts (e.g. loading messages from a runtime API), so bypassing it entirely would silently ignore their customizations on fresh clone.

Chosen approach

Add a NormalModuleReplacementPlugin to frontend-base's webpack configs that transparently substitutes an empty module for the ./messages import when messages.ts is missing. This keeps index.ts as a clean static re-export, produces no webpack warnings, and is invisible to app developers and site operators.

brian-smith-tcril · 2026-04-08T14:47:57Z

tools/webpack/utils/getResolvedSiteI18nPath.ts

+import path from 'path';
+
+export default function getResolvedSiteI18nPath(defaultDirname: string) {
+  const siteI18nPath = process.env.SITE_I18N_PATH;


I was following the pattern from getResolvedSiteConfigPath for this one. Could definitely simplify this if we don't try to handle any process.env stuff.

brian-smith-tcril · 2026-04-08T14:51:55Z

tools/tsconfig.build.json

+  "exclude": [
+    "**/*.test.ts",
+    "**/*.test.js"
+  ]


From 02f65df

Without this the test files were getting added to dist.

brian-smith-tcril · 2026-04-08T14:53:05Z

tools/tsconfig.json

@@ -1,10 +1,11 @@
 {
-  "extends": "@tsconfig/node20/tsconfig.json",
+  "extends": "@tsconfig/node24/tsconfig.json",


Updated because I'm using mkdtempDisposable for some tests. We already have 24 in the .nvmrc so I figured updating this would be fine.

brian-smith-tcril · 2026-04-08T15:22:17Z

package.json

+    "lint:fix": "eslint . --fix; npm run lint:fix:tools",
    "lint:tools": "cd ./tools && eslint . && cd ..",
+    "lint:fix:tools": "cd ./tools && eslint . --fix && cd ..",


I like having lint:fix around as a script (especially when lint is multi-part so appending -- --fix doesn't "just work"), but I can remove it if we don't want it.

brian-smith-tcril · 2026-04-08T15:23:09Z

package.json

  },
  "peerDependencies": {
-    "@openedx/paragon": "^23.4.5",
+    "@openedx/paragon": "^23.20.0",


This is the first version with an atlasTranslations block in its package.json.

arbrandes

Looks awesome! I would've pressed merge, but Claude caught a couple of things I think are worth addressing. The shell injection issue is the main one, but the second one might be worth a quick look, too.

1. Shell injection via `execSync` in `pull.ts`

Severity: Medium - requires a malicious package.json in node_modules or a crafted --atlas-options CLI argument.

In tools/cli/utils/translations/pull.ts#L148:

execSync(`atlas pull ${atlasOptions} ${atlasMappings}`);

Both atlasOptions (from CLI --atlas-options=...) and atlasMappings (from atlasTranslations.path and name fields in dependency package.json files) are interpolated into a shell command string without any sanitization or escaping. Since child_process.execSync runs through /bin/sh, shell metacharacters in these values would be interpreted.

The atlasOptions vector requires a user to pass a malicious --atlas-options value, so it is low-risk in practice. The atlasMappings vector is more concerning: a compromised or malicious package in node_modules with a crafted atlasTranslations.path like foo; rm -rf /; echo would execute arbitrary commands during translations:pull.

Suggested fix: switch execSync to execFileSync('atlas', ['pull', ...args]) with arguments passed as an array, which bypasses shell interpretation entirely. This would also require the caller in tools/cli/commands/translations.ts#L11 to change its execSync signature accordingly.

2. `getResolvedSiteI18nPath` diverges from `getResolvedSiteConfigPath` behavior

Severity: Low - the functions handle missing paths differently, which could confuse future maintainers.

tools/webpack/utils/getResolvedSiteI18nPath.ts calls process.exit(1) when the default src/i18n directory doesn't exist, while getResolvedSiteConfigPath falls through to process.exit(1) only after checking both env var and default paths.

For site.i18n, this means npm run dev or npm run build will hard-abort if src/i18n/ doesn't exist, even though the getI18nMessagesFallbackPlugin was specifically designed to handle missing messages.ts gracefully. A site that hasn't set up src/i18n/index.ts yet will get a webpack-level crash rather than a helpful error. This is especially relevant on fresh clones before translations:pull has been run - the test-site has src/i18n/index.ts committed, but other sites might not.

brian-smith-tcril · 2026-04-08T20:59:05Z

1. Shell injection via execSync in pull.ts

Severity: Medium - requires a malicious package.json in node_modules or a crafted --atlas-options CLI argument.

Addressed in e2b256a

2. getResolvedSiteI18nPath diverges from getResolvedSiteConfigPath behavior

Severity: Low - the functions handle missing paths differently, which could confuse future maintainers.

tools/webpack/utils/getResolvedSiteI18nPath.ts calls process.exit(1) when the default src/i18n directory doesn't exist, while getResolvedSiteConfigPath falls through to process.exit(1) only after checking both env var and default paths.

I think getResolvedSiteI18nPath has the right (or at least my preferred) behavior here (fail immediately on a bad env var). I opened an issue the other day (#210) to bring getResolvedSiteConfigPath in line with this, though that issue is specifically about the error messaging rather than the fallthrough behavior itself.

If we want to only exit(1) after trying both the env var dir and the default dir (silently fall back) I'm open to changing the logic.

For site.i18n, this means npm run dev or npm run build will hard-abort if src/i18n/ doesn't exist, even though the getI18nMessagesFallbackPlugin was specifically designed to handle missing messages.ts gracefully. A site that hasn't set up src/i18n/index.ts yet will get a webpack-level crash rather than a helpful error. This is especially relevant on fresh clones before translations:pull has been run - the test-site has src/i18n/index.ts committed, but other sites might not.

This part feels like a misunderstanding of how this is intended to work:

src/i18n/index.ts: always exists. Checked in to frontend-template-site. Checked in to frontend-apps (for their npm run dev dev sites). Included in the frontend-app migration docs.
openedx translations:pull intentionally never touches src/i18n/index.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…uild Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Atlas places files at <cwd>/<TO> directly. The TO was the bare package name (e.g. @openedx/frontend-app-authn), so files landed at the site root instead of src/i18n/messages/ where prepare expects them. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Add `site.i18n` webpack alias (dev, build, dev-shell configs) using a new `getResolvedSiteI18nPath` utility mirroring `getResolvedSiteConfigPath` - Declare `site.i18n` module in `frontend-base.d.ts` using new `SiteMessages` type (`LocalizedMessages[]`) added to `types.ts` - Remove `App.messages`, `addAppMessages()`, and `shell/i18n/` placeholder — all translations now flow through `initialize({ messages })` from the site-level pull - Add `test-site/src/i18n/index.ts` (empty array) for the dev shell Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- Remove `intl-imports` and `transifex-utils` CLI scripts and their test fixtures — both are superseded by `translations:pull`/`translations:prepare` and the updated extraction pipeline in alpha.16+ - Remove dead per-app i18n placeholder directories (`shell/dev/devHome/i18n/`, `test-site/src/authenticated-page/i18n/`) now that `App.messages` is gone - Update `tools/cli/README.md` and `docs/how_tos/migrate-frontend-app.md` to reflect the new translation workflow Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

23.20.0 added atlasTranslations to package.json, enabling transitive translation resolution via openedx translations:pull. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

When an app runs translations:pull directly, it acts as both the site and a package. Previously only atlasTranslations.dependencies were pulled; atlasTranslations.path on the top-level package was ignored. Refactored the resolver to start from the top-level package.json and recurse into dependencies uniformly, so the running app's own translations are included alongside its dependency translations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

When translations:pull hasn't been run yet, messages.ts doesn't exist and webpack would fail to resolve the import in src/i18n/index.ts. NormalModuleReplacementPlugin now substitutes an empty messages module when the file is missing, so the dev server and build work on a fresh clone without requiring translations to be pulled first. Also documents the required src/i18n/index.ts and messages.d.ts files in the app migration guide. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

arbrandes

Ok, only the potential security issue was really worrisome. It seems the missing-path behavior is not likely to ever happen, in practice. Approved!

openedx-semantic-release-bot · 2026-04-08T21:54:51Z

🎉 This PR is included in version 1.0.0-alpha.22 🎉

The release is available on:

Your semantic-release bot 📦🚀

brian-smith-tcril changed the title ~~feat: add translations:prepare and translations:pull CLI commands~~ feat: add translations:prepare and translations:pull CLI commands Apr 2, 2026

brian-smith-tcril linked an issue Apr 3, 2026 that may be closed by this pull request

i18n pipeline #176

Open

This was referenced Apr 6, 2026

feat: frontend-base i18n support openedx/frontend-app-authn#1661

Draft

feat: add atlasTranslations to package.json openedx/frontend-app-learner-dashboard#824

Draft

chore: test new frontend-base i18n stuff openedx/frontend-template-site#10

Draft

brian-smith-tcril changed the title ~~feat: add translations:prepare and translations:pull CLI commands~~ feat: add translations: CLI commands and update runtime i18n integration Apr 6, 2026

brian-smith-tcril closed this Apr 7, 2026

brian-smith-tcril reopened this Apr 7, 2026

brian-smith-tcril force-pushed the translations-pull branch from bf7e911 to 874086f Compare April 7, 2026 18:58

brian-smith-tcril commented Apr 8, 2026

View reviewed changes

brian-smith-tcril marked this pull request as ready for review April 8, 2026 15:34

brian-smith-tcril requested a review from arbrandes April 8, 2026 15:35

arbrandes requested changes Apr 8, 2026

View reviewed changes

brian-smith-tcril and others added 15 commits April 8, 2026 17:00

chore: add lint:fix and lint:fix:tools scripts

4b93d10

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: upgrade tools tsconfig and @types/node to Node 24

1da5b12

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: add jest types to tools tsconfig and exclude test files from b…

60d2953

…uild Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add translations:prepare and translations:pull CLI commands

465820f

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test: verify stale translation files are replaced by newly pulled ones

9951ee5

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add atlasTranslations to package.json

a9016b1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: bump paragon peer dep to ^23.20.0

95099bc

23.20.0 added atlasTranslations to package.json, enabling transitive translation resolution via openedx translations:pull. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: fix lint errors in pull.ts

77b7e27

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: use execFileSync to prevent shell injection in translations:pull

7e398db

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: rebase onto main and regenerate package-lock

3e4b719

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

brian-smith-tcril force-pushed the translations-pull branch from e2b256a to 3e4b719 Compare April 8, 2026 21:06

arbrandes approved these changes Apr 8, 2026

View reviewed changes

arbrandes merged commit 37e3901 into openedx:main Apr 8, 2026
5 checks passed

openedx-semantic-release-bot added the released on @latest label Apr 8, 2026

		@@ -1,3 +0,0 @@
		# Test i18n directories

		These test files are used by the `src/i18n/scripts/intl-imports.test.js` file.

Conversation

brian-smith-tcril commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Structure

Directory layout (after pull)

CLI usage

Makefile parity

Also included

Implementation log

Before moving out of draft

Uh oh!

brian-smith-tcril left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Approaches considered and rejected

Chosen approach

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arbrandes left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

1. Shell injection via execSync in pull.ts

2. getResolvedSiteI18nPath diverges from getResolvedSiteConfigPath behavior

Uh oh!

brian-smith-tcril commented Apr 8, 2026

1. Shell injection via execSync in pull.ts

2. getResolvedSiteI18nPath diverges from getResolvedSiteConfigPath behavior

Uh oh!

arbrandes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

openedx-semantic-release-bot commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

brian-smith-tcril commented Apr 2, 2026 •

edited

Loading

arbrandes left a comment •

edited

Loading

1. Shell injection via `execSync` in `pull.ts`

2. `getResolvedSiteI18nPath` diverges from `getResolvedSiteConfigPath` behavior

1. Shell injection via `execSync` in `pull.ts`

2. `getResolvedSiteI18nPath` diverges from `getResolvedSiteConfigPath` behavior