Integrate Project model into loader and decompose into composable stages#7023
Integrate Project model into loader and decompose into composable stages#7023ryancbahan wants to merge 19 commits intomainfrom
Conversation
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Coverage report
Test suite run success3916 tests passing in 1504 suites. Report generated by 🧪jest coverage report action from c5eef3c |
6c3198a to
c7fcd8b
Compare
d7d2097 to
ff9497f
Compare
c7fcd8b to
d60ce08
Compare
288c896 to
b17ef80
Compare
|
We detected some changes at Caution DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release. |
b17ef80 to
57e86f9
Compare
d60ce08 to
2edfd38
Compare
18f3516 to
839512e
Compare
509aacf to
8964add
Compare
839512e to
87728e2
Compare
Differences in type declarationsWe detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:
New type declarationsWe found no new type declarations in this PR Existing type declarationspackages/cli-kit/dist/public/node/themes/api.d.ts@@ -5,7 +5,6 @@ export type ThemeParams = Partial<Pick<Theme, 'name' | 'role' | 'processing' | '
export type AssetParams = Pick<ThemeAsset, 'key'> & Partial<Pick<ThemeAsset, 'value' | 'attachment'>>;
export declare function fetchTheme(id: number, session: AdminSession): Promise<Theme | undefined>;
export declare function fetchThemes(session: AdminSession): Promise<Theme[]>;
-export declare function findDevelopmentThemeByName(name: string, session: AdminSession): Promise<Theme | undefined>;
export declare function themeCreate(params: ThemeParams, session: AdminSession): Promise<Theme | undefined>;
export declare function fetchThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<ThemeAsset[]>;
export declare function deleteThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<Result[]>;
packages/cli-kit/dist/public/node/themes/theme-manager.d.ts@@ -8,8 +8,8 @@ export declare abstract class ThemeManager {
protected abstract removeTheme(): void;
protected abstract context: string;
constructor(adminSession: AdminSession);
- findOrCreate(name?: string, role?: Role): Promise<Theme>;
- fetch(name?: string, role?: Role): Promise<Theme | undefined>;
+ findOrCreate(): Promise<Theme>;
+ fetch(): Promise<Theme | undefined>;
generateThemeName(context: string): string;
create(themeRole?: Role, themeName?: string): Promise<Theme>;
}
\ No newline at end of file
|
87728e2 to
490c047
Compare
8964add to
2929234
Compare
Wires the Project domain model into the existing loading pipeline: - getAppConfigurationState uses Project.load() for filesystem discovery - getAppConfigurationContext returns project + activeConfig + state as independent values (project is never nested inside state) - AppLoader reads from Project's pre-loaded data: extension files, web files, dotenv, hidden config, deps, package manager, workspaces - No duplicate filesystem scanning — Project discovers once, loader reads from it - AppConfigurationState no longer carries project as a field - LoadedAppContextOutput exposes project and activeConfig as top-level fields for commands - All extension/web file discovery filtered to active config's directories via config-selection functions Zero behavioral changes. All 3801 existing tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the monolithic loadApp pipeline with composable stages:
- loadApp is now a thin wrapper: getAppConfigurationContext → loadAppFromContext
- loadAppFromContext takes narrow Project + ActiveConfig directly
- getAppConfigurationContext is discovery-only (no parsing/state construction)
- ReloadState replaces passing entire AppLinkedInterface through reloads
- AppLoader takes reloadState? instead of previousApp?
- link() returns {remoteApp, configFileName, configuration} (no state)
- linkedAppContext uses activeConfig directly, no AppConfigurationState
Remove dead code: AppConfigurationState, toAppConfigurationState,
loadAppConfigurationFromState, loadAppUsingConfigurationState,
loadAppConfiguration, getAppConfigurationState, getAppDirectory,
loadDotEnv, loadHiddenConfig, findWebConfigPaths, loadWebsForAppCreation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
490c047 to
e775694
Compare
…Path - Remove orphaned <<<<<<< HEAD marker and dead loadHiddenConfig tests - Re-add isAppConfigSpecification import (used at line 776) - Add missing configPath field to loadedConfiguration in loadAppFromContext - Fix configuration.path references to use configPath (path extracted from config type) - Fix testApp() call signature in use.test.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…path Same bug as the other two configuration.path references fixed in e4487d9 — configuration is Zod-parsed and has no path property. Without this fix, activeConfigFile always returns undefined, causing loadWebs() and createExtensionInstances() to load all project files instead of filtering by the active config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
isaacroldan
left a comment
There was a problem hiding this comment.
Review assisted by pair-review
| }) | ||
| extensionConfigPaths.push(`!${joinPath(appDirectory, '**/node_modules/**')}`) | ||
| const configPaths = await glob(extensionConfigPaths) | ||
| private async createExtensionInstances(appDirectory: string, _extensionDirectories?: string[]) { |
There was a problem hiding this comment.
💡 Improvement: createExtensionInstances takes _extensionDirectories (underscore-prefixed to suppress unused-var linting) but completely ignores it. Extension discovery comes from this.activeConfigFile and extensionFilesForConfig, which reads extension_directories from activeConfig.content internally. The parameter is passed from the call site at line 641 as appConfiguration.extension_directories, but it's never consumed.
Suggestion: Remove the _extensionDirectories parameter from the method signature and the corresponding argument from the call site at line 641.
There was a problem hiding this comment.
This is implemented in the next pr in stack 03-17-remove-project-fields-from-app
| import {AppHiddenConfig, BasicAppConfigurationWithoutModules} from '../app/app.js' | ||
| import {AppConfigurationFileName, AppConfigurationState, getConfigurationPath} from '../app/loader.js' | ||
| import {AppHiddenConfig} from '../app/app.js' | ||
| import {getAppConfigurationFileName} from '../app/loader.js' |
There was a problem hiding this comment.
📐 Design: active-config.ts imports getAppConfigurationFileName from loader.js (line 4), and loader.ts imports selectActiveConfig from active-config.js. Additionally, active-config.ts imports use.js (line 6) which imports getAppConfigurationContext from loader.js, creating a deeper cycle: loader → active-config → use → loader. Node ESM handles this without crashing at startup thanks to live bindings, but it's fragile — reordering imports or adding a top-level side effect in any of these files can produce undefined at import time. It also makes isolated testing harder.
Suggestion: Extract getAppConfigurationFileName into a small, leaf-level utility module (e.g., config-naming.ts) that both loader.ts and active-config.ts can import without forming a cycle. This function is a pure string utility with no dependencies on loader internals.
| }) | ||
| setCurrentConfigPreference(configuration, {configFileName, directory}) | ||
| const {activeConfig} = await getAppConfigurationContext(directory, configFileName) | ||
| setCurrentConfigPreference(activeConfig.file.content as AppConfiguration, {configFileName, directory}) |
There was a problem hiding this comment.
Human comment: Are we planning to validate the active config content using the modules schemas?
AI:
🐛 Bug: The old code called loadAppConfiguration which parsed and validated the config against AppSchema before passing it to setCurrentConfigPreference. The new code reads raw TOML via getAppConfigurationContext and casts it with as AppConfiguration, bypassing all schema validation. A malformed config file (invalid field types, missing nested structure, wrong shapes) will now be silently accepted as the current preference. Previously, such files would produce a validation error. This is a behavioral regression — the app config use command no longer validates configs before adopting them.
Suggestion: Either validate the content against AppSchema before setting the preference, or narrow the type to only what setCurrentConfigPreference actually needs. If full validation isn't feasible here, at minimum validate the fields that setCurrentConfigPreference relies on (like client_id).
There was a problem hiding this comment.
I believe this is the correct behavior. "Use" is just switching active TOML -- it should only validate whether the toml is syntactically correct (which it does). It's not running anything, and therefore should not concern itself with schema validation. Other commands handle that.
There was a problem hiding this comment.
So the validation would happen on the next command run?
The casting is weird though, because you can't be sure that content conforms to AppConfiguration :/
There was a problem hiding this comment.
That's fair, though I'm not sure it's rewriting at the moment since it's just passthrough and any "real" command will catch the errors.
1. Remove dead code fallback in createExtensionInstances — since configPaths is derived from extensionFiles, the find is guaranteed to match. Simplified to iterate extensionFiles directly. 2. Extract getAppConfigurationFileName, getAppConfigurationShorthand, isValidFormatAppConfigurationFileName, and AppConfigurationFileName into a leaf utility module (config-file-naming.ts). This breaks the circular import: loader → active-config → use → loader. Loader re-exports for backward compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Let's add tests for this file :)
| // Used when reloading an app, to avoid some expensive steps during loading. | ||
| previousApp?: AppLinkedInterface | ||
| // Pre-discovered project data — avoids re-scanning the filesystem for dependencies, package manager, etc. | ||
| project: Project |
There was a problem hiding this comment.
Umm, during dev you might add a new extension, that forces an app reload. But this project won't have the new files for the new extension right? :thinking_face:
There was a problem hiding this comment.
Yeah I confirmed this:
app dev appears to work when generating an extension mid-dev, because the watcher detects the changes. And it triggers the reloadApp so that the in-memory App representation is updated. But Since this reload is not re-scanning the file-system is not loading the new extensions and they are not included in the deployed manifest.
We need to re-scan when reloading the app :)
There was a problem hiding this comment.
Context from Slack -- this is a pre-existing issue on main branch. Will make sure it's fixed here though.
There was a problem hiding this comment.
The reload is called as expected in this pr. I added a test to demonstrate this
isaacroldan
left a comment
There was a problem hiding this comment.
The reload behaviour has changed, let's fix that first 🙏
I think some extensive tophat of app dev is necessary before merging, even it appears to work, we need to check the manifest.json and the bundle uploaded to ensure it includes everything
Three new tests verifying that extensions added to disk after initial load are correctly discovered on reload: 1. Project.load re-scans filesystem — fresh glob finds new files 2. reloadApp finds new extensions — full reload path works 3. handleWatcherEvents produces Created event — the event handler correctly calls reloadApp and appDiff classifies the new extension as created Also fixed setupRealApp to include api_version in the function extension TOML (required by strict validation mode used during reload). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two pre-existing bugs prevented `app dev` from handling extensions created mid-session: 1. Chokidar v3 silently ignores non-existent directories. Now the file watcher creates extension directories (with .gitkeep) before starting, so new extensions are detected even if the directory didn't exist at startup. 2. createManifest() only wrote manifest.json during CREATE, not UPDATE. Now it always writes the full manifest before filtering modules for the API payload, keeping the on-disk bundle current. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use AppLinkedInterface cast for reloadApp/handleWatcherEvents calls - Import AbortController from cli-kit for correct type compatibility - Rename short identifier `e` to `ev` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Pulling context from Slack -- we discovered the reload issue with manifest writes is already on main, not introduced in this pr. I've added more tests here and will tophat thoroughly + document. |
Tests for getAppConfigurationFileName, getAppConfigurationShorthand, and isValidFormatAppConfigurationFileName covering default values, valid formats, slugification, shorthand extraction, and edge cases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… cast - Strip glob suffixes (e.g. extensions/**) before mkdir to avoid creating literal ** directories. Chokidar handles globs natively. - Remove .gitkeep creation — chokidar only needs the directory to exist. - Add clarifying comment on client_id cast in app-context.ts. - Add test verifying glob stripping behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ata init - Remove `delete rawConfig.path` — in the new architecture, path lives on TomlFile.path and is never mixed into .content. - Collapse configurationLoadResultMetadata into a single const assignment with usesLinkedConfig: true, since loadAppFromContext is always the linked-app path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Load extension specifications once at module level instead of 8 times per test. This is the heaviest operation in the integration tests (loads all specs from disk + parses Zod schemas). Reduces memory churn that was causing fsevents SIGABRT on macOS CI runners. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The mkdir call added to FileWatcher.start() throws EACCES when the app directory is unwritable (e.g. test fixtures using '/' as directory). This unhandled rejection cascaded into an fsevents SIGABRT on macOS CI. - Wrap mkdir in try/catch since it's non-fatal (chokidar handles missing directories gracefully) - Hoist loadLocalExtensionsSpecifications to module level in integration tests to reduce memory pressure (was called 8x) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents real chokidar/fsevents watchers from being created during tests, which caused native addon crashes on macOS CI runners. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lly) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Multi-config dev tests (multi-config-dev.spec.ts): - Verify -c flag loads the named config (staging TOML, correct scopes) - Verify default config is used when -c is omitted - Assert on info banner, scopes, and Config filename in App info tab Dev hot-reload tests (dev-hot-reload.spec.ts): - Editing app config TOML (scopes change) triggers full reload pipeline - Creating a new extension mid-dev is detected by the file watcher - Deleting an extension mid-dev is detected by the file watcher Key design decisions: - Use flow_trigger extensions (build mode 'none') to avoid theme-check, esbuild, and npm install overhead in e2e temp directories - Start dev without extensions, mutate filesystem mid-dev to avoid dev session API rejection on initial CREATE - Assert on watcher detection messages (Extension created/deleted) rather than API success for extension create/delete tests - App config TOML edit test verifies the full round-trip including API success (App config updated + Updated dev preview) - Clear SHOPIFY_FLAG_CLIENT_ID when using -c flag (mutually exclusive) - Use --skip-dependencies-installation to avoid npm install hangs
…rrentConfigPreference type
1. Remove unused _extensionDirectories parameter from createExtensionInstances.
Extension discovery goes through activeConfigFile + extensionFilesForConfig,
making the parameter dead code.
2. Narrow setCurrentConfigPreference to accept {client_id?: string} instead of
AppConfiguration. The function only reads client_id — full schema validation
is deferred to the next command that loads the app. This removes the unsafe
'as AppConfiguration' cast on raw TOML content and drops the dubious
'asserts configuration is CurrentAppConfiguration' return type.
…NT_ID from env Empty string is still passed to the child process, causing the tunnel setup to use an empty client_id and fail with a Cloudflare unmarshal error. Setting to undefined causes the spawn helper's 'value !== undefined' filter to exclude it from the child process environment entirely.
| vi.useFakeTimers() | ||
| // Prevent real chokidar/fsevents watchers from being created in tests. | ||
| // The hook only needs the AppEventWatcher as an EventEmitter, not real file watching. | ||
| vi.spyOn(AppEventWatcher.prototype, 'start').mockResolvedValue() |
There was a problem hiding this comment.
this was hogging up memory now that tests are sharded
|
Added a bunch more tests around hot reload, file watching, and config switching. Doing a bunch of local tophatting and good so far. |

WHY are these changes introduced?
PR #7022 introduced the
ProjectandActiveConfigdomain models, but nothing consumed them (the loader still ran its own filesystem discovery, constructedAppConfigurationStateas an intermediate, and threaded the entire previousAppInterfacethrough reloads). This PR wires the new models into the loading pipeline and uses that as leverage to decompose the monolithicloadAppinto composable stages with narrow interfaces.WHAT is this pull request doing?
Decomposes the loader into three explicit stages:
getAppConfigurationContext(dir, configName)→{project, activeConfig}— discovery only, no parsing or state constructionloadAppFromContext({project, activeConfig, specifications, …})→AppInterface— validation and assembly, for callers that already hold a ProjectloadApp({directory, configName, …})— thin wrapper composing the two above[Interim state] Replaces
previousAppwith narrowReloadState:Instead of threading the entire
AppLinkedInterfacethrough reloads (just to read 2 fields),reloadAppnow constructs aReloadStatewith onlyextensionDevUUIDs: Map<string, string>andpreviousDevURLs. This is a step change away from broad state passing/mutation, but needs more consideration on "permanent home" as we continue to decompose functionality.Updates all consumers:
linkedAppContextusesactiveConfig.isLinkedandactiveConfig.file.contentdirectly instead of going throughAppConfigurationStatelink()returns{remoteApp, configFileName, configuration}— dropsstatefrom its return typeuse()readsactiveConfig.file.contentinstead of callingloadAppConfigurationloadConfigForAppCreationusesactiveConfigandproject.directorydirectlyRemoves dead code (-400 lines):
AppConfigurationState,AppConfigurationStateBasics,toAppConfigurationState,loadAppConfigurationFromState,loadAppUsingConfigurationState,loadAppConfiguration,getAppConfigurationState,getAppDirectory,loadDotEnv,loadHiddenConfig,findWebConfigPaths,loadWebsForAppCreation,getConfigurationPath(de-exported)How to test your changes?
Measuring impact
Checklist
🤖 Generated with Claude Code