Skip to content

Integrate Project model into loader and decompose into composable stages#7023

Open
ryancbahan wants to merge 19 commits intomainfrom
rcb/project-integration
Open

Integrate Project model into loader and decompose into composable stages#7023
ryancbahan wants to merge 19 commits intomainfrom
rcb/project-integration

Conversation

@ryancbahan
Copy link
Contributor

@ryancbahan ryancbahan commented Mar 16, 2026

WHY are these changes introduced?

PR #7022 introduced the Project and ActiveConfig domain models, but nothing consumed them (the loader still ran its own filesystem discovery, constructed AppConfigurationState as an intermediate, and threaded the entire previous AppInterface through reloads). This PR wires the new models into the loading pipeline and uses that as leverage to decompose the monolithic loadApp into composable stages with narrow interfaces.

WHAT is this pull request doing?

Decomposes the loader into three explicit stages:

  • getAppConfigurationContext(dir, configName){project, activeConfig} — discovery only, no parsing or state construction
  • loadAppFromContext({project, activeConfig, specifications, …})AppInterface — validation and assembly, for callers that already hold a Project
  • loadApp({directory, configName, …}) — thin wrapper composing the two above

[Interim state] Replaces previousApp with narrow ReloadState:

Instead of threading the entire AppLinkedInterface through reloads (just to read 2 fields), reloadApp now constructs a ReloadState with only extensionDevUUIDs: Map<string, string> and previousDevURLs. This is a step change away from broad state passing/mutation, but needs more consideration on "permanent home" as we continue to decompose functionality.

Updates all consumers:

  • linkedAppContext uses activeConfig.isLinked and activeConfig.file.content directly instead of going through AppConfigurationState
  • link() returns {remoteApp, configFileName, configuration} — drops state from its return type
  • use() reads activeConfig.file.content instead of calling loadAppConfiguration
  • loadConfigForAppCreation uses activeConfig and project.directory directly

Removes dead code (-400 lines):

AppConfigurationState, AppConfigurationStateBasics, toAppConfigurationState, loadAppConfigurationFromState, loadAppUsingConfigurationState, loadAppConfiguration, getAppConfigurationState, getAppDirectory, loadDotEnv, loadHiddenConfig, findWebConfigPaths, loadWebsForAppCreation, getConfigurationPath (de-exported)

How to test your changes?

npx vitest run packages/app/src/cli/models/app/loader.test.ts
npx vitest run packages/app/src/cli/services/app-context.test.ts
npx vitest run packages/app/src/cli/services/app/config/link.test.ts
npx vitest run packages/app/src/cli/services/app/config/use.test.ts
npx vitest run packages/app/src/cli/services/context.test.ts
npx vitest run packages/app/src/cli/models/project/

Measuring impact

  • n/a

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes

🤖 Generated with Claude Code

Copy link
Contributor Author

ryancbahan commented Mar 16, 2026

@github-actions
Copy link
Contributor

github-actions bot commented Mar 16, 2026

Coverage report

St.
Category Percentage Covered / Total
🟢 Statements 82.2% 14884/18106
🟡 Branches 74.52% 7353/9867
🟢 Functions 81.38% 3762/4623
🟢 Lines 82.6% 14073/17037

Test suite run success

3916 tests passing in 1504 suites.

Report generated by 🧪jest coverage report action from c5eef3c

@ryancbahan ryancbahan force-pushed the rcb/project-integration branch 2 times, most recently from d7d2097 to ff9497f Compare March 17, 2026 14:04
@ryancbahan ryancbahan force-pushed the rcb/project-integration branch 3 times, most recently from 288c896 to b17ef80 Compare March 17, 2026 14:44
@ryancbahan ryancbahan marked this pull request as ready for review March 17, 2026 14:58
@ryancbahan ryancbahan requested a review from a team as a code owner March 17, 2026 14:58
@github-actions
Copy link
Contributor

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

@ryancbahan ryancbahan force-pushed the rcb/project-integration branch from b17ef80 to 57e86f9 Compare March 17, 2026 15:22
@ryancbahan ryancbahan marked this pull request as draft March 17, 2026 20:12
@ryancbahan ryancbahan changed the title Integrate Project model into loader and app-context Decompose loader into composable stages with narrow interfaces Mar 17, 2026
@ryancbahan ryancbahan changed the title Decompose loader into composable stages with narrow interfaces Integrate Project model into loader and decompose into composable stages Mar 17, 2026
@ryancbahan ryancbahan force-pushed the rcb/project-integration branch from 18f3516 to 839512e Compare March 17, 2026 23:05
@ryancbahan ryancbahan force-pushed the rcb/project-integration branch from 839512e to 87728e2 Compare March 17, 2026 23:18
@github-actions
Copy link
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/themes/api.d.ts
@@ -5,7 +5,6 @@ export type ThemeParams = Partial<Pick<Theme, 'name' | 'role' | 'processing' | '
 export type AssetParams = Pick<ThemeAsset, 'key'> & Partial<Pick<ThemeAsset, 'value' | 'attachment'>>;
 export declare function fetchTheme(id: number, session: AdminSession): Promise<Theme | undefined>;
 export declare function fetchThemes(session: AdminSession): Promise<Theme[]>;
-export declare function findDevelopmentThemeByName(name: string, session: AdminSession): Promise<Theme | undefined>;
 export declare function themeCreate(params: ThemeParams, session: AdminSession): Promise<Theme | undefined>;
 export declare function fetchThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<ThemeAsset[]>;
 export declare function deleteThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<Result[]>;
packages/cli-kit/dist/public/node/themes/theme-manager.d.ts
@@ -8,8 +8,8 @@ export declare abstract class ThemeManager {
     protected abstract removeTheme(): void;
     protected abstract context: string;
     constructor(adminSession: AdminSession);
-    findOrCreate(name?: string, role?: Role): Promise<Theme>;
-    fetch(name?: string, role?: Role): Promise<Theme | undefined>;
+    findOrCreate(): Promise<Theme>;
+    fetch(): Promise<Theme | undefined>;
     generateThemeName(context: string): string;
     create(themeRole?: Role, themeName?: string): Promise<Theme>;
 }
\ No newline at end of file

@ryancbahan ryancbahan marked this pull request as ready for review March 18, 2026 03:00
@ryancbahan ryancbahan changed the base branch from rcb/project-model to graphite-base/7023 March 18, 2026 14:49
@ryancbahan ryancbahan force-pushed the rcb/project-integration branch from 87728e2 to 490c047 Compare March 18, 2026 21:42
@ryancbahan ryancbahan changed the base branch from graphite-base/7023 to main March 18, 2026 21:42
ryancbahan and others added 2 commits March 18, 2026 15:44
Wires the Project domain model into the existing loading pipeline:

- getAppConfigurationState uses Project.load() for filesystem discovery
- getAppConfigurationContext returns project + activeConfig + state
  as independent values (project is never nested inside state)
- AppLoader reads from Project's pre-loaded data: extension files,
  web files, dotenv, hidden config, deps, package manager, workspaces
- No duplicate filesystem scanning — Project discovers once, loader
  reads from it
- AppConfigurationState no longer carries project as a field
- LoadedAppContextOutput exposes project and activeConfig as
  top-level fields for commands
- All extension/web file discovery filtered to active config's
  directories via config-selection functions

Zero behavioral changes. All 3801 existing tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the monolithic loadApp pipeline with composable stages:
- loadApp is now a thin wrapper: getAppConfigurationContext → loadAppFromContext
- loadAppFromContext takes narrow Project + ActiveConfig directly
- getAppConfigurationContext is discovery-only (no parsing/state construction)
- ReloadState replaces passing entire AppLinkedInterface through reloads
- AppLoader takes reloadState? instead of previousApp?
- link() returns {remoteApp, configFileName, configuration} (no state)
- linkedAppContext uses activeConfig directly, no AppConfigurationState

Remove dead code: AppConfigurationState, toAppConfigurationState,
loadAppConfigurationFromState, loadAppUsingConfigurationState,
loadAppConfiguration, getAppConfigurationState, getAppDirectory,
loadDotEnv, loadHiddenConfig, findWebConfigPaths, loadWebsForAppCreation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ryancbahan ryancbahan force-pushed the rcb/project-integration branch from 490c047 to e775694 Compare March 18, 2026 21:44
ryancbahan and others added 3 commits March 18, 2026 16:02
…Path

- Remove orphaned <<<<<<< HEAD marker and dead loadHiddenConfig tests
- Re-add isAppConfigSpecification import (used at line 776)
- Add missing configPath field to loadedConfiguration in loadAppFromContext
- Fix configuration.path references to use configPath (path extracted from config type)
- Fix testApp() call signature in use.test.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…path

Same bug as the other two configuration.path references fixed in
e4487d9 — configuration is Zod-parsed and has no path property.
Without this fix, activeConfigFile always returns undefined, causing
loadWebs() and createExtensionInstances() to load all project files
instead of filtering by the active config.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Contributor

@isaacroldan isaacroldan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Review assisted by pair-review

})
extensionConfigPaths.push(`!${joinPath(appDirectory, '**/node_modules/**')}`)
const configPaths = await glob(extensionConfigPaths)
private async createExtensionInstances(appDirectory: string, _extensionDirectories?: string[]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Improvement: createExtensionInstances takes _extensionDirectories (underscore-prefixed to suppress unused-var linting) but completely ignores it. Extension discovery comes from this.activeConfigFile and extensionFilesForConfig, which reads extension_directories from activeConfig.content internally. The parameter is passed from the call site at line 641 as appConfiguration.extension_directories, but it's never consumed.

Suggestion: Remove the _extensionDirectories parameter from the method signature and the corresponding argument from the call site at line 641.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is implemented in the next pr in stack 03-17-remove-project-fields-from-app

import {AppHiddenConfig, BasicAppConfigurationWithoutModules} from '../app/app.js'
import {AppConfigurationFileName, AppConfigurationState, getConfigurationPath} from '../app/loader.js'
import {AppHiddenConfig} from '../app/app.js'
import {getAppConfigurationFileName} from '../app/loader.js'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Design: active-config.ts imports getAppConfigurationFileName from loader.js (line 4), and loader.ts imports selectActiveConfig from active-config.js. Additionally, active-config.ts imports use.js (line 6) which imports getAppConfigurationContext from loader.js, creating a deeper cycle: loader → active-config → use → loader. Node ESM handles this without crashing at startup thanks to live bindings, but it's fragile — reordering imports or adding a top-level side effect in any of these files can produce undefined at import time. It also makes isolated testing harder.

Suggestion: Extract getAppConfigurationFileName into a small, leaf-level utility module (e.g., config-naming.ts) that both loader.ts and active-config.ts can import without forming a cycle. This function is a pure string utility with no dependencies on loader internals.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will extract.

})
setCurrentConfigPreference(configuration, {configFileName, directory})
const {activeConfig} = await getAppConfigurationContext(directory, configFileName)
setCurrentConfigPreference(activeConfig.file.content as AppConfiguration, {configFileName, directory})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Human comment: Are we planning to validate the active config content using the modules schemas?
AI:
🐛 Bug: The old code called loadAppConfiguration which parsed and validated the config against AppSchema before passing it to setCurrentConfigPreference. The new code reads raw TOML via getAppConfigurationContext and casts it with as AppConfiguration, bypassing all schema validation. A malformed config file (invalid field types, missing nested structure, wrong shapes) will now be silently accepted as the current preference. Previously, such files would produce a validation error. This is a behavioral regression — the app config use command no longer validates configs before adopting them.

Suggestion: Either validate the content against AppSchema before setting the preference, or narrow the type to only what setCurrentConfigPreference actually needs. If full validation isn't feasible here, at minimum validate the fields that setCurrentConfigPreference relies on (like client_id).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is the correct behavior. "Use" is just switching active TOML -- it should only validate whether the toml is syntactically correct (which it does). It's not running anything, and therefore should not concern itself with schema validation. Other commands handle that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the validation would happen on the next command run?
The casting is weird though, because you can't be sure that content conforms to AppConfiguration :/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair, though I'm not sure it's rewriting at the moment since it's just passthrough and any "real" command will catch the errors.

1. Remove dead code fallback in createExtensionInstances — since
   configPaths is derived from extensionFiles, the find is guaranteed
   to match. Simplified to iterate extensionFiles directly.

2. Extract getAppConfigurationFileName, getAppConfigurationShorthand,
   isValidFormatAppConfigurationFileName, and AppConfigurationFileName
   into a leaf utility module (config-file-naming.ts). This breaks the
   circular import: loader → active-config → use → loader.
   Loader re-exports for backward compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ryancbahan ryancbahan requested a review from isaacroldan March 19, 2026 14:13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add tests for this file :)

// Used when reloading an app, to avoid some expensive steps during loading.
previousApp?: AppLinkedInterface
// Pre-discovered project data — avoids re-scanning the filesystem for dependencies, package manager, etc.
project: Project
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Umm, during dev you might add a new extension, that forces an app reload. But this project won't have the new files for the new extension right? :thinking_face:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I confirmed this:
app dev appears to work when generating an extension mid-dev, because the watcher detects the changes. And it triggers the reloadApp so that the in-memory App representation is updated. But Since this reload is not re-scanning the file-system is not loading the new extensions and they are not included in the deployed manifest.

We need to re-scan when reloading the app :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context from Slack -- this is a pre-existing issue on main branch. Will make sure it's fixed here though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reload is called as expected in this pr. I added a test to demonstrate this

Copy link
Contributor

@isaacroldan isaacroldan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reload behaviour has changed, let's fix that first 🙏

I think some extensive tophat of app dev is necessary before merging, even it appears to work, we need to check the manifest.json and the bundle uploaded to ensure it includes everything

ryancbahan and others added 3 commits March 19, 2026 10:41
Three new tests verifying that extensions added to disk after initial
load are correctly discovered on reload:

1. Project.load re-scans filesystem — fresh glob finds new files
2. reloadApp finds new extensions — full reload path works
3. handleWatcherEvents produces Created event — the event handler
   correctly calls reloadApp and appDiff classifies the new extension
   as created

Also fixed setupRealApp to include api_version in the function
extension TOML (required by strict validation mode used during reload).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two pre-existing bugs prevented `app dev` from handling extensions
created mid-session:

1. Chokidar v3 silently ignores non-existent directories. Now the file
   watcher creates extension directories (with .gitkeep) before starting,
   so new extensions are detected even if the directory didn't exist at
   startup.

2. createManifest() only wrote manifest.json during CREATE, not UPDATE.
   Now it always writes the full manifest before filtering modules for
   the API payload, keeping the on-disk bundle current.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use AppLinkedInterface cast for reloadApp/handleWatcherEvents calls
- Import AbortController from cli-kit for correct type compatibility
- Rename short identifier `e` to `ev`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Contributor Author

Pulling context from Slack -- we discovered the reload issue with manifest writes is already on main, not introduced in this pr. I've added more tests here and will tophat thoroughly + document.

ryancbahan and others added 10 commits March 19, 2026 14:01
Tests for getAppConfigurationFileName, getAppConfigurationShorthand,
and isValidFormatAppConfigurationFileName covering default values,
valid formats, slugification, shorthand extraction, and edge cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… cast

- Strip glob suffixes (e.g. extensions/**) before mkdir to avoid creating
  literal ** directories. Chokidar handles globs natively.
- Remove .gitkeep creation — chokidar only needs the directory to exist.
- Add clarifying comment on client_id cast in app-context.ts.
- Add test verifying glob stripping behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ata init

- Remove `delete rawConfig.path` — in the new architecture, path lives on
  TomlFile.path and is never mixed into .content.
- Collapse configurationLoadResultMetadata into a single const assignment
  with usesLinkedConfig: true, since loadAppFromContext is always the
  linked-app path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Load extension specifications once at module level instead of 8 times
per test. This is the heaviest operation in the integration tests
(loads all specs from disk + parses Zod schemas). Reduces memory
churn that was causing fsevents SIGABRT on macOS CI runners.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The mkdir call added to FileWatcher.start() throws EACCES when the app
directory is unwritable (e.g. test fixtures using '/' as directory).
This unhandled rejection cascaded into an fsevents SIGABRT on macOS CI.

- Wrap mkdir in try/catch since it's non-fatal (chokidar handles
  missing directories gracefully)
- Hoist loadLocalExtensionsSpecifications to module level in
  integration tests to reduce memory pressure (was called 8x)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents real chokidar/fsevents watchers from being created during
tests, which caused native addon crashes on macOS CI runners.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lly)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Multi-config dev tests (multi-config-dev.spec.ts):
- Verify -c flag loads the named config (staging TOML, correct scopes)
- Verify default config is used when -c is omitted
- Assert on info banner, scopes, and Config filename in App info tab

Dev hot-reload tests (dev-hot-reload.spec.ts):
- Editing app config TOML (scopes change) triggers full reload pipeline
- Creating a new extension mid-dev is detected by the file watcher
- Deleting an extension mid-dev is detected by the file watcher

Key design decisions:
- Use flow_trigger extensions (build mode 'none') to avoid theme-check,
  esbuild, and npm install overhead in e2e temp directories
- Start dev without extensions, mutate filesystem mid-dev to avoid
  dev session API rejection on initial CREATE
- Assert on watcher detection messages (Extension created/deleted) rather
  than API success for extension create/delete tests
- App config TOML edit test verifies the full round-trip including API
  success (App config updated + Updated dev preview)
- Clear SHOPIFY_FLAG_CLIENT_ID when using -c flag (mutually exclusive)
- Use --skip-dependencies-installation to avoid npm install hangs
…rrentConfigPreference type

1. Remove unused _extensionDirectories parameter from createExtensionInstances.
   Extension discovery goes through activeConfigFile + extensionFilesForConfig,
   making the parameter dead code.

2. Narrow setCurrentConfigPreference to accept {client_id?: string} instead of
   AppConfiguration. The function only reads client_id — full schema validation
   is deferred to the next command that loads the app. This removes the unsafe
   'as AppConfiguration' cast on raw TOML content and drops the dubious
   'asserts configuration is CurrentAppConfiguration' return type.
…NT_ID from env

Empty string is still passed to the child process, causing the tunnel
setup to use an empty client_id and fail with a Cloudflare unmarshal
error. Setting to undefined causes the spawn helper's 'value !== undefined'
filter to exclude it from the child process environment entirely.
vi.useFakeTimers()
// Prevent real chokidar/fsevents watchers from being created in tests.
// The hook only needs the AppEventWatcher as an EventEmitter, not real file watching.
vi.spyOn(AppEventWatcher.prototype, 'start').mockResolvedValue()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was hogging up memory now that tests are sharded

Copy link
Contributor Author

ryancbahan commented Mar 20, 2026

Added a bunch more tests around hot reload, file watching, and config switching. Doing a bunch of local tophatting and good so far.

@ryancbahan ryancbahan requested a review from isaacroldan March 20, 2026 03:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants