Fix observe context for file inputs; add upload eval/example#1718
Draft
Fix observe context for file inputs; add upload eval/example#1718
Conversation
🦋 Changeset detectedLatest commit: 3db73a0 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Greptile SummaryFixed file input observation by capturing Key changes:
Confidence Score: 5/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CDP DOM.getDocument] --> B[extractInputType helper]
B --> C[buildSessionDomIndex]
C --> D[inputTypeByBe Map]
D --> E[collectPerFrameMaps]
E --> F[inputTypeMap per frame]
F --> G[a11yForFrame]
G --> H{isFileInputNode check}
H -->|Yes| I[Keep structural node]
H -->|No| J[Normal pruning]
I --> K[buildHierarchicalTree]
J --> K
K --> L{File input in tree?}
L -->|No| M[appendMissingFileInputNodes]
L -->|Yes| N[Use existing]
M --> O[Synthetic file input node]
N --> P[Final outline with file inputs]
O --> P
P --> Q[observe returns selector]
Q --> R[page.locator xpath .setInputFiles]
Last reviewed commit: ac7939f |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
1 issue found across 11 files
Confidence score: 3/5
- Runtime risk:
packages/evals/tasks/observe_file_input_upload.tsuses a relative ESM import without the.jsextension, which can fail under native ESM resolution and break this task at runtime. - Score reflects a concrete user-impacting runtime failure in an eval task, though the issue is localized and straightforward to fix.
- Pay close attention to
packages/evals/tasks/observe_file_input_upload.ts- missing.jsextension on a relative ESM import.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/evals/tasks/observe_file_input_upload.ts">
<violation number="1" location="packages/evals/tasks/observe_file_input_upload.ts:4">
P2: Missing `.js` extension on relative ESM import. All other eval task files in this directory use `"../types/evals.js"`. Omitting the extension will fail at runtime under native ESM resolution.
(Based on your team's feedback about requiring explicit .js extensions on all relative ESM import specifiers.) [FEEDBACK_USED]</violation>
</file>
Architecture diagram
sequenceDiagram
participant U as User Script
participant SH as Stagehand (v3)
participant CAP as Capture Orchestrator
participant DOM as DOM Tree Service
participant AXS as A11y Tree Service
participant CDP as Browser (CDP)
Note over U,CDP: Observe & File Upload Flow
U->>SH: observe(instruction)
SH->>CAP: tryScopedSnapshot()
CAP->>DOM: domMapsForSession()
DOM->>CDP: DOM.getDocument (depth: -1)
CDP-->>DOM: Full DOM Tree
DOM->>DOM: NEW: extractInputType()
Note right of DOM: Captures 'type' attribute<br/>for <input> elements
DOM-->>CAP: { tagNameMap, inputTypeMap, xpathMap }
CAP->>AXS: a11yForFrame(domMaps)
AXS->>CDP: Accessibility.getFullAXTree()
CDP-->>AXS: AXNodes
AXS->>AXS: buildHierarchicalTree()
Note right of AXS: CHANGED: isFileInputNode() check<br/>prevents pruning file inputs
AXS->>AXS: NEW: appendMissingFileInputNodes()
Note right of AXS: Synthesizes AX nodes if DOM has<br/>file inputs missing from AX tree
AXS-->>CAP: Simplified A11y Tree (with file inputs)
CAP-->>SH: Snapshot Data
SH-->>U: Array of Observations (selectors)
Note over U,CDP: Interaction Phase
U->>U: Unpack observed XPath
U->>CDP: page.locator(xpath).setInputFiles(path)
CDP-->>U: File uploaded successfully
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| import { promises as fs } from "fs"; | ||
| import path from "path"; | ||
| import crypto from "crypto"; | ||
| import { EvalFunction } from "../types/evals"; |
Contributor
There was a problem hiding this comment.
P2: Missing .js extension on relative ESM import. All other eval task files in this directory use "../types/evals.js". Omitting the extension will fail at runtime under native ESM resolution.
(Based on your team's feedback about requiring explicit .js extensions on all relative ESM import specifiers.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tasks/observe_file_input_upload.ts, line 4:
<comment>Missing `.js` extension on relative ESM import. All other eval task files in this directory use `"../types/evals.js"`. Omitting the extension will fail at runtime under native ESM resolution.
(Based on your team's feedback about requiring explicit .js extensions on all relative ESM import specifiers.) </comment>
<file context>
@@ -0,0 +1,103 @@
+import { promises as fs } from "fs";
+import path from "path";
+import crypto from "crypto";
+import { EvalFunction } from "../types/evals";
+
+const FILE_UPLOAD_V2_URL =
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
input[type]metadata in the DOM snapshot pipeline (domMapsForSession+buildSessionDomIndex)observe_file_input_uploadeval task showingobserve -> unpack selector -> page.locator(xpath).setInputFiles(...)packages/core/examples/observe_file_input_upload.tspackages/evals/evals.config.jsonTesting
tests/snapshot-dom-session-builders.test.tstests/snapshot-a11y-tree-utils.test.tstests/snapshot-a11y-resolvers.test.tstests/snapshot-capture-orchestration.test.tsRelated
<input type="date">and<input type="time">confirmed not working #1249Summary by cubic
Fixes observe() context for file upload inputs so the correct input field is returned as a reliable XPath and works with setInputFiles. Adds an example and eval to validate file uploads end-to-end.
Bug Fixes
New Features
Written for commit 3db73a0. Summary will update on new commits. Review in cubic