feat: Restruccturing Simulation eval and Scenario docs according to the latest code#630
feat: Restruccturing Simulation eval and Scenario docs according to the latest code#630sarthakFuture wants to merge 3 commits into
Conversation
…t, Eval, and Summary APIs - Added validation alignment plan for Column Management APIs, addressing security gaps and backend validation inconsistencies. - Created new MCP tools for Run Prompt APIs, enhancing validation and organization filtering. - Established shared validation utilities for Eval APIs, fixing organization filtering issues and ensuring consistent validation across layers. - Implemented organization filtering and bug fixes in Summary APIs to prevent cross-org data leaks and improve error handling. - Introduced centralized API base URL configuration for frontend integration.
There was a problem hiding this comment.
The PR title, desc and the diff don't seem to match..
Have added few comments and also getcallexecutions.mdx, updatetestcomponents.mdx, addemptyrowstodataset.mdx are 0-byte in the diff but the underlying endpoints (GET /simulate/run-tests/{id}/call-executions/, PATCH /simulate/run-tests/{id}/components/, POST /model-hub/develops/{id}/add_empty_rows/) still seem live on prod.
So, were these meant to be deleted, or left as empty placeholders?
Also - there seem to be merge conflicts with the other PR that was merged. Could you pls resolve them?
| {"name": "execution_ids", "in": "query", "required": false, "description": "JSON-encoded array of test execution UUIDs to compare.", "type": "string"} | ||
| ]} | ||
| responseExample={{"execution-uuid-1": {"evaluations": [{"name": "Tone Check", "average_score": 0.85}]}, "execution-uuid-2": {"evaluations": [{"name": "Tone Check", "average_score": 0.92}]}}} | ||
| responseExample={{ |
There was a problem hiding this comment.
prod wraps it in {status, result: {...}}.
Is this an intended new shape, or should it reflect prod?
| </ParamField> | ||
| <ParamField name="404" type="Not Found"> | ||
| No test run found with the specified `run_test_id`. | ||
| ```json |
There was a problem hiding this comment.
Tested the newly added error blocks against Prod - with bogus inputs: a non-matching UUID returns HTTP 500 (not 404) with {"result": "Unable to fetch evaluation summary. Please try again"}, and 401 comes back as {"detail": "..."}.
sarthakFuture
changed the title
KarthikAvinashFI
force-pushed
the
feature/run-test-apis
branch
from
148a286 to
a440927
Compare
|
@sarthakFuture whats the status on this? relevant ? |
|
sir isko aaj kl mein close krdunga....my bad aaj hi krdeta....ek do sanity changes krne the |
3 participants
…t, Eval, and Summary APIs
Pull Request
Description
Describe the changes in this pull request:
Checklist
Linear
Fixes TH-4913
Related Issues
Closes #<issue_number>