Open
Conversation
Code Review SummaryStatus: 10 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)CRITICAL
WARNING
SUGGESTION
Files Reviewed (20 files)
|
There was a problem hiding this comment.
Pull request overview
This PR expands the Keploy v4.0.0 documentation set for advanced API testing workflows, covering suite management, assertions, debugging, reporting, and visualization, and wires the new pages into the v4 sidebar navigation.
Changes:
- Added multiple new
running-keploydocs for API testing workflows (suite actions, selective runs, labeling, reports, schema coverage, debugging, etc.). - Updated the v4.0.0 sidebar to include the new API testing documentation pages.
- Introduced “Fix with AI” documentation (currently not linked from the sidebar).
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| versioned_sidebars/version-4.0.0-sidebars.json | Adds many new API Testing doc entries to the v4 sidebar (but currently misses “Fix with AI”). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-suite-settings.md | New guide for suite panel settings/actions (share/run/bulk edits/variables/functions/CI). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-sharing-reports.md | New guide for sharing run reports within a workspace. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-schema-coverage.md | New guide for schema coverage analysis and generating tests for missing coverage. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-running-selective.md | New guide for selectively running suites via checkbox/bulk actions. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-run-report.md | New guide for understanding and filtering run reports. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-mark-unbuggy.md | New guide for marking suites as unbuggy after fixes. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-local-agent.md | New guide for using the local agent for private/local endpoints. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-generation-history.md | New guide for viewing and using test generation job history. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-fixing-ai.md | New “Fix with AI” doc explaining AI-assisted repair/normalization of failing suites. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-filter-suites.md | New guide for filtering suites by endpoint/method/status/etc. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-edit-suites.md | New guide for editing suites/steps/assertions (contains placeholder/incomplete references + metadata issues). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-edit-assertions.md | New detailed doc for variables + custom assertions/functions (includes conflicting naming guidance). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-custom-assertions.md | New overview doc for assertion types and advanced custom function use. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-bulk-assertions.md | New guide for bulk assertions and schema validation across many tests. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-buggy-suites.md | New guide for diagnosing and fixing buggy suites (contains a typo). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-assertion-tree.md | New guide for visualizing workflows via the Assertion Tree. |
| versioned_docs/version-4.0.0/running-keploy/api-testing-adding-labels.md | New guide for labeling suites (individual + bulk). |
| versioned_docs/version-4.0.0/running-keploy/api-testing-add-suite.md | New guide for adding a suite (front-matter id currently mismatches the sidebar entry). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - Use JavaScript expressions for complex validations | ||
| - Reference external validation logic | ||
|
|
||
| For detailed information on custom functions, [reference here](#). |
There was a problem hiding this comment.
This link is a placeholder ((#)) and will render as a broken link. Please replace it with a real target (e.g., the custom assertions/global functions doc) or remove it until a destination exists.
Suggested change
| For detailed information on custom functions, [reference here](#). | |
| For detailed information on custom functions, refer to the custom functions documentation. |
Comment on lines
+76
to
+79
| ## How to Create Variables | ||
|
|
||
| For in detail usage of variables refer here : | ||
|
|
There was a problem hiding this comment.
This section says "refer here" but provides no link/next step, leaving the variables guidance incomplete. Please add the intended link (or remove the section header until it’s ready).
Suggested change
| ## How to Create Variables | |
| For in detail usage of variables refer here : |
| 3. **Naming Conventions** | ||
| - Use camelCase: `userId`, `apiBaseUrl` | ||
| - Be descriptive: `adminUserToken` instead of `token1` | ||
| - Group related variables: `user_id`, `user_name`, `user_email` |
There was a problem hiding this comment.
This "Naming Conventions" section mixes camelCase (userId) with snake_case examples (user_id). Please pick one convention (or explicitly document when each is used) so readers don’t get conflicting guidance.
Suggested change
| - Group related variables: `user_id`, `user_name`, `user_email` | |
| - Group related variables: `userId`, `userName`, `userEmail` |
Comment on lines
+8
to
+18
| - webhooks | ||
| - integration | ||
| - custom validation | ||
| - policy enforcement | ||
| keywords: | ||
| - webhook | ||
| - API testing | ||
| - PreExecute | ||
| - PostExecute | ||
| - external validation | ||
| - custom logic |
There was a problem hiding this comment.
The front-matter tags include webhook/integration-specific items, but this page is about editing test suites. Please update the tags to reflect the actual topic so tagging/search results aren’t misleading.
Suggested change
| - webhooks | |
| - integration | |
| - custom validation | |
| - policy enforcement | |
| keywords: | |
| - webhook | |
| - API testing | |
| - PreExecute | |
| - PostExecute | |
| - external validation | |
| - custom logic | |
| - test suites | |
| - test editing | |
| - assertions | |
| - variables | |
| keywords: | |
| - API testing | |
| - test suites | |
| - Keploy | |
| - test steps | |
| - assertions | |
| - variables |
Comment on lines
+15
to
+16
| - PreExecute | ||
| - PostExecute |
There was a problem hiding this comment.
The front-matter keywords (e.g., PreExecute, PostExecute) don’t appear to be covered in this doc. Consider replacing them with keywords related to suite/test-step editing, assertions, and variables.
Suggested change
| - PreExecute | |
| - PostExecute | |
| - test suite editing | |
| - test step editing | |
| - assertions | |
| - variables |
Comment on lines
+40
to
+46
| - It will capture the responses and start generating test sheets based on the recorded calls. | ||
|
|
||
| 6. **Troubleshooting Connection Issues** | ||
| - If the local agent fails to connect, you can use the Keplr extension as an alternative. | ||
| - Ensure that the agent is running and the endpoint URL is correct. | ||
|
|
||
| By following these steps, you can efficiently use the local agent to test your APIs and generate test sheets automatically. No newline at end of file |
There was a problem hiding this comment.
"test sheets" looks like a typo/inconsistent term here; elsewhere the docs use "test suites".
Suggested change
| - It will capture the responses and start generating test sheets based on the recorded calls. | |
| 6. **Troubleshooting Connection Issues** | |
| - If the local agent fails to connect, you can use the Keplr extension as an alternative. | |
| - Ensure that the agent is running and the endpoint URL is correct. | |
| By following these steps, you can efficiently use the local agent to test your APIs and generate test sheets automatically. | |
| - It will capture the responses and start generating test suites based on the recorded calls. | |
| 6. **Troubleshooting Connection Issues** | |
| - If the local agent fails to connect, you can use the Keplr extension as an alternative. | |
| - Ensure that the agent is running and the endpoint URL is correct. | |
| By following these steps, you can efficiently use the local agent to test your APIs and generate test suites automatically. |
Comment on lines
172
to
176
| "running-keploy/api-testing-webhook", | ||
| "running-keploy/api-testing-edit-suites", | ||
| "running-keploy/api-testing-custom-assertions", | ||
| "running-keploy/api-testing-assertion-tree", | ||
| "running-keploy/api-testing-auth-setup", |
There was a problem hiding this comment.
The API Testing sidebar list doesn’t include the newly added "Fix with AI" doc (front-matter id: api-testing-fix-with-ai), so it won’t be discoverable via navigation. Consider adding running-keploy/api-testing-fix-with-ai in this list.
Comment on lines
+23
to
+27
| 2. **Download the Keplr Agent** | ||
| - Based on your device configuration, download the Keplr agent: | ||
| - **Windows** | ||
| - **Mac** | ||
| - **Linux** |
There was a problem hiding this comment.
Typo: "Keplr" should be "Keploy" (this appears multiple times in the page and may confuse readers).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request adds comprehensive documentation for advanced API testing features in Keploy, focusing on test suite management, assertion capabilities, test debugging, and workflow visualization. The new docs provide step-by-step guides and best practices for using features such as adding new test suites, bulk and custom assertions, schema validation, assertion trees, and debugging buggy test suites.
New and Enhanced Documentation:
Test Suite Management:
Assertion Capabilities:
Test Workflow Visualization:
Debugging and Troubleshooting: