Skip to content

Fix model tool prompt#9730

Open
dpage wants to merge 3 commits intopgadmin-org:masterfrom
dpage:fix_model_tool_prompt
Open

Fix model tool prompt#9730
dpage wants to merge 3 commits intopgadmin-org:masterfrom
dpage:fix_model_tool_prompt

Conversation

@dpage
Copy link
Contributor

@dpage dpage commented Mar 10, 2026

Make the LLM prompt much clearer about how and when the model should call tools.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed an issue where some LLM models in the AI assistant failed to use database tools, instead outputting text descriptions of tool calls. The assistant now properly invokes database inspection tools to retrieve necessary schema and column information during query generation.

dpage and others added 3 commits March 10, 2026 17:07
…ctions over tool calls.

The previous prompt told the model to "Return ONLY the JSON object, nothing else"
while also providing tool definitions. Models like Qwen 3.5 would follow the text
instruction and never use tools. The updated prompt clearly separates the tool-use
phase from the final JSON response phase, and explicitly instructs the model to
call tools directly rather than describing them in text.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link

coderabbitai bot commented Mar 10, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 461b4485-b0cc-4155-b831-9ae7d8aadb7b

📥 Commits

Reviewing files that changed from the base of the PR and between c8bd75c and f6764cb.

📒 Files selected for processing (2)
  • docs/en_US/release_notes_9_14.rst
  • web/pgadmin/llm/prompts/nlq.py

Walkthrough

This pull request addresses an issue where certain LLM models failed to use database tools in the AI assistant. The fix updates the system prompt with more explicit instructions for tool usage and documents the fix in the release notes.

Changes

Cohort / File(s) Summary
Release Documentation
docs/en_US/release_notes_9_14.rst
Added new bug fix entry for Issue #9729 documenting that some LLM models would not invoke database tools and would output text descriptions instead.
LLM Prompt Configuration
web/pgadmin/llm/prompts/nlq.py
Expanded and clarified NLQ_SYSTEM_PROMPT with more explicit tool usage instructions, including mandatory direct tool invocation, clearer guidelines for database inspection methods (get_database_schema, get_table_columns), and revised response format guidance.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Fix model tool prompt' is directly related to the main change, which updates the NLQ_SYSTEM_PROMPT to fix issues with LLM models not using database tools properly.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant