Adding LLM Naming strategy#1573
Merged
Merged
Conversation
arcuri82
requested changes
Jun 12, 2026
arcuri82
requested changes
Jun 13, 2026
arcuri82
approved these changes
Jun 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test cases can now be named using an LLM, EM will have to be set up using LLM config. Naming strategy involves prompting the LLM stating it should behave as a software engineer specializing in test naming. The prompt will include:
Upon LLM response, a simple validation takes place in which we verify the response does not contain any illegal character, whitespaces and if it honours the max amount of characters the test name should have. If any of those validations fails, the LLM is re-prompted to provide a new name which follows the guidelines.
No disambiguation phase is added since that's already handled when prompting the LLM for the test case name.
Test case source code is generated at this stage only to provide the LLM with context. Since generating source code is just concatenating strings, we can accept the cost of it being called here and later on for the real test case writing.