Skip to content

Adding LLM Naming strategy#1573

Merged
arcuri82 merged 9 commits into
masterfrom
phg/llm-test-case-naming
Jun 14, 2026
Merged

Adding LLM Naming strategy#1573
arcuri82 merged 9 commits into
masterfrom
phg/llm-test-case-naming

Conversation

@Pgarrett

@Pgarrett Pgarrett commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

Test cases can now be named using an LLM, EM will have to be set up using LLM config. Naming strategy involves prompting the LLM stating it should behave as a software engineer specializing in test naming. The prompt will include:

  • Programming language being used (in order to follow naming conventions)
  • Max amount of characters the new name can have
  • Already generated test case names (to avoid repetition)
  • The test case source code

Upon LLM response, a simple validation takes place in which we verify the response does not contain any illegal character, whitespaces and if it honours the max amount of characters the test name should have. If any of those validations fails, the LLM is re-prompted to provide a new name which follows the guidelines.

No disambiguation phase is added since that's already handled when prompting the LLM for the test case name.

Test case source code is generated at this stage only to provide the LLM with context. Since generating source code is just concatenating strings, we can accept the cost of it being called here and later on for the real test case writing.

@Pgarrett Pgarrett marked this pull request as ready for review June 10, 2026 11:13
@Pgarrett Pgarrett requested a review from arcuri82 June 10, 2026 11:13
@Pgarrett Pgarrett requested a review from arcuri82 June 13, 2026 14:40
Comment thread core/src/main/kotlin/org/evomaster/core/llm/Prompts.kt
@Pgarrett Pgarrett requested a review from arcuri82 June 13, 2026 21:29
@arcuri82 arcuri82 merged commit 1539098 into master Jun 14, 2026
29 checks passed
@arcuri82 arcuri82 deleted the phg/llm-test-case-naming branch June 14, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants