Skip to content

Enable parallel MessageCodeGenerator task execution by default, including GenerateMojo multi-version aggregation, with thread-safe state and task-scoped logging#1233

Draft
Copilot wants to merge 9 commits into
masterfrom
copilot/add-parallel-execution-option
Draft

Enable parallel MessageCodeGenerator task execution by default, including GenerateMojo multi-version aggregation, with thread-safe state and task-scoped logging#1233
Copilot wants to merge 9 commits into
masterfrom
copilot/add-parallel-execution-option

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026

MessageCodeGenerator executed version-generation tasks serially and shared mutable state without concurrency guarantees. This change introduces parallel task execution for multi-task generation by default, ensures generator internals are safe under concurrency, and makes concurrent logs separable per task.

  • Parallel execution control

    • Added multi-task execution entrypoint: generate(List<Task>).
    • Added system property toggle: generator.parallelExecution (default true).
    • Kept single-task generate(Task) behavior intact; multi-task execution falls back to sequential when disabled (generator.parallelExecution=false) or when only one task is provided.
  • Thread-safety hardening

    • Replaced shared specification cache with ConcurrentHashMap to prevent races during concurrent dictionary parsing/reuse.
    • Added controlled task submission/collection with failure propagation from worker threads.
  • Task-scoped logging for concurrent runs

    • Introduced per-thread task prefixing via ThreadLocal log context.
    • Updated base and Maven logger paths to emit prefixed messages (e.g. [FIX44] ...) so interleaved logs remain attributable.
    • Added explicit info logging when parallel mode is enabled to report configured worker count and total task count.
    • Added per-task progress logs for multi-task generation:
      • Started task for <taskName> (x / y)
      • Finished task for <taskName> (x / y)
        These are emitted for both sequential and parallel multi-task execution.
  • CLI flow update

    • main now builds all version tasks and dispatches through generate(List<Task>), so parallel mode applies to standard generator CLI usage.
  • Maven plugin flow update (GenerateMojo)

    • Extended GenerateMojo to accept an optional <tasks> list and build all configured generator tasks in a single mojo execution.
    • GenerateMojo now dispatches via generator.generate(List<Task>), so parallel mode applies in Maven plugin usage (not only via main).
    • Preserved backward compatibility for existing single-task mojo configuration (dictFile/packaging etc.).
  • quickfixj-messages-all plugin configuration update

    • Replaced multiple separate quickfixj-codegenerator:generate executions with one aggregated execution containing all FIX-version tasks.
    • This enables effective cross-version parallel generation in quickfixj-messages-all.
  • Targeted regression coverage

    • Added ParallelExecutionOptionTest to verify:
      • sequential execution when generator.parallelExecution=false
      • concurrent execution when generator.parallelExecution=true
      • concurrent execution when generator.parallelExecution is unset (default behavior)
      • presence of parallel worker/task-count log output in parallel mode
      • presence of per-task start/finish progress log output with task index and total count
MessageCodeGenerator generator = new MessageCodeGenerator();
generator.generate(tasks); // runs task list in parallel by default when multiple tasks are provided

System.setProperty("generator.parallelExecution", "false");
generator.generate(tasks); // forces sequential execution

@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 20, 2026

@copilot please add logging output how many parallel tasks are used

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 20, 2026

@copilot please add logging output how many parallel tasks are used

Implemented in commit fde52ce. MessageCodeGenerator.generate(List<Task>) now logs the parallel worker count and total task count when parallel execution is enabled.

Copilot AI requested a review from chrjohn May 20, 2026 18:24
@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 21, 2026

@copilot make generator.parallelExecution true by default.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 21, 2026

@copilot make generator.parallelExecution true by default.

Done in commit d9804f4. generator.parallelExecution is now true by default for multi-task generation; setting it to false forces sequential execution.

Copilot AI changed the title Enable optional parallel MessageCodeGenerator task execution with thread-safe state and task-scoped logging Enable parallel MessageCodeGenerator task execution by default with thread-safe state and task-scoped logging May 21, 2026
@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 21, 2026

@copilot parallel execution might work when calling main() of the code generator. However, it does not work when using the GenerateMojo. So the mojo needs the information for all possible executions for the various fix versions at once. Currently there are several executions in quickfixj-messages/quickfixj-messages-all/pom.xml

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 21, 2026

@copilot parallel execution might work when calling main() of the code generator. However, it does not work when using the GenerateMojo. So the mojo needs the information for all possible executions f...

Implemented in commit 3703f5f. GenerateMojo now accepts a tasks list and invokes generator.generate(List<Task>), and quickfixj-messages-all/pom.xml was switched from multiple generate executions to one aggregated execution containing all FIX-version tasks so parallel generation is applied across versions. No UI changes in this PR.

Copilot AI changed the title Enable parallel MessageCodeGenerator task execution by default with thread-safe state and task-scoped logging Enable parallel MessageCodeGenerator task execution by default, including GenerateMojo multi-version aggregation, with thread-safe state and task-scoped logging May 21, 2026
@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 21, 2026

@copilot looking good so far.

  1. Please add a log entry when processing of one task starts and print the number of the task and the total number of tasks like ... "Started task for FIX44 (x / y)"
  2. and add a log entry when one task finishes and print the number of the task and the total number of tasks like ... "Finished task for FIX44 (x / y)"

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 21, 2026

@copilot looking good so far.

  1. Please add a log entry when processing of one task starts and print the number of the task and the total number of tasks like ... "Started task for FIX44 (x / y)"
    2....

Implemented in commit 1804050. Added per-task progress logs in MessageCodeGenerator.generate(List<Task>): Started task for <taskName> (x / y) before execution and Finished task for <taskName> (x / y) in a finally block after each task completes (sequential and parallel modes). No UI changes in this PR, so no screenshot applies.

@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 21, 2026

@copilot the test ParallelExecutionOptionTest.testParallelExecutionIsEnabledByDefault is flaky. It expects that the tasks always finish in the same order.

Keep the assertions that verify:

  • parallel execution actually happened, and
  • the “parallel task execution enabled” log was emitted.
  • Remove the assertion that requires task-3 to be the last finished task.
  • add a weaker assertion that any finish log for all 4 tasks exists.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 21, 2026

@copilot the test ParallelExecutionOptionTest.testParallelExecutionIsEnabledByDefault is flaky. It expects that the tasks always finish in the same order.

Keep the assertions that verify:
...

Addressed in commit 8e61616. I removed the order-dependent assertion in testParallelExecutionIsEnabledByDefault and replaced it with a weaker check that finish logs exist for all 4 tasks (containsFinishLogForAllTasks(4)), while keeping the assertions that verify parallel execution and the parallel-enabled log output. No UI changes were made, so no screenshot applies.

@chrjohn
Copy link
Copy Markdown
Member

chrjohn commented May 21, 2026

@copilot
Still failing on some OS
org.quickfixj.codegenerator.ParallelExecutionOptionTest.testParallelExecutionIsEnabledByDefault -- Time elapsed: 0.244 s <<< FAILURE!

Java.lang.AssertionError
At org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
At org.junit.Assert.assertTrue(Assert.java:53)
At org.quickfixj.codegenerator.ParallelExecutionOptionTest.testParallelExecutionIsEnabledByDefault(ParallelExecutionOptionTest.java:58)

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 21, 2026

@copilot
Still failing on some OS
org.quickfixj.codegenerator.ParallelExecutionOptionTest.testParallelExecutionIsEnabledByDefault -- Time elapsed: 0.244 s <<< FAILURE!
...

Fixed in commit 1b325a9. The remaining flake was from concurrent writes to infoMessages in the test helper; I switched it to CopyOnWriteArrayList so parallel log capture is thread-safe, keeping the same assertions from your previous guidance. No UI changes were made, so no screenshot applies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants