Implement shared specifications tests by liamhughes · Pull Request #6 · OctopusDeploy/openfeature-provider-java

liamhughes · 2026-03-27T03:30:10Z

Background

We are adding shared specification tests across our three OpenFeature provider libraries as per this ADR.

The one existing test fixture is imported from our specifications repository as a Git submodule. Our intention is to add a separate fixture file for each area of functionality.

Each test fixture set will contain a list of toggles and test cases with expected outcomes.

The toggles list will take the same shape as the response body returned by the OctoToggle API.
The test cases will each contain a description, toggle slug, evaluation context, and expected outcome including any expected error code.

Changes

Modified OctopusConfiguration to have a package-private settable server URI.
Added Server, a wrapper around a WireMock server allowing registration of evaluation payloads for particular client identifiers.
Added SpecificationTests which uses @ParameterizedTest and @MethodSource to run a test for each case in each specification JSON file using the WireMock server as the service for the OpenFeature client to query.

Result

liamhughes · 2026-03-27T03:56:08Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+        );
+
+        assertThat(result.getValue())
+            .as("[%s] %s → value", fileName, description)


Not 100% sure on the necessity of this formatting.

liamhughes · 2026-03-27T04:03:54Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+        public String errorCode;
+    }
+
+    static class RawJsonDeserializer extends JsonDeserializer<String> {


Jackson does not have the equivalent of GetRawText. I could have fetched a generic JSON object and then reserialised it, but I'd prefer to get the raw text from the file to avoid unintentional changes to the "response".

liamhughes · 2026-03-27T04:07:27Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+
+    static Stream<Arguments> fixtureTestCases() throws IOException {
+        ObjectMapper mapper = JsonMapper.builder()
+                .enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION)


Required for the RawJsonDeserializer below.

Copilot

Pull request overview

Adds a shared, submodule-backed specification test harness to validate OpenFeature provider behavior against JSON fixtures.

Changes:

Introduces parameterized SpecificationTests that discover and run JSON fixture cases against a WireMock-backed fake service.
Adds a Server helper wrapping WireMock for per-test-case Bearer-token-based stubbing.
Updates configuration/CI to support the new test approach (package-private server URI override, submodule checkout).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/test/java/com/octopus/openfeature/provider/SpecificationTests.java	New parameterized spec runner that loads fixture JSON and asserts evaluation results.
src/test/java/com/octopus/openfeature/provider/Server.java	WireMock wrapper to serve fixture responses keyed by unique tokens.
src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java	Tests for default server URI and overriding server URI.
src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java	Makes server URI configurable (package-private setter) for tests.
specification	Adds the specification repo as a git submodule pointer.
pom.xml	Adds WireMock test dependency and scopes JUnit to tests.
.github/workflows/ci.yml	Ensures submodules are checked out in CI and updates workflow naming/steps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-27T05:13:14Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+        return jsonFiles.stream().flatMap(path -> {
+            try {
+                String fileContent = Files.readString(path);
+                Fixture fixture = mapper.readValue(fileContent, Fixture.class);
+                String fileName = path.getFileName().toString();
+                return Stream.of(fixture.cases)
+                        .map(c -> Arguments.of(fileName, c.description, fixture.response, c));
+            } catch (IOException e) {


Stream.of(fixture.cases) creates a Stream<FixtureCase[]> (single element: the array), so c.description won’t compile/work. Use an element stream (e.g., Arrays.stream(fixture.cases)) so each FixtureCase becomes a separate Arguments row.

Copilot · 2026-03-27T05:13:14Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+    static class RawJsonDeserializer extends JsonDeserializer<String> {
+        @Override
+        public String deserialize(JsonParser jp, DeserializationContext dc) throws IOException {
+            long begin = jp.currentLocation().getCharOffset();
+            jp.skipChildren();
+            long end = jp.currentLocation().getCharOffset();
+            String json = jp.currentLocation().contentReference().getRawContent().toString();
+            return json.substring((int) begin - 1, (int) end);
+        }
+    }


This “slice raw source by char offsets” approach is fragile (offset semantics vary by parser/source, begin - 1 can underflow, getRawContent().toString() is not guaranteed to be the actual JSON text, and (int) casts can truncate). A more robust approach is to deserialize response as a JsonNode (or Object) and then serialize it when configuring the server (e.g., mapper.writeValueAsString(node)), avoiding any dependency on raw input offsets.

Copilot · 2026-03-27T05:13:14Z

src/test/java/com/octopus/openfeature/provider/SpecificationTests.java

+class SpecificationTests {
+
+    private static Server server;
+
+    @BeforeAll
+    static void startServer() {
+        server = new Server();
+    }
+
+    @AfterAll
+    static void stopServer() {
+        server.stop();
+    }
+
+    @AfterEach
+    void shutdownApi() {
+        OpenFeatureAPI.getInstance().shutdown();
+    }


These tests share global mutable state (OpenFeatureAPI singleton and a static Server with accumulating stubs). If JUnit parallel execution is enabled (locally or in CI), this can cause cross-test interference/flakiness. Consider explicitly forcing same-thread execution for this class (e.g., JUnit 5 @Execution(ExecutionMode.SAME_THREAD)), or otherwise applying a lock/serialization mechanism around these tests.

liamhughes force-pushed the liamhughes/devex-138-implement-specifications-tests-in-java-client branch from 8eb6720 to db7548c Compare March 27, 2026 03:36

liamhughes changed the title ~~Liamhughes/devex 138 implement specifications tests in java client~~ Implement specifications tests Mar 27, 2026

liamhughes changed the title ~~Implement specifications tests~~ Implement shared specifications tests Mar 27, 2026

liamhughes commented Mar 27, 2026

View reviewed changes

liamhughes requested a review from Copilot March 27, 2026 04:07

liamhughes marked this pull request as ready for review March 27, 2026 04:13

liamhughes requested a review from a team March 27, 2026 04:13

dylanlerch approved these changes Mar 27, 2026

View reviewed changes

Base automatically changed from liamhughes/devex-80-java-client-breaks-on-segments-and-new-response-fields to main March 27, 2026 04:37

liamhughes added 16 commits March 27, 2026 15:39

Design doc

e91ebdb

Implementation plan

cb9ae1e

Add wiremock dependency

b2fdbe1

Fix scoping of junit-jupiter dependency

8ac6675

Add specifications submodule

c4e4123

Make OctopusConfiguration server URI configurable

625a1e6

Add WireMock server helper

5f7bb7c

Add SpecificationTests

75b86a7

Checkout submodules in CI

90e62e8

Add check for no JSON files

aa7417d

Maintain raw JSON for response

d551595

Display file name in IDE

135bae3

Run tests in CI

579385d

Remove GenAI docs

9a16513

Self review

34d46a5

Tidy imports

91bab70

liamhughes force-pushed the liamhughes/devex-138-implement-specifications-tests-in-java-client branch from e1c6d2f to 91bab70 Compare March 27, 2026 04:40

liamhughes merged commit 51d5af0 into main Mar 27, 2026
5 checks passed

Copilot started reviewing on behalf of liamhughes March 27, 2026 05:09 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

caitlynstocker mentioned this pull request Mar 30, 2026

test: Implement shared tests from spec submodule OctopusDeploy/openfeature-provider-ts-web#47

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement shared specifications tests#6

Implement shared specifications tests#6
liamhughes merged 16 commits intomainfrom
liamhughes/devex-138-implement-specifications-tests-in-java-client

liamhughes commented Mar 27, 2026 •

edited

Loading

Uh oh!

liamhughes Mar 27, 2026

Uh oh!

liamhughes Mar 27, 2026

Uh oh!

liamhughes Mar 27, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 27, 2026

Uh oh!

Copilot AI Mar 27, 2026

Uh oh!

Copilot AI Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

liamhughes commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Changes

Result

Uh oh!

liamhughes Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

liamhughes Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

liamhughes Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

liamhughes commented Mar 27, 2026 •

edited

Loading