From e91ebdb035c580dd97e6ebadeb0349262485d776 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 12:57:35 +1100 Subject: [PATCH 01/16] Design doc --- .../2026-03-26-specification-tests-design.md | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 docs/superpowers/specs/2026-03-26-specification-tests-design.md diff --git a/docs/superpowers/specs/2026-03-26-specification-tests-design.md b/docs/superpowers/specs/2026-03-26-specification-tests-design.md new file mode 100644 index 0000000..270ff6a --- /dev/null +++ b/docs/superpowers/specs/2026-03-26-specification-tests-design.md @@ -0,0 +1,74 @@ +# Specification Tests — Design + +**Date:** 2026-03-26 +**Linear:** DEVEX-138 +**ADR:** [Sharing client library tests](https://whimsical.com/octopusdeploy/adr-sharing-client-library-tests-8QPKL2XSw9SFec4zJENBo9) +**C# reference:** [openfeature-provider-dotnet#44](https://github.com/OctopusDeploy/openfeature-provider-dotnet/pull/44) + +## Context + +The OpenFeature provider specification repository (`OctopusDeploy/openfeature-provider-specification`) holds shared JSON fixture files used to drive cross-language integration tests. The Java client needs a test harness that reads these fixtures and runs each case as a separate named test against a fake HTTP server, mirroring the approach taken in the C# client. + +## Scope + +Implement the Java test harness only. Adding new fixture files to the specification repository is out of scope for this work. + +## Design + +### 1. Production code change — configurable server URI + +`OctopusConfiguration.getServerUri()` currently returns a hardcoded `https://features.octopus.com`. A private `serverUri` field will be added, defaulting to that value. A package-private setter `void setServerUri(URI uri)` will allow tests in the same package to override it without exposing the setter to library consumers (Java equivalent of C# `internal`). + +### 2. New test classes + +Both in `src/test/java/com/octopus/openfeature/provider/`. + +#### `Server` + +Wraps a `WireMockServer` started and stopped once per test class. Exposes: + +``` +String configure(String responseJson) +``` + +This stores the JSON keyed by a randomly generated UUID and returns that UUID as the client identifier token. A custom WireMock `ResponseTransformer` intercepts every request to `/api/featuretoggles/v3/`, reads the `Authorization: Bearer ` header, looks up the stored JSON, and returns it with a `ContentHash` response header (stable base64-encoded value). Requests with an unrecognised token receive a 401. + +The background refresh thread in `OctopusContextProvider` polls the check endpoint (`/api/featuretoggles/check/v3/`) only after the configured `cacheDuration` elapses (default: 1 minute). Since each test completes and shuts down the provider in milliseconds, the check endpoint will not be called in practice. + +#### `SpecificationTests` + +```java +@ParameterizedTest(name = "{0}") +@MethodSource("fixtureTestCases") +void evaluate(String description, String responseJson, FixtureCase testCase) { ... } +``` + +The `fixtureTestCases()` static method: +1. Walks `specification/Fixtures/*.json` +2. Deserialises each file with Jackson (existing dependency) into a `Fixture` record containing a raw `response` JSON string and a `FixtureCase[]` +3. Yields one `Arguments.of(description, responseJson, testCase)` per case + +The `description` field becomes the parameterised test display name (argument `{0}`). + +Each test invocation: +1. Calls `server.configure(responseJson)` to get a unique client identifier +2. Creates `OctopusConfiguration` with that identifier, then calls `setServerUri` to point at the WireMock base URL +3. Calls `OpenFeatureAPI.getInstance().setProvider(provider)` and sets the evaluation context from the fixture +4. Evaluates the flag via `client.getBooleanDetails(slug, defaultValue)` +5. Asserts `result.getValue()` and `result.getErrorCode()` match the fixture expectations +6. Calls `OpenFeatureAPI.getInstance().shutdown()` to stop the background refresh thread + +Error codes are mapped from fixture strings (e.g. `"FLAG_NOT_FOUND"`) to `ErrorCode` enum values. An unrecognised string throws `IllegalArgumentException` — a fixture bug should fail loudly. + +### 3. Infrastructure changes + +| Change | Detail | +|---|---| +| Git submodule | `specification/` → `https://github.com/OctopusDeploy/openfeature-provider-specification` | +| `pom.xml` | Add `org.wiremock:wiremock` (test scope) | +| CI workflow | Add `submodules: true` to the `actions/checkout` step | + +## Out of scope + +- Adding new fixture files to the specification repository (separate ticket) +- Removing existing `OctopusContextTests` cases now covered by fixtures (can be done once the fixture set is expanded) From cb9ae1ef00002d9bf60e15ea551a8159ed7abe2e Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 13:51:29 +1100 Subject: [PATCH 02/16] Implementation plan --- .../plans/2026-03-26-specification-tests.md | 499 ++++++++++++++++++ 1 file changed, 499 insertions(+) create mode 100644 docs/superpowers/plans/2026-03-26-specification-tests.md diff --git a/docs/superpowers/plans/2026-03-26-specification-tests.md b/docs/superpowers/plans/2026-03-26-specification-tests.md new file mode 100644 index 0000000..d5a1679 --- /dev/null +++ b/docs/superpowers/plans/2026-03-26-specification-tests.md @@ -0,0 +1,499 @@ +# Specification Tests Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a parameterised integration test harness that reads shared JSON fixture files from a git submodule and runs each test case against a fake HTTP server. + +**Architecture:** A `Server` test helper wraps WireMock and maps unique Bearer tokens to fixture response bodies via per-token stubs. `SpecificationTests` uses JUnit 5 `@ParameterizedTest` + `@MethodSource` to load fixture files from `specification/Fixtures/*.json`, yielding one named test per case. Each test configures the fake server, points `OctopusProvider` at it, evaluates the flag, and asserts value and error code. + +**Tech Stack:** JUnit 5 (already present), WireMock 3.x (`org.wiremock:wiremock`, new test dependency), Jackson (already present), OpenFeature Java SDK (already present) + +--- + +## File Map + +| Action | File | Purpose | +|---|---|---| +| Modify | `pom.xml` | Add WireMock test dependency | +| Shell | `git submodule add ...` | Clone spec repo to `specification/` | +| Modify | `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` | Add `serverUri` field + package-private setter | +| Create | `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java` | Test for configurable server URI | +| Create | `src/test/java/com/octopus/openfeature/provider/Server.java` | WireMock wrapper for fake toggle API | +| Create | `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java` | Parameterised specification tests + fixture models | +| Modify | `.github/workflows/ci.yml` | Add `submodules: true` to checkout step | + +--- + +## Task 1: Add WireMock dependency + +**Files:** +- Modify: `pom.xml` + +- [ ] **Step 1: Add WireMock to pom.xml** + +In `pom.xml`, add the following inside ``: + +```xml + + org.wiremock + wiremock + 3.5.4 + test + +``` + +- [ ] **Step 2: Verify existing tests still pass** + +Run: `mvn test -B` + +Expected output includes: +``` +Tests run: 26, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +- [ ] **Step 3: Commit** + +```bash +git add pom.xml +git commit -m "build: add WireMock test dependency" +``` + +--- + +## Task 2: Add git submodule + +**Files:** +- Shell: `git submodule add` + +- [ ] **Step 1: Add the specification submodule** + +Run from the project root: +```bash +git submodule add https://github.com/OctopusDeploy/openfeature-provider-specification.git specification +``` + +- [ ] **Step 2: Verify the fixture file is present** + +Run: `ls specification/Fixtures/` + +Expected output: +``` +simple-value-only-toggles.json +``` + +- [ ] **Step 3: Commit** + +```bash +git add .gitmodules specification +git commit -m "chore: add openfeature-provider-specification as git submodule" +``` + +--- + +## Task 3: Make OctopusConfiguration server URI configurable + +**Files:** +- Create: `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java` +- Modify: `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` + +- [ ] **Step 1: Write the failing test** + +Create `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java`: + +```java +package com.octopus.openfeature.provider; + +import org.junit.jupiter.api.Test; + +import java.net.URI; + +import static org.assertj.core.api.Assertions.assertThat; + +class OctopusConfigurationTests { + + @Test + void defaultServerUriIsOctopusCloud() { + var config = new OctopusConfiguration("test-client"); + assertThat(config.getServerUri()).isEqualTo(URI.create("https://features.octopus.com")); + } + + @Test + void serverUriCanBeOverridden() { + var config = new OctopusConfiguration("test-client"); + var customUri = URI.create("http://localhost:8080"); + config.setServerUri(customUri); + assertThat(config.getServerUri()).isEqualTo(customUri); + } +} +``` + +- [ ] **Step 2: Run to confirm failure** + +Run: `mvn test -Dtest=OctopusConfigurationTests -B` + +Expected: `COMPILATION ERROR` or `BUILD FAILURE` — `setServerUri` does not yet exist. + +- [ ] **Step 3: Implement setServerUri in OctopusConfiguration** + +Replace the content of `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` with: + +```java +package com.octopus.openfeature.provider; + +import java.net.URI; +import java.time.Duration; + +public class OctopusConfiguration { + private final String clientIdentifier; + private static final String DEFAULT_SERVER_URI = "https://features.octopus.com"; + private URI serverUri = URI.create(DEFAULT_SERVER_URI); + private Duration cacheDuration = Duration.ofMinutes(1); + + public OctopusConfiguration(String clientIdentifier) { + this.clientIdentifier = clientIdentifier; + } + + public String getClientIdentifier() { return clientIdentifier; } + + public URI getServerUri() { return serverUri; } + + // Package-private: visible to tests in same package, not to library consumers. + void setServerUri(URI serverUri) { this.serverUri = serverUri; } + + public Duration getCacheDuration() { + return cacheDuration; + } + + public Duration setCacheDuration(Duration cacheDuration) { + this.cacheDuration = cacheDuration; + return this.cacheDuration; + } +} +``` + +- [ ] **Step 4: Run the tests to confirm they pass** + +Run: `mvn test -B` + +Expected output includes: +``` +Tests run: 28, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +- [ ] **Step 5: Commit** + +```bash +git add src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java \ + src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java +git commit -m "feat: make OctopusConfiguration server URI configurable (package-private)" +``` + +--- + +## Task 4: Create the Server test helper + +**Files:** +- Create: `src/test/java/com/octopus/openfeature/provider/Server.java` + +- [ ] **Step 1: Create Server.java** + +Create `src/test/java/com/octopus/openfeature/provider/Server.java`: + +```java +package com.octopus.openfeature.provider; + +import com.github.tomakehurst.wiremock.WireMockServer; + +import java.util.Base64; +import java.util.UUID; + +import static com.github.tomakehurst.wiremock.client.WireMock.*; +import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; + +/** + * Fake HTTP server for specification tests. + * + * Each call to {@link #configure(String)} registers a stub for a unique Bearer token + * and returns that token as the client identifier. Stubs accumulate over the server's + * lifetime (one per test case), which is harmless since each token is unique. + * + * Note: parallel test execution is not supported because SpecificationTests uses + * the OpenFeatureAPI singleton. + */ +class Server { + + private static final String CONTENT_HASH = Base64.getEncoder().encodeToString(new byte[]{0x01}); + private final WireMockServer wireMock; + + Server() { + wireMock = new WireMockServer(wireMockConfig().dynamicPort()); + wireMock.start(); + // Fallback: return 401 for any request that does not match a registered token. + wireMock.stubFor(any(anyUrl()) + .atPriority(100) + .willReturn(aResponse().withStatus(401))); + } + + /** + * Registers the given JSON as the response body for a new unique client token. + * + * @param responseJson the JSON array that the toggle API would return + * @return the client identifier (Bearer token) to use in OctopusConfiguration + */ + String configure(String responseJson) { + String token = UUID.randomUUID().toString(); + wireMock.stubFor(get(urlPathEqualTo("/api/featuretoggles/v3/")) + .withHeader("Authorization", equalTo("Bearer " + token)) + .willReturn(aResponse() + .withStatus(200) + .withHeader("Content-Type", "application/json") + .withHeader("ContentHash", CONTENT_HASH) + .withBody(responseJson))); + return token; + } + + String baseUrl() { + return wireMock.baseUrl(); + } + + void stop() { + wireMock.stop(); + } +} +``` + +- [ ] **Step 2: Verify it compiles** + +Run: `mvn test-compile -B` + +Expected: `BUILD SUCCESS` + +- [ ] **Step 3: Commit** + +```bash +git add src/test/java/com/octopus/openfeature/provider/Server.java +git commit -m "test: add WireMock Server helper for specification tests" +``` + +--- + +## Task 5: Create SpecificationTests + +**Files:** +- Create: `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java` + +- [ ] **Step 1: Create SpecificationTests.java** + +Create `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java`: + +```java +package com.octopus.openfeature.provider; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import dev.openfeature.sdk.Client; +import dev.openfeature.sdk.ErrorCode; +import dev.openfeature.sdk.EvaluationContext; +import dev.openfeature.sdk.FlagEvaluationDetails; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.OpenFeatureAPI; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.io.IOException; +import java.io.UncheckedIOException; +import java.net.URI; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static org.assertj.core.api.Assertions.assertThat; + +class SpecificationTests { + + private static Server server; + + @BeforeAll + static void startServer() { + server = new Server(); + } + + @AfterAll + static void stopServer() { + server.stop(); + } + + @AfterEach + void shutdownApi() throws Exception { + OpenFeatureAPI.getInstance().shutdown(); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("fixtureTestCases") + void evaluate(String description, String responseJson, FixtureCase testCase) throws Exception { + String token = server.configure(responseJson); + OctopusConfiguration config = new OctopusConfiguration(token); + config.setServerUri(URI.create(server.baseUrl())); + + OpenFeatureAPI api = OpenFeatureAPI.getInstance(); + api.setProviderAndWait(new OctopusProvider(config)); + Client client = api.getClient(); + + EvaluationContext ctx = buildContext(testCase.configuration.context); + FlagEvaluationDetails result = client.getBooleanDetails( + testCase.configuration.slug, + testCase.configuration.defaultValue, + ctx + ); + + assertThat(result.getValue()) + .as(description + " → value") + .isEqualTo(testCase.expected.value); + assertThat(result.getErrorCode()) + .as(description + " → errorCode") + .isEqualTo(mapErrorCode(testCase.expected.errorCode)); + } + + static Stream fixtureTestCases() throws IOException { + ObjectMapper mapper = new ObjectMapper(); + List jsonFiles; + try (Stream files = Files.list(Path.of("specification", "Fixtures"))) { + jsonFiles = files + .filter(p -> p.getFileName().toString().endsWith(".json")) + .collect(Collectors.toList()); + } + return jsonFiles.stream().flatMap(path -> { + try { + Fixture fixture = mapper.readValue(path.toFile(), Fixture.class); + String responseJson = fixture.response.toString(); + return Stream.of(fixture.cases) + .map(c -> Arguments.of(c.description, responseJson, c)); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + }); + } + + private static EvaluationContext buildContext(Map context) { + MutableContext ctx = new MutableContext(); + if (context != null) { + context.forEach(ctx::add); + } + return ctx; + } + + private static ErrorCode mapErrorCode(String code) { + if (code == null) return null; + switch (code) { + case "FLAG_NOT_FOUND": return ErrorCode.FLAG_NOT_FOUND; + case "PARSE_ERROR": return ErrorCode.PARSE_ERROR; + case "TYPE_MISMATCH": return ErrorCode.TYPE_MISMATCH; + case "TARGETING_KEY_MISSING": return ErrorCode.TARGETING_KEY_MISSING; + case "PROVIDER_NOT_READY": return ErrorCode.PROVIDER_NOT_READY; + case "INVALID_CONTEXT": return ErrorCode.INVALID_CONTEXT; + case "PROVIDER_FATAL": return ErrorCode.PROVIDER_FATAL; + case "GENERAL": return ErrorCode.GENERAL; + default: throw new IllegalArgumentException("Unknown error code in fixture: " + code); + } + } + + // ---- Fixture model classes ---- + + static class Fixture { + public JsonNode response; + public FixtureCase[] cases; + } + + static class FixtureCase { + public String description; + public FixtureConfiguration configuration; + public FixtureExpected expected; + } + + static class FixtureConfiguration { + public String slug; + public boolean defaultValue; + public Map context; + } + + static class FixtureExpected { + public boolean value; + public String errorCode; + } +} +``` + +- [ ] **Step 2: Run the specification tests** + +Run: `mvn test -B` + +Expected output includes: +``` +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: ... -- in com.octopus.openfeature.provider.SpecificationTests +Tests run: 32, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +The 4 new tests correspond to the 4 cases in `specification/Fixtures/simple-value-only-toggles.json`. + +- [ ] **Step 3: Commit** + +```bash +git add src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +git commit -m "test: add specification tests harness (DEVEX-138)" +``` + +--- + +## Task 6: Update CI workflow + +**Files:** +- Modify: `.github/workflows/ci.yml` + +- [ ] **Step 1: Add submodules: true to the checkout step** + +Replace the content of `.github/workflows/ci.yml` with: + +```yaml +name: Build and Test + +on: + push: + branches: [ "main" ] + pull_request: + branches: [ "main" ] + +jobs: + build: + + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + with: + submodules: true + - name: Set up JDK 11 + uses: actions/setup-java@v4 + with: + java-version: '11' + distribution: 'temurin' + cache: maven + - name: Build with Maven + run: mvn -B package --file pom.xml +``` + +- [ ] **Step 2: Commit** + +```bash +git add .github/workflows/ci.yml +git commit -m "ci: fetch git submodules on checkout" +``` From b2fdbe155dfc916c67b06c9a004f3ba1342fc586 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 13:56:08 +1100 Subject: [PATCH 03/16] Add wiremock dependency --- pom.xml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/pom.xml b/pom.xml index e1aa131..f9c91f2 100644 --- a/pom.xml +++ b/pom.xml @@ -126,6 +126,12 @@ 3.27.3 test + + org.wiremock + wiremock + 3.5.4 + test + \ No newline at end of file From 8ac6675ac7e73b823b0e441425e804f840078ffc Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 13:57:02 +1100 Subject: [PATCH 04/16] Fix scoping of junit-jupiter dependency --- pom.xml | 1 + 1 file changed, 1 insertion(+) diff --git a/pom.xml b/pom.xml index f9c91f2..1c58b21 100644 --- a/pom.xml +++ b/pom.xml @@ -119,6 +119,7 @@ org.junit.jupiter junit-jupiter 5.11.1 + test org.assertj From c4e41233c577e49f1de8d2af2494f4fd2618fdb5 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 13:59:46 +1100 Subject: [PATCH 05/16] Add specifications submodule --- .gitmodules | 3 +++ specification | 1 + 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 160000 specification diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..4f9a6e1 --- /dev/null +++ b/.gitmodules @@ -0,0 +1,3 @@ +[submodule "specification"] + path = specification + url = https://github.com/OctopusDeploy/openfeature-provider-specification.git diff --git a/specification b/specification new file mode 160000 index 0000000..57495a9 --- /dev/null +++ b/specification @@ -0,0 +1 @@ +Subproject commit 57495a9dc1155e4c079aba1b91663a4ee501dca7 From 625a1e6e8cb63d465f1c6a7b1d06d40fb28d358b Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 14:03:39 +1100 Subject: [PATCH 06/16] Make OctopusConfiguration server URI configurable --- .../provider/OctopusConfiguration.java | 14 +++++++---- .../provider/OctopusConfigurationTests.java | 24 +++++++++++++++++++ 2 files changed, 33 insertions(+), 5 deletions(-) create mode 100644 src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java diff --git a/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java b/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java index 9cab049..431fb26 100644 --- a/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java +++ b/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java @@ -5,21 +5,25 @@ public class OctopusConfiguration { private final String clientIdentifier; - private static final String DEFAULT_SERVER_URI = "https://features.octopus.com"; - private Duration cacheDuration = Duration.ofMinutes(1); + private static final URI DEFAULT_SERVER_URI = URI.create("https://features.octopus.com"); + private URI serverUri = DEFAULT_SERVER_URI; + private Duration cacheDuration = Duration.ofMinutes(1); public OctopusConfiguration(String clientIdentifier) { this.clientIdentifier = clientIdentifier; } public String getClientIdentifier() { return clientIdentifier; } - - public URI getServerUri() { return URI.create(DEFAULT_SERVER_URI); } + + public URI getServerUri() { return serverUri; } + + // Package-private: visible to tests in same package, not to library consumers. + void setServerUri(URI serverUri) { this.serverUri = serverUri; } public Duration getCacheDuration() { return cacheDuration; } - + public Duration setCacheDuration(Duration cacheDuration) { this.cacheDuration = cacheDuration; return this.cacheDuration; diff --git a/src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java b/src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java new file mode 100644 index 0000000..aa17d27 --- /dev/null +++ b/src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java @@ -0,0 +1,24 @@ +package com.octopus.openfeature.provider; + +import org.junit.jupiter.api.Test; + +import java.net.URI; + +import static org.assertj.core.api.Assertions.assertThat; + +class OctopusConfigurationTests { + + @Test + void defaultServerUriIsOctopusCloud() { + var config = new OctopusConfiguration("test-client"); + assertThat(config.getServerUri()).isEqualTo(URI.create("https://features.octopus.com")); + } + + @Test + void serverUriCanBeOverridden() { + var config = new OctopusConfiguration("test-client"); + var customUri = URI.create("http://localhost:8080"); + config.setServerUri(customUri); + assertThat(config.getServerUri()).isEqualTo(customUri); + } +} From 5f7bb7c879ce9ad01a15b16c617c4bcec22f7988 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 14:07:57 +1100 Subject: [PATCH 07/16] Add WireMock server helper --- .../octopus/openfeature/provider/Server.java | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 src/test/java/com/octopus/openfeature/provider/Server.java diff --git a/src/test/java/com/octopus/openfeature/provider/Server.java b/src/test/java/com/octopus/openfeature/provider/Server.java new file mode 100644 index 0000000..49f3c77 --- /dev/null +++ b/src/test/java/com/octopus/openfeature/provider/Server.java @@ -0,0 +1,62 @@ +package com.octopus.openfeature.provider; + +import com.github.tomakehurst.wiremock.WireMockServer; + +import java.util.Base64; +import java.util.UUID; + +import static com.github.tomakehurst.wiremock.client.WireMock.*; +import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; + +/** + * Fake HTTP server for specification tests. + * + * Each call to {@link #configure(String)} registers a stub for a unique Bearer token + * and returns that token as the client identifier. Stubs accumulate over the server's + * lifetime (one per test case), which is harmless since each token is unique. + * + * Note: parallel test execution is not supported because SpecificationTests uses + * the OpenFeatureAPI singleton. + */ +class Server { + + // A fixed hash is safe here because each test shuts down the provider via OpenFeatureAPI.shutdown() + // before the background refresh thread can poll the check endpoint and compare hashes. + private static final String CONTENT_HASH = Base64.getEncoder().encodeToString(new byte[]{0x01}); + private final WireMockServer wireMock; + + Server() { + wireMock = new WireMockServer(wireMockConfig().dynamicPort()); + wireMock.start(); + // Fallback: return 401 for any request that does not match a registered token. + wireMock.stubFor(any(anyUrl()) + .atPriority(100) + .willReturn(aResponse().withStatus(401))); + } + + /** + * Registers the given JSON as the response body for a new unique client token. + * + * @param responseJson the JSON array that the toggle API would return + * @return the client identifier (Bearer token) to use in OctopusConfiguration + */ + String configure(String responseJson) { + String token = UUID.randomUUID().toString(); + wireMock.stubFor(get(urlPathEqualTo("/api/featuretoggles/v3/")) + .withHeader("Authorization", equalTo("Bearer " + token)) + .willReturn(aResponse() + .withStatus(200) + .withHeader("Content-Type", "application/json") + .withHeader("ContentHash", CONTENT_HASH) + .withBody(responseJson))); + return token; + } + + String baseUrl() { + return wireMock.baseUrl(); + } + + void stop() { + wireMock.stop(); + } +} From 75b86a7bcef21924a2cce65dac6f2cd64df669f2 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 14:18:18 +1100 Subject: [PATCH 08/16] Add SpecificationTests --- .../provider/OctopusConfiguration.java | 2 +- .../provider/SpecificationTests.java | 146 ++++++++++++++++++ 2 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 src/test/java/com/octopus/openfeature/provider/SpecificationTests.java diff --git a/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java b/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java index 431fb26..f9ab3b2 100644 --- a/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java +++ b/src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java @@ -17,7 +17,7 @@ public OctopusConfiguration(String clientIdentifier) { public URI getServerUri() { return serverUri; } - // Package-private: visible to tests in same package, not to library consumers. + // Note: package-private by default. Visible to tests in same package, but not to library consumers. void setServerUri(URI serverUri) { this.serverUri = serverUri; } public Duration getCacheDuration() { diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java new file mode 100644 index 0000000..969ac97 --- /dev/null +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -0,0 +1,146 @@ +package com.octopus.openfeature.provider; + +import com.fasterxml.jackson.annotation.JsonIgnoreProperties; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import dev.openfeature.sdk.Client; +import dev.openfeature.sdk.ErrorCode; +import dev.openfeature.sdk.EvaluationContext; +import dev.openfeature.sdk.FlagEvaluationDetails; +import dev.openfeature.sdk.MutableContext; +import dev.openfeature.sdk.OpenFeatureAPI; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.io.IOException; +import java.io.UncheckedIOException; +import java.net.URI; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static org.assertj.core.api.Assertions.assertThat; + +class SpecificationTests { + + private static Server server; + + @BeforeAll + static void startServer() { + server = new Server(); + } + + @AfterAll + static void stopServer() { + server.stop(); + } + + @AfterEach + void shutdownApi() { + OpenFeatureAPI.getInstance().shutdown(); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("fixtureTestCases") + void evaluate(String description, String responseJson, FixtureCase testCase) { + String token = server.configure(responseJson); + OctopusConfiguration config = new OctopusConfiguration(token); + config.setServerUri(URI.create(server.baseUrl())); + + OpenFeatureAPI api = OpenFeatureAPI.getInstance(); + api.setProviderAndWait(new OctopusProvider(config)); + Client client = api.getClient(); + + EvaluationContext ctx = buildContext(testCase.configuration.context); + FlagEvaluationDetails result = client.getBooleanDetails( + testCase.configuration.slug, + testCase.configuration.defaultValue, + ctx + ); + + assertThat(result.getValue()) + .as(description + " → value") + .isEqualTo(testCase.expected.value); + assertThat(result.getErrorCode()) + .as(description + " → errorCode") + .isEqualTo(mapErrorCode(testCase.expected.errorCode)); + } + + static Stream fixtureTestCases() throws IOException { + ObjectMapper mapper = new ObjectMapper(); + List jsonFiles; + try (Stream files = Files.list(Path.of("specification", "Fixtures"))) { + jsonFiles = files + .filter(p -> p.getFileName().toString().endsWith(".json")) + .collect(Collectors.toList()); + } + return jsonFiles.stream().flatMap(path -> { + try { + Fixture fixture = mapper.readValue(path.toFile(), Fixture.class); + String responseJson = mapper.writeValueAsString(fixture.response); + return Stream.of(fixture.cases) + .map(c -> Arguments.of(c.description, responseJson, c)); + } catch (IOException e) { + throw new UncheckedIOException(e); + } + }); + } + + private static EvaluationContext buildContext(Map context) { + MutableContext ctx = new MutableContext(); + if (context != null) { + context.forEach(ctx::add); + } + return ctx; + } + + private static ErrorCode mapErrorCode(String code) { + if (code == null) return null; + switch (code) { + case "FLAG_NOT_FOUND": return ErrorCode.FLAG_NOT_FOUND; + case "PARSE_ERROR": return ErrorCode.PARSE_ERROR; + case "TYPE_MISMATCH": return ErrorCode.TYPE_MISMATCH; + case "TARGETING_KEY_MISSING": return ErrorCode.TARGETING_KEY_MISSING; + case "PROVIDER_NOT_READY": return ErrorCode.PROVIDER_NOT_READY; + case "INVALID_CONTEXT": return ErrorCode.INVALID_CONTEXT; + case "PROVIDER_FATAL": return ErrorCode.PROVIDER_FATAL; + case "GENERAL": return ErrorCode.GENERAL; + default: throw new IllegalArgumentException("Unknown error code in fixture: " + code); + } + } + + // ---- Fixture model classes ---- + + @JsonIgnoreProperties(ignoreUnknown = true) + static class Fixture { + public JsonNode response; + public FixtureCase[] cases; + } + + @JsonIgnoreProperties(ignoreUnknown = true) + static class FixtureCase { + public String description; + public FixtureConfiguration configuration; + public FixtureExpected expected; + } + + @JsonIgnoreProperties(ignoreUnknown = true) + static class FixtureConfiguration { + public String slug; + public boolean defaultValue; + public Map context; + } + + @JsonIgnoreProperties(ignoreUnknown = true) + static class FixtureExpected { + public boolean value; + public String errorCode; + } +} From 90e62e80c0b08f4892de3fc0e0f6ca0f36c2a4ed Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 14:19:54 +1100 Subject: [PATCH 09/16] Checkout submodules in CI --- .github/workflows/ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5126c4c..b1bcbdd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -13,6 +13,8 @@ jobs: steps: - uses: actions/checkout@v4 + with: + submodules: true - name: Set up JDK 11 uses: actions/setup-java@v4 with: From aa7417de958ee57f5fd6be79bf5ae2ea49e08d90 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 14:22:43 +1100 Subject: [PATCH 10/16] Add check for no JSON files --- .../com/octopus/openfeature/provider/SpecificationTests.java | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java index 969ac97..69bc954 100644 --- a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -81,6 +81,11 @@ static Stream fixtureTestCases() throws IOException { .filter(p -> p.getFileName().toString().endsWith(".json")) .collect(Collectors.toList()); } + if (jsonFiles.isEmpty()) { + throw new IllegalStateException( + "No fixture files found under 'specification/Fixtures/'. " + + "Ensure the git submodule is initialised: git submodule update --init"); + } return jsonFiles.stream().flatMap(path -> { try { Fixture fixture = mapper.readValue(path.toFile(), Fixture.class); From d551595eca84c3ae912d86d4ec758afa0181fd60 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 15:00:11 +1100 Subject: [PATCH 11/16] Maintain raw JSON for response --- .../provider/SpecificationTests.java | 32 +++++++++++++++---- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java index 69bc954..1543866 100644 --- a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -1,8 +1,13 @@ package com.octopus.openfeature.provider; import com.fasterxml.jackson.annotation.JsonIgnoreProperties; -import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.core.JsonFactory; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.StreamReadFeature; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.JsonDeserializer; import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.annotation.JsonDeserialize; import dev.openfeature.sdk.Client; import dev.openfeature.sdk.ErrorCode; import dev.openfeature.sdk.EvaluationContext; @@ -74,7 +79,9 @@ void evaluate(String description, String responseJson, FixtureCase testCase) { } static Stream fixtureTestCases() throws IOException { - ObjectMapper mapper = new ObjectMapper(); + ObjectMapper mapper = new ObjectMapper( + JsonFactory.builder().enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION).build() + ); List jsonFiles; try (Stream files = Files.list(Path.of("specification", "Fixtures"))) { jsonFiles = files @@ -88,10 +95,10 @@ static Stream fixtureTestCases() throws IOException { } return jsonFiles.stream().flatMap(path -> { try { - Fixture fixture = mapper.readValue(path.toFile(), Fixture.class); - String responseJson = mapper.writeValueAsString(fixture.response); + String fileContent = Files.readString(path); + Fixture fixture = mapper.readValue(fileContent, Fixture.class); return Stream.of(fixture.cases) - .map(c -> Arguments.of(c.description, responseJson, c)); + .map(c -> Arguments.of(c.description, fixture.response, c)); } catch (IOException e) { throw new UncheckedIOException(e); } @@ -125,7 +132,8 @@ private static ErrorCode mapErrorCode(String code) { @JsonIgnoreProperties(ignoreUnknown = true) static class Fixture { - public JsonNode response; + @JsonDeserialize(using = RawJsonDeserializer.class) + public String response; public FixtureCase[] cases; } @@ -148,4 +156,16 @@ static class FixtureExpected { public boolean value; public String errorCode; } + + static class RawJsonDeserializer extends JsonDeserializer { + @Override + public String deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException { + long begin = jp.currentLocation().getCharOffset(); + jp.skipChildren(); + long end = jp.currentLocation().getCharOffset(); + String json = jp.currentLocation().contentReference().getRawContent().toString(); + return json.substring((int) begin - 1, (int) end); + } + } + } From 135bae3c3bd401333c5ab870f9b704adc8806618 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Thu, 26 Mar 2026 16:30:45 +1100 Subject: [PATCH 12/16] Display file name in IDE --- .../openfeature/provider/SpecificationTests.java | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java index 1543866..5b1cf6c 100644 --- a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -52,9 +52,9 @@ void shutdownApi() { OpenFeatureAPI.getInstance().shutdown(); } - @ParameterizedTest(name = "{0}") + @ParameterizedTest(name = "[{0}] {1}") @MethodSource("fixtureTestCases") - void evaluate(String description, String responseJson, FixtureCase testCase) { + void evaluate(String fileName, String description, String responseJson, FixtureCase testCase) { String token = server.configure(responseJson); OctopusConfiguration config = new OctopusConfiguration(token); config.setServerUri(URI.create(server.baseUrl())); @@ -71,10 +71,10 @@ void evaluate(String description, String responseJson, FixtureCase testCase) { ); assertThat(result.getValue()) - .as(description + " → value") + .as("[%s] %s → value", fileName, description) .isEqualTo(testCase.expected.value); assertThat(result.getErrorCode()) - .as(description + " → errorCode") + .as("[%s] %s → errorCode", fileName, description) .isEqualTo(mapErrorCode(testCase.expected.errorCode)); } @@ -97,8 +97,9 @@ static Stream fixtureTestCases() throws IOException { try { String fileContent = Files.readString(path); Fixture fixture = mapper.readValue(fileContent, Fixture.class); + String fileName = path.getFileName().toString(); return Stream.of(fixture.cases) - .map(c -> Arguments.of(c.description, fixture.response, c)); + .map(c -> Arguments.of(fileName, c.description, fixture.response, c)); } catch (IOException e) { throw new UncheckedIOException(e); } From 579385d6d0f60c1fe08370310fe59a8520a92eca Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Fri, 27 Mar 2026 14:33:17 +1100 Subject: [PATCH 13/16] Run tests in CI --- .github/workflows/ci.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b1bcbdd..be0a865 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,10 +1,9 @@ -name: Build and Test +name: Build, Test, and Package on: push: branches: [ "main" ] pull_request: - branches: [ "main" ] jobs: build: @@ -15,12 +14,14 @@ jobs: - uses: actions/checkout@v4 with: submodules: true + - name: Set up JDK 11 uses: actions/setup-java@v4 with: java-version: '11' distribution: 'temurin' cache: maven - - name: Build with Maven + + - name: Build, test and package run: mvn -B package --file pom.xml From 9a165131b76a3e44f03df50b45d7ae57dfc3ba0c Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Fri, 27 Mar 2026 14:48:35 +1100 Subject: [PATCH 14/16] Remove GenAI docs --- .../plans/2026-03-26-specification-tests.md | 499 ------------------ .../2026-03-26-specification-tests-design.md | 74 --- 2 files changed, 573 deletions(-) delete mode 100644 docs/superpowers/plans/2026-03-26-specification-tests.md delete mode 100644 docs/superpowers/specs/2026-03-26-specification-tests-design.md diff --git a/docs/superpowers/plans/2026-03-26-specification-tests.md b/docs/superpowers/plans/2026-03-26-specification-tests.md deleted file mode 100644 index d5a1679..0000000 --- a/docs/superpowers/plans/2026-03-26-specification-tests.md +++ /dev/null @@ -1,499 +0,0 @@ -# Specification Tests Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add a parameterised integration test harness that reads shared JSON fixture files from a git submodule and runs each test case against a fake HTTP server. - -**Architecture:** A `Server` test helper wraps WireMock and maps unique Bearer tokens to fixture response bodies via per-token stubs. `SpecificationTests` uses JUnit 5 `@ParameterizedTest` + `@MethodSource` to load fixture files from `specification/Fixtures/*.json`, yielding one named test per case. Each test configures the fake server, points `OctopusProvider` at it, evaluates the flag, and asserts value and error code. - -**Tech Stack:** JUnit 5 (already present), WireMock 3.x (`org.wiremock:wiremock`, new test dependency), Jackson (already present), OpenFeature Java SDK (already present) - ---- - -## File Map - -| Action | File | Purpose | -|---|---|---| -| Modify | `pom.xml` | Add WireMock test dependency | -| Shell | `git submodule add ...` | Clone spec repo to `specification/` | -| Modify | `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` | Add `serverUri` field + package-private setter | -| Create | `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java` | Test for configurable server URI | -| Create | `src/test/java/com/octopus/openfeature/provider/Server.java` | WireMock wrapper for fake toggle API | -| Create | `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java` | Parameterised specification tests + fixture models | -| Modify | `.github/workflows/ci.yml` | Add `submodules: true` to checkout step | - ---- - -## Task 1: Add WireMock dependency - -**Files:** -- Modify: `pom.xml` - -- [ ] **Step 1: Add WireMock to pom.xml** - -In `pom.xml`, add the following inside ``: - -```xml - - org.wiremock - wiremock - 3.5.4 - test - -``` - -- [ ] **Step 2: Verify existing tests still pass** - -Run: `mvn test -B` - -Expected output includes: -``` -Tests run: 26, Failures: 0, Errors: 0, Skipped: 0 -BUILD SUCCESS -``` - -- [ ] **Step 3: Commit** - -```bash -git add pom.xml -git commit -m "build: add WireMock test dependency" -``` - ---- - -## Task 2: Add git submodule - -**Files:** -- Shell: `git submodule add` - -- [ ] **Step 1: Add the specification submodule** - -Run from the project root: -```bash -git submodule add https://github.com/OctopusDeploy/openfeature-provider-specification.git specification -``` - -- [ ] **Step 2: Verify the fixture file is present** - -Run: `ls specification/Fixtures/` - -Expected output: -``` -simple-value-only-toggles.json -``` - -- [ ] **Step 3: Commit** - -```bash -git add .gitmodules specification -git commit -m "chore: add openfeature-provider-specification as git submodule" -``` - ---- - -## Task 3: Make OctopusConfiguration server URI configurable - -**Files:** -- Create: `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java` -- Modify: `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` - -- [ ] **Step 1: Write the failing test** - -Create `src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java`: - -```java -package com.octopus.openfeature.provider; - -import org.junit.jupiter.api.Test; - -import java.net.URI; - -import static org.assertj.core.api.Assertions.assertThat; - -class OctopusConfigurationTests { - - @Test - void defaultServerUriIsOctopusCloud() { - var config = new OctopusConfiguration("test-client"); - assertThat(config.getServerUri()).isEqualTo(URI.create("https://features.octopus.com")); - } - - @Test - void serverUriCanBeOverridden() { - var config = new OctopusConfiguration("test-client"); - var customUri = URI.create("http://localhost:8080"); - config.setServerUri(customUri); - assertThat(config.getServerUri()).isEqualTo(customUri); - } -} -``` - -- [ ] **Step 2: Run to confirm failure** - -Run: `mvn test -Dtest=OctopusConfigurationTests -B` - -Expected: `COMPILATION ERROR` or `BUILD FAILURE` — `setServerUri` does not yet exist. - -- [ ] **Step 3: Implement setServerUri in OctopusConfiguration** - -Replace the content of `src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java` with: - -```java -package com.octopus.openfeature.provider; - -import java.net.URI; -import java.time.Duration; - -public class OctopusConfiguration { - private final String clientIdentifier; - private static final String DEFAULT_SERVER_URI = "https://features.octopus.com"; - private URI serverUri = URI.create(DEFAULT_SERVER_URI); - private Duration cacheDuration = Duration.ofMinutes(1); - - public OctopusConfiguration(String clientIdentifier) { - this.clientIdentifier = clientIdentifier; - } - - public String getClientIdentifier() { return clientIdentifier; } - - public URI getServerUri() { return serverUri; } - - // Package-private: visible to tests in same package, not to library consumers. - void setServerUri(URI serverUri) { this.serverUri = serverUri; } - - public Duration getCacheDuration() { - return cacheDuration; - } - - public Duration setCacheDuration(Duration cacheDuration) { - this.cacheDuration = cacheDuration; - return this.cacheDuration; - } -} -``` - -- [ ] **Step 4: Run the tests to confirm they pass** - -Run: `mvn test -B` - -Expected output includes: -``` -Tests run: 28, Failures: 0, Errors: 0, Skipped: 0 -BUILD SUCCESS -``` - -- [ ] **Step 5: Commit** - -```bash -git add src/main/java/com/octopus/openfeature/provider/OctopusConfiguration.java \ - src/test/java/com/octopus/openfeature/provider/OctopusConfigurationTests.java -git commit -m "feat: make OctopusConfiguration server URI configurable (package-private)" -``` - ---- - -## Task 4: Create the Server test helper - -**Files:** -- Create: `src/test/java/com/octopus/openfeature/provider/Server.java` - -- [ ] **Step 1: Create Server.java** - -Create `src/test/java/com/octopus/openfeature/provider/Server.java`: - -```java -package com.octopus.openfeature.provider; - -import com.github.tomakehurst.wiremock.WireMockServer; - -import java.util.Base64; -import java.util.UUID; - -import static com.github.tomakehurst.wiremock.client.WireMock.*; -import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; - -/** - * Fake HTTP server for specification tests. - * - * Each call to {@link #configure(String)} registers a stub for a unique Bearer token - * and returns that token as the client identifier. Stubs accumulate over the server's - * lifetime (one per test case), which is harmless since each token is unique. - * - * Note: parallel test execution is not supported because SpecificationTests uses - * the OpenFeatureAPI singleton. - */ -class Server { - - private static final String CONTENT_HASH = Base64.getEncoder().encodeToString(new byte[]{0x01}); - private final WireMockServer wireMock; - - Server() { - wireMock = new WireMockServer(wireMockConfig().dynamicPort()); - wireMock.start(); - // Fallback: return 401 for any request that does not match a registered token. - wireMock.stubFor(any(anyUrl()) - .atPriority(100) - .willReturn(aResponse().withStatus(401))); - } - - /** - * Registers the given JSON as the response body for a new unique client token. - * - * @param responseJson the JSON array that the toggle API would return - * @return the client identifier (Bearer token) to use in OctopusConfiguration - */ - String configure(String responseJson) { - String token = UUID.randomUUID().toString(); - wireMock.stubFor(get(urlPathEqualTo("/api/featuretoggles/v3/")) - .withHeader("Authorization", equalTo("Bearer " + token)) - .willReturn(aResponse() - .withStatus(200) - .withHeader("Content-Type", "application/json") - .withHeader("ContentHash", CONTENT_HASH) - .withBody(responseJson))); - return token; - } - - String baseUrl() { - return wireMock.baseUrl(); - } - - void stop() { - wireMock.stop(); - } -} -``` - -- [ ] **Step 2: Verify it compiles** - -Run: `mvn test-compile -B` - -Expected: `BUILD SUCCESS` - -- [ ] **Step 3: Commit** - -```bash -git add src/test/java/com/octopus/openfeature/provider/Server.java -git commit -m "test: add WireMock Server helper for specification tests" -``` - ---- - -## Task 5: Create SpecificationTests - -**Files:** -- Create: `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java` - -- [ ] **Step 1: Create SpecificationTests.java** - -Create `src/test/java/com/octopus/openfeature/provider/SpecificationTests.java`: - -```java -package com.octopus.openfeature.provider; - -import com.fasterxml.jackson.databind.JsonNode; -import com.fasterxml.jackson.databind.ObjectMapper; -import dev.openfeature.sdk.Client; -import dev.openfeature.sdk.ErrorCode; -import dev.openfeature.sdk.EvaluationContext; -import dev.openfeature.sdk.FlagEvaluationDetails; -import dev.openfeature.sdk.MutableContext; -import dev.openfeature.sdk.OpenFeatureAPI; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.params.ParameterizedTest; -import org.junit.jupiter.params.provider.Arguments; -import org.junit.jupiter.params.provider.MethodSource; - -import java.io.IOException; -import java.io.UncheckedIOException; -import java.net.URI; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.List; -import java.util.Map; -import java.util.stream.Collectors; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class SpecificationTests { - - private static Server server; - - @BeforeAll - static void startServer() { - server = new Server(); - } - - @AfterAll - static void stopServer() { - server.stop(); - } - - @AfterEach - void shutdownApi() throws Exception { - OpenFeatureAPI.getInstance().shutdown(); - } - - @ParameterizedTest(name = "{0}") - @MethodSource("fixtureTestCases") - void evaluate(String description, String responseJson, FixtureCase testCase) throws Exception { - String token = server.configure(responseJson); - OctopusConfiguration config = new OctopusConfiguration(token); - config.setServerUri(URI.create(server.baseUrl())); - - OpenFeatureAPI api = OpenFeatureAPI.getInstance(); - api.setProviderAndWait(new OctopusProvider(config)); - Client client = api.getClient(); - - EvaluationContext ctx = buildContext(testCase.configuration.context); - FlagEvaluationDetails result = client.getBooleanDetails( - testCase.configuration.slug, - testCase.configuration.defaultValue, - ctx - ); - - assertThat(result.getValue()) - .as(description + " → value") - .isEqualTo(testCase.expected.value); - assertThat(result.getErrorCode()) - .as(description + " → errorCode") - .isEqualTo(mapErrorCode(testCase.expected.errorCode)); - } - - static Stream fixtureTestCases() throws IOException { - ObjectMapper mapper = new ObjectMapper(); - List jsonFiles; - try (Stream files = Files.list(Path.of("specification", "Fixtures"))) { - jsonFiles = files - .filter(p -> p.getFileName().toString().endsWith(".json")) - .collect(Collectors.toList()); - } - return jsonFiles.stream().flatMap(path -> { - try { - Fixture fixture = mapper.readValue(path.toFile(), Fixture.class); - String responseJson = fixture.response.toString(); - return Stream.of(fixture.cases) - .map(c -> Arguments.of(c.description, responseJson, c)); - } catch (IOException e) { - throw new UncheckedIOException(e); - } - }); - } - - private static EvaluationContext buildContext(Map context) { - MutableContext ctx = new MutableContext(); - if (context != null) { - context.forEach(ctx::add); - } - return ctx; - } - - private static ErrorCode mapErrorCode(String code) { - if (code == null) return null; - switch (code) { - case "FLAG_NOT_FOUND": return ErrorCode.FLAG_NOT_FOUND; - case "PARSE_ERROR": return ErrorCode.PARSE_ERROR; - case "TYPE_MISMATCH": return ErrorCode.TYPE_MISMATCH; - case "TARGETING_KEY_MISSING": return ErrorCode.TARGETING_KEY_MISSING; - case "PROVIDER_NOT_READY": return ErrorCode.PROVIDER_NOT_READY; - case "INVALID_CONTEXT": return ErrorCode.INVALID_CONTEXT; - case "PROVIDER_FATAL": return ErrorCode.PROVIDER_FATAL; - case "GENERAL": return ErrorCode.GENERAL; - default: throw new IllegalArgumentException("Unknown error code in fixture: " + code); - } - } - - // ---- Fixture model classes ---- - - static class Fixture { - public JsonNode response; - public FixtureCase[] cases; - } - - static class FixtureCase { - public String description; - public FixtureConfiguration configuration; - public FixtureExpected expected; - } - - static class FixtureConfiguration { - public String slug; - public boolean defaultValue; - public Map context; - } - - static class FixtureExpected { - public boolean value; - public String errorCode; - } -} -``` - -- [ ] **Step 2: Run the specification tests** - -Run: `mvn test -B` - -Expected output includes: -``` -Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: ... -- in com.octopus.openfeature.provider.SpecificationTests -Tests run: 32, Failures: 0, Errors: 0, Skipped: 0 -BUILD SUCCESS -``` - -The 4 new tests correspond to the 4 cases in `specification/Fixtures/simple-value-only-toggles.json`. - -- [ ] **Step 3: Commit** - -```bash -git add src/test/java/com/octopus/openfeature/provider/SpecificationTests.java -git commit -m "test: add specification tests harness (DEVEX-138)" -``` - ---- - -## Task 6: Update CI workflow - -**Files:** -- Modify: `.github/workflows/ci.yml` - -- [ ] **Step 1: Add submodules: true to the checkout step** - -Replace the content of `.github/workflows/ci.yml` with: - -```yaml -name: Build and Test - -on: - push: - branches: [ "main" ] - pull_request: - branches: [ "main" ] - -jobs: - build: - - runs-on: ubuntu-latest - - steps: - - uses: actions/checkout@v4 - with: - submodules: true - - name: Set up JDK 11 - uses: actions/setup-java@v4 - with: - java-version: '11' - distribution: 'temurin' - cache: maven - - name: Build with Maven - run: mvn -B package --file pom.xml -``` - -- [ ] **Step 2: Commit** - -```bash -git add .github/workflows/ci.yml -git commit -m "ci: fetch git submodules on checkout" -``` diff --git a/docs/superpowers/specs/2026-03-26-specification-tests-design.md b/docs/superpowers/specs/2026-03-26-specification-tests-design.md deleted file mode 100644 index 270ff6a..0000000 --- a/docs/superpowers/specs/2026-03-26-specification-tests-design.md +++ /dev/null @@ -1,74 +0,0 @@ -# Specification Tests — Design - -**Date:** 2026-03-26 -**Linear:** DEVEX-138 -**ADR:** [Sharing client library tests](https://whimsical.com/octopusdeploy/adr-sharing-client-library-tests-8QPKL2XSw9SFec4zJENBo9) -**C# reference:** [openfeature-provider-dotnet#44](https://github.com/OctopusDeploy/openfeature-provider-dotnet/pull/44) - -## Context - -The OpenFeature provider specification repository (`OctopusDeploy/openfeature-provider-specification`) holds shared JSON fixture files used to drive cross-language integration tests. The Java client needs a test harness that reads these fixtures and runs each case as a separate named test against a fake HTTP server, mirroring the approach taken in the C# client. - -## Scope - -Implement the Java test harness only. Adding new fixture files to the specification repository is out of scope for this work. - -## Design - -### 1. Production code change — configurable server URI - -`OctopusConfiguration.getServerUri()` currently returns a hardcoded `https://features.octopus.com`. A private `serverUri` field will be added, defaulting to that value. A package-private setter `void setServerUri(URI uri)` will allow tests in the same package to override it without exposing the setter to library consumers (Java equivalent of C# `internal`). - -### 2. New test classes - -Both in `src/test/java/com/octopus/openfeature/provider/`. - -#### `Server` - -Wraps a `WireMockServer` started and stopped once per test class. Exposes: - -``` -String configure(String responseJson) -``` - -This stores the JSON keyed by a randomly generated UUID and returns that UUID as the client identifier token. A custom WireMock `ResponseTransformer` intercepts every request to `/api/featuretoggles/v3/`, reads the `Authorization: Bearer ` header, looks up the stored JSON, and returns it with a `ContentHash` response header (stable base64-encoded value). Requests with an unrecognised token receive a 401. - -The background refresh thread in `OctopusContextProvider` polls the check endpoint (`/api/featuretoggles/check/v3/`) only after the configured `cacheDuration` elapses (default: 1 minute). Since each test completes and shuts down the provider in milliseconds, the check endpoint will not be called in practice. - -#### `SpecificationTests` - -```java -@ParameterizedTest(name = "{0}") -@MethodSource("fixtureTestCases") -void evaluate(String description, String responseJson, FixtureCase testCase) { ... } -``` - -The `fixtureTestCases()` static method: -1. Walks `specification/Fixtures/*.json` -2. Deserialises each file with Jackson (existing dependency) into a `Fixture` record containing a raw `response` JSON string and a `FixtureCase[]` -3. Yields one `Arguments.of(description, responseJson, testCase)` per case - -The `description` field becomes the parameterised test display name (argument `{0}`). - -Each test invocation: -1. Calls `server.configure(responseJson)` to get a unique client identifier -2. Creates `OctopusConfiguration` with that identifier, then calls `setServerUri` to point at the WireMock base URL -3. Calls `OpenFeatureAPI.getInstance().setProvider(provider)` and sets the evaluation context from the fixture -4. Evaluates the flag via `client.getBooleanDetails(slug, defaultValue)` -5. Asserts `result.getValue()` and `result.getErrorCode()` match the fixture expectations -6. Calls `OpenFeatureAPI.getInstance().shutdown()` to stop the background refresh thread - -Error codes are mapped from fixture strings (e.g. `"FLAG_NOT_FOUND"`) to `ErrorCode` enum values. An unrecognised string throws `IllegalArgumentException` — a fixture bug should fail loudly. - -### 3. Infrastructure changes - -| Change | Detail | -|---|---| -| Git submodule | `specification/` → `https://github.com/OctopusDeploy/openfeature-provider-specification` | -| `pom.xml` | Add `org.wiremock:wiremock` (test scope) | -| CI workflow | Add `submodules: true` to the `actions/checkout` step | - -## Out of scope - -- Adding new fixture files to the specification repository (separate ticket) -- Removing existing `OctopusContextTests` cases now covered by fixtures (can be done once the fixture set is expanded) From 34d46a5b2a0e710f51f94ecae0c3d3c5049a712e Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Fri, 27 Mar 2026 15:06:56 +1100 Subject: [PATCH 15/16] Self review --- .../provider/SpecificationTests.java | 69 ++++++++++--------- 1 file changed, 38 insertions(+), 31 deletions(-) diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java index 5b1cf6c..28fd7f6 100644 --- a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -5,9 +5,11 @@ import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.core.StreamReadFeature; import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.DeserializationFeature; import com.fasterxml.jackson.databind.JsonDeserializer; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.annotation.JsonDeserialize; +import com.fasterxml.jackson.databind.json.JsonMapper; import dev.openfeature.sdk.Client; import dev.openfeature.sdk.ErrorCode; import dev.openfeature.sdk.EvaluationContext; @@ -65,33 +67,35 @@ void evaluate(String fileName, String description, String responseJson, FixtureC EvaluationContext ctx = buildContext(testCase.configuration.context); FlagEvaluationDetails result = client.getBooleanDetails( - testCase.configuration.slug, - testCase.configuration.defaultValue, - ctx + testCase.configuration.slug, + testCase.configuration.defaultValue, + ctx ); assertThat(result.getValue()) - .as("[%s] %s → value", fileName, description) - .isEqualTo(testCase.expected.value); + .as("[%s] %s → value", fileName, description) + .isEqualTo(testCase.expected.value); assertThat(result.getErrorCode()) - .as("[%s] %s → errorCode", fileName, description) - .isEqualTo(mapErrorCode(testCase.expected.errorCode)); + .as("[%s] %s → errorCode", fileName, description) + .isEqualTo(mapErrorCode(testCase.expected.errorCode)); } static Stream fixtureTestCases() throws IOException { - ObjectMapper mapper = new ObjectMapper( - JsonFactory.builder().enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION).build() - ); + ObjectMapper mapper = JsonMapper.builder() + .enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION) + .disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES) + .build(); + List jsonFiles; try (Stream files = Files.list(Path.of("specification", "Fixtures"))) { jsonFiles = files - .filter(p -> p.getFileName().toString().endsWith(".json")) - .collect(Collectors.toList()); + .filter(p -> p.getFileName().toString().endsWith(".json")) + .collect(Collectors.toList()); } if (jsonFiles.isEmpty()) { throw new IllegalStateException( - "No fixture files found under 'specification/Fixtures/'. " + - "Ensure the git submodule is initialised: git submodule update --init"); + "No fixture files found under 'specification/Fixtures/'. " + + "Ensure the git submodule is initialised: git submodule update --init"); } return jsonFiles.stream().flatMap(path -> { try { @@ -99,7 +103,7 @@ static Stream fixtureTestCases() throws IOException { Fixture fixture = mapper.readValue(fileContent, Fixture.class); String fileName = path.getFileName().toString(); return Stream.of(fixture.cases) - .map(c -> Arguments.of(fileName, c.description, fixture.response, c)); + .map(c -> Arguments.of(fileName, c.description, fixture.response, c)); } catch (IOException e) { throw new UncheckedIOException(e); } @@ -117,42 +121,45 @@ private static EvaluationContext buildContext(Map context) { private static ErrorCode mapErrorCode(String code) { if (code == null) return null; switch (code) { - case "FLAG_NOT_FOUND": return ErrorCode.FLAG_NOT_FOUND; - case "PARSE_ERROR": return ErrorCode.PARSE_ERROR; - case "TYPE_MISMATCH": return ErrorCode.TYPE_MISMATCH; - case "TARGETING_KEY_MISSING": return ErrorCode.TARGETING_KEY_MISSING; - case "PROVIDER_NOT_READY": return ErrorCode.PROVIDER_NOT_READY; - case "INVALID_CONTEXT": return ErrorCode.INVALID_CONTEXT; - case "PROVIDER_FATAL": return ErrorCode.PROVIDER_FATAL; - case "GENERAL": return ErrorCode.GENERAL; - default: throw new IllegalArgumentException("Unknown error code in fixture: " + code); + case "FLAG_NOT_FOUND": + return ErrorCode.FLAG_NOT_FOUND; + case "PARSE_ERROR": + return ErrorCode.PARSE_ERROR; + case "TYPE_MISMATCH": + return ErrorCode.TYPE_MISMATCH; + case "TARGETING_KEY_MISSING": + return ErrorCode.TARGETING_KEY_MISSING; + case "PROVIDER_NOT_READY": + return ErrorCode.PROVIDER_NOT_READY; + case "INVALID_CONTEXT": + return ErrorCode.INVALID_CONTEXT; + case "PROVIDER_FATAL": + return ErrorCode.PROVIDER_FATAL; + case "GENERAL": + return ErrorCode.GENERAL; + default: + throw new IllegalArgumentException("Unknown error code in fixture: " + code); } } - // ---- Fixture model classes ---- - - @JsonIgnoreProperties(ignoreUnknown = true) static class Fixture { @JsonDeserialize(using = RawJsonDeserializer.class) public String response; public FixtureCase[] cases; } - @JsonIgnoreProperties(ignoreUnknown = true) static class FixtureCase { public String description; public FixtureConfiguration configuration; public FixtureExpected expected; } - @JsonIgnoreProperties(ignoreUnknown = true) static class FixtureConfiguration { public String slug; public boolean defaultValue; public Map context; } - @JsonIgnoreProperties(ignoreUnknown = true) static class FixtureExpected { public boolean value; public String errorCode; @@ -160,7 +167,7 @@ static class FixtureExpected { static class RawJsonDeserializer extends JsonDeserializer { @Override - public String deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException { + public String deserialize(JsonParser jp, DeserializationContext dc) throws IOException { long begin = jp.currentLocation().getCharOffset(); jp.skipChildren(); long end = jp.currentLocation().getCharOffset(); From 91bab700185fa72e4f21e1139add889def00ec42 Mon Sep 17 00:00:00 2001 From: Liam Hughes Date: Fri, 27 Mar 2026 15:39:00 +1100 Subject: [PATCH 16/16] Tidy imports --- .../octopus/openfeature/provider/SpecificationTests.java | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java index 28fd7f6..e6fe337 100644 --- a/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java +++ b/src/test/java/com/octopus/openfeature/provider/SpecificationTests.java @@ -1,7 +1,5 @@ package com.octopus.openfeature.provider; -import com.fasterxml.jackson.annotation.JsonIgnoreProperties; -import com.fasterxml.jackson.core.JsonFactory; import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.core.StreamReadFeature; import com.fasterxml.jackson.databind.DeserializationContext; @@ -10,12 +8,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.annotation.JsonDeserialize; import com.fasterxml.jackson.databind.json.JsonMapper; -import dev.openfeature.sdk.Client; -import dev.openfeature.sdk.ErrorCode; -import dev.openfeature.sdk.EvaluationContext; -import dev.openfeature.sdk.FlagEvaluationDetails; -import dev.openfeature.sdk.MutableContext; -import dev.openfeature.sdk.OpenFeatureAPI; +import dev.openfeature.sdk.*; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeAll;