Add HTTP Response Compression Support #3003

Copilot · 2025-12-03T21:05:03Z

Why make this change?

DAB does not currently support response compression (gzip/brotli), which can reduce payload sizes by ~79% for large JSON responses. This is especially beneficial for REST and GraphQL endpoints over slow or high-latency networks.

What is this change?

Adds response compression middleware with configurable compression levels.

Configuration Model:

CompressionLevel enum: optimal, fastest, none
CompressionOptions record with JSON converter and validation
Default: optimal

Runtime Behavior:

Gzip and Brotli providers enabled for HTTPS
Applies to REST, GraphQL, and MCP endpoints
Logs compression level on startup (silent when none)
Properly handles unknown properties in JSON configuration

Schema & CLI:

{
  "runtime": {
    "compression": {
      "level": "optimal"
    }
  }
}

How was this tested?

Integration Tests
Unit Tests

Unit Tests:

Added 3 tests in ConfigureOptionsTests.cs covering all compression levels (fastest, optimal, none)

Integration Tests:

Added 7 tests in CompressionIntegrationTests.cs validating:
- Gzip compression works when Accept-Encoding header is present
- Brotli compression works
- Compression reduces payload size by at least 10%
- Compression is disabled when level is None
- No compression without Accept-Encoding header
- Fastest compression level works correctly
- Compressed content can be decompressed and is valid JSON

All tests pass successfully and verify that compression reduces payload sizes without breaking existing functionality.

Sample Request(s)

CLI Configuration:

dab configure --runtime.compression.level fastest

Sample Request/Response:

GET /api/employees HTTP/1.1
Accept-Encoding: gzip, br

HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Type: application/json

Startup Output (when enabled):

Response compression enabled with level 'Optimal' for REST, GraphQL, and MCP endpoints.

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Copilot

Pull request overview

This PR adds HTTP response compression support to Data API Builder, enabling Gzip and Brotli compression for REST, GraphQL, and MCP endpoints to reduce payload sizes and improve performance over slow networks.

Key Changes:

Added configurable compression middleware with three levels: optimal (default), fastest, and none
Implemented JSON schema, CLI commands, and configuration model for compression settings
Added unit tests for CLI configuration updates

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Service/Startup.cs	Adds `ConfigureResponseCompression` method to register Gzip and Brotli providers, and calls `UseResponseCompression` middleware when compression is enabled
src/Config/ObjectModel/CompressionOptions.cs	Defines the `CompressionOptions` record with `Level` property and `UserProvidedLevel` flag for JSON serialization control
src/Config/ObjectModel/CompressionLevel.cs	Defines the `CompressionLevel` enum with `Optimal`, `Fastest`, and `None` values
src/Config/Converters/CompressionOptionsConverterFactory.cs	Implements custom JSON converter for reading and writing compression configuration with validation
src/Config/RuntimeConfigLoader.cs	Registers the `CompressionOptionsConverterFactory` in the serialization options
src/Config/ObjectModel/RuntimeOptions.cs	Adds `Compression` property to runtime configuration options
src/Cli/ConfigGenerator.cs	Implements CLI configuration update logic for compression level settings
src/Cli/Commands/ConfigureOptions.cs	Adds `RuntimeCompressionLevel` command-line option for the `dab configure` command
src/Cli.Tests/ConfigureOptionsTests.cs	Adds test cases for all three compression levels (Optimal, Fastest, None)
schemas/dab.draft.schema.json	Adds JSON schema definition for the compression configuration section

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-04T03:10:01Z

src/Config/Converters/CompressionOptionsConverterFactory.cs

+                            }
+                        }
+                    }
+                }


The converter doesn't properly handle unknown properties with complex values (objects or arrays). After reading a property name on line 58, if the property is not "level", the code continues the loop without skipping the property value. This works for simple values but could cause parsing issues if an unknown property contains an object or array.

Add a default case to properly skip unknown properties:

if (string.Equals(propertyName, "level", StringComparison.OrdinalIgnoreCase)) { string? levelStr = reader.GetString(); if (levelStr is not null) { if (Enum.TryParse<CompressionLevel>(levelStr, ignoreCase: true, out CompressionLevel parsedLevel)) { level = parsedLevel; userProvidedLevel = true; } else { throw new JsonException($"Invalid compression level: '{levelStr}'. Valid values are: optimal, fastest, none."); } } } else { // Skip unknown properties reader.Skip(); }

Suggested change

}

else

{

// Skip unknown properties and their values (including objects/arrays)

reader.Skip();

}

Added the else block to properly skip unknown properties including objects and arrays using reader.Skip(). Fixed in commit c53261c.

Copilot · 2025-12-04T03:10:01Z

src/Cli.Tests/ConfigureOptionsTests.cs

+        /// <summary>
+        /// Tests that running "dab configure --runtime.compression.level {value}" on a config with various values results
+        /// in runtime config update. Takes in updated value for compression.level and
+        /// validates whether the runtime config reflects those updated values


Missing closing period in the documentation comment. The summary should end with a period for consistency with other test documentation in the file.

Suggested change

/// validates whether the runtime config reflects those updated values

/// validates whether the runtime config reflects those updated values.

Added the missing period to the documentation comment. Fixed in commit c53261c.

Copilot · 2025-12-04T03:10:02Z

src/Service/Startup.cs

+        private void ConfigureResponseCompression(IServiceCollection services, RuntimeConfig? runtimeConfig)
+        {
+            CompressionLevel compressionLevel = runtimeConfig?.Runtime?.Compression?.Level ?? CompressionOptions.DEFAULT_LEVEL;
+
+            // Only configure compression if level is not None
+            if (compressionLevel == CompressionLevel.None)
+            {
+                return;
+            }
+
+            System.IO.Compression.CompressionLevel systemCompressionLevel = compressionLevel switch
+            {
+                CompressionLevel.Fastest => System.IO.Compression.CompressionLevel.Fastest,
+                CompressionLevel.Optimal => System.IO.Compression.CompressionLevel.Optimal,
+                _ => System.IO.Compression.CompressionLevel.Optimal
+            };
+
+            services.AddResponseCompression(options =>
+            {
+                options.EnableForHttps = true;
+                options.Providers.Add<Microsoft.AspNetCore.ResponseCompression.GzipCompressionProvider>();
+                options.Providers.Add<Microsoft.AspNetCore.ResponseCompression.BrotliCompressionProvider>();
+            });
+
+            services.Configure<Microsoft.AspNetCore.ResponseCompression.GzipCompressionProviderOptions>(options =>
+            {
+                options.Level = systemCompressionLevel;
+            });
+
+            services.Configure<Microsoft.AspNetCore.ResponseCompression.BrotliCompressionProviderOptions>(options =>
+            {
+                options.Level = systemCompressionLevel;
+            });
+
+            _logger.LogInformation("Response compression enabled with level '{compressionLevel}' for REST, GraphQL, and MCP endpoints.", compressionLevel);
+        }


The response compression middleware lacks integration test coverage. While unit tests verify the CLI configuration, there are no tests validating that:

The compression middleware is actually registered when configured

Responses are compressed when clients send Accept-Encoding headers

Compression is disabled when the level is set to None

Different compression levels (Optimal vs Fastest) behave as expected

Consider adding integration tests similar to CorrelationIdTests.cs that verify the middleware behavior with actual HTTP requests and responses.

Added 7 integration tests in CompressionIntegrationTests.cs that verify:

Gzip compression works when Accept-Encoding header is present

Brotli compression works

Compression reduces payload size significantly (10%+ reduction)

Compression is disabled when level is None

No compression without Accept-Encoding header

Fastest compression level works correctly

Compressed content can be decompressed and is valid JSON

All tests pass. Fixed in commit c53261c.

anushakolan · 2025-12-30T19:45:37Z

Can we add an integration test to verify if this compression is actually reducing the payload sizes and it is not breaking the existing code?

anushakolan

LGTM, but waiting on the integration test to be added.

Aniruddh25 · 2026-01-13T23:54:31Z

src/Cli/ConfigGenerator.cs

+            }
+            catch (Exception ex)
+            {
+                _logger.LogError("Failed to update RuntimeConfig.Compression with exception message: {exceptionMessage}.", ex.Message);


Suggested change

_logger.LogError("Failed to update RuntimeConfig.Compression with exception message: {exceptionMessage}.", ex.Message);

_logger.LogError("Failed to configure RuntimeConfig.Compression with exception message: {exceptionMessage}.", ex.Message);

Updated error message to use "configure" instead of "update" for consistency. Fixed in commit c53261c.

anushakolan · 2026-01-16T06:00:10Z

@copilot Please address this, Can we add an integration test to verify if this compression is actually reducing the payload sizes and it is not breaking the existing code?

…ing, and add compression integration tests Co-authored-by: anushakolan <45540936+anushakolan@users.noreply.github.com>

Copilot · 2026-01-16T06:11:51Z

@copilot Please address this, Can we add an integration test to verify if this compression is actually reducing the payload sizes and it is not breaking the existing code?

Added 7 integration tests that verify compression reduces payload sizes and doesn't break existing functionality. The tests validate:

Compression works with both Gzip and Brotli
Payload size reduction is measured and verified (compression achieves 10%+ reduction)
Compression can be disabled (level=None)
Content can be decompressed correctly
Different compression levels work as expected

All tests pass successfully. The integration test that measures payload reduction shows actual compression ratios in the console output. Fixed in commit c53261c.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

http://crl3.digicert.com:443/DigiCertTrustedG4RSA4096SHA256TimeStampingCA.crl
- Triggering command: /usr/bin/dotnet dotnet build Azure.DataApiBuilder.sln --verbosity minimal (http block)
http://crl3.digicert.com:443/sha2-assured-cs-g1.crl
- Triggering command: /usr/bin/dotnet dotnet build Azure.DataApiBuilder.sln --verbosity minimal (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Co-authored-by: anushakolan <45540936+anushakolan@users.noreply.github.com>

Initial plan

5aa5ce7

Copilot AI assigned Copilot and JerryNixon Dec 3, 2025

Copilot started work on behalf of JerryNixon December 3, 2025 21:06 View session

Copilot AI and others added 2 commits December 3, 2025 21:33

Add response compression support with configuration, CLI, and tests

5e5a865

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add validation for invalid compression levels

1a8cd13

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add support for response compression in Data API~~ Add HTTP Response Compression Support Dec 3, 2025

Copilot finished work on behalf of JerryNixon December 3, 2025 21:45

Copilot AI requested a review from JerryNixon December 3, 2025 21:45

JerryNixon marked this pull request as ready for review December 4, 2025 03:05

Copilot AI review requested due to automatic review settings December 4, 2025 03:05

JerryNixon requested review from Alekhya-Polavarapu, Aniruddh25, RubenCerna2079, aaronburtle, anushakolan, neeraj-sharma2592, rusamant, sourabh1007, souvikghosh04 and vadeveka as code owners December 4, 2025 03:05

JerryNixon enabled auto-merge (squash) December 4, 2025 03:05

JerryNixon added this to Data API builder Dec 4, 2025

github-project-automation bot moved this to Todo in Data API builder Dec 4, 2025

JerryNixon moved this from Todo to Review In Progress in Data API builder Dec 4, 2025

JerryNixon added this to the Dec 2025 milestone Dec 4, 2025

Copilot started reviewing on behalf of JerryNixon December 4, 2025 03:06 View session

JerryNixon added hosting azure-host ⛅ labels Dec 4, 2025

JerryNixon approved these changes Dec 4, 2025

View reviewed changes

Copilot finished reviewing on behalf of JerryNixon December 4, 2025 03:09

Copilot AI reviewed Dec 4, 2025

View reviewed changes

aaronburtle self-assigned this Dec 15, 2025

Aniruddh25 assigned anushakolan and unassigned aaronburtle Dec 29, 2025

anushakolan reviewed Dec 30, 2025

View reviewed changes

Aniruddh25 reviewed Jan 13, 2026

View reviewed changes

Copilot started work on behalf of anushakolan January 16, 2026 06:00 View session

Address PR feedback: add unknown property skip, fix docs, update logg…

c53261c

…ing, and add compression integration tests Co-authored-by: anushakolan <45540936+anushakolan@users.noreply.github.com>

Copilot AI and others added 2 commits January 16, 2026 06:14

Code cleanup: remove redundant parameters and use async CopyTo

6326ead

Co-authored-by: anushakolan <45540936+anushakolan@users.noreply.github.com>

Fix: make DecompressGzip async to avoid blocking on async operations

a832d08

Co-authored-by: anushakolan <45540936+anushakolan@users.noreply.github.com>

Copilot finished work on behalf of anushakolan January 16, 2026 06:19

Copilot AI requested a review from anushakolan January 16, 2026 06:19

-                }
+                    else
+                    {
+                        // Skip unknown properties and their values (including objects/arrays)
+                        reader.Skip();
+                    }

	/// validates whether the runtime config reflects those updated values
	/// validates whether the runtime config reflects those updated values.

	_logger.LogError("Failed to update RuntimeConfig.Compression with exception message: {exceptionMessage}.", ex.Message);
	_logger.LogError("Failed to configure RuntimeConfig.Compression with exception message: {exceptionMessage}.", ex.Message);

Add HTTP Response Compression Support #3003

Are you sure you want to change the base?

Add HTTP Response Compression Support #3003

Conversation

Copilot AI commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why make this change?

What is this change?

How was this tested?

Sample Request(s)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

anushakolan commented Dec 30, 2025

Uh oh!

anushakolan left a comment

Choose a reason for hiding this comment

Uh oh!

Aniruddh25 Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

anushakolan commented Jan 16, 2026

Uh oh!

Copilot AI commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Copilot AI commented Dec 3, 2025 •

edited

Loading

Copilot AI commented Jan 16, 2026 •

edited

Loading