Skip to content

Include the length of time we have spent downloading DataStreams in the error message raised when we can't get the entire DataStream#707

Merged
LukeButters merged 3 commits intomainfrom
luke/include-time-spent-downloading-datastreams
Mar 27, 2026
Merged

Include the length of time we have spent downloading DataStreams in the error message raised when we can't get the entire DataStream#707
LukeButters merged 3 commits intomainfrom
luke/include-time-spent-downloading-datastreams

Conversation

@LukeButters
Copy link
Copy Markdown
Contributor

@LukeButters LukeButters commented Mar 26, 2026

Background

ref CLOUDPT-11160
ref EFT-3141

In the issue we are looking at, it would be ideal to understand how long the service has been receiving DataStreams for. Since the client has given up waiting for the service to download the data streams.

Results

After

---> Halibut.Transport.Protocol.ProtocolException: Data stream reading failed. Message Id: IReadDataStreamService::SendDataAsync[1] / 0441e6e9-8aa4-4b57-b1ce-0925340cb159, Stream ID: e517a67d-e8ad-4af7-be18-f9748110fbc0, Expected length: 100, Actual bytes read: 18. Total length of all DataStreams to be sent is 100. Stream with length 100 was closed after only reading 18 bytes. Time elapsed downloading all streams: 44999ms.

How to review this PR

Quality ✔️

Pre-requisites

  • I have read How we use GitHub Issues for help deciding when and where it's appropriate to make an issue.
  • I have considered informing or consulting the right people, according to the ownership map.
  • I have considered appropriate testing for my change.

@LukeButters LukeButters requested a review from a team as a code owner March 26, 2026 20:07
Copy link
Copy Markdown
Contributor

@rhysparry rhysparry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approving. I think there is an opportunity to make this clearer by separating log context information from the data required to actually perform the work.

I'll leave it to you whether you want to do that in a follow up PR, this PR or if you think it's completely unnecessary.

}

async Task ReadStreamAsync(string messageId, IReadOnlyList<DataStream> deserializedStreams, CancellationToken cancellationToken)
async Task ReadStreamAsync(string messageId, IReadOnlyList<DataStream> deserializedStreams, Stopwatch stopWatchForDownloadingAllStreams, CancellationToken cancellationToken)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're passing around a lot of extra context information for the purposes of logging/enriching the exception message.

  • Message Id
  • Stopwatch
  • Total size of data streams

It might help make this clearer if we wrapped all these in a small class. This would clearly separate data that is for logging and what is actually used in the logic. I would imagine it could also provide a ToString() implementation to simplify the addition of this context to all the various places.

@LukeButters LukeButters enabled auto-merge (squash) March 27, 2026 00:25
@LukeButters LukeButters merged commit 263338e into main Mar 27, 2026
17 checks passed
@LukeButters LukeButters deleted the luke/include-time-spent-downloading-datastreams branch March 27, 2026 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants