[uss_qualifier/astm/utm] Validate operational intent changes are always published to DSS by mickmis · Pull Request #1393 · interuss/monitoring

mickmis · 2026-03-11T16:31:35Z

Closes #1379

This adds:

a simple cache field (implemented as a dist) to instances of scenario
methods 'is_equivalent' to several entities in geo.py and geotemporal.py aimed at implementing comparison with some tolerances

…ys published to DSS

BenjaminPelletier

I suspect addressing these comments (particularly the cache key one) will resolve the CI issues.

BenjaminPelletier · 2026-03-11T21:06:33Z

monitoring/monitorlib/geotemporal.py

 )
 from monitoring.monitorlib.transformations import Transformation

+TIME_TOLERANCE = timedelta(seconds=1)


This is probably too lenient since we're just talking about numeric precision/representation errors

Suggested change

TIME_TOLERANCE = timedelta(seconds=1)

TIME_TOLERANCE = timedelta(milliseconds=10)

BenjaminPelletier · 2026-03-11T21:21:05Z

monitoring/uss_qualifier/scenarios/astm/utm/evaluation.py

+    def append_err(name: str):
+        errors_text.append(
+            f"{name} reported by USS does not match the one published to the DSS"


It seems like it isn't that reported doesn't match DSS, it's that the details corresponding to a particular single version of an op intent are now reported differently than they previously were. So, I would expect a message more like (changing old_oi and new_oi types to OperationalIntent):

Suggested change

def append_err(name: str):

errors_text.append(

f"{name} reported by USS does not match the one published to the DSS"

def append_err(field_name: str):

errors_text.append(

f"The value {get_attr(new_oi.details, field_name)} for `{field_name}` reported by USS for details of operational intent {new_oi.reference.id} at version {new_oi.reference.version} (OVN {new_oi.reference.ovn if 'ovn' in new_oi.reference else 'empty'}) does not match the value {get_attr(old_oi.details, field_name)} previously indicated by the USS for the same operational intent with the same version"

BenjaminPelletier · 2026-03-11T21:22:05Z

monitoring/uss_qualifier/scenarios/astm/utm/evaluation.py

    return "; ".join(errors_text) if len(errors_text) > 0 else None
+
+
+def validate_equivalent_op_intent_details(


nit: It seems like this would be clearer in communicating what this function will return:

Suggested change

def validate_equivalent_op_intent_details(

def errors_for_nonequivalent_op_intent_details(

BenjaminPelletier · 2026-03-11T21:27:19Z

monitoring/uss_qualifier/scenarios/astm/utm/test_steps.py

+            [self._flight_planner.participant_id],
+        ) as check:
+            cache_key = (
+                f"op_intent_details:{oi_full.reference.version}:{oi_full.reference.ovn}"


Seems like the ID is important here, but actually the OVN can't be because it's not ok to change the OVN for the same version. In fact, the OVN is one of the elements of the details that 1) can only be obtained from the USS and 2) shouldn't change. So, reference.ovn should actually be checked in validate_equivalent_op_intent_details.

The rest of the reference fields don't need to be checked there because a different check (#1378) ensures they match the DSS, and the DSS won't change those fields without synchronizing correctly.

Suggested change

f"op_intent_details:{oi_full.reference.version}:{oi_full.reference.ovn}"

f"op_intent_details:{oi_full.reference.id}:{oi_full.reference.version}"

BenjaminPelletier · 2026-03-11T21:36:12Z

monitoring/uss_qualifier/scenarios/astm/utm/test_steps.py

+            cache_key = (
+                f"op_intent_details:{oi_full.reference.version}:{oi_full.reference.ovn}"
+            )
+            old_details = OperationalIntentDetails(self._scenario.cache.get(cache_key))


I think we want to cache the full OperationalIntent (and probably query timestamp; see other comment).

I think the API unfortunately makes the language a little less clear than it could: in the text of the standard, the full information about the op intent is the "details" -- that is, "details" is inclusive of the information in the field reference. The API (somewhat confusingly) names the second field in OperationalIntent "details" rather than something more pedantically accurate like information_that_when_added_to_reference_forms_details. I've made a note to ASTM about perhaps making this more clear in the next version.

Caching the full OperationalIntent is the only way to directly detect if the USS has incorrectly changed ovn without informing the DSS since a client can't compare the ovn value reported by the DSS to the correct OVN in the DSS since the DSS won't provide the correct OVN to anyone other than the Entity's manager.

BenjaminPelletier · 2026-03-11T21:53:36Z

monitoring/uss_qualifier/scenarios/astm/utm/validate_shared_operational_intent.md

+If the operational intent details exposed by the USS have changed without the USS having updated the operational intent reference in the DSS, this check will fail per:
+- **[astm.f3548.v21.USS0105,1](../../../requirements/astm/f3548/v21.md)**  because the USS did not implement the operation _getOperationalIntentDetails_ correctly; and
+- **[astm.f3548.v21.USS0005](../../../requirements/astm/f3548/v21.md)** because the USS did not make the operational intent correctly discoverable by the DSS.
+
+Out of clarity, this check specifically targets cases where the USS changes details of the operational intent (volumes or priority) without a version or OVN change.


Suggested change

If the operational intent details exposed by the USS have changed without the USS having updated the operational intent reference in the DSS, this check will fail per:

- **[astm.f3548.v21.USS0105,1](../../../requirements/astm/f3548/v21.md)** because the USS did not implement the operation _getOperationalIntentDetails_ correctly; and

- **[astm.f3548.v21.USS0005](../../../requirements/astm/f3548/v21.md)** because the USS did not make the operational intent correctly discoverable by the DSS.

Out of clarity, this check specifically targets cases where the USS changes details of the operational intent (volumes or priority) without a version or OVN change.

If the operational intent details reported by the USS have changed without the USS having updated the operational intent reference in the DSS, this check will fail because the USS did not make the changes to the operational intent discoverable using the DSS, per **[astm.f3548.v21.USS0005](../../../requirements/astm/f3548/v21.md)**. Per **[astm.f3548.v21.USS0105,1](../../../requirements/astm/f3548/v21.md)** (definition of `version` field), a USS must update the operational intent reference in the DSS, even when no data fields of the reference have changed, to make changes to the operational intent discoverable using the DSS.

To be clear, this check specifically targets cases where the USS changes details of the operational intent without a version change.

BenjaminPelletier · 2026-03-11T21:59:48Z

monitoring/uss_qualifier/scenarios/astm/utm/test_steps.py

+                    check.record_failed(
+                        summary="Operational intent details have changed without the change being published to the DSS",
+                        details=error_text,
+                        query_timestamps=[oi_full_query.request.timestamp],


It would be very useful to include the timestamp of the old query in addition to the new one. Fundamentally, we're saying we asked the USS for the same thing multiple times and they gave us different answers, so it would be super useful to quickly point a report reader to the multiple times we asked. Ideally, the combination of the test report and scenario documentation should be sufficient for most readers to be able to clearly and quickly identify the problem themselves. If we didn't include the "before" timestamp, I would expect a reader to ask how we know the details changed (how do we know the finding isn't a false positive?). With a timestamp, the answer to that is straightforward: "Look at your response A and your response B; they're different."

BenjaminPelletier · 2026-03-11T22:05:14Z

monitoring/uss_qualifier/scenarios/astm/utm/evaluation.py

+    # this function assumes all fields required by the OpenAPI definition are present as the format validation
+    # should have been performed by OpIntentValidator._evaluate_op_intent_validation before


I think no need to make this note, and it reduces reusability of the function.

The function's type annotations indicate that it should be provided valid OperationalIntentDetails, and all required fields have necessarily been validated in order for the provided value to actually be a valid OperationalIntentDetails. But, this function doesn't care where that validation is performed; when the function is reused, the validation might not happen where indicated here.

[uss_qualifier/astm/utm] Validate operational intent changes are alwa…

ee477c2

…ys published to DSS

BenjaminPelletier reviewed Mar 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[uss_qualifier/astm/utm] Validate operational intent changes are always published to DSS#1393

[uss_qualifier/astm/utm] Validate operational intent changes are always published to DSS#1393
mickmis wants to merge 1 commit intointeruss:mainfrom
Orbitalize:1379/details-change-check

mickmis commented Mar 11, 2026

Uh oh!

BenjaminPelletier left a comment

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

BenjaminPelletier Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	TIME_TOLERANCE = timedelta(seconds=1)
	TIME_TOLERANCE = timedelta(milliseconds=10)

		return "; ".join(errors_text) if len(errors_text) > 0 else None


		def validate_equivalent_op_intent_details(

	def validate_equivalent_op_intent_details(
	def errors_for_nonequivalent_op_intent_details(

	f"op_intent_details:{oi_full.reference.version}:{oi_full.reference.ovn}"
	f"op_intent_details:{oi_full.reference.id}:{oi_full.reference.version}"

		# this function assumes all fields required by the OpenAPI definition are present as the format validation
		# should have been performed by OpIntentValidator._evaluate_op_intent_validation before

Conversation

mickmis commented Mar 11, 2026

Uh oh!

BenjaminPelletier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants