Loading in leapseconds kernel in Metakernel API by bryan-harter · Pull Request #1172 · IMAP-Science-Operations-Center/sds-data-manager

bryan-harter · 2026-03-11T18:18:36Z

Change Summary

Overview

This is a simple fix to load in the leapsecond kernel in the metakernel API before converting times between datetime and J2000. It only looks like a lot because I moved some of the functions around. The SPICE indexer lambda code already had a function to do what I wanted in the metakernel, so I broke off that code into a spice_utilities.py file. Now both the metakernel API and the SPICE indexer will just import that function from spice_utilities.py.

File changes

sds_data_manager/constructs/sds_api_manager_construct.py

Added two environment variables that spice_metakernel_api needs to successfully download/furnish the latest leapseconds kernel like the spice_indexer does.
Reduced timeout to 1 minute, down from 5
Increased RAM to 5000. This is a little unrelated to this PR, but I figured this would be temporarily helpful while I further optimize the function. Increasing the RAM will increase the number of CPUs available as well, so the whole thing should run faster before the API times out.

sds_data_manager/lambda_code/SDSCode/api_lambdas/spice_metakernel_api.py

Added a few lines to furnish the leapsecond kernel before converting datetime to J2000

sds_data_manager/lambda_code/SDSCode/pipeline_lambdas/spice_indexer.py

Took out two functions to place in spice_utilities.py

sds_data_manager/lambda_code/SDSCode/spice_utilities.py

A new file that takes two functions out of spice_indexer.py

Testing

I un-skipped the unit test that already checked for this, and I put the leapsecond file in the mock S3 bucket.

…nds-kernel

Copilot

Pull request overview

This PR aims to ensure SPICE leapseconds (LSK) are loaded before converting YYYYMMDD-style inputs to J2000 in the metakernel API, and factors shared SPICE download/furnish logic out of spice_indexer.py into a reusable utility module.

Changes:

Added leapseconds-kernel furnishing before datetime2et conversion in spice_metakernel_api.
Extracted shared S3 download + “furnish latest kernel” logic into a new spice_utilities.py module and updated spice_indexer to use it.
Updated the metakernel API Lambda config to include S3/data-dir environment variables and S3 read permissions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
sds_data_manager/lambda_code/SDSCode/spice_utilities.py	New shared utilities for downloading kernels from S3 and furnishing the “latest” kernel.
sds_data_manager/lambda_code/SDSCode/pipeline_lambdas/spice_indexer.py	Removes inlined helpers and imports the shared `furnish_best_spice_file`.
sds_data_manager/lambda_code/SDSCode/api_lambdas/spice_metakernel_api.py	Attempts to load leapseconds kernel before converting YYYYMMDD to J2000.
sds_data_manager/constructs/sds_api_manager_construct.py	Adds env vars + S3 permissions for metakernel API Lambda; adjusts timeout/memory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…nds-kernel

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tech3371

looks good besides one minor request.

tech3371 · 2026-03-12T16:11:46Z

+            logger.info(
+                "Attempt failed, continuing in hopes that it is already loaded."
+            )


This message confused me little bit. can we rephrase this? or add more context of why we can continue?

I got rid of it since I'm checking if an LSK is loaded now. If nothing is loaded, it should just throw an error!

tech3371 · 2026-03-12T17:39:32Z

+        logger.error()
+        raise FileNotFoundError(
+            f"Failed to download {s3_key} from bucket {bucket_name}: {e}"
+        ) from e


nice refactor and clean up of error raise!

tech3371 · 2026-03-12T17:40:23Z

+    logger.info(f"Downloaded SPICE file: {highest_version_spice_file}")
+    # Furnish the SPICE file
+    spiceypy.furnsh(str(highest_version_spice_file))
+    return highest_version_spice_file


tmplummer

Just two suggestions to consider.

tmplummer · 2026-03-12T20:32:27Z

+logger.setLevel(logging.INFO)
+
+
+def download_from_s3(s3_key: str, bucket_name: Optional[str] = None) -> Path:


Is there a reason that this is not using the imap-data-access library to download files?

I think it would simplify the code a little bit, I just didn't know if it was worth invoking the API and whatnot from inside of our own SDC. I do see now that some of the ialirt lambdas use the .download() function from there though. I'm fine with it either way!

tmplummer · 2026-03-12T20:38:04Z

+            )
+            furnish_best_spice_file("leapseconds")
+        except FileNotFoundError:
+            logger.info(


You could think about using spiceypy tools to check to see if a leapsecond kernel is furnished here. I suppose it doesn't hurt to furnish it more than once, but the flow could be:

check if lsk kernel is furnished

if not furnished try to furnish one

except: raise error

I forgot you could check what has already been loaded, I switched it to use that logic!

…nds-kernel

Loading in leapseconds before converting

8a41420

bryan-harter requested review from Copilot and tech3371 March 11, 2026 18:18

bryan-harter linked an issue Mar 11, 2026 that may be closed by this pull request

BUG: metakernel API error out on leapseconds kernel #1167

Closed

Copilot started reviewing on behalf of bryan-harter March 11, 2026 18:19 View session

bryan-harter added 2 commits March 11, 2026 18:20

Fixing some formatting

171739f

Merge branch 'dev' into 1167-bug-metakernel-api-error-out-on-leapseco…

66ca2dd

…nds-kernel

Copilot AI reviewed Mar 11, 2026

View reviewed changes

bryan-harter added 2 commits March 11, 2026 19:59

Fixing the unit tests

2514c63

Merge branch 'dev' into 1167-bug-metakernel-api-error-out-on-leapseco…

f67f60c

…nds-kernel

bryan-harter requested a review from Copilot March 11, 2026 20:08

bryan-harter changed the title ~~Loading in leapseconds before converting~~ Loading in leapseconds kernel in Metakernel API Mar 11, 2026

Copilot started reviewing on behalf of bryan-harter March 11, 2026 20:08 View session

Copilot AI reviewed Mar 11, 2026

View reviewed changes

Comment thread tests/lambda_endpoints/test_spice_metakernel_api.py

tech3371 requested a review from tmplummer March 12, 2026 17:35

tech3371 approved these changes Mar 12, 2026

View reviewed changes

tmplummer reviewed Mar 12, 2026

View reviewed changes

bryan-harter added 4 commits March 12, 2026 22:52

Using spiceypy to determine if leapseconds are loaded

98ec1f7

Merge branch 'dev' into 1167-bug-metakernel-api-error-out-on-leapseco…

6ae084c

…nds-kernel

fixing the ktotal call

a9bf918

Making fixes to the linting

a36c537

bryan-harter merged commit 7f812a9 into dev Mar 13, 2026
2 checks passed

bryan-harter deleted the 1167-bug-metakernel-api-error-out-on-leapseconds-kernel branch March 13, 2026 17:20

		logger.setLevel(logging.INFO)


		def download_from_s3(s3_key: str, bucket_name: Optional[str] = None) -> Path:

Conversation

bryan-harter commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Summary

Overview

File changes

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

tech3371 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmplummer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bryan-harter commented Mar 11, 2026 •

edited

Loading