Skip to content

fix: support datetime variables in Dataset.interp#11081

Open
emmanuel-ferdman wants to merge 7 commits intopydata:mainfrom
emmanuel-ferdman:main
Open

fix: support datetime variables in Dataset.interp#11081
emmanuel-ferdman wants to merge 7 commits intopydata:mainfrom
emmanuel-ferdman:main

Conversation

@emmanuel-ferdman
Copy link

PR Summary

Dataset.interp() silently dropped datetime64 and timedelta64 variables. Now they are interpolated by converting to float64 and back, with NaT handled like NaN.

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
Copy link
Member

@spencerkclark spencerkclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @emmanuel-ferdman—this largely looks good.

Could you add "Mm" to the valid dtypes on the DataArray.interp side as well? There is no need now to raise in that circumstance, which is great.

One edge case to consider is what to do in the scenario that extrapolation leads to float values outside the range that can be represented by 64-bit integers. Maybe we punt on that for now though, since it could be messy to handle in a robust way.

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
@emmanuel-ferdman
Copy link
Author

@spencerkclark Thanks, updated :) Regarding the offset-based approach, I think we could subtract the minimum datetime before converting to float, which gives nanosecond precision - but it would break dask laziness since we'd need to compute .min().

Copy link
Member

@spencerkclark spencerkclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @emmanuel-ferdman! This looks good from my perspective.

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dataset.interp() silently dropping time-like data arrays

2 participants