Merge the 'chardata' feature-branch#7098
Draft
pp-mo wants to merge 2 commits into
Draft
Conversation
* Initial tests. * Get 'create_cf_data_variable' to call 'create_generic_cf_array_var': Mostly working? Get 'create_cf_data_variable' to call 'create_generic_cf_array_var': Mostly working? * Reinstate decode on load, now in-Iris coded. * Revert and amend. * Hack to preserve the existing order of attributes on saved Coords and Cubes. * Fix for dataless; avoid FUTURE global state change from temporary tests. * Further fix to attribute ordering. * Fixes for data packing. * Latest test-chararrays. * Fix search+replace error. * Tiny fix in crucial place! (merge error?). * Extra mock property prevents weird test crashes. * Fix another mock problem. * Initial dataset wrappers. Rename; addin parts of old investigation; add temporary notes. * Various notes, choices + changes: Beginnings of encoded-dataset testing. * Replace use of encoding functions with test-specific function: Test for overlength writes. * Radically simplify 'make_bytesarray', by using a known specified bytewidth. * Add read tests. * Remove iris width control (not in this layer). * more notes * Remove temporary test code. * Use iris categorised warnings for unknown encodings. * Clarify the temporary load/save exercising tests (a bit). * Use bytecoded_datasets in nc load+save, begin fixes. * Further attempt to satisfy warning cateogry checker. * Fix overlength error tests. * Get temporary iris load/save exercises working (todo: proper tests). * Put encoding information into separate converter class, for use in proxies. * First proper testing (reads working). * Encoded reading ~working; new ideas for switching (untested). * Check loads when coords do/not share a string dim with data. * Fix nondecoded reference loads in test_byecoded_datasets. * Test writing of string data: various encodings, from strings or bytes. * Fix write proxy; tmp_path in stringdata tests; tidy stringdata tests. * Fix for non-string data. * Pre-clear load problems. * Fix mock patches. * Fix patches in test_CFReader. * Fix variable creation in odd cases. * Ignore attribute reordering in scaling-packed saves. * Fix test for refactored proxy constructor. * Fix get_cf_var_data to support vlen-string. * Add back new test results, folder removed in error. * Fix string-type check in cf to suit any of the new dtypes. * Remove non-working no-unit for label variables. * Separate asserts for ruff PT018. * Make encoding controls public API. * Fix old label-loading tests for new chardata handling. * Review changes, stylistic only. * Fix test for new dataset type. * Remove obsolete not-really-a-test. * Odd pre-commit fixes, and autoupdate. * Replace all use of 'CFLabelVariable.cf_label_data' with standard '_get_cf_var_data' function. * Fix problem with lazy string writing; add tests for lazy writes. * Test fixes. * Make context handler error-safe. * Define the supported encodings, and test them all. * Fix utf-16 nchars/nbytes relation. * Rationalise error handling + improve messages. * Fix type constructions of EncodedDataset.groups() and similar. * Removed obsolete reminder notes. * Allow test_bytecoding_datasets to import netCDF4. * Update lib/iris/fileformats/netcdf/saver.py Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk> * Correct type wrapping existing datasets in netcdf.saver. * Update lib/iris/fileformats/netcdf/saver.py Update comment from review Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk> --------- Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
![scitools-ci[bot]](https://avatars.githubusercontent.com/in/212324?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Templating
This PR includes changes that may be worth sharing via templating. For each file listed below, please either:
- Action the suggestion via a pull request editing/adding the relevant file in the SciTools/.github
templates/directory. 1 - Raise an issue against the SciTools/.github repo for the above action if you really don't have 10mins spare right now. Include an assignee, to avoid it being forgotten.
- Dismiss the suggestion if the changes are not suitable for templating.
You will need to dismiss this review before this PR can be merged. Recommend the reviewer does this as their final action before merging, as this text will continually update as commits come in.
Templated files
The following changed files are templated:
Footnotes
-
Include this text in the PR body to avoid any notifications about applying the template changes back to the source repo!
@scitools-templating: please no update notification on: iris↩
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7098 +/- ##
==========================================
+ Coverage 90.13% 90.20% +0.07%
==========================================
Files 91 92 +1
Lines 24957 25137 +180
Branches 4684 4698 +14
==========================================
+ Hits 22496 22676 +180
- Misses 1683 1687 +4
+ Partials 778 774 -4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Just as a PoC
-- don't merge !
(and when we finally do, it should be a merge commit not a squash)