Support UK radar data by BernardClaxton · Pull Request #1825 · MetOffice/CSET

BernardClaxton (BernardClaxton) · 2025-11-27T11:55:59Z

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

Documentation has been updated to reflect change.
New code has tests, and affected old tests have been updated.
All tests and CI checks pass.
Ensured the pull request title is descriptive.
Conda lock files have been updated if dependencies have changed.
Attributed any Generative AI, such as GitHub Copilot, used in this PR.
Marked the PR as ready to review.

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-01-20T16:31:03Z

for info if the upgrade of you branch with main causes problems then I have created a branch of your branch and merged main in now including the addition of plotting transverse mercator. So you should be able to plot the radar data.
Branch: #1759_import_nimrod_merge_main

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-01-20T18:22:28Z

#https://github.com/MetOffice/CSET/tree/1759_import_nimrod_merge_main adds nimrod radar switch into rose gui and enables nimrod switch in flow.cylc for the workflow.

James Frost (jfrost-mo)

src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py is the main file for this change, as the stuff in fetch_nimrod is largely unused.

What is our plan with this? Are we going to try and reuse the RES capability in cube_utils? Or do we not really need it given the existing capabilities of CSET?

The recipe and such looks broadly sensible, though I think we can get away with hard-coding some more for the initial simple recipe.

github-actions · 2026-01-29T10:53:16Z

#1759

The radar configuration was moved to the Observation rose meta directory.

Using a positional argument was only added in python 3.13.

This matches the new location of the other observation settings.

James Frost (jfrost-mo)

Looking sensible. I've left quite a few comments, but most of them are mechanical cleanup; I'll apply some myself to get the comment count down.

It would be good to see some tests for fetch_nimrod; we can work on them together to validate it is doing the correct things.

We also need to have a think about where fetch_nimrod will live. Its is a reasonable size piece of functionality, which makes me want to have it in the public repository, both for better integration into tests, as well as to allow it to serve as an example for others adding their own radar. On the other hand, it contains a number of Met Office specific tasks, and more broadly I think NIMROD is a Met Office specific format, which will limit the ability for it to be run elsewhere. My suggestion is to factor out the paths and such (which are mostly in the nimrod_dict already) into a separate data file and have that in the restricted repository, but leave the bulk of the code here.

James Frost (jfrost-mo) · 2026-04-20T09:31:01Z

+values="rainaccum_comp_hour", ""
+value-titles=rainaccum_comp_hour, off


Should this be a boolean toggle? Or are there cases when we will have multiple options?

James Frost (jfrost-mo) · 2026-04-20T12:27:48Z

+    * NIMROD_FIELDs
+    #* ANALYSIS_OFFSET
+    #* ANALYSIS_LENGTH
+    #* CYLC_TASK_CYCLE_POINT
+    * DATA_PATH
+    #* DATA_PERIOD
+    #* DATE_TYPE
+    #* MODEL_IDENTIFIER
+    * ROSE_DATAC


Check this list is up-to-date.

James Frost (jfrost-mo) · 2026-04-20T12:34:48Z

+                    # Write the Nimrod weights to a NetCDF file.
+                    filename_wei_nc = (
+                        nimrod_dir_wei
+                        + "/"
+                        + f"{date_use.strftime('%Y%m%d%H%M')}"
+                        + "_"
+                        + nimrod_dict[nimrod_field]["obs_id"]
+                        + "_weights"
+                    )
+                    filename_wei_nc = Path(filename_wei_nc).with_suffix(".nc")
+                    iris.save(nimrod_cube_weights, filename_wei_nc)


Do we need to save the weights if we have already applied them to the data?

James Frost (jfrost-mo) · 2026-04-20T12:43:11Z

+    return variables
+
+
+def _nimrod_dictionary() -> dict:


Should this read from a data file that can be put into the restricted repository as a site-specific file?

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

This is mostly to satisfy my type checker that nimrod_dir_wei is set when used.

The number of radar sites can change, preventing the cubes from merging. Therefore the respective cube attributes are stripped.

James Frost (jfrost-mo) · 2026-04-20T16:06:46Z

Be warned that plotting spatial plots for 5 minutely data takes ages.

James Frost (jfrost-mo) · 2026-04-21T07:53:50Z

Rainfall rate Composite will also want a colourbar, so the 5 minutely data displays sensibly.

My run: https://wwwspice/~james.frost/CSET-radar-test/

BernardClaxton (BernardClaxton) · 2026-04-24T14:19:44Z

Nimrod 5 minute rainfall rates now plots using the same logarithmic colourbar as the other rainfall plots.

BernardClaxton (BernardClaxton) added 15 commits October 8, 2025 11:58

add Nimrod radar page to CSET GUI #1759

09b28e4

put nimrod functions in fetch_data.py to grab radar files #1759

af67bf9

added job in cylc workflow to grab nimrod obs #1759

b7eea4a

defined fetchnimrod task as fetch-nimrod.py #1759

0369efb

put in script fetch-nimrod.py #1759

98fb38b

edits to fetch_data.py #1759

450ef6c

edits to fetch-nimrod.py #1759

d6aaf0a

edits to flow-cylc #1759

97c72ad

edits to meta/rose-meta.conf #1759

68b99a7

fetch-nimrod.py update #1759

518c017

constants.py added #1759

1ce6494

added nimrod colorbar #1759

065afab

added rose-meta colorbar #1759

9fe96cd

adding cube_utils

7e146ed

recipe for nimrod #1759

6963f5d

BernardClaxton (BernardClaxton) self-assigned this Nov 27, 2025

BernardClaxton (BernardClaxton) added the enhancement New feature or request label Nov 27, 2025

BernardClaxton (BernardClaxton) requested a review from James Frost (jfrost-mo) November 27, 2025 11:56

BernardClaxton (BernardClaxton) added 3 commits January 20, 2026 14:43

debugging code added to fetch-nimrod.py at L715 #1759

8d73e22

debugging code added to fetch-nimrod.py at L715 #1759

b76224e

Merge remote-tracking branch 'origin/main' into 1759_import_nimrod

671f904

James Frost (jfrost-mo) changed the title ~~supporting UK radar data~~ Support UK radar data Jan 23, 2026

James Frost (jfrost-mo) reviewed Jan 23, 2026

View reviewed changes

Nimrod fetching code moved into fetch_nimrod.py #1759

a63cefc

tidied the Nimrod radio buttons in GUI #1759

a84e98f

Stephen Gallagher (SGallagherMet) added the R2O Trials Items of significance to the Met Office Research to Operations Team. label Feb 16, 2026

rewrote fetch_nimrod.py to output Nimrod data as NetCDF #1759

8102db4

BernardClaxton (BernardClaxton) added 2 commits April 10, 2026 13:27

adding the radar loader file ..../loaders/radar.py #1759

014eb23

tidied the labelling of the colour bar for model rainfall spatial plots

df62a34

#1759

James Frost (jfrost-mo) marked this pull request as ready for review April 13, 2026 09:52

James Frost (jfrost-mo) added 5 commits April 17, 2026 16:30

Remove unused cube_utils.py file

3e3a6d7

Merge branch 'main' into 1759_import_nimrod

fe51529

The radar configuration was moved to the Observation rose meta directory.

Initialise namespace through unpacking to support python 3.12

f1b8eab

Using a positional argument was only added in python 3.13.

Update namespace of NIMROD radar obs

ae2ac6f

This matches the new location of the other observation settings.

Combine comprehensions

c504ae7

James Frost (jfrost-mo) requested changes Apr 20, 2026

View reviewed changes

James Frost (jfrost-mo) added 7 commits April 20, 2026 14:01

Remove commented out/useless/print code

7e4f255

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

Avoid unnecessary array creation

c760bea

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

Reformat string templates

826f416

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

Update terminology in comments

ee06dda

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

Remove prints

0733c77

Co-authored-by: James Frost <james.frost@metoffice.gov.uk>

Clean up radar recipe files

78ff5aa

Create directory for weights when needed

7e8590b

This is mostly to satisfy my type checker that nimrod_dir_wei is set when used.

James Frost (jfrost-mo) force-pushed the 1759_import_nimrod branch from 9bb07f8 to 7e8590b Compare April 20, 2026 13:48

Merge branch 'main' into 1759_import_nimrod

f671af7

James Frost (jfrost-mo) reviewed Apr 20, 2026

View reviewed changes

Comment thread src/CSET/cset_workflow/meta/observations/rose-meta.conf

James Frost (jfrost-mo) added 5 commits April 20, 2026 15:49

Add NIMROD normalisation callback

4c7b118

The number of radar sites can change, preventing the cubes from merging. Therefore the respective cube attributes are stripped.

Add configuration for 5 minute rainfall rate

ac824fb

Rearrange loader to use simpler structure

785ffc7

Add 5 minute rainfall rate to loader

7453503

Remove troublesome attributes from 5 minute rainrate cubes

0f960ff

BernardClaxton (BernardClaxton) added 3 commits April 23, 2026 15:58

changed Composite to composite in name of Nimrod_5min variable #1759

4e72b1a

Added entry in _colorbar_definition.json for Nimrod_5min data #1759

d87bcd2

Added log rainfall colorbar for Nimrod_5min 2-D field plots #1759

3c6a742

		values="rainaccum_comp_hour", ""
		value-titles=rainaccum_comp_hour, off

Conversation

BernardClaxton (BernardClaxton) commented Nov 27, 2025 • edited by jfrost-mo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution checklist

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jan 20, 2026

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

James Frost (jfrost-mo) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

James Frost (jfrost-mo) left a comment

Choose a reason for hiding this comment

Uh oh!

James Frost (jfrost-mo) Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

James Frost (jfrost-mo) Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

James Frost (jfrost-mo) Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

James Frost (jfrost-mo) Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

James Frost (jfrost-mo) commented Apr 20, 2026

Uh oh!

James Frost (jfrost-mo) commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BernardClaxton (BernardClaxton) commented Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

BernardClaxton (BernardClaxton) commented Nov 27, 2025 •

edited by jfrost-mo

Loading

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jan 20, 2026 •

edited

Loading

github-actions Bot commented Jan 29, 2026 •

edited

Loading

James Frost (jfrost-mo) commented Apr 21, 2026 •

edited

Loading