Completes LETKF task, adds WCDA CI test #1287

AndrewEichmann-NOAA · 2024-09-19T19:08:02Z

Completes LETKF task, adds WCDA CI test

Mutually dependent on NOAA-EMC/global-workflow#2944

guillaumevernieres

👍

test/gw-ci/CMakeLists.txt

CoryMartin-NOAA · 2024-09-30T20:26:56Z

parm/soca/letkf/letkf.yaml.j2

-      basename: ./ens/mem%mem%
-      state variables: [socn, tocn, ssh, uocn, vocn, cicen]
+      basename: ./ens/mem%mem%/
+      state variables: [socn, tocn, ssh, hocn,  uocn, vocn, cicen]


Suggested change

state variables: [socn, tocn, ssh, hocn, uocn, vocn, cicen]

state variables: [socn, tocn, ssh, hocn, uocn, vocn, cicen]

CoryMartin-NOAA · 2024-09-30T20:28:28Z

parm/soca/letkf/letkf.yaml.j2

@@ -38,32 +41,31 @@ local ensemble DA:
    mult: 1.1

 output:
-  datadir: data_output/
+  datadir: Data/


Wouldn't a more descriptive output path be useful than Data/ for all of these?

As I recall this was to match existing code but I don't see a reason why it can't be changed

Data sounds good to me, but up to you @AndrewEichmann-NOAA .

CoryMartin-NOAA · 2024-09-30T20:30:18Z

parm/soca/letkf/letkf_save.yaml.j2

+   {% set COMOUT_OCEAN_LETKF_MEM = COM_OCEAN_LETKF_TMPL | replace_tmpl(tmpl_dict) %}
+   {% set COMOUT_ICE_LETKF_MEM = COM_ICE_LETKF_TMPL | replace_tmpl(tmpl_dict) %}
+
+   - ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.letkf.nc"]


Suggested change

- ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.letkf.nc"]

- ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.analysis.nc"]

seems strange to call this file gdas.ocean.t00z.letkf.nc, we don't say gdas.ocean.t00z.var.nc do we?

CoryMartin-NOAA · 2024-09-30T20:30:30Z

parm/soca/letkf/letkf_save.yaml.j2

+   {% set COMOUT_ICE_LETKF_MEM = COM_ICE_LETKF_TMPL | replace_tmpl(tmpl_dict) %}
+
+   - ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.letkf.nc"]
+   - ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.letkf.nc"]


Suggested change

- ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.letkf.nc"]

- ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.analysis.nc"]

CoryMartin-NOAA · 2024-09-30T20:31:46Z

parm/soca/letkf/letkf_save.yaml.j2

+
+   - ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.letkf.nc"]
+   - ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.letkf.nc"]
+{% endfor %}


CoryMartin-NOAA · 2024-09-30T20:34:41Z

parm/soca/obs/config/sst_viirs_npp_l3u.yaml

@@ -11,6 +11,9 @@ obs space:
  simulated variables: [seaSurfaceTemperature]
  io pool:
    max pool size: 1
+  distribution:


Same comment x n

CoryMartin-NOAA · 2024-09-30T20:35:38Z

parm/soca/soca_fix_stage_025.yaml.j2

@@ -1,4 +1,3 @@
-# TODO(AFE): make resolution dependent


Why are these not made resolution dependent? Seems like a lot of redundancy here that can be simplified with some Jinja

This got broken out into different yamls - @guillaumevernieres is there a particular approach you're committed to?

Yes, the one that we have now. We might decide to combine these at some point, but it's pretty far down the priority list.

CoryMartin-NOAA · 2024-09-30T20:35:53Z

test/gw-ci/CMakeLists.txt

@@ -17,12 +17,40 @@ function(add_cycling_tests pslot YAML_PATH HOMEgfs RUNTESTS PROJECT_SOURCE_DIR T
  list(GET DATES_LIST 0 HALF_CYCLE)
  list(GET DATES_LIST 1 FULL_CYCLE)

+  # stage ics
+  message(STATUS "preparing gdasstage_ic  for ${pslot} ctest")


Suggested change

message(STATUS "preparing gdasstage_ic for ${pslot} ctest")

message(STATUS "preparing gdasstage_ic for ${pslot} ctest")

CoryMartin-NOAA · 2024-09-30T20:36:10Z

test/gw-ci/CMakeLists.txt

+
+  # stage ensemble ics
+  if (letkf)
+    message(STATUS "preparing enkfgdasstage_ic  for ${pslot} ctest")


Suggested change

message(STATUS "preparing enkfgdasstage_ic for ${pslot} ctest")

message(STATUS "preparing enkfgdasstage_ic for ${pslot} ctest")

CoryMartin-NOAA · 2024-09-30T20:36:57Z

test/gw-ci/CMakeLists.txt

+
+  # 1/2 cycle enkfgdasfcst
+  if (letkf)
+  set(ENS_MEMS  "mem001" "mem002" "mem003")


indent this if block to be consistent with the others

AndrewEichmann-NOAA added 10 commits August 21, 2024 18:11

debugging for ens WCDA

8032f4b

Merge branch 'develop' into feature/ens-WCDA-ctest

9e283e2

model_data 2 model

3156f05

Merge branch 'develop' into feature/ens-WCDA-ctest

818984d

adding ensemble WCDA run to ctests

bca25b5

Merge branch 'develop' into feature/ens-WCDA-ctest

4b1932a

adding letkf ctest, making ita llwork

129d601

Merge branch 'develop' into feature/ens-WCDA-ctest

cfcfd17

tweaks

533d8ea

wrapping up

fe9462e

AndrewEichmann-NOAA requested review from guillaumevernieres, CoryMartin-NOAA, apchoiCMD and givelberg September 19, 2024 19:08

AndrewEichmann-NOAA added 2 commits September 19, 2024 19:24

restore adt yaml

f768e79

retry with adts

aa60925

AndrewEichmann-NOAA mentioned this pull request Sep 19, 2024

Add run and finalize methods to marine LETKF task NOAA-EMC/global-workflow#2944

Open

16 tasks

guillaumevernieres reviewed Sep 19, 2024

View reviewed changes

test/gw-ci/CMakeLists.txt Outdated Show resolved Hide resolved

AndrewEichmann-NOAA mentioned this pull request Sep 20, 2024

Add enkfgdas.20210324 to ICSDIR/C48mx500 NOAA-EMC/global-workflow#2945

Closed

4 tasks

AndrewEichmann-NOAA added 3 commits September 20, 2024 14:53

the great deawkwarding

45c76c9

re-adding test

dc23cfd

Merge branch 'develop' into feature/ens-WCDA-ctest

7e54d9c

CoryMartin-NOAA reviewed Sep 30, 2024

View reviewed changes

AndrewEichmann-NOAA added 2 commits October 3, 2024 15:51

merge debugging

5394e24

Merge branch 'develop' into feature/ens-WCDA-ctest

ef98951

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Completes LETKF task, adds WCDA CI test #1287

Completes LETKF task, adds WCDA CI test #1287

AndrewEichmann-NOAA commented Sep 19, 2024

guillaumevernieres left a comment

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

AndrewEichmann-NOAA Oct 1, 2024

guillaumevernieres Oct 1, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

AndrewEichmann-NOAA Oct 1, 2024

guillaumevernieres Oct 1, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

CoryMartin-NOAA Sep 30, 2024

	state variables: [socn, tocn, ssh, hocn, uocn, vocn, cicen]
	state variables: [socn, tocn, ssh, hocn, uocn, vocn, cicen]

	- ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.letkf.nc"]
	- ["{{ DATA }}/Data/ocn.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_OCEAN_LETKF_MEM }}/{{ RUN }}.ocean.t{{ cyc }}z.analysis.nc"]

	- ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.letkf.nc"]
	- ["{{ DATA }}/Data/ice.letkf.ens.{{ mem }}.{{ timestr }}.PT3H.nc", "{{ COMOUT_ICE_LETKF_MEM }}/{{ RUN }}.ice.t{{ cyc }}z.analysis.nc"]

	message(STATUS "preparing gdasstage_ic for ${pslot} ctest")
	message(STATUS "preparing gdasstage_ic for ${pslot} ctest")

	message(STATUS "preparing enkfgdasstage_ic for ${pslot} ctest")
	message(STATUS "preparing enkfgdasstage_ic for ${pslot} ctest")

Completes LETKF task, adds WCDA CI test #1287

Are you sure you want to change the base?

Completes LETKF task, adds WCDA CI test #1287

Conversation

AndrewEichmann-NOAA commented Sep 19, 2024

guillaumevernieres left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment