-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DATA REQUEST] Add COSIMA Panantarctic / GFDL_OM4 Builder & Data #175
Comments
Following on from COSIMA/cosima-recipes#369 , I am suggesting maybe adding OM4_025.JRA_RYF to the intake catalog. @dougiesquire - As this is a different model configuration, I guess this would require a new datastore "builder", so maybe its not worth the effort? The runs are used in cosima recipes to show examples of handling MOM6 data. @adele-morrison - Are their companion runs to OM4_025.JRA_RYF which also should be added? Can you help with the "Description of the data product" and "Location of the data product on Gadi" sections, and then I will edit the original post please? |
I tried using the access-om3 builder, and got these errors when using builder.parser:
|
Ah, yet another permutation of file naming. It might be safest just to write a dedicated builder, which is straightforward. I guess it would be an Is this output structured in a similar way to the regional MOM6 output? If so, it may be worth thinking about writing a builder that handles both? |
Apologies for being slow. Yes, lets add the panan experiments to Intake. We'd still to like delete a bunch of the daily data for the 1/20th panan, is that ok to do after it's added to Intake? After that frees up space on ol01 ideally I'd also like to move the 1/10th panan from ik11 to ol01. But the current locations are as follows: |
I think if we know this is going to happen then it would be better to wait until it is done. We can get a Builder set up and ready to go though. |
@anton-seaice could you please add the precise location(s) of the data on Gadi? |
@adele-morrison is more on top of it than I am ? Noting the comments above about possibly moving it.
|
Yes that’s the right location. Would be great to get this in the catalog so we can keep switching all the COSIMA recipes over. What do you need in terms of documentation? |
I don’t think there’s any plans to move OM4_025.JRA_RYF. The panan data location is still in flux. I will try to keep that moving forward. |
OK, I'll start taking a look at the current data structure and builders to see what needs to happen to get these data ingested. Stay tuned... |
The filenames all look pretty coherent, but there's a couple of things I haven't been able to work out on my own:
|
It contains fields that do not change in frequency, such as grid-related data. It is saved once per run.
contains annually-averaged 2d fields
contains annually-averaged 0d fields |
I think we want all of those files - there is a frequency = 'fx' for the static files which exists in OM2 and OM3 datastores (and maybe others) |
Ah yes, I've found the |
Dumping this here so I can find it later (for building workable test data): https://stackoverflow.com/questions/15141563/python-netcdf-making-a-copy-of-all-variables-and-attributes-but-one |
I now have what I think is a functional |
@marc-white, we definitely don't want to call this I'd suggest seeing if the data mentioned in this comment can use the same builder. If so, then we could possibly call the builder |
I've updated the Builder to be able to read the filenames found in these directories. However, I've come across an interesting conundrum whilst trying to test the resulting catalog; the data in those three directories are, when ingested in to the catalog, pretty much identical, to the point where I can't figure out how to, say, get only the data from For the uninitiated like myself, what is the difference between these three runs, and how can I differentiate between them in an |
@marc-white, each of the experiments should be separate intake-esm datastores within the catalog. |
HI @anton-seaice and @adele-morrison , I'm now at the point where I'm ready to try an all-up ingest of the data. However, the |
I've updated the We're not quite ready to add the panan simulations ending in But we could add |
@AndyHoggANU any chance you want to create the |
I've confirmed with @AndyHoggANU and |
OK, there are |
I'm about ready to try testing a catalog creation - @dougiesquire what's the best practice for getting this set up?
|
@marc-white I think you should be able to use the
|
Description of the data product
<Please replace this text with a description of the data product to add to the ACCESS-NRI catalog. What data does it contain? What format is it in? Who is it useful for?>
Location of the data product on Gadi
Checklist
Add a "x" between the brackets to all that apply
The text was updated successfully, but these errors were encountered: