Skip to content

Commit

Permalink
Updated for new release - 1.1.6; CMOR3.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@durack1
durack1 committed Nov 25, 2019
1 parent 9231fe5 commit ac641df
Show file tree
Hide file tree
Showing 17 changed files with 201 additions and 112 deletions.
4 changes: 2 additions & 2 deletions CMOR/drive_input4MIPs_bcs.json
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"institution_id": "PCMDI",
"source_id": "PCMDI-AMIP-1-1-5",
"source": "PCMDI-AMIP 1.1.5: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"source_id": "PCMDI-AMIP-1-1-6",
"source": "PCMDI-AMIP 1.1.6: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"contact ": "PCMDI ([email protected])",
"further_info_url": "https://pcmdi.llnl.gov/mips/amip",

Expand Down
4 changes: 2 additions & 2 deletions CMOR/drive_input4MIPs_obs.json
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"institution_id": "PCMDI",
"source_id": "PCMDI-AMIP-1-1-5",
"source": "PCMDI-AMIP 1.1.5: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"source_id": "PCMDI-AMIP-1-1-6",
"source": "PCMDI-AMIP 1.1.6: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"contact ": "PCMDI ([email protected])",
"further_info_url": "https://pcmdi.llnl.gov/mips/amip",

Expand Down
6 changes: 3 additions & 3 deletions README.md
100755 → 100644
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# amipbcs - AMIP dataset prepared for input4MIPs
Code to generate boundary condition data for the AMIP (Atmospheric Model Intercomparison Project) experiment as part of CMIP6 and the input4MIPs project

Generated data can be obtained from the [PCMDI ESGF data portal](https://esgf-node.llnl.gov/search/input4mips/?source_version=1.1.5)
Generated data can be obtained from the [PCMDI ESGF data portal](https://esgf-node.llnl.gov/search/input4mips/?source_version=1.1.6)

The code depends upon the following packages:
- [**CDAT 8.0+ with cd77**](https://github.com/CDAT/cdat) (Available through [conda](https://anaconda.org/CDAT/repo))
- [**CMOR 3.4.0+**](https://github.com/PCMDI/cmor) (Available through [conda](https://anaconda.org/conda-forge/cmor/files))
- [**CDAT 8.1+ with cd77**](https://github.com/CDAT/cdat) (Available through [conda](https://anaconda.org/CDAT/repo))
- [**CMOR 3.5.0+**](https://github.com/PCMDI/cmor) (Available through [conda](https://anaconda.org/conda-forge/cmor/files))
- [**gfortran 7.3+**](https://gcc.gnu.org/wiki/GFortran)

The optional libraries of CDAT 8.1+ are required (these are bundled with cd77):
Expand Down
43 changes: 36 additions & 7 deletions README.txt
100755 → 100644
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ Installing flex solves issues
***
This solves issues with wrapit77

Hit issues with wrapit77 -
Hit issues with wrapit77 -

WRAPIT Version: 120209
OPERATING SYSTEM: Linux
nbits = 64
FORTRAN COMPILER (f90c): gfortran
WRAPIT Version: 120209
OPERATING SYSTEM: Linux
nbits = 64
FORTRAN COMPILER (f90c): gfortran
FORTRAN COMPILER OPTIONS (fopts): -m64 -fPIC -v -c -fno-second-underscore
/export/durack1/anaconda2/envs/cdat80cmor333nclnco/bin/wrapit77: error while loading shared libraries: libfl.so.2: cannot open shared object file: No such file or directory
FATAL ERROR: wrapit77 failed
Expand Down Expand Up @@ -104,7 +104,7 @@ drwxr-xr-x 30 durack1 climate 4.0K Jan 18 2019 ..

9. Run
# Create/prepare output directory
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9643]> mkdir 360x180_v1.1.5
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9643]> mkdir 360x180_v1.1.5
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9639]> mkhurrell1
*****
# If errors are hit, add LD_LIBRARY_PATH to env
Expand Down Expand Up @@ -171,7 +171,7 @@ lrwxrwxrwx 1 durack1 climate 42 Jun 2 2016 input4MIPs-cmor-tables -> /expor
> Will need to involve Sasha/Jiwoo for this step


14. Update github repo
14a. Update github repo
> cd /export/durack1/git/amipbcs
# Update all source files
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/*.f .
Expand All @@ -196,3 +196,32 @@ lrwxrwxrwx 1 durack1 climate 42 Jun 2 2016 input4MIPs-cmor-tables -> /expor
# Commit version
> git commit -am 'Updated for new release - 1.1.5; CMOR3.4.0'
> git push

14b. Update github repo for 1.1.6 release (symlink to v1.2.0)
> cd /export/durack1/git/amipbcs
[durack1@detect amipbcs]$ pwd
/export/durack1/git/amipbcs
# Update all source files
# Due to recent clone rsync doesn't work (timestamp conflicts) so use cp
> cp /work/durack1/Shared/150219_AMIPForcingData/*.f .
> cp -R /work/durack1/Shared/150219_AMIPForcingData/src .
> cp /work/durack1/Shared/150219_AMIPForcingData/*.py .
> chmod 755 *.py *.f
> chmod 644 *.txt *.md
# Update CMOR info
> cp -R /work/durack1/Shared/150219_AMIPForcingData/CMOR .
> chmod 644 CMOR/*.json
# Update data contributed (*.nc files are not hosted in git - .gitignore)
> #rm -rf SST_1-1-5b ; # Purge local dir (if offical release replaces it)
> rsync -vrutz --copy-links /work/durack1/Shared/150219_AMIPForcingData/SST_1-1-6 . ; # Copy remote dir
> chmod 755 SST_1-1-6
> chmod 755 SST_1-1-6/*.csh SST_1-1-6/*.dat SST_1-1-6/*.f SST_1-1-6/*.ncl SST_1-1-6/*.sh
> chmod 644 SST_1-1-6/*.bz2 SST_1-1-6/*.nc SST_1-1-6/*.txt
# Update this file once everything is updated
# > rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/README.txt . ; # for v1.1.6 updates live to amipbcs/README.txt
# Update README.md
# Add new dir to repo
> git add SST_1-1-6
# Commit version
> git commit -am 'Updated for new release - 1.1.6; CMOR3.5.0'
> git push
Binary file removed SST_1-1-5b/190118.tar.bz2
View file
Binary file not shown.
16 changes: 0 additions & 16 deletions SST_1-1-5b/190118_log.txt
View file

This file was deleted.

Binary file added SST_1-1-6/190702.tar.bz2
View file
Binary file not shown.
23 changes: 23 additions & 0 deletions SST_1-1-6/190702_log.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
2019-07-02 14:31:30 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/ [34787] -> ".listing" [1]
2019-07-02 14:31:30 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201901.gz [197266] -> "oiv2mon.201901.gz" [1]
2019-07-02 14:31:31 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201902.gz [194872] -> "oiv2mon.201902.gz" [1]
2019-07-02 14:31:32 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201903.gz [195429] -> "oiv2mon.201903.gz" [1]
2019-07-02 14:31:32 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201904.gz [196418] -> "oiv2mon.201904.gz" [1]
2019-07-02 14:31:33 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201905.gz [197338] -> "oiv2mon.201905.gz" [1]
FINISHED --2019-07-02 14:31:33--
Downloaded: 5 files, 958K in 0s (1827857 GB/s)
2019-07-02 14:31:34 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/ [34787] -> ".listing" [1]
2019-07-02 14:31:35 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201801.gz [197480] -> "oiv2mon.201801.gz" [1]
2019-07-02 14:31:36 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201802.gz [182429] -> "oiv2mon.201802.gz" [1]
2019-07-02 14:31:37 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201803.gz [197062] -> "oiv2mon.201803.gz" [1]
2019-07-02 14:31:38 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201804.gz [196674] -> "oiv2mon.201804.gz" [1]
2019-07-02 14:31:39 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201805.gz [198604] -> "oiv2mon.201805.gz" [1]
2019-07-02 14:31:40 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201806.gz [198245] -> "oiv2mon.201806.gz" [1]
2019-07-02 14:31:41 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201807.gz [196809] -> "oiv2mon.201807.gz" [1]
2019-07-02 14:31:42 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201808.gz [196791] -> "oiv2mon.201808.gz" [1]
2019-07-02 14:31:43 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201809.gz [196503] -> "oiv2mon.201809.gz" [1]
2019-07-02 14:31:44 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201810.gz [198035] -> "oiv2mon.201810.gz" [1]
2019-07-02 14:31:46 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201811.gz [199391] -> "oiv2mon.201811.gz" [1]
2019-07-02 14:31:46 URL: ftp://ftp.emc.ncep.noaa.gov/cmb/sst/oimonth_v2/oiv2mon.201812.gz [199317] -> "oiv2mon.201812.gz" [1]
FINISHED --2019-07-02 14:31:46--
Downloaded: 12 files, 2.2M in 0s (4390888 GB/s)
10 changes: 5 additions & 5 deletions SST_1-1-5b/SSTICE.Update.unf.csh → SST_1-1-6/SSTICE.Update.unf.csh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ set date=`date +%y%m%d` ; # Set date for dynamic run
# ==============================NCL====================================

### setenv NCARG_ROOT /usr/local/bin
setenv NCARG_ROOT /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex
setenv PATH /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/bin:${PATH} ; # Add wrapit77 to PATH
setenv NCARG_ROOT /export/durack1/anaconda2/envs/py2cdmscd77Ncmor340Nnclncoflex
setenv PATH /export/durack1/anaconda2/envs/py2cdmscd77Ncmor340Nnclncoflex/bin:${PATH} ; # Add wrapit77 to PATH

cat >! main.ncl << "END_MAIN_NCL"
;; Not needed from 6.2 onwarded
Expand Down Expand Up @@ -83,9 +83,9 @@ begin ; MAIN NCL DRIVER
;diri = "/work/durack1/Shared/150219_AMIPForcingData/SST_NEW/$date/"
;###### UPDATE : REQUIRES UPDATING ##
diri = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b/190118/" ; ## UPDATE : REQUIRES UPDATING ##
diro = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b/" ; ## UPDATE : REQUIRES UPDATING ##
dirm = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b/" ; ## UPDATE : REQUIRES UPDATING - Last edit required ##
diri = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-2-0/190702/" ; ## UPDATE : REQUIRES UPDATING ##
diro = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-2-0/" ; ## UPDATE : REQUIRES UPDATING ##
dirm = "/work/durack1/Shared/150219_AMIPForcingData/SST_1-2-0/" ; ## UPDATE : REQUIRES UPDATING - Last edit required ##
;######
film = "lstags.onedeg.dat" ; sst mask [not used]
Expand Down
0 SST_1-1-5b/SST_COMPARE.ncl → SST_1-1-6/SST_COMPARE.ncl
View file
File renamed without changes.
0 SST_1-1-5b/coast_land_NearNbor.f → SST_1-1-6/coast_land_NearNbor.f
View file
File renamed without changes.
0 SST_1-1-5b/consistent.f → SST_1-1-6/consistent.f
View file
File renamed without changes.
28 changes: 16 additions & 12 deletions SST_1-1-5b/download.sh → SST_1-1-6/download.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
# PJD 18 Oct 2018 - Updated for V1.1.5 and using conda env cdat80cmor333nclnco
# PJD 18 Oct 2018 - Updated from 201803 to 201809
# PJD 18 Jan 2019 - Updated to reflect latest conda envs, latest downloads and relevant nco indexing
# PJD 2 Jul 2019 - Updated to reflect latest conda envs, latest downloads and relevant nco indexing

# USER WILL NEED TO SET:
# ENVIRONMENT VARIABLES: NCARG_ROOT (and PATH to include ncl path)
Expand All @@ -29,12 +30,12 @@
# Doc: http://doi.org/10.1175/2008JCLI2292.1 (Hurrell et al., 2008)

### USER TO SET ###
setenv NCARG_ROOT /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex
setenv PATH /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/bin:${PATH} ; # Add wrapit77, ncl, nco to PATH
setenv NCARG_ROOT /export/durack1/anaconda2/envs/py2cdmscd77Ncmor340Nnclncoflex
setenv PATH /export/durack1/anaconda2/envs/py2cdmscd77Ncmor340Nnclncoflex/bin:${PATH} ; # Add wrapit77, ncl, nco to PATH

###### UPDATE : PATH REQUIRES UPDATING ##
set oi2path=/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b/ ; ## UPDATE : PATH REQUIRES UPDATING ##
set oi2oldpath=/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-4/ ; ## UPDATE : PATH REQUIRES UPDATING ##
set oi2path=/work/durack1/Shared/150219_AMIPForcingData/SST_1-2-0/ ; ## UPDATE : PATH REQUIRES UPDATING ##
set oi2oldpath=/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b/ ; ## UPDATE : PATH REQUIRES UPDATING ##
######

set date=`date +%y%m%d`
Expand Down Expand Up @@ -80,9 +81,11 @@ cd ${oi2path}
# 9,14 (Sept 2017 through March 2018)
# 15,20 (April 2018 through September 2018)
# 3,8 (April 2018 through September 2018) note 0 indexing ; Run 190118
# 9,14 (October 2018 through March 2019) note 0 indexing ; Run 190702

###### UPDATE : YEARS REQUIRE UPDATING ##
ncks -O -h -d time,3,8 MODEL.OI2.ice.mnly.201801-201812.unf.nc ICE.update.nc ; # Extract months ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncks -O -h -d time,3,8 MODEL.OI2.sst.mnly.201801-201812.unf.nc SST.update.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncks -O -h -d time,9,14 MODEL.OI2.ice.mnly.201801-201905.unf.nc ICE.update.nc ; # Extract months ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncks -O -h -d time,9,14 MODEL.OI2.sst.mnly.201801-201905.unf.nc SST.update.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
# Make sure updates went correctly ... only the 12 new months, Check 'date' variable
######
# Check times of new updates
Expand All @@ -93,7 +96,7 @@ ncdump -v date SST.update.nc
# Check times of older files
echo '**********'
echo 'Previous file time extent'
ncdump -v date ${oi2oldpath}MODEL.SST.HAD187001-198110.OI198111-201803.nc
ncdump -v date ${oi2oldpath}MODEL.SST.HAD187001-198110.OI198111-201809.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##


### Step 6 - Append new months onto existing data ###
Expand All @@ -107,18 +110,18 @@ ncdump -v date ${oi2oldpath}MODEL.SST.HAD187001-198110.OI198111-201803.nc

###### UPDATE : FILENAME/YEARS REQUIRE UPDATING ##
# V1.1.2 - extend to 201703
ncrcat ${oi2oldpath}MODEL.ICE.HAD187001-198110.OI198111-201803.nc ICE.update.nc MODEL.ICE.HAD187001-198110.OI198111-201809.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncrcat ${oi2oldpath}MODEL.SST.HAD187001-198110.OI198111-201803.nc SST.update.nc MODEL.SST.HAD187001-198110.OI198111-201809.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncrcat ${oi2oldpath}MODEL.ICE.HAD187001-198110.OI198111-201809.nc ICE.update.nc MODEL.ICE.HAD187001-198110.OI198111-201903.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncrcat ${oi2oldpath}MODEL.SST.HAD187001-198110.OI198111-201809.nc SST.update.nc MODEL.SST.HAD187001-198110.OI198111-201903.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
# Make sure updates went correctly
echo '**********'
echo 'Updated file time extent'
#ncdump -v date MODEL.ICE.HAD187001-198110.OI198111-201609.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncdump -v date MODEL.SST.HAD187001-198110.OI198111-201809.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
ncdump -v date MODEL.SST.HAD187001-198110.OI198111-201903.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
# Purge partial new files
rm -f ICE.update.nc ; # These may not purge do to a file handle being unreleased by ncrcat
rm -f MODEL.OI2.ice.mnly.201801-201812.unf.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
rm -f MODEL.OI2.ice.mnly.201801-201905.unf.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
rm -f SST.update.nc
rm -f MODEL.OI2.sst.mnly.201801-201812.unf.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
rm -f MODEL.OI2.sst.mnly.201801-201905.unf.nc ; ## UPDATE : YEARS IN FILENAMES REQUIRE UPDATING ##
######


Expand All @@ -137,6 +140,7 @@ if ( $1 != "" ) then
\rm -f ${1}_log.txt
endif


: <<--
comments
--
0 SST_1-1-5b/lstags.onedeg.dat → SST_1-1-6/lstags.onedeg.dat
View file
File renamed without changes.
Loading

0 comments on commit ac641df

Please sign in to comment.