Updates for SMRF #70

scotthavens · 2020-07-17T17:36:19Z

SMRF underwent some major changes that had implications for AWSM.

The main change is that SMRF is now installed through the requirements via pip.
Updated pysnobal to the master branch as there were some cython issues with v0.2.0 not being compiled for python 3.7.
New travis testing for multiple version's and OS
New Dockerfile, almost 2GB smaller than before
Removed the knn package which did weather generation from the database. Since the database was removed from SMRF this was also removed in AWSM since it was experimental anyways
The use of the IPW version of isnobal will still work with a locally installed version of iSnobal but the tests were removed and it will be fully removed in a later release.
Gold files were updated and changed because of the changes within SMRF for solar radiation and topocalc

Gold file differences

Images of the differences and the summary below.
smrf_dev.zip

goldmeister.utilities INFO File/variable: snow.nc/thickness Difference Statistics
goldmeister.utilities INFO ======================================================
goldmeister.utilities INFO max                           0.0004470646381378174
goldmeister.utilities INFO min                           -0.00028274208307266235
goldmeister.utilities INFO mean                          9.933655746863224e-07
goldmeister.utilities INFO std                           2.3112850612960756e-05
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/snow_density Difference Statistics
goldmeister.utilities INFO =========================================================
goldmeister.utilities INFO max                           3.009857177734375   
goldmeister.utilities INFO min                           -0.5888671875       
goldmeister.utilities INFO mean                          0.024581022560596466
goldmeister.utilities INFO std                           0.13861200213432312 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/specific_mass Difference Statistics
goldmeister.utilities INFO ==========================================================
goldmeister.utilities INFO max                           0.05183982849121094 
goldmeister.utilities INFO min                           -0.025712966918945312
goldmeister.utilities INFO mean                          0.001318870927207172
goldmeister.utilities INFO std                           0.004474854562431574
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/liquid_water Difference Statistics
goldmeister.utilities INFO =========================================================
goldmeister.utilities INFO max                           0.012703202664852142
goldmeister.utilities INFO min                           -0.04171094298362732
goldmeister.utilities INFO mean                          -2.0873520043096505e-05
goldmeister.utilities INFO std                           0.0011108004255220294
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/temp_surf Difference Statistics
goldmeister.utilities INFO ======================================================
goldmeister.utilities INFO max                           0.058060646057128906
goldmeister.utilities INFO min                           -0.19327130913734436
goldmeister.utilities INFO mean                          -0.0068933055736124516
goldmeister.utilities INFO std                           0.015677690505981445
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/temp_lower Difference Statistics
goldmeister.utilities INFO =======================================================
goldmeister.utilities INFO max                           0.0                 
goldmeister.utilities INFO min                           0.0                 
goldmeister.utilities INFO mean                          0.0                 
goldmeister.utilities INFO std                           0.0                 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/temp_snowcover Difference Statistics
goldmeister.utilities INFO ===========================================================
goldmeister.utilities INFO max                           0.058060646057128906
goldmeister.utilities INFO min                           -0.19327130913734436
goldmeister.utilities INFO mean                          -0.0068933055736124516
goldmeister.utilities INFO std                           0.015677690505981445
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/thickness_lower Difference Statistics
goldmeister.utilities INFO ============================================================
goldmeister.utilities INFO max                           0.0                 
goldmeister.utilities INFO min                           0.0                 
goldmeister.utilities INFO mean                          0.0                 
goldmeister.utilities INFO std                           0.0                 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: snow.nc/water_saturation Difference Statistics
goldmeister.utilities INFO =============================================================
goldmeister.utilities INFO max                           0.11317205429077148 
goldmeister.utilities INFO min                           -0.07741556316614151
goldmeister.utilities INFO mean                          -3.4419968869769946e-05
goldmeister.utilities INFO std                           0.003968303557485342
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/net_rad Difference Statistics
goldmeister.utilities INFO ==================================================
goldmeister.utilities INFO max                           3.071197509765625   
goldmeister.utilities INFO min                           -7.217500686645508  
goldmeister.utilities INFO mean                          -0.5476462841033936 
goldmeister.utilities INFO std                           1.0057307481765747  
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/sensible_heat Difference Statistics
goldmeister.utilities INFO ========================================================
goldmeister.utilities INFO max                           3.043153762817383   
goldmeister.utilities INFO min                           -1.57342529296875   
goldmeister.utilities INFO mean                          0.22111770510673523 
goldmeister.utilities INFO std                           0.41072872281074524 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/latent_heat Difference Statistics
goldmeister.utilities INFO ======================================================
goldmeister.utilities INFO max                           3.081019878387451   
goldmeister.utilities INFO min                           -1.6431560516357422 
goldmeister.utilities INFO mean                          0.2250952124595642  
goldmeister.utilities INFO std                           0.41739389300346375 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/snow_soil Difference Statistics
goldmeister.utilities INFO ====================================================
goldmeister.utilities INFO max                           1.2364089488983154  
goldmeister.utilities INFO min                           -0.4286336898803711 
goldmeister.utilities INFO mean                          0.05252968519926071 
goldmeister.utilities INFO std                           0.11697542667388916 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/precip_advected Difference Statistics
goldmeister.utilities INFO ==========================================================
goldmeister.utilities INFO max                           0.35845232009887695 
goldmeister.utilities INFO min                           -0.138840913772583  
goldmeister.utilities INFO mean                          0.014016466215252876
goldmeister.utilities INFO std                           0.030695520341396332
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/sum_EB Difference Statistics
goldmeister.utilities INFO =================================================
goldmeister.utilities INFO max                           3.0695877075195312  
goldmeister.utilities INFO min                           -4.340187072753906  
goldmeister.utilities INFO mean                          -0.03488721325993538
goldmeister.utilities INFO std                           0.24159379303455353 
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/evaporation Difference Statistics
goldmeister.utilities INFO ======================================================
goldmeister.utilities INFO max                           0.003923313692212105
goldmeister.utilities INFO min                           -0.0020947493612766266
goldmeister.utilities INFO mean                          0.0002866920840460807
goldmeister.utilities INFO std                           0.0005317896720953286
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/snowmelt Difference Statistics
goldmeister.utilities INFO ===================================================
goldmeister.utilities INFO max                           0.012823767960071564
goldmeister.utilities INFO min                           -0.009303182363510132
goldmeister.utilities INFO mean                          2.1667812688974664e-05
goldmeister.utilities INFO std                           0.0005146563635207713
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/SWI Difference Statistics
goldmeister.utilities INFO ==============================================
goldmeister.utilities INFO max                           0.020421847701072693
goldmeister.utilities INFO min                           -0.04546251893043518
goldmeister.utilities INFO mean                          -0.0001418040628777817
goldmeister.utilities INFO std                           0.002005483489483595
goldmeister.utilities INFO 
goldmeister.utilities INFO File/variable: em.nc/cold_content Difference Statistics
goldmeister.utilities INFO =======================================================
goldmeister.utilities INFO max                           1946.712890625      
goldmeister.utilities INFO min                           -5040.1875          
goldmeister.utilities INFO mean                          -167.69265747070312 
goldmeister.utilities INFO std                           416.5120849609375

…the weather db which has been deprecated from SMRF

…ersion

Makefile

awsm/interface/ipysnobal.py

awsm/interface/smrf_ipysnobal.py

requirements.txt

tests/RME/gold/make_gold_stations

Hedrick-ARS

The Dockerfile was the only file to have major changes, and now that Scott explained those changes I am happy.

Consider this pull request APPROVED.

micahjohnson150 · 2020-08-05T17:56:44Z

Goldmeister! I am happy to see you found it useful!

scotthavens · 2020-08-05T18:00:19Z

Goldmeister! I am happy to see you found it useful!

Once we figured it out! We've used it in SMRF as well.

micahjohnson150 · 2020-08-05T18:10:35Z

hahah isn't that the nature of all my code?! lol glad it helps.

scotthavens added 15 commits July 9, 2020 09:02

working on getting the smrf-dev into awsm

8768646

added pysnobal as a requirement

603608c

pysnobal import and logging import from SMRF

5adf78d

tests are running but not passing as smrf has changed

ddfb2ec

tests passing, new gold files

fef01e0

ignore the goldmeister outputs

31798da

smrf 0.11.2 working within awsm and tests pass

4776a06

working on dockerfile and had to remove the knn because it was using …

fb30bd2

…the weather db which has been deprecated from SMRF

dockerfile built

041ffe3

setup tox and travis

0d56f44

local env has IPW installed so didn't know awsm was testing the IPW v…

71c49de

…ersion

updated pysnobal to master

4a7e4a6

updated pysnobal imports and tox

843801c

Merge branch 'master' into smrf_dev

dbfd58a

fixing merge issues

6292a36

scotthavens requested review from jomey, etrujil and Hedrick-ARS July 17, 2020 18:46

scotthavens mentioned this pull request Jul 17, 2020

Travis Build Issues around new config and threading. #60

Closed

jomey requested changes Jul 17, 2020

View reviewed changes

addressing review comments

9271324

jomey mentioned this pull request Jul 17, 2020

Makefile for gold test files USDA-ARS-NWRC/smrf#158

Open

jomey approved these changes Jul 17, 2020

View reviewed changes

Hedrick-ARS approved these changes Jul 17, 2020

View reviewed changes

scotthavens merged commit f178d6a into master Aug 3, 2020

scotthavens deleted the smrf_dev branch August 3, 2020 17:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates for SMRF #70

Updates for SMRF #70

scotthavens commented Jul 17, 2020

Hedrick-ARS left a comment

micahjohnson150 commented Aug 5, 2020

scotthavens commented Aug 5, 2020

micahjohnson150 commented Aug 5, 2020

Updates for SMRF #70

Updates for SMRF #70

Conversation

scotthavens commented Jul 17, 2020

Gold file differences

Hedrick-ARS left a comment

Choose a reason for hiding this comment

micahjohnson150 commented Aug 5, 2020

scotthavens commented Aug 5, 2020

micahjohnson150 commented Aug 5, 2020