Shared data

[malmans2]

@atb299

I moved data in #25 to make it easier to reproduce this setup even for people that don't have access to Archer2.
All symlinks now point to data in /work/n01/shared/malmans/immerse_wp6.2_data/, with the exception of JRA data that point to /work/n01/shared/nemo/FORCING/JRA/.

If we assume users will download JRA themselves, we could upload /work/n01/shared/malmans/immerse_wp6.2_data/ on Zenodo as it's just 14GB. (The JRA folder below only contains weights)

du -h /work/n01/shared/malmans/immerse_wp6.2_data
41M	/work/n01/shared/malmans/immerse_wp6.2_data/1_INPUT
2.3G	/work/n01/shared/malmans/immerse_wp6.2_data/INPUT/JRA
14G	/work/n01/shared/malmans/immerse_wp6.2_data/INPUT
892K	/work/n01/shared/malmans/immerse_wp6.2_data/2_INPUT
14G	/work/n01/shared/malmans/immerse_wp6.2_data

For non-Archer2 users, the workflow would be this:

git clone [email protected]:immerse-project/wp6.2-demonstrator.git
cd wp6.2-demonstrator
# Download our data from zenodo
cd data/INPUT/JRA
# Download JRA data

[malmans2]

Other than that, I think users "only" have to add the arch files in wp6.2-demonstrator/arch and edit the compilation and submission scripts in wp6.2-demonstrator/scripts

[malmans2]

Although non-Archer2 users probably won't use this repo/scripts, but just the nemo-simsar we'll eventually create.

[atb299]

@malmans2 , the other option for the /shared/ data would be to do something similar to what @jdha has done with e.g. the ORCHESTRA input files. It's less permanent than Zenodo, but still publicly accessible.

[malmans2]

@jdha has done with e.g. the ORCHESTRA input files

Do you mean make it public on Jasmin? I think it makes sense if we have to share this configuration before it's definitive.

When we are done and we want to publish/make a report, we should probably go for zenodo, and we should add the files that are currently generated using the tools.

[atb299]

Do you mean make it public on Jasmin? I think it makes sense if we have to share this configuration before it's definitive.

Yes. I'd probably go for the Zenodo option only once we're satisfied that we've got a simulation we want to use for the report/paper.

[jdha]

sounds like a plan - once everything is locked down - then look to publish the forcing etc

[jdha]

I think in one of my setups (possibly a branch of SE-NEMO) if have a ARCHER2 flag - to link to local data - if not grab from remote source