Setting Up Run-time Environment for WRF&WRF-Hydro Coupled Training v5.1.2 on CyberGIS-Jupyter for Water

The HydroShare project is pleased to bring you this notebook that can set up a run-time environment on the CyberGIS-Jupyter for Water (CJW) platform for WRF&WRF-Hydro Coupled Testcase Online Lesson (v5.1.2). In contrast to the Docker-based local setup, this HydroShare solution does not require installation or downloading of any software or data onto your local computer, and it enables you to access to more powerful computing resources in a clould-based CJW environment. All necessary materials required to complete this training are remotely accessible through a browser (Google Chrome recommended).

This notebook retrieves the WRFHydro model codes and relevant data from different official repos on Github and Google Drive managed by the NCAR/UCAR WRFHydro Development Team, and puts them in certain directory structure (same as the Docker-based local setup) required by the training notebooks. Specifically, two new folders will be created (wrf-hydro-training, and WRF_WPS) alongside. The training notebooks are stored in wrf-hydro-training --> lessons as shown below.

alt text

The following table lists all the data (and sources) used by this training:

Repo/File Link Commit/Tag/SHA-1
wrf_hydro_nwm_public https://github.com/NCAR/wrf_hydro_nwm_public v5.1.2
wrf_hydro_training https://github.com/NCAR/wrf_hydro_training 57a05ca277a1618185fb5e1646757124d3b3c49e
front_range_CO_example_testcase_coupled.tar.gz https://github.com/NCAR/.../testcase_coupled.tar.gz ed5ca115acaf979f00801100ae23614baee46be1

To run this notebook: Cell (top menu) --> Run All

Note: You are supposed to run this notebook only once. Each run will set up a fresh new environment, and any pre-existing training setup (codes, notebooks and model data) and your edits will be removed.

After running this notebook successfully, the lesson notebooks will have been downloaded into the workspace and you can start the training at Lesson-run, or view all notebooks here. Until this is done these links are inactive.

Troubleshooting: If any error message occurred in this notebook, it might be caused by some previously executed training notebooks still running and referencing model data or files on disk . Try one of the two actions below, and run this notebook again.

  • Stop all other training notebooks: Go to each notebook --> Kernel (top menu) --> Shutdown;
  • Restart Jupyter server: Control Panel (upper-righer corner) --> Stop My Server --> Start My Server;
In [ ]:
%%bash
rm -rf ./wrf-hydro-training 2>/dev/null 
rm -rf ./WRF_WPS 2>/dev/null 
rm -rf ~/wrf-hydro-training 2>/dev/null 
rm -rf ~/WRF_WPS 2>/dev/null 
rm -rf /home/docker/wrf-hydro-training 2>/dev/null 
rm -rf /home/docker/WRF_WPS 2>/dev/null
In [ ]:
! ls $(pwd)
In [ ]:
%%bash
mkdir -p ./wrf-hydro-training
mkdir -p ./WRF_WPS

if [ -d "/WRF_WPS_coupled" ] 
then
    ln -sf /WRF_WPS_coupled/WPS $(pwd)/WRF_WPS/WPS
    ln -sf /WRF_WPS_coupled/WRF $(pwd)/WRF_WPS/WRF
    ln -sf /WRF_WPS_coupled/wrf_hydro_nwm_public-5.1.2 $(pwd)/WRF_WPS/wrf_hydro_nwm_public-5.1.2
else
    ln -sf $EBROOTWPS/WPS-$EBVERSIONWPS $(pwd)/WRF_WPS/WPS
    ln -sf /data/cigi/cjw-easybuild/WRF_WRFHydro/WRF4.2.1_WHydro5.1.2_coupled $(pwd)/WRF_WPS/WRF
    ln -sf /data/cigi/cjw-easybuild/WRF_WRFHydro/wrf_hydro_nwm_public-5.1.2 $(pwd)/WRF_WPS/wrf_hydro_nwm_public-5.1.2
    echo "Using binary from easybuild...."
fi

ln -sf $(pwd)/wrf-hydro-training ~/wrf-hydro-training
ln -sf $(pwd)/wrf-hydro-training /home/docker/wrf-hydro-training
ln -sf $(pwd)/WRF_WPS ~/WRF_WPS
ln -sf $(pwd)/WRF_WPS /home/docker/WRF_WPS
In [ ]:
training_version="57a05ca277a1618185fb5e1646757124d3b3c49e"
In [ ]:
!wget https://github.com/NCAR/wrf_hydro_nwm_public/releases/download/v5.1.2/front_range_CO_example_testcase_coupled.tar.gz
!tar -xf *coupled*.tar.gz
!mv ./example_case_coupled ./wrf-hydro-training/
!rm -rf *coupled*.tar.gz
In [ ]:
!git clone https://github.com/NCAR/wrf_hydro_training
!cd ./wrf_hydro_training && git checkout {training_version}
!mv ./wrf_hydro_training/lessons/coupled_training ./wrf-hydro-training/lessons
!rm -rf ./wrf_hydro_training
In [ ]:
import sys
!{sys.executable} -m pip install nbformat
In [ ]:
import nbformat
import glob

# Set notebooks to the WRFHydro kernel on CJW
for fp in glob.glob("./wrf-hydro-training/lessons/*.ipynb"):
    nb = nbformat.read(fp, as_version=4)
    if nb.metadata.kernelspec.display_name == "Python 3" and nb.metadata.kernelspec.name == "python3":
        print("changing kernal for {}".format(fp))
        nb.metadata.kernelspec.display_name = "WRFHydro"
        nb.metadata.kernelspec.name = "wrfhydro-2021-09"
        nbformat.write(nb, fp, version=nbformat.NO_CONVERT)
    elif nb.metadata.kernelspec.display_name == "Bash" and nb.metadata.kernelspec.name == "bash":
        print("changing kernal for {}".format(fp))
        nb.metadata.kernelspec.display_name = "Bash-2021-09"
        nb.metadata.kernelspec.name = "bash-2021-09"
        nbformat.write(nb, fp, version=nbformat.NO_CONVERT)
In [ ]:
%%bash
gdown https://drive.google.com/uc?id=1X71fdaSEJ5GWyNY2MDIy9cC6E7A0kihl 
tar -xzvf geog_conus.tar.gz
rm geog_conus.tar.gz
mv ./geog_conus ./WRF_WPS/geog_conus
In [ ]:
print("Done")

Next - Start from Lesson-run or view All Lessons

If there was any error occurred in this notebook, you can try to re-run it by: Kernel (top menu) --> Restart & Run All. If the same error persisits, you can try restarting the jupyter environment: Control Panel (upper-right corner) --> Stop My Server --> Start My Server, and then run this notebook again.

In [ ]: