Technically, you could use the customization we gave in Chapter 1 to test various configuration and namelist options for CLM. Sometimes, it’s also useful to have automated tests though to test that restarts give exactly the same results as without a restart. It’s also useful to have automated tests to run over a wide variety of configurations, resolutions, and namelist options. To do that we have several different types of scripts set up to make running comprehensive testing of CLM easy. There are two types of testing scripts for CLM. The first are the CESM test scripts, which utilize the create_newcase scripts that we shown how to use in this User’s Guide. The second are a set of stand-alone scripts that use the CLM configure and build-namelist scripts to build and test the model as well as testing the CLM tools as well. Below we will go into further details of how to use both methods.
18.104.22.168. CIME Testing scripts¶
We first introduce the test scripts that work for all CESM components. The CIME script create_test runs a specific type of test, at a given resolution, for a given compset using a given machine. See CIME Chapter on Testing for how to use it to run single tests as well as lists of tests. The standard testname for CLM is “aux_clm” for cheyenne with intel and gnu compilers as well as the CGD machine hobart for intel, nag, and pgi compilers. There’s also a shorter test list called “clm_short”. Also see the CTSM Wiki on Testing.
22.214.171.124. CTSM Tools Testing¶
$CTSMROOT/clm/test/tools/README 06/08/2018 Scripts for testing the CLM support tools with many different configurations and run-time options. I. MAIN SCRIPTS: test_driver.sh - Test the CLM offline tools To use... ./test_driver.sh -i on cheyenne qcmd -l walltime=06:00:00 -- ./test_driver.sh -i >& run.out & Intended for use on NCAR machines cheyenne, geyser (DAV) and hobart. II. RUNNING test_driver.sh TOOLS TESTING: Basic use: ./test_driver.sh -i ./test_driver.sh -h # to get help on options Important environment variables (just used by test_driver.sh) BL_ROOT ---------------- Root directory of CLM baseline code to compare to (if not set BL test will not be performed) BL_TESTDIR ------------- Root directory of where to put baseline tests CLM_INPUT_TESTS -------- Filename of file with list of tests to perform CLM_TESTDIR ------------ Root directory of where to put most tests CLM_RETAIN_FILES ------- If set to TRUE -- don't cleanup files after testing CLM_FC ----------------- Use given compiler CLM_JOBID -------------- Job identification number to use (rather than process ID) CLM_THREADS ------------ Number of open-MP threads to use (by default this is set differently by machine) CLM_SOFF --------------- If set to TRUE -- stop on first failed test (default FALSE) Important files for test_driver tools testing: test_driver.sh ------- Main test script for tools nl_files ------------- Directory with various namelists to test config_files --------- Directory with various configurations to test input_tests_master --- Master list of tests tests_pretag_* ------- Tests for specific machines to do by default before a tag is done tests_posttag_* ------ Tests for specific machines to do for more extensive testing after a tag is done CLM_compare.sh ------- Compares output history files between two cases T*.sh ---------------- Basic test script to do a specific type of test gen_test_table.sh ---- Creates HTML table of tests Makefile ------------- Will build the HTML table of tests ../../tools/README.testing - Information on how the testing works for the CLM tools
126.96.36.199. CTSM Fortran Unit Tests¶
# To run all CTSM unit tests on caldera, run the following command: # # The creation of a temporary directory ensures that you are doing a completely # clean build of the unit tests. (The use of the --clean flag to run_tests.py # cleans most, but not all of the files created by the unit test build.) For # rerunning the tests after an incremental change, you can instead use an # existing build directory. # From a standalone CTSM checkout: ../cime/scripts/fortran_unit_testing/run_tests.py --build-dir `mktemp -d --tmpdir=. unit_tests.XXXXXXXX` # If you are within a full CESM checkout, you would instead do: # ../../../cime/scripts/fortran_unit_testing/run_tests.py --build-dir `mktemp -d --tmpdir=. unit_tests.XXXXXXXX`
188.8.131.52. CTSM Build-namelist Tests¶
Run the following perl tester that
> cd bld/unit_testers > ./build-namelist_test.pl
184.108.40.206. Testing PTCLM¶
PTCLM/README 04/10/2015 PTCLMmkdata is a python tool built on top of CLM tools and CESM scripts for building datasets to run CLM "I" cases for data from Ameriflux Tower-sites, or other user-supplied single-point datasets. Original Authors: Daniel M. Ricciuto, Dali Wang, Peter E. Thornton, Wilfred M. Post Environmental Sciences Division, Oak Ridge National Laboratory (ORNL) R. Quinn Thomas Cornell University Modified by: Erik Kluzek (NCAR) General Directory structure: PTCLM/PTCLMmkdata ----- Main script PTCLM/PTCLM_sitedata - Site data files of static information latitude, longitude, soil info., and PFT information for each site Also different "groups" of site-data lists, and the script to convert the transient years landuse_timeseries files into landuse_timeseries text files that mksurfdata can use. PTCLM/mydatafiles ----- Default location of data files that will be created by PTCLMmkdata. Sites will be built in their own subdirectories under here. Optionally you can give your own location you'd like to use for your data. PTCLM/PTCLMsublist --------- Script to submit a list of PTCLM sites to the batch que (only setup for a few machines). PTCLM/PTCLMsublist_prog.py - Python module to support submit list script. Handles command line arguments and such. PTCLM/batchque.py ---------- Python module for batch submital. PTCLM/buildtools ----------- Script to build the CLM tools needed to run PTCLMmkdata (mksurfdata_map and gen_domain). Works on cheyenne. Quickstart: # ASSUMPTIONS: # For this example I'm running a I1PtClm50SpGs case on cheyenne using # CSMDATA in the standard location # Finally we use the 6-digit AmeriFlux site code for the University of Mich. Biological # Station US-UMB (data for this station is checked into the inputdata repository). # I also assume you are using UNIX C-shell, and GNU make is called gmake setenv CSMDATA /glade/p/cesm/cseg/inputdata setenv SITE US-UMB cd PTCLM setenv MYDATAFILES `pwd`/mydatafiles # Next build all of the clm tools you will need # The following script assumes cheyenne, hobart, or izumi for other machines # you'll need to build each tool by hand ./buildtools # next run PTCLMsublist which will submit PTCLMmkdata to batch queue (NOTE -- MAKE SURE python, NCO AND NCL IS IN YOUR PATH) # PTCLMsublist is only setup for a few batch machines, you'll need to update them to add new machines # or create your own batch submission script. # NOTE: Every day you run PTCLMmkdata it will remake the map called # renamemapfiles to rename files with todays creation date. # This makes running PTCLMmkdata a reasonable amount of time. # However, you can use the script in mydatafiles # qcmd -l walltime=02:00:00 -- ./PTCLMsublist -l $SITE -d $CSMDATA --account=XXXXXXXXX --mach=cheyenne # NOTE: To submit several sites at once, make the "-l" option a comma delimited # list of site names. # Next copy the towersite meterology datafiles into your $MYDATAFILES space # (For the US-UMB station you can skip this step as the .build step will bring the data over) cd $MYDATAFILES/1x1pt_$SITE mkdir $MYDATAFILES/1x1pt_$SITE/CLM1PT_data # Copy meteorology data NetCDF files into 1x1pt_$SITE sub-directory # (with filenames of yyyy-mm.nc) # The variables assumed to be on the files are: # ZBOT, TBOT, RH, WIND, PRECTmms, FSDS, PSRF, FLDS # (if other fields are available or with different names this can be changed by # adding a user_nl_datm.streams.txt file as we outline below) # Make sure your data has time with the attribute: calendar="gregorian" # Make sure the forcing directory points to the location of your data # (PTCLMmkdata should already do this) ./xmlchange DIN_LOC_ROOT_CLMFORC=$MYDATAFILES/1x1pt_$SITE # Then create a case using the data you just created setenv MYCASE "testPTCLM" cd $CTSMROOT/cime setenv CIMEROOT `pwd` cd $CIMEROOT/scripts ./create_newcase --user-mods-dir $MYDATAFILES/1x1pt_$SITE --case $MYCASE --res CLM_USRDAT --compset I1PtClm50SpGs --mach cheyenne # Next setup as normal cd $MYCASE ./case.setup # If you need to customize your list of fields uncomment and do the following... # cp CaseDocs/datm.streams.txt.CLM1PT.CLM_USRDAT user_datm.streams.txt.CLM1PT.CLM_USRDAT # chmod u+w user_datm.streams.txt.CLM1PT.CLM_USRDAT # $EDITOR user_datm.streams.txt.CLM1PT.CLM_USRDAT # ./preview_namelists # Finally build, and run the case as normal ./case.build ./case.submit
To run on cheyenne, you do the following:
#!/bin/bash cd .. . /glade/u/apps/ch/opt/lmod/7.2.1/lmod/lmod/init/sh module load ncl ./buildtools cd test ./run_PTCLM_tests