1.9.1. Testing

Technically, you could use the customization we gave in Setting Up and Running a Case to test various configuration and namelist options for CLM. Sometimes, it’s also useful to have automated tests though to test that restarts give exactly the same results as without a restart. It’s also useful to have automated tests to run over a wide variety of configurations, resolutions, and namelist options. To do that we have several different types of scripts set up to make running comprehensive testing of CLM easy. There are two types of testing scripts for CLM. The first are the CESM test scripts, which utilize the create_newcase scripts that we shown how to use in this User’s Guide. The second are a set of stand-alone scripts that use the CLM configure and build-namelist scripts to build and test the model as well as testing the CLM tools as well. Below we will go into further details of how to use both methods.

1.9.1.1. CIME Testing scripts

We first introduce the test scripts that work for all CESM components. The CIME script create_test runs a specific type of test, at a given resolution, for a given compset using a given machine. See CIME Chapter on Testing for how to use it to run single tests as well as lists of tests. The standard testname for CLM is “aux_clm” for cheyenne with intel and gnu compilers as well as the CGD machine hobart for intel, nag, and pgi compilers. There’s also a shorter test list called “clm_short”. Also see the CTSM Wiki on Testing.

1.9.1.2. CTSM Tools Testing

$CTSMROOT/clm/test/tools/README                                 06/08/2018

Scripts for testing the CLM support tools with many different 
configurations and run-time options.

I.   MAIN SCRIPTS:

test_driver.sh - Test the CLM offline tools

To use...

./test_driver.sh -i

on Derecho

qcmd -l walltime=08:00:00 -- ./test_driver.sh -i >& run.out &

And to for example to compare to another baseline code (in this case ctsm5.1.dev066, which would need to be cloned at the given
path) ...

qcmd -l walltime=08:00:00 -- env BL_ROOT=/glade/scratch/erik/ctsm5.1.dev066 ./test_driver.sh -i >& run.out &

on izumi

nohup ./test_driver.sh -i >& run.out &

release tests

qcmd -l walltime=10:00:00 -- env CLM_INPUT_TESTS=`pwd`/tests_posttag_nompi_regression \
./test_driver.sh -i >& run_regress.out &

To run neon-specific tests, please use login nodes:
env CLM_INPUT_TESTS=`pwd`/tests_pretag_nompi_neon ./test_driver.sh -i > & run_neon.out &


Intended for use on NCAR machines Derecho, Casper (DAV) and izumi.

II. RUNNING test_driver.sh TOOLS TESTING:

Basic use:

./test_driver.sh -i
./test_driver.sh -h   # to get help on options
 
Important environment variables (just used by test_driver.sh)

BL_ROOT ---------------- Root directory of CLM baseline code to compare to 
                         (if not set BL test will not be performed)
BL_TESTDIR ------------- Root directory of where to put baseline tests
CLM_INPUT_TESTS -------- Filename of file with list of tests to perform
CLM_TESTDIR ------------ Root directory of where to put most tests
CLM_RETAIN_FILES ------- If set to TRUE -- don't cleanup files after testing
CLM_FC ----------------- Use given compiler
CLM_JOBID -------------- Job identification number to use (rather than process ID)
CLM_THREADS ------------ Number of open-MP threads to use 
                         (by default this is set differently by machine)
CLM_SOFF --------------- If set to TRUE -- stop on first failed test (default FALSE)

Important files for test_driver tools testing:

test_driver.sh ------- Main test script for tools
nl_files ------------- Directory with various namelists to test
config_files --------- Directory with various configurations to test
input_tests_master --- Master list of tests
tests_pretag_* ------- Tests for specific machines to do by default before a tag is done
tests_posttag_* ------ Tests for specific machines to do for more extensive testing 
                       after a tag is done
CLM_compare.sh ------- Compares output history files between two cases
T*.sh ---------------- Basic test script to do a specific type of test
gen_test_table.sh ---- Creates HTML table of tests
Makefile ------------- Will build the HTML table of tests

../../tools/README.testing - Information on how the testing works for the CLM tools

1.9.1.3. CTSM Fortran Unit Tests

# To run all CTSM unit tests, run the following command:
#
# This reuses the existing 'unit_tests.temp' directory (if present) in
# order to do an incremental rebuild of the unit tests from the last
# time you ran them from this directory.

# From a standalone CTSM checkout:
../cime/scripts/fortran_unit_testing/run_tests.py --build-dir unit_tests.temp

# If you are within a full CESM checkout, you would instead do:
# ../../../cime/scripts/fortran_unit_testing/run_tests.py --build-dir unit_tests.temp

1.9.1.4. CTSM Build-namelist Tests

Test the namelist build script by running the following:

> cd bld/unit_testers
> ./build-namelist_test.pl 1>namelist_test.log 2>&1

When that’s complete, inspect namelist_test.log (e.g., with less namelist_test.log). If you see Successfully ran all testing for build-namelist but nothing like # Looks like you failed 4 tests of 1999., then everything went fine.

If something went wrong, you can find the failing tests like so:

> grep -E "^[0-9]+/[0-9]+ < [a-zA-Z]+" namelist_test.log | grep -v "PASS"

1.9.1.5. Testing PTCLM

To run on cheyenne, you do the following: