1.3.6. Creating Surface Datasets

When just creating a replacement file for an existing one, the relevant tool should be used directly to create the file. When you are creating a set of files for a new resolution there are some dependencies between the tools that you need to keep in mind when creating them. The main dependency is that you MUST create a SCRIP grid file first as the SCRIP grid dataset is then input into the other tools. Also look at Table 1.4.3.1.1 which gives information on the files required and when. Figure 1.3.2 shows an overview of the general data-flow for creation of the fsurdat datasets.

Figure 1.3.2 Data Flow for Creation of Surface Datasets from Raw SCRIP Grid Files

Starting from a SCRIP grid file that describes the grid you will run the model on, you first run `mkmapdata.sh to create a list of mapping files. See Figure 1.3.1 for a more detailed view of how mkmapdata.sh works. The mapping files tell mksurfdata_esmf how to map between the output grid and the raw datasets that it uses as input. The output of mksurfdata_esmf is a surface dataset that you then use for running the model. See Figure 2.27.3 for a more detailed view of how mksurfdata_esmf works.

Figure 1.3.3 is the legend for this figure (Figure 1.3.2) and other figures in this chapter (Figure 1.3.4 and Figure 1.3.5).

Figure 1.3.3 Legend for Data Flow Figures

Green arrows define the input to a program, while red arrows define the output. Cylinders define files that are either created by a program or used as input for a program. Boxes are programs.

You start with a description of a SCRIP grid file for your output grid file and then create mapping files from the raw datasets to it. Once, the mapping files are created mksurfdata_esmf is run to create the surface dataset to run the model.

1.3.6.1. Creating a Complete Set of Files for Input to CLM

1. Create SCRIP grid datasets (if NOT already done)

   First you need to create a descriptor file for your grid, that includes the locations of cell centers and cell corners. There is also a “mask” field, but in this case the mask is set to one everywhere (i.e. all of the masks for the output model grid are “nomask”). An example SCRIP grid file is: $CSMDATA/lnd/clm2/mappingdata/grids/SCRIPgrid_10x15_nomask_c110308.nc. The mkmapgrids and mkscripgrid.ncl NCL script in the $CTSMROOT/tools/mkmapgrids directory can help you with this. SCRIP grid files for all the standard CLM grids are already created for you. See the Section called Creating an output SCRIP grid file at a resolution to run the model on for more information on this.

Todo

Update the below, as domain files aren’t needed with nuopc.

2. Create domain dataset (if NOT already done)

   Next use gen_domain to create a domain file for use by DATM and CLM. This is required, unless a domain file was already created. See the Section called Creating a domain file for CLM and DATM for more information on this.

3. Create mapping files for mksurfdata_esmf (if NOT already done)

   Create mapping files for mksurfdata_esmf with mkmapdata.sh in $CTSMROOT/tools/mkmapdata. See the Section called Creating mapping files that mksurfdata_esmf will use for more information on this.

4. Create surface datasets

   Next use mksurfdata_esmf to create a surface dataset, using the mapping datasets created on the previous step as input. There is a version for either clm4_0 or CTSM1 for this program. See the Section called Using mksurfdata_esmf to create surface datasets from grid datasets for more information on this.

5. Enter the new datasets into the build-namelist XML database The last optional thing to do is to enter the new datasets into the build-namelist XML database. See Chapter 3 for more information on doing this. This is optional because the user may enter these files into their namelists manually. The advantage of entering them into the database is so that they automatically come up when you create new cases.

The $CTSMROOT/tools/README.md goes through the complete process for creating input files needed to run CLM. We repeat that file here:

# CTSM Tools for Preprocessing of Input Datasets or Postprocessing of History Output
#### $CTSMROOT/tools/README

CTSM tools for analysis of CTSM history files -- or for creation or
modification of CTSM input files.

I.  General directory structure:

    `$CTSMROOT/tools`
        mksurfdata_esmf -- Create surface datasets.

        crop_calendars --- Regrid and process GGCMI sowing and harvest date files for use in CTSM.

        mkmapgrids ------- Create regular lat/lon SCRIP grid files

        site_and_regional  Scripts for handling input datasets for site and regional cases.
                           These scripts both help with creation of datasets using the
                           standard process as well as subsetting existing datasets and overwriting
                           some aspects for a specific case.

        modify_input_files Scripts to modify CTSM input files. Specifically modifying the surface
                           datasets and mesh files.

        contrib ---------- Miscellaneous tools for pre or post processing of CTSM.
                           Typically these are contributed by anyone who has something
                           they think might be helpful to the community. They may not
                           be as well tested or supported as other tools.

II. Notes on building/running for each of the above tools:

    mksurfdata_esmf has a cime configure and CMake based build using the following files:

        gen_mksurfdata_build ---- Build mksurfdata_esmf
        src/CMakeLists.txt ------ Tells CMake how to build the source code
        Makefile ---------------- GNU makefile to link the program together
        cmake ------------------- CMake macros for finding libraries

    mkmapgrids and site_and_regional only contain scripts that do not need build files.

    Some tools have copies of files from other directories -- see the README.filecopies
    file for more information on this.

    Tools may also have files with the directory name followed by namelist to provide sample namelists.

        <directory>.namelist ------ Namelist to create a global file.

    These files are also used by the test scripts to test the tools (see the
    README.testing) file.

> [!NOTE]
> Be sure to change the path of the datasets referenced by these namelists to
> point to where you have exported your CESM inputdata datasets.

III. Process sequence to create input datasets needed to run CTSM

    1. Create ESMF MESH grid files (if needed)

       a. For standard resolutions these files will already be created. (done)

       b. Run `tools/site_and_regional/subset_data point` to create single-point datasets

       This creates just the fsurdat file as MESH files are NOT needed for single-point cases.

       c. Run `tools/site_and_regional/subset_data region` to create regional datasets subset from a global dataset

       This creates both the fsurdat file and MESH file needed to run.

       d. General custom grid

        You'll need to convert or create MESH grid files on your own (using scripts
        or other tools) for the general case where you have an unstructured grid, or
        a grid that is not regular in latitude and longitude, and that grid is custom
        and not merely subset from one of the global grids.

    2. Create surface datasets with mksurfdata_esmf on Derecho
        (See mksurfdata_esmf/README.md for more help on doing this)

       - gen_mksurfdata_build to build
       - gen_mksurfdata_namelist to build the namelist
       - gen_mksurfdata_jobscript_single to build a batch script to run on Derecho
       - Submit the batch script just created above

       - This step uses the results of step (1) entered into the XML database.
       - If datasets were NOT entered into the XML database, set the resolution
         by entering the mesh file using the options: --model-mesh --model-mesh-nx --model-mesh-ny

       Example: for 0.9x1.25 resolution for 1850

``` shell
       # On Derecho
       cd mksurfdata_esmf
       ./gen_mksurfdata_build
       ./gen_mksurfdata_namelist --res 0.9x1.25 --start-year 1850 --end-year 1850
       ./gen_mksurfdata_jobscript_single --number-of-nodes 2 --tasks-per-node 128 --namelist-file target.namelist
       qsub mksurfdata_jobscript_single.sh
```

    3. Add new files to XML data or using user_nl_clm (optional)

       See notes on doing this in step (1) above.

IV.  Notes on which input datasets are needed for CTSM

       global or regional grids
         - need fsurdata
         - need mesh files in env_run.xml ATM_DOMAIN_MESH and LND_DOMAIN_MESH

       single-point grids
        - Just need fsurdata