1.7.1. What is PTCLMmkdata?

PTCLMmkdata (pronounced Pee-Tee Cee-L-M make data is a Python script to help you set up PoinT CLM simulations.

It runs the CLM tools for you to get datasets set up, and copies them to a location you can use them including the changes needed for a case to use the dataset with namelist and XML changes.

Then you run create_newcase and point to the directory so that the namelist and XML changes are automatically applied.

PTCLMmkdata has a simple ASCII text file for storing basic information for your sites.

We also have complete lists for AmeriFlux and Fluxnet-Canada sites, although we only have the meteorology data for one site.

For other sites you will need to obtain the meteorology data and translate it to a format that the CESM datm model can use.

But, even without meteorology data PTCLMmkdata is useful to setup datasets to run with standard CLM_QIAN data.

The original authors of PTCLMmkdata are: Daniel M. Ricciuto, Dali Wang, Peter E. Thornton, Wilfred M. Post all at Environmental Sciences Division, Oak Ridge National Laboratory (ORNL) and R. Quinn Thomas at Cornell University. It was then modified fairly extensively by Erik Kluzek at NCAR. We want to thank all of these individuals for this contribution to the CESM effort. We also want to thank the folks at University of Michigan Biological Stations (US-UMB) who allowed us to use their Fluxnet station data and import it into our inputdata repository, especially Gil Bohrer the PI on record for this site.

1.7.2. Details of PTCLMmkdata

To get help on PTCLM2_180611 use the “–help” option as follows.

> cd $CTSMROOT/tools/PTCLM
> ./PTCLMmkdata --help

The output to the above command is as follows:

 Usage: PTCLM.py [options] -d inputdatadir -m machine -s sitename

 Python script to create cases to run single point simulations with tower site data.

    --version             show program's version number and exit
    -h, --help            show this help message and exit

Required Options:
  -d CCSM_INPUT, --csmdata=CCSM_INPUT
                      Location of CCSM input data
                      Machine, valid CESM script machine (-m list to list valid
  -s MYSITE, --site=MYSITE
                      Site-code to run, FLUXNET code or CLM1PT name (-s list to list
                      valid names)

Configure and Run Options:
                      Compset for CCSM simulation (Must be a valid 'I' compset [other
                      than IG compsets], use -c list to list valid compsets)
  --coldstart         Do a coldstart with arbitrary initial conditions
                      Unique identifier to include as a prefix to the case name
                      Root CESM directory (top level directory with models and scripts
  --debug             Flag to turn on debug mode so won't run, but display what would
  --finidat=FINIDAT   Name of finidat initial conditions file to start CLM from
  --list              List all valid: sites, compsets, and machines
                      List of namelist items to add to CLM namelist (example:
  --QIAN_tower_yrs    Use the QIAN forcing data year that correspond to the tower
  --rmold             Remove the old case directory before starting
  --run_n=MYRUN_N     Number of time units to run simulation
                      Time units to run simulation (steps,days,years, etc.)
  --quiet             Print minimul information on what the script is doing
                      Name of the group of sites to search for you selected site in
                      (look for prefix group names in the PTCLM_sitedata directory)
  --stdurbpt          If you want to setup for standard urban namelist settings
  --useQIAN           use QIAN input forcing data instead of tower site meterology data
  --verbose           Print out extra information on what the script is doing

Input data generation options:
  These are options having to do with generation of input datasets.  Note: When
  running for supported CLM1PT single-point datasets you can NOT generate new
  datasets.  For supported CLM1PT single-point datasets, you MUST run with the
  following settings: --nopointdata And you must NOT set any of these: --soilgrid
  --pftgrid --owritesrf

  --nopointdata       Do NOT make point data (use data already created)
  --owritesrf         Overwrite the existing surface datasets if they exist (normally
                      do NOT recreate them)
  --pftgrid           Use pft information from global gridded file (rather than site
  --soilgrid          Use soil information from global gridded file (rather than site

Main Script Version Id: $Id: PTCLM.py 47576 2013-05-29 19:11:16Z erik $ Scripts URL: $HeadURL: https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM1_130529/PTCLM.py $:

Here we give a simple example of using PTCLMmkdata for a straightforward case of running at the US-UMB Fluxnet site on cheyenne where we already have the meteorology data on the machine. Note, see the Section called Converting AmeriFlux Data for use by PTCLMmkdata for permission information to use this data. Example 6-1. Example of running PTCLMmkdata for US-UMB on cheyenne

> setenv CSMDATA   $CESMDATAROOT/inputdata
> setenv MYDATAFILES `pwd`/mydatafiles
> setenv SITE      US-UMB
> setenv MYCASE    testPTCLM

# Next build all of the clm tools you will need
> cd $CTSMROOT/tools/PTCLM
> buildtools
# next run PTCLM (NOTE -- MAKE SURE python IS IN YOUR PATH)
> cd $CTSMROOT/tools/PTCLM
# Here we run it using qcmd so that it will be run on a batch node
> qcmd --  ./PTCLMmkdata --site=$SITE --csmdata=$CSMDATA --mydatadir=$MYDATAFILES >& ptclmrun.log &
> cd $CIMEROOT/scripts
> ./create_newcase --user-mods-dir $MYDATAFILES/1x1pt_$SITE --case $MYCASE --res CLM_USRDAT --compset I1PtClm50SpGs
# Next setup, build and run as normal
> cd $MYCASE
> ./case.setup

PTCLMmkdata includes a README file that gives some extra details and a simple example.

PTCLM/README                04/10/2015

PTCLMmkdata is a python tool built on top of CLM tools and CESM scripts
for building datasets to run CLM "I" cases for data from Ameriflux Tower-sites,
or other user-supplied single-point datasets.

Original Authors:

Daniel M. Ricciuto, Dali Wang, Peter E. Thornton, Wilfred M. Post

Environmental Sciences Division, Oak Ridge National Laboratory (ORNL)

R. Quinn Thomas

Cornell University

Modified by:

Erik Kluzek (NCAR)

General Directory structure:

  PTCLM/PTCLMmkdata ----- Main script
  PTCLM/PTCLM_sitedata  - Site data files of 
        static information latitude, longitude, soil info., and PFT information 
        for each site Also different "groups" of site-data lists, and the script to
        convert the transient years landuse_timeseries files into landuse_timeseries text files that
        mksurfdata can use.
  PTCLM/mydatafiles ----- Default location of
        data files that will be created by PTCLMmkdata. Sites will be built
        in their own subdirectories under here. Optionally you can give your
        own location you'd like to use for your data.

  PTCLM/PTCLMsublist --------- Script to submit a list of PTCLM
        sites to the batch que (only setup for a few machines).
  PTCLM/PTCLMsublist_prog.py - Python module to support submit
        list script. Handles command line arguments and such.
  PTCLM/batchque.py ---------- Python module for batch submital.
  PTCLM/buildtools ----------- Script to build the CLM
        tools needed to run PTCLMmkdata (mksurfdata_map and gen_domain). Works on cheyenne.


# For this example I'm running a I1PtClm50SpGs case on cheyenne using  
# CSMDATA in the standard location
# Finally we use the 6-digit AmeriFlux site code for the University of Mich. Biological
# Station US-UMB (data for this station is checked into the inputdata repository).
# I also assume you are using UNIX C-shell, and GNU make is called gmake
setenv CSMDATA   /glade/p/cesm/cseg/inputdata
setenv SITE      US-UMB

setenv MYDATAFILES `pwd`/mydatafiles

# Next build all of the clm tools you will need
# The following script assumes cheyenne, hobart, or izumi for other machines
# you'll need to build each tool by hand
# next run PTCLMsublist which will submit PTCLMmkdata to batch queue (NOTE -- MAKE SURE python, NCO AND NCL IS IN YOUR PATH)
#       PTCLMsublist is only setup for a few batch machines, you'll need to update them to add new machines
#       or create your own batch submission script.
# NOTE: Every day you run PTCLMmkdata it will remake the map called
#       renamemapfiles to rename files with todays creation date.
#       This makes running PTCLMmkdata a reasonable amount of time.
#       However, you can use the script in mydatafiles
qcmd -l walltime=02:00:00 -- ./PTCLMsublist -l $SITE -d $CSMDATA --account=XXXXXXXXX --mach=cheyenne

# NOTE: To submit several sites at once, make the "-l" option a comma delimited
#       list of site names.

# Next copy the towersite meterology datafiles into your $MYDATAFILES space
# (For the US-UMB station you can skip this step as the .build step will bring the data over) 
mkdir $MYDATAFILES/1x1pt_$SITE/CLM1PT_data
# Copy meteorology data NetCDF files into 1x1pt_$SITE sub-directory 
# (with filenames of yyyy-mm.nc)
# The variables assumed to be on the files are: 
# (if other fields are available or with different names this can be changed by
#  adding a user_nl_datm.streams.txt file as we outline below)
# Make sure your data has time with the attribute: calendar="gregorian"

# Make sure the forcing directory points to the location of your data
# (PTCLMmkdata should already do this)

# Then create a case using the data you just created
setenv MYCASE "testPTCLM"
cd $CTSMROOT/cime
setenv CIMEROOT `pwd`
cd $CIMEROOT/scripts
./create_newcase --user-mods-dir $MYDATAFILES/1x1pt_$SITE --case $MYCASE --res CLM_USRDAT --compset I1PtClm50SpGs --mach cheyenne

# Next setup as normal

# If you need to customize your list of fields uncomment and do the following...
# cp CaseDocs/datm.streams.txt.CLM1PT.CLM_USRDAT user_datm.streams.txt.CLM1PT.CLM_USRDAT
# chmod u+w user_datm.streams.txt.CLM1PT.CLM_USRDAT
# $EDITOR user_datm.streams.txt.CLM1PT.CLM_USRDAT
# ./preview_namelists

# Finally build, and run the case as normal