GFDL - Geophysical Fluid Dynamics Laboratory

Skip to main content

Moab Tips

Submitting Jobs with msub

  • msub -q eslogin scriptname to run scripts on the login nodes.
  • msub -q batch scriptname to run scripts in the batch queue.
  • msub -q ldtn scriptname (or msub -l partition=es -q ldtn scriptname) to run jobs on the local data transfer nodes.

  • msub -I to request an interactive session (default gives you 24 cores for 1 hour)
  • msub -I -l walltime=2:00:00,size=48 to request an interactive session with 48 cores for 2 hours
  • msub -I -X -l walltime=2:00:00,size=48 to request an interactive session with 48 cores for 2 hours and forward xwindow sessions

  • msub -l partition=c1ms to submit a job to the Gaea compute resources
  • msub -l partition=t1ms to submit a job to the TDS (test and development system)
  • msub -V -l partition=gfdl to submit a job to GFDL's local compute resources

Checking Jobs, System Information

  • showq like qstat, lists jobs in the system. It has a -u option like qstat and a -v option which will show you the partition information.
  • showq -n show the job name instead of the job number
  • showq -n -v show the job name and job number, with a wider column
    • Documentation of options by Garrett Power: ShowqExplanation
    • Utility to print job number, job name, number of processors, and job state: ~Amy.Langenhorst/local/bin/mq

  • checkjob jobnumber is like qstat -f jobnumber
  • checkjob -v jobnumber verbose, more complete job information, including the stdout file location
    • Utility to print the job stdout path given the job number: ~Amy.Langenhorst/local/bin/mstdout JOBID

  • mdiag -f to show available queues

Job Control

  • mjobctl -c jobnumber cancels a job
  • mjobctl -h jobnumber places a job on hold
  • mjobctl -u jobnumber removes a hold from a job

Inserting Scheduler Options in Scripts

Here is an example of msub options at the top of a script. 
#PBS -o [LOCATION FOR STDOUT]

#PBS -N [JOBNAME]

#PBS -q batch

#PBS -v [VARIABLE]=[VALUE]

#PBS -A [PROJECT NAME]

#PBS -l walltime=12:00:00

#PBS -l size=[NUMBER OF CORES]

#PBS -m abe

#PBS -j oe

  • -q batch the queue to send the job to
  • -v [VARIABLE]=[VALUE] sets a variable that can be used in the job script
  • -m abe send email to the user on abort, begin, and end of running the job
  • -j oe write script stdout and stderr to same file