- 31621240Introduction
- Example of a Job Script
How to Convert a Task ID to a More Useful Set of Parameters
- How to Consolidate Small Jobs into Fewer Larger Jobs Using Job Arrays
- Rules for Submitting Job Arrays that use Parallel Environments (like MPI)
...
| one job script with embedded directives | is replaced by two files, a qsub_XXX.sou and a XXX.sh |
|---|
| Code Block |
|---|
| language | bash |
|---|
| title | demo.job |
|---|
| #--------
#$ -q mThC.q
#$ -pe orte 20
#$ -l mres=4G,h_data=4G,h_vmem=4G
#$ -cwd -y j -N demo -o demo.$TASK_ID.log
#$ -t 1-100
#-------
module load some/thing
#-------
echo + `date` job $JOB_NAME started in $QUEUE with jobID=$JOB_ID on $HOSTNAME
echo + taskID=$SGE_TASK_ID
echo + NSLOTS=$NSLOTS distributed over:
cat $PE_HOSTFILE
#
mpirun -np $NSLOTS crunch -i input.$SGE_TASK_ID -o output.$SGE_TASK_ID
#
echo = `date` job $JOB_NAME done |
|
| Code Block |
|---|
| language | bash |
|---|
| title | qsub_demo.sou |
|---|
| qsub \
-q mThC.q \
-pe orte 20 \
-l mres=4G,h_data=4G,h_vmem=4G \
-cwd -y j -N demo -o 'demo.$TASK_ID.log' \
-t 1-100 \
-b y $PWD/demo.sh |
 no spaces after the '\'
|
| Code Block |
|---|
| #!/bin/sh
# any embedded directives here will be ignored
#-------
source /etc/profile.d/modules.sh
module load some/thing
#-------
echo + `date` job $JOB_NAME started in $QUEUE with jobID=$JOB_ID on $HOSTNAME
echo + taskID=$SGE_TASK_ID
echo + NSLOTS=$NSLOTS distributed over:
cat $PE_HOSTFILE
#
mpirun -np $NSLOTS crunch -i input.$SGE_TASK_ID -o output.$SGE_TASK_ID
#
echo = `date` job $JOB_NAME done |
 this This can be any type of executable script, the line "but if you use:
#!/bin/sh or #!/bin/bash, you'll need source /etc/profile.d/modules.sh"
is bash/sh specific tho- to access modules,
#!/bin/sh -l or #!/bin/bash -l, module will be defined, but the script will read ~/.profile,#!/bin/csh or #!/bin/tcsh, module will be defined,#!/bin/csh -f or #!/bin/tcsh -f, you'll need source /etc/profile.d/modules.csh to access modules.
 Don't you love Linux? (It's all in the man pages, tho).
|
Before submitting the job array, make sure the script is executable:
...
