Versions Compared

Old Version 77

changes.mady.by.user Korzennik, Sylvain

Saved on

compared with

New Version 78

changes.mady.by.user Korzennik, Sylvain

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

and the explanation of these parameters can be found in

   % man 5 sge_conf

 


To check the queue specific resource limits, use

...

You can also inquire about a specific resource (qquota -l mem_res), and use the PERL wrapper qquota.pl tolocal tools (module load tools/local) qquota+ to

    • get a nicer printout of the reserved memory,
    • get the % of usage with respect to its limit

like in

   % ~hpc/sbin/qquota.pl qquota+ +% -l slots -u hpc

(more info via qquota+ -help or man qquota+.)

To check the limits of a specific queue (CPU and memory), use

...

  • You can limit how many jobs you submit with the following trick:

    Code Block
    languagebash
    titleHow to limit the number of jobs submitted, using C-shell syntax
    # define how many jobs to queue 
@ NMAX = 250 
#
loop:
  @ N = `qstat -u $USER | tail --lines=+3 | wc -l`
  if ($N >= $NMAX) then
    sleep 180
    goto loop
  endif
#

    This example counts how many jobs you have in the queue (running and waiting) using the command qstat  (and tail and  wc -l) and pauses for 3 minutes (180 seconds) if that count is 250 or higher.

    You would include these lines in a script that submits a slew of jobs, but should not queue more than a given number at any time (to count only the queued jobs, add -s p to qsub).

  • Or you can use the tool q-wait (needs the module tools/local), that takes an argument and two options:
       % q-wait blah
    will pause until you have no job whose name has the string 'blah' left queued or running.
  • The options allow you to specify the number of jobs, and how often to check, i.e.:
       % q-wait -N 125 -wait 3600 crunch
    will pause until there are 250 or fewer jobs whose name has the string 'crunch' left queued or running, checking once an hour.

  • (warning) Avoid using the -V flag to qsub
    • The -V flag passes all the active environment variables to the script.
    • While it may be convenient in some instances, it creates a dependency on the precise environment configuration when submitting the job,
      thus the same job script may fail when it is submitted at a later time (or from a different log in) from a different configuration. 

Anchor
Examples
Examples
8. Examples

You can find examples of simple/trivial test cases with source code, Makefile, job script files and resulting log files under ~hpc/examples.

  • The examples are organized as follows:
     

     miscmisc/

    		miscellaneous
    serial/simple (hello world) serial job
    mpi/using MPI
    openmp/using OpenMP
    idl/running IDL
    gdl/running GDL
    java/running JAVA
    lapack/linking with LAPACK and Intel's MKL
    memtest/large memory use and reservation

    c++11/ 

    		using the C++11 extension

    You can use the command find to get a list of all the subdirectories under ~hpc/examples, i.e.: 

       % find ~hpc/examples -type d -print

...

If your computation will useyour job script needs toqsub option needed/recommended
more than one CPU (parallel jobs need)request a PE and N slots-pe <pe-name> N or -pe <pe-name> N-M
more than 2GB/CPU of memoryreserve the required memory-l mres=X,h_data=X,h_vmem=X
more than 6GB/CPU of memory

use a high-memory queue, and

reserve the required memory

-l mres=X,h_data=X,h_vmem=X,himem
up to T hours of CPU (per CPU)specify the required amount-l s_cpu=T:0:0

 or specify the queue-q mThC.q
no idea how much CPUuse an unlimited, low priority queue-q uThC.q -l lopri

...

Use the command qconf -srqs or qquota, see how to check under resource limits.

The local tool check-qwait allows you to visualize the queue quota resources and which jobs are waiting.

...

Last Updated  SGKUpdated   SGK