Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • The accounting information is at regular intervals in a data base.
    • The ingestion is run every few five minutes, so the database is not instantaneously synchronized.
  • That database can be queried with qacct+
    • The output of qacct+  can be better controlled than the output of qacct, and
    • qacct+  computes derived values.
  • How to use qacct+ is explained in the man page (man qacct+) and is described by:

...

Code Block
languagetext
titleqacct+ -help
usage: qacct+ [options] where options are  
  
  -j     <jobid>                    limit to
a          -j     <jobid>    given job  ID ((s),    single value, comma separated list, OK)or range like in n:m
         -t     <taskid>   limit to a given task ID(s), or a range ofsingle taskvalue IDor (if range like in n:m)
         -o     <owner>    limitgiven toowner, a given owner      exact match
         -q     <queue>    limit to given queue,        ( regexp okOK, like ".ThC.q")
         -peN    <PE> <name>     given limitjob to given PEname,      regexp OK, like "crunch*"
         -pe    <pe>       given pe,           (like mthread)  
  -from   exact match, like mthread
         -xSQL  <string>   extra SQL constrain        like 'slots <= 2'

         -nSearch <n>      jobs that ended >= (now - n) days, def. 31
         -since <date>     limitjobs to submitted  >= date,   like "4/27/2016 10:00AM"
    -to     -from  <date>     limitjobs tothat submittedended <>= date,   like "4/27/2016 1110:00AM"
       -max  -until <number><date>   max number ofjobs entriessubmitted to show<= date,    -wdt like "4/27/2016 11:00AM"
 <number>   width of columns in tabular mode-to (def: 15)  <date>   -f  jobs that ended <dbname><= date,  use the given DB file name, (def: /home/hpc/cron/data/dbs/qacct+.db)like "4/27/2016 11:00AM"

          -all              show all that match, not just the latest
-v         -max   <number>   max number of entries to show

         -v[=value]        verbose mode (repeat to increase verbosity, -v=2 equiv to -v -v)
         -n                dry run, use w/ -v to check the resulting SQL search
         -dateFmt <n>      how to format dates, n=-1,0,1 for YYYY/MM/DD, Mon DD YYY, MM/DD/YYYY, def. 0

   -help      -db    <dbname>   use the given DB [engine:name]
 shows this help      -init  <file>     mysql initialization file, def. /scratch/admin/mysql/qacct+/.my.cnf, /home/sylvain/.my.cnf

         -show  <string>   specify what to show        (  def.: -show simple)
                      where <string> can be "simple", "simple+", "tab", "tab+", or "raw",
                                         or "help", "fields" or "stats",
                                        or a custom specification
                           use -show help to get additional help on this option
       Ver 1.5/1 Oct 22 2019

 -wdt   <number>   width of columns in tabular mode def: 15)

         -help             shows this help
  • and by
Code Block
languagetext
titleqacct+ -show help
  -show  <string> specify what to show  
  
  i.e.     The  default  output (-show) is 'simple', it foris simplea formatkeyed output (def.i.e., field = output) with only a subset of the -show simple+ for available
       fields, in a "friendly" output.

       There are three other preset output formats: 'simple+', 'tab', 'tab+', 'raw' and 'stats'.

          'simple+' is an extension of the 'simple': formatmore fields,
     -show     'tab'     foris a tabular formatoutput: one line per accounting entry,
 -show          'tab+'    foris an extension of the tabular format 'tab': more fields,
          'raw'     returns everything available with no formatting,
          'stats'   return statistics about the DB.

       Moreover, you can customize the output, by specifying which fields to print out and how to format them.

       The option '-show rawfields' lists all the available forfields.
raw
format       The option '-show statshelp' explains how showto onlyuse the DB-show stats<string> option to produce a custom output,  or where  <string>  is  a
       list what keyword to show and anoptionally optionalhow to format where it.

       This string is either
           +field1[=format1][,field2[=format2]]  for keyed format
         or
      @field1     #field1=[format1][,field2[=format2]]  for tabular format

       the  format  is either a C format specification (see man printf) like %d, %s, %.1f,  
  or the values &DATE@DATE, &MEM@MEM, or &AGE@AGE to
convert to date, memory or age  convert the value useto -showa fieldsdate, toa listmemory allor thean availableage. fieldsExamples are:

     like in:       -show @qname#qname,slots,ruelapsed_wallclocktime,cpu=%-15.1f,granted_pe
         or
           -show +qname,slots,ruelapsed_wallclocktime=\&AGE@AGE,cpu=%.1f

...



  • The back-end database server was changed to mysql to speed it up. 

Anchor
Quota+
Quota+
3 A better "quota": quota+

...

The following tools are available when loading the module tools/local;

chage+

substitute to chage, to query LDAP properties

ie: chage+ $USER

check-disks-usage

check disks usage, and print warning when usage exceed a threshold

ie: check-disks-usage -w 10

check-gpuse

print the cluster GPUs usage, for the hosts in the GPU host group


check-hi-memusecheck for memory use versus reservation and CPU usage efficiency, for a specific job or jobs
check-memrescheck for jobs that use less memory than the amount reserved
check-memuseprint the cluster usage (memory and CPUs), for the hosts in a specific host group
check-qlogsreturn a report on either oversubscribed or inefficient jobs, parsing existing log files report
check-qwaitshow job(s) waiting in the queue and associated queue quota limits
disk-usagereturn information on disk usage
find-all-zombiesfind zombies
get-gpu-infoprint information about GPU on the local host or a specific compute node
monitor-code-usagehelps you monitor your code usage
parse-disk-quota-reportsreturn report by parsing the disk quota report
plot-disk-usage plot the usage of a given disk (volume)
plot-qmemuseplot memory and cpu usage as a function of time for a given jobID
plot-qsnapshotplot a snapshot of cluster usage
plot-qssduseplot SSD usage as a function of time for a given jobID
plot-qssduse-summaryplot SSD usage summary as a function of time
plot-uptimeplot the load (uptime) of the head and login nodes
qacct+a "better" qacctshow accounting information for completed jobs
qchainchains a set of jobs by adding the -hold_jid <jobID> to qsub for you
qhost+convenience wrapper around qhost to use the shorthand "NN-MM" instead of typing "-h compute-NN-MM"
qquota+a "better" qquotashow queue quota wrt queue limits
qstat+a "better" qstatshow queue status
quota+a "better" quota quotashow disk quota information for all type (NFS, GPFS, NAS)
q-waitwait until some jobs are not found in the queue
rkillremote kill
rkillallremote killall
rtop+remote top
show-ge-reportingparse and print part of the GE reporting file
show-qmemuseshow  memory use statistics
show-qslotsshows how many slots are available in the queue(s)
show-qssduselog statistics for plot-qssduse

Each tool has a man page, accessible after you load the module tools/local.

...

Each tool has a man page, accessible after you load the module tools/local+.


...

Last Updated  SGKUpdated   SGK.