Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • By default, NUMPY is build to use multi-threading, namely some numerical operations will use all the available CPUs on the node it is running on.
    • (warning) This is NOT the way to use a shared resource, like Hydra,
    • The symptom is that your job is oversubscribed.
  • The solution is to tell NUMPY how many threads to use, using the respective module:

    Code Block
    languagebash
    titleserial case
    module load tools/single-thread-numpy

    or

    Code Block
    languagebash
    titlemulti-thread case
    module load tools/mthread-numpy

    use module show <module-name> to see what is done.

    1. Example:

      Code Block
      languagebash
      titledemo-mthread-numpy.job
      #
      # this example uses 4 threads
      #$ -pe mthread 4
      #$ -cwd -j y -o demo-mthread-numpy.log -N demo-mthread-numpy
      #
      echo + `date` $JOB_NAME started on $HOSTNAME in $QUEUE with id=$JOB_ID
      echo NSLOTS = $NSLOTS
      #
      module load tools/mthread-numpy
      python my-big-data-cruncher.py
      #
      echo = `date` $JOB_NAME done.


...

...

Last update 19 Nov     SGK/MPK