Versions Compared

Old Version 8

changes.mady.by.user Korzennik, Sylvain

Saved on

compared with

New Version Current

changes.mady.by.user Korzennik, Sylvain

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Hardware
    • Add 16 new nodes (40c/380GB) for a total of 640 cores,
    • Decommission oldest nodes (old compute-6* and compute-4-*),
    • Add 1.5PB of high performance storage (parallel file system GPFS, aka Spectrum Scale),
    • This is in addition to the 500TB of NetApp and the  950TB of near-line storage (NAS),
    • The /scratch disk will be moved to the GPFS and increase in size from 100TB to 900TB (450TB+450TB for /scratch/genomics and /scratch/sao)
    • Move all the nodes to a 10Gb/s Ethernet connection (was 1Gb/s).

...

Upgrade schedule is 8/26 to 9/2/2019

Note
titlePlease Note
  • While we are making every effort to set up the new configuration as backward compatible as possible, there will be changes.
  • The upgrade will take place from August 26th through September 2nd, 2019.
  • During this time, Hydra will be inaccessible to users and as of 9am on Monday August 26th any remaining running jobs will be killed.
  • We hope to have Hydra back up before September 2nd, but that decision won’t be made until the upgrade work is completed.
  • During the down time, access to files stored on Hydra will be limited, and at times unavailable. Note that none of your files will be deleted.

Plan your use of Hydra accordingly.

...

  • For biology and genomics software issues and/or incompatibilities that arise, please contact SI-HPC@si.edu,
  • SAO users please contact Sylvain at hpc@cfa.harvard.edu.

...

List of Software Changes 

...

  • Job Scheduler
    • We are switching to UGE, or Univa Grid Engine; that
      • is backward compatible with SGE;
      • offers some additional features;
      • Note that the output of some commands will look different, and

      • some have different options 

  •  The list of queues and their limits will not changed,
    • a few complex values will change (to use local SSD and GPU usage)

    • (warning) One important change (warning) is that 

      Code Block
      languagetext
      titlethe memory reservation value will no longer be specified per slots, but per jobs:
      the specification  "-pe mthread 10 -l mres=20G,h_data=20G,h_vmem=20G,himem"
must be changed to "-pe mthread 10 -l mres=200G,h_data=200G,h_vmem=200G,himem"
to reserve 200GB of memory for this job


  • Compilers
    • The default compiler versions will change, and
    • they had to be reloaded.
    • For MPI jobs the compiler/flavor/version combos will change.

...