Welcome the Smithsonian Institution High Performance Computing Wiki.

This Wiki holds information for the use of HPC resources at the Smithsonian.

Central to the SI HPC resources is the Smithsonian Institution High Performance Cluster (SI/HPC), named Hydra.

  • High performance computing is administered by the Office of Research Computing within ODT and in collaboration with OCIO.
  • OCIO's Herndon Data Center in Herndon, VA houses the high performance computing cluster, Hydra.

What's New

  • Jul 31, 2025
    • Hydra will be shut down Wednesday September 10 at 9am ET to be moved to the new data center

    • All running and queued jobs will be deleted and logins to either login nodes disabled. During the move there will be no access to any of Hydra's resources, including compute nodes, R server, Globus, or any file stored on Hydra. If you anticipate needing access to some of these files, now is the time to copy them somewhere else.

    • If all goes well we anticipate that the move will be done by Monday September 22, although we can't guarantee it. This is a complex operation involving over a 100 pieces of equipment that need to be physically moved to a new location. We will send an update during the week of September 15.

    • In preparation for the move we will soon start making a second copy of the data stored on Hydra on our new GPFS system. This may slow down I/O operations on the disk systems being copied.

    • Also, a lot is stored on Hydra, hence we are asking you to delete anything you no longer need ASAP. 

    • All scrubbed files will be permanently deleted ten days after being scrubbed, except for what is scrubbed during the last 2 weeks prior to the shutdown, so do not delay requests to restore files.

  • May 21, 2025

    • OCIO had to decommission the listserv we used to communicate with our users (aka HPCC-L) as of Friday 5/16 at COB.

    • We have set up a Google group email hosted at SAO (hppc-l@cfa.harvard.edu) to send such emails.

      • You should have received a test email saying "New HPCC distribution list/method"

      • Please make sure that your mail reader is not sending those to spam.

    • This is a moderated groups email, hence users can post messages/questions to everyone on the list, and
      • the group moderator will distribute the message if deemed appropriate.  
    • As before, you can still contact the Hydra support team via the following email addresses, based on your needs:
        SI-HPC-Admin@si.edu  for SysAdmin related issues,
        SI-HPC@si.edu             for Bioinformatic/Genomics questions,
        hpc@cfa.harvard.edu    for SAO/CfA users who need help.
  • May 8, 2025

    • R Studio Server in a Browser

      • The Hydra Team is excited to announce a new interactive R Studio environment that is accessible directly via a browser, at https://galaxy.si.edu/R4.

      • Hydra users can leverage this server to test, debug, and develop R based workflows using the interactive R Studio GUI (currently running R 4.4.3).

      • By logging in with your Hydra account credentials, users will have access to the storage under /pool , /scratch  and /store .

        • This server offers resources totaling 192 CPUs and 1.5 TB of RAM.

NOTES:

      • This is a shared resource and should be used accordingly.
        • Long running jobs or jobs requiring the entire resources of the server would be more appropriate as a job submission.
      • This server is only accessible from trusted computers, not on the public internet.
      • This is a new resource - please be patient as we test this offering with our user community.
        • We will evaluate this test once Hydra is moved to the new data-center and decide whether it should be kept or altered in any way.
  • Mar 5, 2025
    • Hydra will be down while moving to a new data center location.
    • Current best guess
      • a 2 to 4 weeks downtime,
      • sometime between June July and September 2025.
    • See updates under 2025 Data Center Move
  • Feb 13, 2025
    • We have increased the limit per user on the number of concurrent interactive sessions from one to four (still up to 6 slots/CPUs/cores). 
    • (lightbulb) We may reduce this number if the interactive queue gets filled or may decide to add more nodes to that queue.
  • Feb 10, 2025
    • To address a problem that arose lately (i.e., job that creates an excessive number of threads), we have changed a section of Hydra's Usage Policy.
  • Nov 22, 2024
    • MATLAB runtime R2024a and R2024b are now available,
    • load the matlab/R2024b or matlab/2024a module to access them.
  • Nov 7, 2024
    • The command dos2unix is accessible without the need to load any module and the man page is available (man dos2unix).
    • We have added a workflow manager ("WFM") special queue, please consult the relevant documentation.
  • Oct 28, 2024
    • A new web-based SSH terminal (WeTTY) for accessing Hydra through a browser (e.g., https://telework.si.edu). 

      This new interface uses improved website technology for a better user experience: faster display of text to the screen; better support for pasting text into the browser, etc. compared to the web-based SSH interface we have been offering until now (shell-in-a-box).

      Both the new and old systems are currently available, either on the Hydra landing page (https://hydra.si.edu) or via the "Hydra" option at https://telework.si.edu. 
      The new system is labeled "New Web SSH terminal (WeTTY)".

      If you're a user of the web-based SSH terminal, please try out the new system and let us know if you encounter any issues.
      We will remove the old interface by December 1st if no major issues are identified.

  • Oct 26, 2024
    • We have added a page explaining the automatic emails sent to users.
  • Sep 16, 2024

    • We have installed the following:

      • gnuplot on all the compute nodes (v5.2p4), not just the login nodes;
      • the latest version of awscli (v 2.17.43):
        • to access it load the tools/awscli module,
        • version 2.15.27 is available via the tools/awscli/2.15.27 module;
      • newest version of the compilers:
        • gnu 14.2.0 via the gcc/14.2.0 module,
          • the default version remains 8.5.0;
        • Nvidia 24.3 and 24.5, via the nvidia/24.3 or nvidia/24.5 modules,
          • the default version remains 23.9;
        • Intel 2024.1 and 2024.2, via the intel/2024.1 and intel/2024.2 modules,
          • the default version remains 2024.0.
      • The various MPI flavors (vendor's, OpenMPI and MVAPICH) are available for each compiler version, including several versions whenever possible.
        • refer to the module list, or use:
          • module whatis gcc/14.2/ 
          • module whatis nvidia/24/
          • module whatis intel/24/
      • The Nvidia compilers come with two different versions of cuda: 11.8 and 12.x.
      • The Intel compilers come with Intel's distribution of python and conda.

*: these pages are only accessible from trusted machines: within SI or SAO networks, or using VPN.

 Reminder

  • References to Hydra (publications, proposals, etc.) should mention the cluster as:

Smithsonian Institution High Performance Computing Cluster. Smithsonian Institution. https://doi.org/10.25572/SIHPC.

  • or as an acknowledgment:

"(Some of) [Tt]he computations in this paper were conducted on the Smithsonian High Performance Cluster (SI/HPC), Smithsonian Institution. https://doi.org/10.25572/SIHPC". 


Last updated  SGK/MPK.

Search this documentation


Recently Updated Pages

Show More Please wait