(this is still a draft, some items need to be fixed)
All the disk space available on the cluster is mounted off a dedicated device (aka appliance or server), a NetApp filer.
The current disk configuration is as follow:
Maximum | Quotas per user | NetApp | |||
---|---|---|---|---|---|
disk | disk space |
| snapshots | ||
Disk name | capacity | soft/hard | soft/hard | enabled? | What disk shall I use? |
|
| 50/ | 1.8/2M |
| For your basic configuration files, scripts and job files - your limit is low but you can recover old stuff up to 4 weeks. |
/ |
|
| 4/5M |
| For the bulk of your storage - your limit is high, and disk space is released right away, for SAO users. |
|
| 1.8/2.0TB | 1.8/2M |
| For the bulk of your storage - your limit is high, and disk space is released right away, for non-SAO users. |
|
|
| 1/2M |
| For important but relatively small files like final results, etc. - your limit is medium, you can recover old stuff, but disk space is not released right away. For SAO users |
|
| 1.0/2.0TB | 1/2M |
| For important but relatively small files like final results, etc. - your limit is medium, you can recover old stuff, but disk space is not released right away. For non-SAO users. |
|
|
| 1/1M |
| If you need more than what you can keep in - SAO/non-SAO user should use The FIFO model (first in first out) purging has yet to be implemented as we tune the system. |
The sizes of the file systems (aka the disks) on the NetApp will "auto-grow" until they reach the listed maximum capacity, so the size shown by the command df
does always not reflect the maximum size.
To prevent the disks to fill up and hose the cluster:
inodes
": the sum of number of files and number of directories).The Linux command quota
is not (yet) working with the NetApp filer. We compile a daily quota report and provide tools to query the quotas and parse the quota report. (need to insert links to these tools)
Once we secure more space for /scratch
, we will implement a FIFO (first in first out) model, where old files are deleted without warning to make space.
/scratch
from filling up by running a scrubber regularly.
Some of the disks on the NetApp filer have the so called "snapshot mechanism" enabled:
To recover an old version or a deleted file, foo.dat, that was (for example) in /data/genomics/frandsen/important/results/
:
cd /data/genomics/.snapshot/XXXX/frandsen/important/results cp -pi foo.dat /data/genomics/frandsen/important/results/foo.dat
cd /data/genomics/.snapshot/XXXX/frandsen/important/results cp -pi foo.dat /data/genomics/frandsen/important/results/old-foo.dat
-p
will preserve the file creation date and the -i
will prevent overwriting an existing file. XXXX
is to be replaced by either:hourly.YYYY-MM-DD_HHMM
daily.YYYY-MM-DD_0010
weekly.YYYY-MM-DD_0015
YYY-MM-DD
is a date specification (i.e., 2015-11-01
).snapshot
are read-only:cp
, tar
or rsync
; butmv
) or deleted (rm
).The following tools can be used to monitor disk usage
(more to come)
(more to come)
Last Updated SGK.