Storage
Types of Storage
Super Computing Wales offers two types of storage: home directories and scratch.
The scratch disks are large, have no quota, are on fast storage but only intended for short term storage of temporary data. Please don’t leave data on the scratch disk once you are finished with it. If the disk fills up it will negatively impact on other users ability to run jobs.
Home directories have a default quota of 50GB per user, they are slower to access and are intended for longer term storage of data. However, please note that you should backup important data elsewhere as we do not formally offer an SCW backup service.
Checking Storage Quotas
Quotas are enabled on Supercomputing Wales home directories.
These limit the amount of storage space and number of files that a user or group can store in their respective home directory location.
To view your allocation, use the ‘myquota’ command, like this:
[ade.fewings@csc001 ~]$ myquota Disk quotas for group ade.fewings (gid 16778693): Filesystem blocks quota limit grace files quota limit grace cfsfs001-s05:/nfsshare/exports/space05 402G 500G 525G 192k 1000k 1050k
In this example, we see six items of data:
- The first set of three items represent storage space:
- ‘blocks’ is the amount of storage space utilised, in this case 402GB
- ‘quota’ is the ‘soft limit’ assigned, in this case 500GB
- ‘limit’ is the ‘hard limit’ assigned, in this case 525GB
- The second set of three items represent file count:
- ‘files’ is the number of files present, in this case 192000
- ‘quota’ is the ‘soft limit’ assigned, in this case 1000000
- ‘limit’ is the ‘hard limit’ assigned, in this case 1050000
A ‘soft limit’ can be exceeded for a period of maximum 7 days, after which utilisation must be reduced below the threshold or error will result. In this case, the ‘grace’ item in the ‘myquota’ output will display the number of days remaining in this state.
A ‘hard limit’ cannot be exceeded and attempts to do so will result in error.
Increases in quota allocations are possible on a case-by-case basis, please contact the Support Desk to request. However, it is worth noting that quotas are implemented to reduce the unfair use of the system by a few users and thus provide capacity for new users.
Shared Areas
We are able to create two types of shared area for multiple users to access:
Project Shared Areas are a way for the users of a particular project to share data. They are separate from user home directories and quotas as such.
Collaboration Shared Areas are also possible that cross project and institutional boundaries as separate areas from home directories for data sharing among multiple users.
Please contact Support to discuss either of these if this would be useful.