Guidelines to long-term data storage
Data files stored in the archive system are handled by a »Hierarchical Storage Manager« (HSM) type of software, specifically developed to manage large quantities of data and utilizing different media types for storage. It is operated on dedicated hardware and is based on the Linux operating system.
The HSM system at LUIS consists of a spinning disk storage array and a robotic tape library. Data files may be kept on disk for a certain amount of time, but will in any event be copied to tape. Upon retrieval, the HSM system copies the files back to disk for easy access. A particular feature is that tape storage is transparent to the user - files remain visible in the file system even if they have been migrated to tape.
Time span of data retention
Data will be retained in the archive system for ten years. This corresponds to common requirements regarding best practice in science.
Storage space byte count quota (user accounts)
Storage capacity space is measured in bytes. There is a byte quota for files stored into the archive system, which is referred to as the byte count quota. This effectively limits how much you can store in your archive system user account.
Currently, the byte count quota parameters are the following:
500 GB (soft quota), with an option to exceed this quota for another 200 GB (hard quota) with a grace period of 30 days
For user acounts, an increase is possible on request, but subject to decision of the service administrator.
File count quota (user accounts)
File count, like byte count in the previous section, is under quota as well. This means you will only be able to store a specific number of files in the archive system, regardless of file size. Even if you have not yet reached the byte count quota, you may find it impossible to create any more files in the system unless you delete already existing files in your archive system user account.
This is necessary to manage tape based storage efficiently within the Hierarchical Storage Manager (HSM) system. Please store fewer but larger files instead of many small ones. To achieve this, it is recommended to use file archiving and compression tools (such as TAR or Zip).
Currently, the file count quota parameters are:
2000 files (soft quota), with an option to exceed this quota for another 200 files (hard quota) with a grace period of 30 days
Group accounts
Group accounts are handled with a different quota. Please contact us for more infomation.