Dear HPC user,

On Sep 1, 2024 RC system engineers will reinstate a file purging policy for personal /scratch folders on the Afton and Rivanna high-performance computing (HPC) systems. From Sep 1 forward, files that have not been accessed for over 90 days will be permanently deleted on a daily rolling basis. This is not a new policy; it is a reactivation of an established policy that follows general HPC best practices.

The /scratch filesystem is intended as a temporary work directory. It is not backed up and old files must be removed periodically to maintain a stable HPC environment.

How should I prepare?

We encourage you, if you are using scratch, to identify and move important files to more persistent storage solutions on a regular basis. The FAQ “How can I find out what files will be purged?” provides instructions to identify the scratch files marked for deletion. RC offers several low-cost storage options to researchers, including 10TB of Research Standard storage for each eligible PI at no charge. For more detailed descriptions of our storage options, visit [https://www.rc.virginia.edu/userinfo/storage/].

What to expect on Sep 1, 2024?

Starting on Sep 1, 2024, a script will be launched on the Afton and Rivanna systems to monitor on a daily basis if scratch files have been accessed within the past 90 days. Last access times are determined as the last time a file was opened (read) or modified (written to). Scratch files that have not been accessed since Jun 3 (90 days before Sep 1) will be removed from the filesystem. The 90-day file purging window will move forward on a daily basis.

If you have any questions about the file purging policy or process, you may contact our user services team.

With regards, 

Karsten Siller
Director, Research Computing User Services
Information Technology Services
University of Virginia

Scratch is intended as a temporary work directory, not for long-term storage. It is not backed up, and old files need to be purged periodically to maintain system stability and filesystem performance. This is generally an established best practice at HPC centers.

Starting Sep 1, 2024, the Afton and Rivanna systems execute a daily script that identifies the last access time for each scratch file. Each day the script will permanently delete those files identified with an access time older than 90 days. Directories that are left empty as a result of the file purging process will be removed as well.

A file’s last access corresponds to the date and time that file was last opened (read) or modified (written to).

Log in to Afton/Rivanna and from the command line, run:

check-scratch-for-purge > outfile

where outfile is the path of the file to which you wish to save the results. This will save to your outfile a list of files ordered from the oldest last accessed to the most recently accessed that will be purged.

Alternatively, you can access the Check Scratch For Purge tool available in the Utilities dropdown on Open OnDemand. From there, you can view a list of files being purged and save the list to /home, /scratch, or download it locally.

We encourage users to back up their important data. Data can be transferred to either your home storage (50G) or leased storage (if applicable). For more details, see “What storage options does RC provide?"

Learn more about available data transfer tools.

No. Scratch is a high-performance filesystem without any snapshots or backups. Deleted files cannot be restored.

RC offers several low-cost storage options to researchers. Home directory storage provides up to 50G with daily and weekly snapshots of data. Eligible PIs can request 10TB of Research Standard storage at no charge. PIs also have the option to purchase additional leased storage.

Learn more about storage options and how to purchase storage.

If you have any questions, please contact our user services team, or join us for our virtual office hours every Tuesday, 3-5 p.m. and Thursday, 10-12 p.m..