Afton
The Afton cluster is comprised of 300 compute nodes, each with 96 compute cores based on the AMD EPYC 9454 architecture for a total of 28,800 cores. Each Afton node boasts a minimum of 750 Gigabytes of memory, with some supporting up to 1.5 Terabytes of RAM memory. Afton also provides access to 55 nodes with NVIDIA general purpose GPU accelerators (RTX2080, RTX3090, A6000, V100, A40, A100, H200, RTX Pro 6000 and B200), including an NVIDIA BasePOD.
Rivanna
Rivanna supercomputer currently contains 603 nodes with over 20,476 compute cores.
Get started with Afton or Rivanna
Storage for Afton and Rivanna
The Afton and Rivanna platforms share storage systems of over 8 Petabytes and a stack of pre-installed software packages available for computational research across many disciplines.
Instructional Use
Instructors can request instructional allocations on Afton/Rivanna for classes and extended workshops. These allocations are time-limited and expire at the end of the teaching period, but are otherwise equivalent to any allocation.
The instructional allocation is 1,000,000 Service Units (SUs) for the semester during which the course is conducted. For workshops, the allocation will persist during the workshop and for two days afterwards.
Using the Allocation
Instructors are recommended to have their students use the Interactive partition during class period. Job size and duration are more limited on this partition. This is intended to reduce queueing delays and allow for uninterrupted access to compute resources for interactive work, including during lecture. Outside of class, students are encouraged to use any of the partitions.
Already have an account?
System Details
A high performance computing cluster is typically made up of at least four service layers:
- Login nodes - Where you log in, interact with data and code, and submit jobs.
- Compute nodes - Where production jobs are run.
- Storage - Where files are stored, accessible by all nodes in the cluster.
- Resource Manager - A software system that accepts job requests, schedules the jobs on a node or set of nodes, then manages their execution.
Hardware Configuration
| System | Cores/Node | Memory/Node | Specialty Hardware | GPU memory | GPU devices | # of Nodes |
| Rivanna | 40 | 375GB | - | - | - | 182 |
| Rivanna | 40 | 750GB | - | - | - | 40 |
| Afton | 96 | 750GB | - | - | - | 280 |
| Afton | 96 | 1464GB | - | - | - | 20 |
| Afton | 128 | 976GB | GPU: A100 | 40GB | 8 | 2 |
| Afton | 128 | 1953GB | GPU: A100 | 80GB | 8 | 18 |
| Afton | 96 | 1464GB | GPU: A40 | 48GB | 8 | 12 |
| Afton | 48 | 250GB | GPU: A6000 | 48GB | 8 | 14 |
| Afton | 40 | 374GB | GPU: RTX2080Ti | 11GB | 10 | 2 |
| Afton | 32 | 125GB | GPU: RTX3090 | 24GB | 4 | 4 |
| Afton | 36 | 375GB | GPU: V100 | 32GB | 4 | 2 |
Afton | 96 | 1953GB | GPU:H200 | 140GB | 8 | 1 |
Afton | 128 | 2TB | GPU:RTX6000 | 96GB | 8 | 1 |
Afton | 120 | 1953GB | GPU:B200 | 180GB | 8 | 7 |
Afton | 128 | 1464GB | GPU:RTX6000 | 96GB | 8 | 6 |
Job Partitions
Rivanna and Afton are managed resources; users must submit jobs to queues controlled by a resource manager, also known as a queueing system. The manager in use on Rivanna and Afton is Slurm. Slurm refers to queues as partitions because they divide the total hardware into sets of resources. There is no default partition, and each job must request a specific partition. Partitions and access policies are subject to change, but the following table shows the current structure.
| Partition | Purpose | Max time / job | Max nodes / job | Max cores / job | Max cores / node | Default memory / core | Max memory / node |
| standard | For jobs on a single compute node | 7 days | 1 | 96 | 96 | 9GB | 1462GB |
| parallel | For large parallel jobs on up to 64 nodes | 3 days | 64 | 6000 | 96 | 8GB | 750GB |
| gpu | For jobs using general purpose graphical processing units | 3 days | 4 | 32GPU | 128 | 6GB | 1953GB |
| gpu-mig | For smaller GPU jobs | 3 days | 1 | 1MIG/job=1GPU + 4CPU | 16 MIGs / user | 4000MB | 2TB |
| interactive | For quick interactive and debugging sessions | 12 hours | 2 | 24 / 2GPU | 96 | 6GB | 216GB |
Service Unit Charge Rate
| System | Memory | Specialty Hardware | SU/CPU-hr | SU/GB-hr | SU/GPU-hr |
| Rivanna | 375GB | - | 0.177 | 0,0143 | - |
| Afton | 750GB | - | 0.464 | 0.0284 | - |
| Rivanna | 750GB | - | 0.13 | 0.009 | - |
| Afton | 1464GB | - | 0.418 | 0.0244 | - |
| Afton | 976GB | GPU: A100: 40GB | 0.311 | 0.0292 | 46.381 |
| Afton | 1953GB | GPU: A100: 80GB | 0.403 | 0.043 | 50.89 |
| Afton | 1464GB | GPU: A40 | 0.34 | 0.02 | 18.669 |
| Afton | 250GB | GPU: A6000 | 0.383 | 0.062 | 14.273 |
| Afton | 374GB | GPU: RTX2080Ti | 0.388 | 0.041 | 7.757 |
| Afton | 125GB | GPU: RTX3090 | 0.303 | 0.076 | 11.323 |
| Afton | 375GB | GPU: V100 | 0.058 | 0.005 | 2.096 |
| Afton | 1953GB | GPU:B200 | 0.44 | 0.044 | 76.17 |
Afton | 1464GB | GPU:RTX6000 | 0.43 | 0.049 | 33.43 |
Remarks
- Standard partition maximum aggregate CPU cores allowed for a single user’s running jobs is 1000.
- Parallel partition requires at least 2 nodes and 4 CPU cores.
- Slurm’s default memory unit is in MB. Different units may be specified, e.g. --mem=100G, where 1G = 1024M.
- The gpu partition is dedicated to jobs that can utilize a general purpose graphics processing unit (GPGPU). In Slurm scripts, you must request at least one GPU device through --gres=gpu. Jobs that do not utilize any GPUs are not allowed in this partition.
- Interactive partition maximum aggregate CPU cores (GPUs) is 24 (2) for a single user.
- The NVIDIA DGX BasePOD offers high-performance GPUs that bring new AI and ML functionality to support parallel GPU computing and large deep-learning models.
- Running jobs on gpu-mig partition will not incur any SU charges.