Most terms here are used and explained thoroughly during the course. This section is intended to be a reference/dictionary you can use to freshen your memory while reading.
CPU or central processing unit can be considered the brain of the computer. It performs logical, arithmetic, controlling and input/output operations determined by the program's given instructions.
Modern CPUs/processors consist of multiple processing units on a single integrated circuit. These processing units are commonly referred to as cores. General household computers such as laptops, desktops, phones, tablets etc. have around 4-8 cores. Server processors, such as the ones used in supercomputers have up to 64 cores at the moment. With time these numbers will go up.
GPU or graphics processing unit is a type of processing unit specialized in very simple but fast arithmetic. As the name infers, originally graphics cards were intended in rendering computer graphics, such as in video games and visual effects. When features needed for e.g. number crunching were added the term GPGPU or general purpose graphics processing unit was coined. Nowadays we simply use the term GPU in the HPC context.
HPC applications and simulations can often be pared down to a vast collection of simple arithmetic. Due to this, GPUs are often good tools to "accelerate" the massive amount of calculations done in HPC and for this reason they are sometimes called accelerators.
Disk refers to long term storage used by a computer. Reading and writing to disk is generally slow, but speed varies depending on the type of the disk. Examples of disk devices are hard disk drive(HDD), solid state drive(SSD), and a USB drive which is a type of SSD. In addition to disks, a tape drive is also sometimes used for long term storage.
Memory refers normally to random access memory (RAM). Memory is temporary storage which holds the program code and the data during the runtime of a program. Accessing memory is much faster than accessing a disk, but the caviat is that when the computer is shut off the memory is wiped clean.
Cache is a special memory which is usually directly on the CPU, allowing extremely quick access for the cores. Capacity of cache is, however, small. Cache is used as intermediate storage, when accessing the memory data is first read/write to cache.
A node contains CPUs and memory, and it is the basic building block of a supercomputer. If you take out a single node from a supercomputer and hook up a monitor, keyboard and mouse to it, it would be equivalent to having a desktop computer. Albeit a very powerful computer, but a computer nonetheless. Similarly you could treat an everyday laptop as a node and connect a collection of them into a supercomputer.
The supercomputer or cluster is formed by connecting a set of nodes together via a fast network. As explained in the Node section, you could link up a collection of laptops to make up a cluster.
Interconnect is the high-speed network connecting hundreds or thousands of nodes allowing them to communicate with minimal latency. Your home router letting your phone and laptop communicate could be considered an interconnect. Of course, in a supercomputer the interconnect is much more advanced and optimised than what a generic router is.
Virtualization means simulating the necessary hardware for a guest operating system. For example, you could take a single node consisting of a single 64-core processor, and by virtualizing each core as a single independent CPU you can run independent operating systems on all of those cores. Now, in a certain sense, each of these virtual machines could be treated as independent node. Ultimately you end up with 64 independent "computers" inside a single computer. These 64 virtual machines could be linked up with a virtual interconnect and thus we're left with a virtual cluster inside a node.
A virtual machine is an independent computer that is simulated using virtualization technology. A good example of virtual machines are running emulators of old video game consoles on modern computers. Another example of a virtual machine is running an instance of Linux on your Windows computer.
MPI or message passing interface is a programming model allowing multiple nodes or cores to communicate with each other, and execute computation in parallel. MPI allocates isolated memory for each task, meaning a task cannot access the information an other task uses. MPI can be run on different levels of abstraction. For example, you could execute multiple tasks with independent memory within a single CPU. More generally, in supercomputer MPI is needed for communication between the nodes, since they do not share the same memory.
OpenMP is a parallel programming approach that is based on a shared memory model. Program is split into multiple tasks which can exchange information by directly reading and writing to the common shared memory. Since OpenMP requires shared memory it cannot be utilised over multiple nodes, but only within a single node.