Many subproblems can then be solved simultaneously by multiple processing units. In modern CPUs, the single execution unit is typically a CPU core.
This, for example, cuts down the computation time of any given problem. For example, weather forecasts which rely heavily on weather simulations wouldn't be ready in time if computed with a single core. If we divide the problem into a multitude of subtasks and compute them separately we cut down on computation time significantly. Thus we can predict next weeks’ weather before it happens.
Parallel computing has recently become mainstream because of the supercomputers’ increasing power consumption. The heat generation stays manageable when multiple subtasks are computed simultaneously using many different processing units. Later, the subtask results are combined in order to solve the actual problem at hand. Parallel computing also has other benefits that will be examined more closely during this course.
How a problem is split into smaller subtasks depends entirely on the problem. There are various paradigms and programming approaches to doing this. Later we will discuss some of them in more detail.
You can get more information by watching the video Introduction to Parallel Programming.
Let us assume that Alice and Bob want to compute the sum of 20 numbers, say, the prices of their groceries. Either Alice or Bob could do it alone, just like a single CPU core could compute it in serial. But how could they do it in parallel?
In order to compute the sum in parallel, Alice and Bob could divide the numbers in two, and then each computes the sum of their own half (figure below). After completing these partial sums, all they need to do is add them up to have the total sum of all numbers. If the calculation by Alice alone would take 20 seconds at best, together Alice and Bob could solve the problem in only 10, provided they have around the same mathematical capabilities. Also, if the maximum amount of numbers each of them can compute on their own is 20, together, the two could process 40 numbers. Such limitation could come from the amount of memory available for the processing unit in a computer.
The problem of Alice and Bob illustrates the two benefits of parallel computing: making computations faster and enabling larger computations.