What is GPGPU and what does it do?

CPU microarchitectures have come a long way, but even if they are used for generic processing, they are not the best at everything. That is why a new paradigm was born a few years ago, such as heterogeneous computing. And one of the most important points of this type of computing is the so-called GPGPU, which we will focus on in this article.

Although until heterogeneous computing appeared it had no name, now that it does exist it has been given the name homogeneous computing. That is, these are those systems in which the CPU is in charge of computing, using processors or similar units for this purpose.

What is heterogeneous computing?

However, unlike homogeneous or traditional computing, more and more types of processors are being integrated into the system, so that they work together and each one does what it does best, to put it simply. For example, the CPU could process the graphics, but it would take a long time, that’s why there’s the GPU.

Currently there are many different types of processors, each with its advantages and disadvantages, and each one standing out in processing a type of application. For example, we have the CPU, GPU, DPU, VPU, DSP, FGPA as dedicated accelerators, ASICs, NPUs, etc.

In heterogeneous computing, each of them will be assigned different workloads for which they are specialized or have higher performance and/or energy efficiency. For example, if executing a vector instruction that is usually processed on the CPU could be done in less time on a GPU, why not? Well, this is precisely what this heterogeneous computation is based on.

Thanks to this, performance and energy efficiency could improve exponentially. And this is what has been exploited especially in the HPC sector in recent years. The way to achieve maximum performance per watt, and minimizing memory accesses. For example, imagine a supercomputer that uses GPU, CPU, and DPU. So the benefits of heterogeneous computing are already being exploited in it, reducing execution time and consumption, something extremely important in these data centers.

Imagine you’re running workloads like artificial intelligence, machine vision, or deep learning. These tasks need to process a large amount of data. For this reason, they can be transferred to graphic format and the GPU can take care of them. And it is that these chips, although they work at a lower frequency than the CPU, have more processing cores.

By the same logic, the use of cores based on a different ISA or microarchitecture can also be a good way to take advantage of this heterogeneous computing, as when we talk about units like the Arm that usually use clusters of two Cortex A-Series core types. high performance and efficiency, something like what Intel is currently using in its x86. In this way, when the workload is low, the E-Cores are used, while when it is high, the P-Cores are used, thus improving efficiency.

It’s something that industry insiders have been looking for in various ways, to try to unleash the potential of current processors by using them in another way. But of course, this also implies adapting the software to be able to take advantage of this paradigm.

What is GPGPU?

GPGPU (General-Purpose Graphics Processing Unit) is a term that refers to a modern heterogeneous computing methodology where the GPU is used for more than just graphics, such as performing non-specialized general-purpose computations that traditionally should be process CPU.

However, given the GPU architecture made for parallelism, you could run them faster and with better power efficiency. In short, taking advantage of the potential of the GPU for general purposes, thus improving parallelism. However, leveraging the GPU as GPGPU involves adapting the software for it, so it poses some challenges as well.

As for the hardware, the truth is that all modern GPUs are GPGPUs, they do not need any type of physical change for it, the processor is programmable and you can take advantage of all those thousands of cores that they include to perform calculations simultaneously, that is to say , with massive parallelism and efficiently. It is true that GPUs were not originally designed for this, but for processing graphics, but as has been seen, they have a lot of potential if used for certain scientific applications.

How to trick the GPU into acting like a GPGPU?

As I have mentioned before, for a current GPU or graphics card to work as a GPGPU, it must be “tricked”, that is, that the data from the applications that we want to accelerate arrives as if they were graphics data, so that it processes them as it would. with the normal graphics and thus obtain the result. And for this there are two outstanding APIs:

CUDA is a parallel computing platform and programming model created by NVIDIA. The SDK that developers use to build applications for GPGPU only works for graphics cards or GPUs of this brand.

OpenCL is a low-level Khronos Group (initially created by Apple) API for heterogeneous computing that uses GPGPU for processing. With it, developers can use the GPU’s compute cores for general purpose writing in the C/C++ programming language, similar to the previous example with CUDA.