Memory, Compute (CPU), or I/O Bound?

Whenever your system runs slow, it is almost always due to the slowest component or chain of components in your system. This can sometimes be caused by software, but often, it’s the hardware that’s the culprit.

For example, if you have a very old and slow disk – for example, a still somewhat common 5400 RPM spinning disk, often referred to as an HDD, or Hard Disk Drive – that disk may be the bottleneck in your system.

Think about it like pipes with water running through them. Picture your system’s memory, computing unit (your CPU, or Central Processing Unit – the main chip/processor in your system), and disks (as part of the I/O, or Input/Output system) all as water pipes. Now, imagine that the slowest components are a smaller pipe, and the fastest components are a bigger pipe. If you needed to transfer 10L through each pipe, the slowest pipe will take much longer than the largest one.

On Linux, the main bottlenecks are Memory (RAM), Compute (CPU), or I/O (disk operations). In the case of memory, speed can be a factor, but it’s running out of it that’s a big problem. For CPU, if you’re on older hardware, each CPU core works a lot slower, and there may not be enough of them. For I/O, reading from slow hard drives as well as excessive disk writes can be the issue.

There are tools you can use that make it easy to find out if a system is Memory, Compute (CPU), or I/O Bound. All you need is htop and iotop, two semi-graphical tools, which can be easily installed on Linux.

Installing htop and iotop

To install htop and iotop on your Debian/Apt based Linux distribution (Like Ubuntu and Mint), do:

sudo apt install htop iotop

To install htop and iotop on your RedHat/Yum based Linux distribution (Like RedHat and Fedora), do:

sudo yum install htop iotop

CPU Bound

It is easy to see if a system is CPU bound or not. Simply type htop at the command line and press enter. Then look at the colorful CPU bars at the top of the screen. If your processor has 16 threads, there will be 16 bars.

The simple question to answer is whether they are all nearly ‘full’ (nearing 100%), or whether there is ample of room to move:

If the bars are as good as full, the system is clearly CPU bound. Note also that the memory (Mem) and the swap (Swp) bars are by no means full: this is not a memory related performance issue.

For a bit more information and trending, you can next look at the ‘Load average’ number. While this number is highly arbitrary, a little familiarity with your system and the general understanding that if any of these three numbers goes higher than twice the number of threads in your system, the system is struggling to keep up goes a long way here.

The first load average number is a 1 minute average, the next a 5 minute average, and the last number a 15 minute average. In this case, the 1 minute load is 270, which is almost 17 times the number of threads: our system is heavily CPU bound.

Finally, an interesting number to check out is the number of tasks (and to a lesser extent threads). While the exact low and high water marks depend on the capabilities of the underlying hardware/machine, if the tasks number is excessively high, your CPU may be context switching (changing from processing one task to another) heavily.

If you would like to learn more about what the various colors in htop indicate, see Color Bars in htop – What Do They Mean?

Memory bound

Immediately on accessing htop, it is easy to see if a system is Memory bound or not. Simply look at the memory (Mem) and swap (Swp) bars mentioned earlier.

If the Memory bar is completely full, and the Swap bar is for example 50% full, the system is almost definitely swapping heavily. Swapping is the process of exchanging main memory contents with disk (using a special Swap file or Swap partition) because it is full, and it is generally speaking super-slow. Once a system starts and continues swapping, it will become prohibitively slow.

It is easy to see when you start running out of memory, as the bar will become full. However, the swap space usage can sometimes be a little ambiguous.

For example, 20% may be in use, but there is plenty of memory left. This may indicate that the operating system has moved some low frequency use memory areas to disk in order to optimize main memory. As plenty of memory remains free, this situation is fine and not a cause of any concern.

There is also an exception to a memory bar that looks quite full, and that is caching. Your system may be set up to reserve x amount of memory for caching.

Another way to quickly check this is by running free -g at the command line (or free -m for machines with smaller amounts of memory like a Raspberry Pi):

This is easy to read: 62 Gigabytes of memory, 25 in use, 12 free, and 24 currently assigned to buffers and cache. The 32 available is an approximate total of actual free (12) and whatever is assigned to buffers and cache (24) minus what is already in use (not shown), or in other words 12 + 24 = 36 and 32 is available, so approx 4 gigabytes is used by buffers and caching.

Note we can also see how much swap space was reserved (10 Gigabytes) and how much is being used here: 0 currently, and thus 10 free.

I/O bound

Let’s say you are checking htop and see this:

The system looks busy, but not busy enough to be deemed CPU-bound. The used/free memory and swap bars look fine too. Let’s checkout iotop next. To do so, you need to use sudo iotop to start iotop as iotop requires sudo.

The top two bars are the most helpful in quickly analyzing whether a system is struggling with disk throughput and is thus I/O bound.

While the M/s number is not super-high in terms of modern SSD, constantly reading and writing several Megabytes per second to a slow HDD drive is quite intense I/O!

This number, when watched for a little while, along with the processlist under it (to see who the top users are), and the top section of the htop output (in terms of CPU and Memory) gives a good overall feel for whether a system is Memory, CPU or I/O Bound.

Mitigating Performance Issues

The system changes required to mitigate performance issues are always specific to the system as well as the specific situation experienced. A few examples:

Is the system Disk/IO bound? It may make sense to stop some heavy-write log services, to upgrade the I/O system (for example by adding an SATA card in an old computer), to change to a faster storage device (like an NVMe based disk instead of a HDD), or to simply find a faster SSD.

Is the system Memory/Swap bound? It may, for example, make sense to run less Virtual Machines, run less memory intensive processes, or to add more physical hardware memory modules.

Is the system CPU bound? Use the bottom process list in htop to find the process which is hogging the CPU. You can even terminate it directly from within htop by using the F9 key.

If the issue is the CPU itself (i.e. the CPU is clearly not keeping up with the most basic tasks associated with the system), changing hardware is a bit more complex. One needs to find a faster CPU, still compatible with the socket on the motherboard, and even in that case performance improvements may be small. It may be time to upgrade the system overall.

More Than One Performance Bottleneck?

Coming back to our water pipe analogy, keep in mind that sometimes a bottleneck can be caused by a combination of various components.

For example, if an old or cheap I/O controller card requires 80% of CPU time just to process data, and the disk attached to it is slow HDD drive which is used to 80% of its capabilities, even with the throughput of the cheaper I/O card, then both are creating an overall issue which will not be resolved by addressing either. Both will need to be fixed before the system will become performant again.

Wrapping up

Whether you are a DevOps engineer or a home computer Linux user, knowing how to quickly analyze whether your system is Memory, CPU, or I/O Bound will help you implement better software and hardware changes to cater to the performance issue being experienced.