vCPU vs CPU core: what you are really buying

Two servers are sold as 4 vCPU. Log in to either one and nproc prints 4, and lscpu may well report four cores on one socket. Yet the same job can run at a steady pace on one of them and slow down on the other every evening, and nothing on the plan page explains why.

The reason is that "CPU" is used for three different things: a physical core, a hardware thread on that core, and a vCPU, which is what a hypervisor hands your virtual machine. Each is a smaller and more shared slice than the one before it. Once you can see that ladder, the numbers on a price list, the output of lscpu and the behaviour of a busy server stop contradicting each other.

Three things that all get called a CPU

A core is a complete processing engine. It fetches instructions, decodes them and runs them on its own execution units, the circuits that add integers, multiply floating point numbers and move data to and from memory. The processor this guide uses as its example, the AMD EPYC 4584PX, has 16 cores in a single package (one socket).¹

A hardware thread is one instruction stream running on a core. A processor with simultaneous multithreading, or SMT, runs more than one of them on each core at once, two on processors such as this one; Hyper-Threading is Intel's name for its version. In Intel's description, each physical core then offers two execution contexts and behaves like two "logical cores".² The AMD EPYC 4584PX has 32 threads, two on each of its 16 cores.¹

Linux calls each hardware thread a CPU. The kernel keeps a file for every CPU that lists the "CPUs within the same core",⁶ a phrase that only makes sense because a CPU, to Linux, is a thread. Boot Linux directly on a AMD EPYC 4584PX with SMT on and it reports 32 CPUs.

A vCPU is a virtual processor that a hypervisor presents to a virtual machine. How much hardware stands behind one is the provider's choice, and AWS spells out its own: on its instances with SMT, it counts every hardware thread as one vCPU. Its m5.xlarge has two cores with two threads on each, and is sold as four vCPUs.³ Two vCPUs, one core.

Physical cores on one AMD EPYC 4584PX

16cores

Two per core, so Linux on the bare chip sees 32 CPUs

32threads

Max boost, the most any single core can reach

5.7GHz

L3 cache in total on the package

128MB

Two threads, one core: what SMT shares

A core often has to wait. When an instruction needs data that is not in a cache, the data has to come from main memory, and every instruction that depends on it has to wait too. With one thread on the core, the core can often find little else to do, and many of its execution units sit idle until the data arrives.

SMT gives the core a second thread to take work from. Intel describes the gain the same way: Hyper-Threading raises throughput by using time in which the core would otherwise be waiting.² AWS describes the same mechanism from the other side, as SMT letting several threads run concurrently on one core.³ The drawing at the top of this page is one such moment: thread 0 is stalled on memory, and thread 1's instructions are using units that would otherwise do nothing.

Each thread keeps its own execution context, Intel's term for the state that records where a thread is in its program.² Almost everything that does the work is shared between the two threads: the execution units, the core's caches, and even some small internal buffers.⁵ The kernel's core scheduling documentation notes that two tasks may do well on one core when they need different hardware resources, and better on separate cores when they need the same ones.⁴

SMT raises the total amount of work a core gets through; it does not make any single thread faster, and when both threads want the same unit at the same moment, one of them waits. How much a second thread adds depends on what the two threads need. Even the kernel's documentation for a feature that lets trusted tasks share a core warns that performance will not always improve, and tells you to measure your own workloads.⁴

Sharing a core is also a security question

Because the two threads share internal buffers, code on one thread can in some cases learn about data the other thread touched. The kernel documentation gives MDS and L1TF as examples of these cross-thread attacks, and says that switching Hyper-Threading off is the only way to stop them completely.⁴ That is one reason a host may prefer never to put two customers' vCPUs on the two threads of one core, and one reason cores allocated to a single customer are worth something beyond speed.

A vCPU is a task on the host, and the label does not say where it runs

On a KVM host, each vCPU of your virtual machine is an ordinary software thread on the host, a task its scheduler places on whichever hardware thread it chooses. KVM's documentation notes that a vCPU's thread can move between the host's logical processors, and that another task can be using the same one.⁹ What is a VPS? covers that side, including the steal time you see when a vCPU is ready and no hardware thread is free. What matters here is where a vCPU lands.

Unless the host pins it, it can land anywhere. In libvirt, a toolkit for managing KVM virtual machines, a vCPU that has no pinning setting is allowed onto every one of the host's physical CPUs.¹⁰

Take a 4 vCPU server on a AMD EPYC 4584PX host. The host has 32 hardware threads, so the four vCPUs are an eighth of them (4 / 32). At any given moment they could be on four different cores, each sharing its core with whatever a neighbour is running on the other thread. They could be on two cores, two to a core, competing with each other for the same execution units. Or, at a quiet hour, they could have four cores to themselves with the sibling threads idle. The label "4 vCPU" describes all three, and the host's scheduler is free to move from one to another.

This is why the same 4 vCPU plan can feel different from hour to hour. At the quiet end, each vCPU has a core to itself. At the busy end, each shares a core with someone else's thread, and some of the time waits for one. Neither is a fault. It is what a shared vCPU is: time on a hardware thread, wherever one is free.

What changes when cores are allocated to you

An allocated core, often sold as a dedicated core, is a host core that runs your vCPUs and nobody else's. On KVM this can be done with pinning: libvirt's vcpupin setting ties one of a guest's vCPUs to a chosen list of host CPUs.¹⁰ But pinning one guest reserves nothing: to make the cores yours, the host also has to keep every other guest's vCPUs off them. The right half of the drawing above is that arrangement.

Three things change.

• Your vCPUs stop waiting behind other guests. A core that only runs your work is free whenever your work is ready, so the steal time that comes from busy neighbours goes away.
• The sibling thread is yours or idle. Nobody else's thread competes for your core's execution units, so the speed of one thread no longer depends on what a stranger happens to be running. The clock and the shared L3 still move with the load on the rest of the processor, as the next section shows. If both threads of an allocated core carry your vCPUs, as in the drawing, your own two vCPUs share that core's units, so four allocated vCPUs counted this way are two cores of execution units, not four.
• No stranger shares your core. The cross-thread attacks described above need the attacker on the other thread of the same core. On an allocated core, there is no other customer's code in that position.

What does not change is the rest of the processor. Allocated cores are still cores on a shared processor, and they still share an L3 cache with other cores. The memory controller and its 2 DDR5 channels,¹ the disks and the network port serve every guest on the machine, so a neighbour streaming through a large data set still takes a share of the memory bandwidth your database uses. If you need all of that to yourself as well, you need the whole machine.

Clock speed, boost and cache: what one thread feels

Core and thread counts tell you how much work can happen at once. How fast any single piece of work goes depends on two other things: how many cycles per second its core runs, and how often it has to stop and wait for memory.

Base and boost clocks

AMD publishes two clocks for the AMD EPYC 4584PX: a base clock of 4.2 GHz and a max boost clock of up to 5.7 GHz. A footnote on the same page defines max boost as the highest frequency that any one core on the processor can reach in normal server operation.¹ The arithmetic: 5.7 / 4.2 is about 1.36, so a core at full boost runs about 36% more cycles each second than one at base.

The words "any single core" are the important part. The kernel documentation for AMD's frequency driver treats the top performance level as a ceiling for one processor under ideal conditions: it may not last, and it may need other processors to be idle. Below it sits a nominal level, which all the cores should be able to hold together. The same page says that only some cores can reach the top frequency, and that a feature called preferred core lets the scheduler favour them.⁸ So a single busy thread on a quiet machine can run near the boost clock, while a machine with every core busy should be expected to run them lower. Neither figure is a promise about your vCPU at any particular moment.

Cache: shared by more than you think

The other half of single-thread speed is how often the core waits. Caches are small, fast memories close to the cores that hold recently used data, and the AMD EPYC 4584PX has 128 MB of L3 cache.¹ That figure is the total for the package. L3 is shared by groups of cores rather than set aside for each core, and Linux lists, for every cache, the logical CPUs that share it.⁷ On a bare metal server, cat /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_list shows which CPUs share cpu0's L3; inside a virtual machine it shows the layout the guest was given. Work whose hot data fits in cache avoids the trips to memory that stall a thread. Work that streams through gigabytes gets little from it, and while it streams it pushes out other data held in the same cache, including a neighbour's.

One trap when you look: the cache sizes in lscpu's summary are added up over all CPUs, while versions before 2.34 showed the size per core,¹¹ so the same machine can show quite different cache figures depending on which version you run.

When one fast thread matters more than many

Plenty of server software does its most important work on one thread. Redis is a clear example: its design is mostly single-threaded, so a single process handles every client's requests, strictly one after another.¹³ Node.js runs your JavaScript on one event loop thread, with a pool of workers for some other tasks, and a callback that hogs the event loop holds up every other client until it returns.¹⁴ Intel makes the same point about games: some older game engines lean on single-threaded performance, mostly use one core, and gain a lot from a higher clock.² Game servers, and how their main loop's tick rate shapes what players feel, have their own guide.

For work like this, more vCPUs do nothing for the thread that is the bottleneck. What helps is everything in the last section: a higher clock, data that stays in cache, and a core whose other thread is not busy with someone else's work. Redis's documentation treats a busy neighbour on the same core as a real risk: when you pin Redis to cores, it warns that its CPU-hungry background jobs, such as writing a snapshot, must never share a core with the main event loop.¹³ On a shared VPS, you do not choose what runs on your core's other thread.

You can see which kind of load you have. In top, press 1 to show each CPU on its own line and H to list threads instead of processes. One thread sitting near 100% while the CPUs still have room means that thread is the limit, and a bigger plan with more vCPUs will not fix it. Every CPU busy with your own work (not steal or iowait, which top also shows) means the work is spread out, and more vCPUs or cores of your own are likely to help.

Reading it from inside your server

Two commands answer most questions, as long as you know what each one is counting. Start with nproc. It answers a narrower question than "how many processors are there": it counts the ones this process is allowed to use, which can be fewer than are online. The OMP_NUM_THREADS and OMP_THREAD_LIMIT environment variables change its answer, and so can a CPU quota, such as the limit a container runtime sets. nproc --all ignores both and counts every installed processor.¹² Because it counts what this process may use, a process started with a narrower CPU affinity, for example under taskset, gets a smaller answer too.

$ nproc4$ nproc --all4

On a whole VPS the two usually agree. Inside a container with a CPU limit, nproc can come out lower than nproc --all, and a program that sizes its worker pool from the second number will start more workers than it is allowed to keep busy.

lscpu goes further. It reads sysfs and /proc/cpuinfo and summarises them: CPU, thread, core, socket and NUMA node counts, then the caches.¹¹ Here is the kind of thing it prints in a small KVM guest:

$ lscpuArchitecture:             x86_64CPU(s):                   4  On-line CPU(s) list:    0-3Vendor ID:                AuthenticAMD  Model name:             AMD EPYC 4584PX 16-Core Processor    Thread(s) per core:   1    Core(s) per socket:   4    Socket(s):            1Virtualization features:  Hypervisor vendor:      KVM  Virtualization type:    full

Read it line by line.

• CPU(s): 4 is your four vCPUs.
• Model name can be the host's real processor. libvirt's host-passthrough mode shows the guest exactly the host CPU. Other modes show a model the host chose: in host-model, the best model that host can offer a guest; in custom, whatever model the host set, so that a guest sees the same hardware on any host.¹⁰ Either way, the name tells you less than it seems.
• Thread(s) per core, Core(s) per socket, Socket(s) are the layout the hypervisor chose to present. libvirt lets the host set the sockets, cores and threads a guest sees,¹⁰ and the lscpu manual itself warns that inside a virtual machine the output describes the guest as configured, which usually differs from the host underneath, and that cores per socket can be wrong on virtual hardware.¹¹ "4 cores, 1 thread per core" here does not mean four physical cores.
• Hypervisor vendor: KVM tells you that you are in a virtual machine, and which hypervisor runs it.

For a per-CPU view, lscpu -e prints the same information as a table,¹¹ one row per CPU with its socket, core and cache columns. And to ask the kernel directly which CPUs share a core, read the topology file mentioned earlier, core_cpus_list (thread_siblings_list is the older, deprecated name for the same list):⁶

$ lscpu -eCPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE  0    0      0    0 0:0:0:0          yes  1    0      0    1 1:1:1:0          yes  2    0      0    2 2:2:2:0          yes  3    0      0    3 3:3:3:0          yes$ cat /sys/devices/system/cpu/cpu0/topology/core_cpus_list0

Each vCPU appears to have a core of its own, and CPU 0 has no sibling. On a physical machine with SMT the same file lists two CPUs, and which two depends on how that machine numbers them. Inside a virtual machine, every one of these answers describes the layout the guest was given, not where its vCPUs run on the host at this moment. What you can see from inside is the effect: steal time, and how the speed of the same job changes over a day.

Mistakes people make with CPU numbers

Choosing: shared vCPUs, allocated cores or a whole machine

Shared vCPUs suit work that is bursty or mostly waiting: a website, an API, a small database, a VPN, a bot, a staging server. Most of the time such work needs far less processor than its vCPU count, and the moments when it needs a burst are rarely the moments every neighbour needs one too.

Allocated cores suit work that is steady, or that cannot tolerate a slow moment: a database under constant load, a request handler with a latency target, a game server with a full crowd, anything where one thread is the bottleneck. If steal time climbs every time your server is busy, or the same job is fast in the morning and slow in the evening, that is the sign. And if you need the L3 cache, the memory bandwidth and every thread of the processor to yourself, the only answer is the whole machine.