PCI Express Basics: Chapter 2 - Topology and the Role of PCIe SWITCHES- | NEXTY Electronics, a trading company for electronic components and semiconductors

In the previous article, "Chapter 1: Basic Knowledge of PCI Express: The Expressway for Electronic Devices," we explained that PCI Express (PCIe) is a high-speed serial interface that uses point-to-point connections, that the bandwidth is determined by the number of lanes, and that communication speeds have improved with each generation.

So, what kind of structure does PCIe use to connect devices in an actual system? Also, as you continue to add NVMe SSDs, GPUs, and other devices, how do you expand PCIe lanes if you run out of them?

This chapter explains the topology (hierarchical structure) of PCIe connections, and explains the role and use of PCIe SWITCHES, which are important elements for efficiently using limited lane resources. Read on to learn the basics for understanding the design of modern high-performance systems, such as AI servers and automotive systems.

Previous article: PCI Express Basics Chapter 1: Electronic Device Expressway

PCIe topology (connection form)

PCIe uses a hierarchical topology that connects multiple devices in a "tree structure." Starting from the CPU (or SoC), the structure branches out to lower-level devices via SWITCHES and bridges, making it easy to see which devices are connected to which locations.

To understand this hierarchy, let's first organize the main components.

Root Complex

The Root Complex (RC) acts as the top (root) level that connects the CPU and memory subsystem with PCIe devices. In many cases, it is integrated into the CPU or SoC and is the starting point of the PCIe hierarchy.
A root complex has multiple root ports, each of which has a PCIe link extending from it to connect to a device or SWITCHES, and this is where the tree structure begins.

endpoint

An Endpoint (EP) is a device that provides the actual functionality over PCIe.
Typical examples include NVMe SSDs (high-speed storage), GPUs, NICs (network interface cards), and FPGAs.
The endpoints may be directly connected to the root complex or may be connected by hanging below a PCIe SWITCHES.

bridge

A bridge is a component that connects PCIe with other interface standards.
A bridge acts as an "interpreter" and is used to connect devices that use communication methods different from PCIe, such as USB or SATA, to the PCIe side.

SWITCHES

SWITCHES switch is a component that "branches" a single PCIe link to multiple downstream devices. Because PCIe is essentially a point-to-point connection, without SWITCHES, only one device can be connected to one root port.
On the other hand, SWITCHES allows you to distribute a single upstream link to multiple downstream devices.

Why PCIe SWITCHES are Needed

There is a limit to the number of PCIe lanes that a CPU (or SoC) can have.

As you sequentially add more NVMe SSDs, Wi-Fi modules, Thunderbolt controllers, etc., the available lanes decrease, making it difficult to achieve both scalability and bandwidth.

In recent years, in particular, configurations that consolidate multiple GPUs and numerous NVMe SSDs into a single system have become more common for AI processing applications, making the lack of PCIe lanes a more realistic issue.

A PCIe SWITCHES is effective in such cases.

As an example, let's consider a case where you want to equip an AI training server with the following devices:

GPU (16 lanes) x 2
NIC (16 lanes) x 1
NVMe SSD (4 lanes) x 4

If there are only 16 PCIe lanes available from the CPU, connecting one GPU (16 lanes) will prevent you from adding other devices. However, if you use a PCIe SWITCHES that can receive 16 lanes upstream and allocate a total of 64 lanes downstream, the following configuration is possible.

In this way, using SWITCHES allows you to efficiently utilize limited lane resources.

How PCIe SWITCHES Work - Why Don't They Congest?

PCIe SWITCHES ports can be broadly divided into two types:

Upstream Port: The port connected to the root complex side
Downstream Port: A port that connects to an endpoint or a lower SWITCHES

A PCIe SWITCHES acts like a hub, distributing upstream communications to the appropriate downstream ports. Furthermore, by connecting another SWITCHES to the downstream ports, the PCIe bus can be expanded hierarchically.

When connecting multiple devices using a PCIe SWITCHES, you may wonder, "Will communication congestion occur?" and "How does the switch determine which device to connect to?"

PCIe SWITCHES have built-in mechanisms to prevent these issues.

Address Space Management

SWITCHES holds the configuration information distributed from the root complex by ^enumeration (bus discovery) at startup, and manages the address space and identification information used by the endpoints under it.

This makes it appear from the host's perspective that the devices under SWITCHES are directly connected.

Packet routing

With PCIe, data is transferred in a packet format called a Transaction Layer Packet (TLP). The TLP header contains information indicating the type of communication and its destination, and the PCIe SWITCHES uses this information to determine the transfer destination.
SWITCHES reads the BDF ^* information contained in the header of the received TLP and compares it with the "correspondence between downstream ports and connected devices" stored internally during enumeration, thereby accurately forwarding the packet to the appropriate downstream port.

Enumeration/BDF
In PCIe, enumeration (bus discovery) is performed by the root complex at boot time. During enumeration, the root complex sends Configuration Request packets downstream to discover SWITCHES and endpoints. It then assigns a unique BDF (bus number/device number/function number) to each device port and writes this information into each device's configuration space.

As an example, we illustrated an image of the AI training server mentioned earlier, where the TPL from the root complex is routed to device GPU0 using BDF information.

In this way, by combining the information set in address space management with the destination information contained in the TLP header, PCIe SWITCHES achieve high-speed data transfer while preventing communication collisions and misdelivery, even in configurations where many devices are connected.

PCIe SWITCHES Usage Examples

PCIe SWITCHES are widely used in a variety of applications, from data centers to automotive systems and general-purpose PC accessories. Here we will introduce three typical examples.

1. Data Center/Server: NVMe SSD Expansion

In data centers, NVMe SSDs are used as core storage, supplying large amounts of data to CPUs and GPUs with low latency and high parallelism. In recent years, there has been an increase in cases where a single server handles tens of TB of data, and entire systems handle petabytes of data. However, because there is a limit to the capacity of a single NVMe SSD, it is essential to use multiple SSDs in combination to ensure the required total capacity and performance.

By using an expansion card with a PCIe SWITCHES, you can connect multiple NVMe SSDs through a single x4 slot. To the CPU, each SSD appears to be directly connected, enabling a high-density, high-speed storage configuration.

2. In-vehicle systems: parking assistance cameras and LiDAR high-speed data transmission

In-vehicle systems require low-latency transmission of large volumes of high-resolution data generated by sensors such as cameras and millimeter-wave radar to a central SoC. ADAS and autonomous driving, in particular, require real-time integration and processing of data from multiple sensors, leaving conventional CAN and low-speed buses with insufficient bandwidth. Infotainment SoCs also handle high-definition video and navigation data, and bandwidth requirements are increasing year by year.

By using a PCIe SWITCHES, multiple sensors and SoCs can be connected at high speed, with each device recognized as an independent PCIe endpoint, enabling in-vehicle systems to achieve high-speed, low-latency data transmission configurations similar to those of a server.

3. Consumer: External GPU (eGPU), etc.

Due to the constraints of the chassis and power consumption, thin laptops may not be able to incorporate a high-performance GPU. Therefore, a configuration in which an external GPU (eGPU) is connected via USB4 or Thunderbolt is used.

The eGPU box uses a PCIe SWITCHES inside to distribute and relay the PCIe connection from the PC to the GPU. From the CPU's perspective, the external GPU is recognized as a PCIe device, just like the internal connection, allowing for GPU-intensive processing performance expansion for video editing, 3D rendering, generative AI, and other applications without sacrificing portability.

PCIe limitations – not infinitely scalable

PCIe SWITCHES are very useful, but they do not offer unlimited scalability.

There are limits to the number of ports, lanes, and address space, and the deeper the hierarchy, the greater the latency due to passing through SWITCHES and the more restrictions there may be on BIOS settings.

Also, if multiple devices are operated simultaneously under high load, performance may not be achieved as expected.

This is because bottlenecks occur in host software processes such as issuing and completing I/O commands, interrupt handling, and the OS storage stack. As the number of devices increases, the load on the CPU and drivers increases, making it more likely that waiting times will occur.

It is important to note that PCIe performance can only be maximized when SWITCHES, device, and host are designed in a balanced manner.

Summary

PCIe SWITCHES are an important technology for efficiently expanding limited PCIe lane resources and connecting multiple devices with high speed and low latency. Their applications are expanding to data centers, automotive, consumer devices, and more.

However, there are limitations on the number of ports, hierarchical depth, and host-side processing power, so a design that takes into account the overall balance of the system is required.The optimal SWITCHES must be selected after taking into account the PCIe generation, lane allocation, and important points for each application.

We can consider configurations and propose devices that suit your needs. Please feel free to contact us.