Loading...
Loading...
An intelligent lossless network is an advanced networking approach that combines flow control and congestion control to improve network performance and reduce latency. It also enables deeper integration between the network layer and application layer through technologies such as intelligent lossless storage networking. Together, these capabilities deliver a low-packet-loss, low-latency, high-throughput environment for demanding workloads like AI, centralized and distributed storage, and HPC — accelerating both compute and storage efficiency while creating a unified, high-performance data center infrastructure.
As data center technology continues to evolve, the limitations of traditional networks are becoming harder to ignore. Emerging workloads such as high-performance computing, AI, and distributed storage demand far more from the network than before. The traditional TCP/IP protocol stack, however, consumes significant resources and introduces high latency at critical stages such as data transmission — and it simply can't keep up with these new requirements.
RDMA (Remote Direct Memory Access) is a high-speed network interconnect technology designed to minimize processing delay and resource consumption on both the sending and receiving ends during data transfer. RDMA allows computers to access the memory of remote systems directly and transfer data at the memory level, without constant CPU involvement — significantly boosting network communication performance.
Today, distributed storage, HPC, and AI workloads commonly rely on RoCEv2 (RDMA over Converged Ethernet version 2) as the transport protocol over Ethernet, reducing both transmission latency and CPU load. Compared to traditional TCP/IP communication, RDMA not only reduces resource consumption during data transfer but also cuts down processing delay.
That said, RDMA runs over UDP and is connectionless — meaning it lacks a built-in packet-loss protection mechanism and is highly sensitive to any packet loss on the network. Compounding this, distributed high-performance applications typically generate many-to-one "Incast" traffic patterns. On Ethernet devices, this kind of traffic can easily trigger sudden bursts of congestion — or even packet loss — within a device's internal queue buffers, leading to higher latency, lower throughput, and degraded performance for distributed applications overall.
To fully realize RDMA's performance potential and overcome the bottlenecks that limit large-scale distributed systems in the data center, it's essential to build a lossless network environment defined by three qualities: zero packet loss, low latency, and high throughput.
Flow Control
Flow control regulates the rate of data transmission, primarily to prevent a sender from transmitting faster than the receiver can process. Without it, the receiver's buffer can overflow, causing packet loss.
Congestion Control
Congestion control is the core technology behind intelligent lossless networks, designed to prevent or mitigate network congestion. When traffic volume exceeds a device's processing capacity, the network risks delayed or dropped packets.
ECN (Explicit Congestion Notification) signals congestion along a transmission path by marking the DS field in the IP packet header. Endpoints that support ECN can read this marker to detect congestion and adjust their transmission rate accordingly — preventing the situation from worsening.
Traffic Scheduling
RoCE traffic must run within a lossless queue to perform reliably. This lossless queue relies on PFC, which can send Pause frames to a specific queue, forcing the upstream device to temporarily halt transmission. When a device forwards a message, it's placed into the queue corresponding to its priority for scheduling and forwarding. If the send rate for a given priority exceeds the receive rate — leaving insufficient buffer space at the receiving end — the device signals the previous hop with a PFC PAUSE frame. Upon receiving this frame, the upstream device stops sending traffic at that priority level until it receives a PFC XON frame, or until a defined aging timer expires and transmission resumes automatically.
Switching chips operate on fixed pipelines, with buffer management sitting between the in-chip and out-of-chip processes. By the time a message reaches this stage, both its ingress and egress information are already known, so buffer management is logically split into ingress and egress directions and handled separately.
The PFC watermark is triggered based on ingress buffer management. Each chip provides eight ingress queues, allowing traffic of different priorities to be mapped to different queues — each with its own buffer allocation strategy.
For each queue, buffer allocation is divided into three categories based on use case:
Configuring PFC buffer thresholds correctly helps prevent problems such as tail-drop on the send buffer due to insufficient space, or excessive queue depth on inbound traffic. PFC currently supports the following threshold types:
In AI training scenarios, PFC delivers lossless transmission guarantees for RoCEv2/RDMA traffic through buffer watermarks and queue scheduling, while ECN enables fast, end-to-end congestion detection and mitigation — working together with intelligent traffic scheduling to optimize link load across the network. This combined approach unlocks the full performance advantages of RDMA — low latency and low CPU overhead — while avoiding congestion and deadlock issues, ultimately delivering the low-latency, high-throughput, stable, and reliable network foundation that large-scale AI clusters require.
