AI data centers are placing extraordinary demands on network infrastructure. GPU clusters now reach tens of thousands of units, training jobs are highly sensitive to even a single dropped packet, and power budgets have grown to rival those of small cities. The pressure on network fabric has never been greater, and the industry's response is arriving in the form of 1.6 Terabit Ethernet.
To understand what this means for organizations planning or operating AI infrastructure, The Tolly Group spoke with Celestica's Matt Roman, Senior Director of Product Line Management. The company earlier in 2026 announced that its DS6000 series 1.6T switching platforms are available for order. The platforms are powered by Broadcom's Tomahawk 6 ASIC and SONiC™ Distribution by Celestica.
Why 1.6T Is More Than a Speed Bump
The most significant architectural shift that 1.6TbE enables has less to do with raw port speed and more to do with what it allows organizations to eliminate. Those building large-scale AI clusters on legacy infrastructure today are effectively locked into a three-tier design: leaf, aggregation, and spine. That third tier exists because the port density at lower speeds cannot support the scale of modern GPU clusters within a two-tier architecture.
"With 1.6T technology and specifically the 102.4 terabit ASIC capacity with 512 radix, it allows you to collapse from a three-tier architecture down to two-tier and support up to 32,000 GPUs," Roman explained. "It reduces the number of switches as well as optical hops, which lowers end to end latency, improves GPU utilization, and ultimately reduces total cost of ownership (TCO)."
Each network hop in a typical AI fabric adds roughly 500 to 600 nanoseconds of latency. Moving from five hops in a 400G or less environment down to three hops in a 1.6T fabric translates to a 40% reduction in end-to-end latency, based on Celestica's analysis. For synchronized GPU collectives running operations like all-reduce, where every node waits on the slowest response, that reduction carries significant operational value.
Power efficiency compounds the benefit. By doubling the port capacity without linear increase in power consumption, the 1.6T technology improves the power-per-bit metric. Celestica's analysis confirms meaningful reduction in power per gigabit compared to legacy alternatives, an advantage for modern data centers where power grids and cooling capacities have become the primary constraints on cluster expansions.
A third benefit is frequently overlooked: fewer physical connections. Every optical transceiver in a network represents a potential failure point. Collapsing a tier removes an entire layer of those connections, raising the mean time between failures for the fabric as a whole.
Engineering Challenges Are Real
Roman was candid about the complexity involved in bringing 1.6T products to market. Two challenges stand out above the rest.
The first is signal integrity. SerDes interfaces running at 224G per lane demand extremely precise PCB design, careful attention to trace lengths, and rigorous validation. Getting that signaling right at these speeds is not a trivial undertaking.
The second is thermal management. High-density switch ASICs at this capacity generate substantial heat, and the optical interfaces compound the problem. Celestica's first 1.6T platform ships in two configurations to address this directly: a fully air-cooled version and a hybrid model that uses cold plates liquid cooling on both the Tomahawk 6 ASIC and the front-panel optical interfaces.
Celestica has also been contributing to the Open Compute Project's working group on open cluster designs for AI, helping define reference architectures for liquid-cooled AI racks. The goal is to lower the barriers for organizations that need to move toward liquid cooling but lack established reference points for doing so.
Misconceptions That Can Derail a Deployment
Roman was direct in addressing the assumption that adding bandwidth to an AI fabric automatically improves performance. "Just adding raw bandwidth, it doesn't always solve the problem," he said. "In GPU clusters, things are very much dictated by packet loss, protocol efficiency, quality of service, buffering, and those types of things."
The reason is structural. GPU training operations using frameworks like NVIDIA Collective Communications Library (NCCL) function as collectives. A single packet drop or retransmission does not affect one GPU; it stalls the entire group waiting on that operation. The cost of that stall, measured in idle GPU cycles against the capital cost of those GPUs, dwarfs the cost of the network infrastructure itself.
As a result, delivered performance depends as much on software configuration as on hardware capability. In RDMA over Converged Ethernet (RoCE) v2 fabrics, misconfigured Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) thresholds can cause pause storms and tail-latency spikes. Additional port speed cannot resolve these issues, which makes disciplined configuration and validation as important as raw capacity.
Migration Paths: Greenfield Deployments and Staged Upgrades
For organizations moving from 400G infrastructure, 1.6T switches support four-by-400 breakout cabling, allowing existing leaf switches to connect to the new spine without immediate replacement. The third aggregation tier can be eliminated while the leaf layer remains in place.
From 800G, the migration follows a similar staged approach: replace the spine with 1.6T platforms using two-by-800G breakout cables, then refresh the leaf layer in parallel with GPU server upgrades.
Roman identified transceiver interoperability as the most persistent friction point in real-world migrations, even in deployments involving earlier generations. "That's probably one of the biggest things we see our current customers face, even with 800G switching, is just getting these transceivers up and running," he said. Celestica maintains its own pluggable transceiver qualification process and sells branded optics specifically to reduce that friction.
Higher-layer interoperability considerations around advanced QoS features and NIC compatibility, particularly with NVIDIA ConnectX and Broadcom Thor series cards, also require attention. Celestica's SONiC-based platforms include streaming telemetry capabilities that give operators real-time insight into fabric performance, which Roman cited as critical for the ongoing tuning these environments require.
Early Adoption Patterns
Early adoption follows a predictable pattern. Tier-one hyperscalers are already moving to 1.6T, and Roman noted that in many cases those conversations require little persuasion. The need is well established and the technical readiness exists.
The NeoCloud category, GPU-as-a-service providers that lease AI compute capacity, represents the next wave. These organizations are heavy 800G consumers today and are actively evaluating 1.6T platforms. High-performance computing environments, including major universities and Department of Energy national laboratories, round out the early adopter profile.
The Path to Broader Adoption
The adoption of 1.6T Ethernet is following the familiar technology diffusion model seen across the networking industry. Hyperscalers are the initial adopters, absorbing the higher early costs and validating high-radix architectures at massive scale. Their high-volume demand will ultimately drive down the cost curve, making the technology accessible to the broader enterprise market. For organizations currently on 100G or 400G infrastructure, transitioning to 1.6T requires proactive planning beyond a simple hardware refresh. Building a foundational network that can support next-generation, high-density requirements necessitates strategic considerations, such as implementing pod-based scaling models and granular telemetry.
The main catalyst for this shift is the growing need to run AI workloads on-premises. In this environment, 400G fabrics quickly encounter practical limits regarding latency and port density. However, 1.6T offers a compelling transition path, enhanced by key features that simplify deployment: OSFP backward compatibility protects existing cabling investments, and SONiC NOS-based solutions increase accessibility. This evolution is also expected to improve efficiency. As Matt Roman said, "As multiple generations of this technology come out, it's going to provide lower power, and in theory, higher reliability from an optical interface perspective." Ultimately, the maturation of 1.6T technology will enable enterprises to achieve a more sustainable and high-performance fabric tailored for their evolving data center needs.
Key Takeaways
1.6TbE enables a shift from three-tier to two-tier AI fabric architecture, reducing hop count from five to three and cutting latency by roughly 40%
Power efficiency improves significantly per gigabit, a critical factor as data center power constraints continue to tighten
Maximizing AI fabric efficiency requires pairing 1.6T bandwidth with advanced traffic management features. Implementing Priority-based Flow Control (PFC), Explicit Congestion Notification (ECN), Link-level Retry (LLR), and Credit-Based Flow Control (CBFC) etc ensures zero packet loss, ultra-low tail latency, and peak performance across AI clusters.
Transceiver interoperability is the most common friction point in real-world migration projects
Early adopters include hyperscalers, NeoCloud providers, and HPC environments; broader enterprise adoption will follow on a longer timeline
Co-packaged optics and linear pluggable optics are the technologies most likely to accelerate mainstream adoption
Learn More
For additional information about Celestica's 1.6T switching platforms and SONiC-based network solutions, visit celestica.com. Connect with Matt Roman and Arun Solleti on LinkedIn for deeper discussions about AI fabric architecture and migration planning.
The SONiC Distribution by logo and word mark are trademarks or registered trademarks of The Linux Foundation, used under license.
