400G in the data center: Densification and campus architecture

James Young James Young June 23, 2020

CommScope_MTDC_Image_2_smallHigher bandwidth and capacity demands are driving ever-higher fiber counts.  15 years ago, most fiber backbones in the data center used no more than 96 strands, including coverage for diverse and redundant routing. Current fiber counts of 144, 288, and 864 are becoming the norm, while interconnect cables and those used across hyper- and cloud-scale data centers are migrating to 3,456 strands. Several fiber cable manufacturers now offer 6,912-fiber cables, and 7,776 fibers are already on the horizon.

CLICK TO TWEET: CommScope’s Jim Young explains how data centers are responding to the challenges of densification and campus architecture.

New fiber packaging and design increases density

The higher fiber-count cabling takes up valuable space in the raceways, and their larger diameter presents performance challenges regarding limited bend radii. To combat these issues, cable OEMs are moving toward rollable-ribbon construction and 200-micron fiber. Whereas traditional ribbon fiber bonds 12 strands along the entire length of the cable, rollable ribbon fiber is intermittently bonded — allowing the fiber to be rolled rather than lay flat. On average, this type of design enables 3,456 strands to fit into a two-inch duct compared to a flat design that can accommodate only 1,728 in the same space.

The 200-micron fiber retains the standard 125-micron cladding, which is fully backward compatible with current and emerging optics; the difference is that the typical 250-micron coating is reduced to 200 microns. When paired with rollable ribbon fiber, the decreased fiber diameter enables cabling OEMs to keep the cable size the same while doubling the number of fibers compared to a traditional 250-micron flat ribbon cable.

200 micron imageTechnologies like rollable ribbon and 200-micron fiber are deployed by hyperscale data centers to support the increased demand for inter-data center connectivity. Within the data center, where leaf-to-server connection distances are much shorter and densities much higher, the primary consideration is the capital and operating cost of optic modules. For this reason, many data centers are sticking with lower cost vertical-cavity surface-emitting laser (VCSEL) transceivers, which are supported by multimode fiber. Others opt for a hybrid approach—using singlemode in the upper mesh network layers while multimode connects servers to the tier one leaf switches. As more facilities adopt 400GE, network managers will need these options to balance cost and performance as 50G and 100G optic connections to servers become the norm.  

80 km DCI space: Coherent vs. direct detection

As the trend to regional data center clusters continues, the need for high-capacity, low-cost data center interconnect (DCI) links becomes increasingly critical. New IEEE standards are emerging to provide a variety of lower-cost options that offer plug-and-play, point-to-point deployments. Transceivers based on traditional four-level pulse amplitude modulation (PAM4) for direct detection will be available to provide links up to 40 km while being directly compatible with the recent 400G data center switches. Still other developments are targeting similar functionality for traditional DWDM transport links.

PoIDM image

As link distances increase beyond 40 km to 80 km and beyond, coherent systems offering enhanced support for long-haul transmission are likely to capture most of the high-speed market. Coherent optics overcome limitations like chromatic and polarization dispersion, making them an ideal technical choice for longer links. They have traditionally been highly customized (and expensive), requiring custom “modems” as opposed to plug-and-play optic modules. As technology advances, coherent solutions likely will become smaller and cheaper to deploy. Eventually, the relative cost differences may decrease to the point that shorter links will benefit from this technology. 

Taking a holistic approach to continual high-speed migration

The continual journey to higher speeds in the data center is a step-process; as applications and services evolve, storage and server speeds must also increase. Adopting a patterned approach to handle the repeated periodic upgrades can help reduce the time and cost needed to plan and implement the changes. We recommend a holistic approach in which switches, optics and fiber cabling operate as a single coordinated transmission path. Ultimately, how all these components work together will dictate the network’s ability to reliably and efficiently support new and future applications. Today’s challenge is 400G; tomorrow, it will be 800G and 1.6T. The fundamental requirement for high-quality fiber infrastructure remains constant, even as network technologies continue to change. 

In the final installment of this three-part blog, we discuss the changing role of the data center in a 5G/IoT-connected world. Stay tuned.

Part 1: 400G in the data center: options for optical transceivers

About the Author

James Young

James Young

James currently serves as the Director of CommScope’s Enterprise Data Center division, overseeing strategy and providing leadership to product and field teams globally. Formerly James has been involved in a variety of roles including sales, marketing and operations for communication solutions working with Tyco Electronics/AMP, Anixter, Canadian Pacific and TTS in Canada. 

James has gained extensive experience in the sale of OEM products, network solutions and value-added services through direct and indirect channel sales environments. His sales experience includes electronic transmission components, telephony systems, network systems, LAN infrastructure products and fibre transmission system products. James has garnered substantial experience in OEM and channel marketing, as well as network operations as assistant director of CP’s computers and communications group. 

James graduated with a Bachelor of Science from the University of Western Ontario.  He is a registered Communication Distribution Designer (RCDD) and certified Data Center Design Professional (CDCP).

Categories

Enterprise Networks