By Bernard HL Lee
When I was a researcher in the United Kingdom over twenty years ago, one of my colleagues often used the term “A dog’s dinner” to describe the tangle of optical cables and patch cords running around the experiment tables and racks, interconnecting the test equipment with various optical components. That may have been when I first seriously asked myself “Isn’t there a more organized way to do this? Is there a set of guidelines somewhere?”
When I first joined the telecoms industry, I was introduced to the concept of ‘structured’ cabling, and I have been learning ever since. However, it is not until I started pursuing my RCDD® that I came to truly appreciate the amount of time, effort and thought that were put into the development of the associated guidelines and standards. These documents incorporate the collective knowledge and experience accrued by generations of network engineers and ICT professionals and cover everything from how the cables are routed, mounted to even how they are labelled, thus providing structure to what may have been once considered ‘a dog’s dinner’.
But what is structured cabling? It can be summarised as building or campus telecommunications cabling infrastructure comprising several standardized smaller elements (structured). A properly designed and installed structured cabling system is essential to a cabling infrastructure that: 1) delivers predictable performance and the flexibility to accommodate changes, 2) that maximizes system availability, 3) that provides redundancy, and 4) that future-proofs the usability of the cabling system (thus maximising your initial investment).
What are the benefits of structured cabling? “It is such a nuisance”. “Can’t I just use a point-to-point connection? it will still do the job”. These are the common sentiments we encounter. Although a point-to-point connection will work (in most cases), structured cabling provides a long-term solution especially, when we need to move, add, or re-route the network. By having an ‘organised’ inventory the potential for downtime is reduced as is the potential for human error, like pulling out the wrong connection (“Whoops, I just unplugged the bank next door”).
Of course, with today’s networks, typically comprising tens of thousands of connections, cable and port tracing is like looking for a needle in a haystack. At the very least, structured cabling offers a neater alternative to a point-to-point arrangement.
Another important aspect of structured cabling is that it continually adapts to the evolving needs and functions of the network. There are few better examples of this than modern data centers, which have scaled both in size (to the hyperscale) and in architecture from hierarchical “Fat Tree” networks to “Leaf-and-Spine” networks, to bring the “Cloud” to the people.
While the older hierarchical architectures consisted of three network layers (i.e., the Core, the Distribution, and the Access layers), the Leaf-and-Spine topologies only consist of two namesake switching layers. As shown in Figure 1 the switches on the Leaf layer are connected to all switches on the higher Spine layer, which enables a flatter hierarchy.
But what prompted this evolution? In a nutshell, exponential growth in demand for digital information. The nature of the older fat tree hierarchies, which had low, fixed switch radices (numbers of ports) was such that, as the number of servers in the data center increased, the number of switching layers needed to increase with the link bandwidth (i.e., link data rates) going up with each layer. Therefore, in order to accommodate the huge numbers of nodes in modern hyperscale data centers, the link bandwidths would have needed to become prohibitively large, well beyond the capabilities of modern transceivers. Thus, a new architecture – the Leaf-and-Spine architecture – was devised, in which the link bandwidths were the same for each layer (allowing traditional high-volume transceivers to be used), but instead the number of ports per switch went up. It is this trend, which is now pushing switch ASICs to the phenomenal aggregate bandwidths, such as 104 Tb/s anticipated within the next few years. New radical disruptions in optical integration, such as “co-packaged optics”, are required. A Leaf-and-Spine network not only allows data to be transferred through multiple paths to cope with the required bandwidth, but it also provides enough cross-connections to allow any node to reach any other node more efficiently and with the lowest possible and predictable latency by effectively placing multiple traffic streams in parallel. While traditional designs aggregate traffic by stacking multiple traffic streams onto a single link and cross-connecting all the devices within a single device or carrying the streams serially, the leaf-and-spine does it parallelly and with less hops thus reduced latency.
Once of the key innovations in optics that enables the Leaf-and-Spine design is the introduction of multi-engine optical transceivers like the QSFP-DD and OSFP, which allows a single high bandwidth transceiver (e.g., 400 Gbps) to be individually connected to multiple lower bandwidth transceivers (i.e., 100 Gbps) like the one shown in Figure 2. This enables increased density at the Leaf switches as connections from multiple servers can be aggregated into a single transceiver. Another key feature of the multi-engine transceiver is the ability to interconnect the Leaf-and-Spine switches in a full mesh topology as shown in Figure 3 directly from the transceiver without the need of additional components. This capability not only reduces the number of components but also improves the link performance between the leaf and spine switches.
Another recent advancement in the structured cabling methodology is the elimination of expensive and cumbersome fiber breakout systems, such as LC/MPO cassette modules. With the introduction of Very Small Form Factor (VSFF) connectors such as the SN® connector, the jumpers from the servers can be connected directly to a Uniboot version of the SN connector, which serve as the trunk between the patch panel at the server racks to the patch panel at the Leaf switch rack (Figure 4). This method removes the need for the MPO connection between the LC/MPO cassette and the MPO trunk. By eliminating this connection, we can:
- Lower the link loss by reducing a connection
- Minimise point of failure due to reduction of connection point
- Reduce inventory and logistics of heavy cassettes
More recently, VSFF versions of multi-fiber ferrules, such as the SN-MT® (Figure 5) have been introduced to further increase the achievable fiber count at the front faceplate. The benefits of this include:
- Reduced space and weight in cable trunking and
- Better air flow due to smaller cables and absence of breakout cassettes
Dr. Lee is currently the Director of Technology & Innovation at SENKO Advanced Components and previously he was the Assistant General Manager at Group Business Strategy Division in Telekom Malaysia. Bernard is an expert at the International Electrotechnical Commission (IEC), a Chartered Engineer accredited by the British Engineering Council and a BICSI Registered Communications Distributions Designer (RCDD). He was formerly the President of the Fiber-To-The-Home Council APAC and currently the Country Chair for BICSI Malaysia.