High Performance Datacenter Networks: Architectures, Algorithms, & Opportunities (Synthesis Lectures on Computer Architecture)
Format: PDF / Kindle (mobi) / ePub
Datacenter networks provide the communication substrate for large parallel computer systems that form the ecosystem for high performance computing (HPC) systems and modern Internet applications. The design of new datacenter networks is motivated by an array of applications ranging from communication intensive climatology, complex material simulations and molecular dynamics to such Internet applications as Web search, language translation, collaborative Internet applications, streaming video and voice-over-IP. For both Supercomputing and Cloud Computing the network enables distributed applications to communicate and interoperate in an orchestrated and efficient way. This book describes the design and engineering tradeoffs of datacenter networks. It describes interconnection networks from topology and network architecture to routing algorithms, and presents opportunities for taking advantage of the emerging technology trends that are influencing router microarchitecture. With the emergence of "many-core" processor chips, it is evident that we will also need "many-port" routing chips to provide a bandwidth-rich network to avoid the performance limiting effects of Amdahl's Law. We provide an overview of conventional topologies and their routing algorithms and show how technology, signaling rates and cost-effective optics are motivating new network topologies that scale up to millions of hosts. The book also provides detailed case studies of two high performance parallel computer systems and their networks. Table of Contents: Introduction / Background / Topology Basics / High-Radix Topologies / Routing / Scalable Switch Microarchitecture / System Packaging / Case Studies / Closing Remarks
required for a non-blocking network. k 2 For example, routing pipeline stage is often independent of the radix while the switch allocation is dependent on the radix. 3The time of flight is also dependent on the packaging of the system but we ignore packaging in this analysis. 28 4. HIGH-RADIX TOPOLOGIES Optimal Radix (k) 1000 2010 100 2003 1996 10 1991 1 10 100 1000 10000 Aspect Ratio Figure 4.2: Relationship between the optimal radix for minimum latency and router aspect ratio. The
optical and electrical cables. the advantage of the lower per-unit cost of optical fibers. The high-radix topologies described in the following section exploits this relationship between cost and distance and thus, exploiting the availability of high-radix routers. 4.3 HIGH-RADIX TOPOLOGY 4.3.1 HIGH-DIMENSION HYPERCUBE, MESH, TORUS The direct networks described earlier in Chapter 3 can use high-radix routers to create high-dimension topologies, including hypercube, mesh, and torus. The
channel and one terminal channel, this ratio maintains balance. Additional details of routing and load-balancing on the dragonfly topology will be discussed in Chapter 5. Because global channels are expensive, deviations from this 2:1 ratio should be done in a manner that overprovisions local and terminal channels, so that the expensive global channels remain fully utilized. That is, the network should be balanced so that a ≥ 2h, 2p ≥ 2h. The scalability of a balanced dragonfly is shown in Figure
0.20 0.40 0.60 0.80 1.00 offered load Figure 8.8: Offered load versus latency for an ideal M/D/1 queue model. As traffic flows through the network it merges with newly injected packets and traffic from other directions in the network (Figure 8.9). This merging of traffic from different sources causes packets that have further to travel (more hops) to receive geometrically less bandwidth. For example, consider the 8-ary 1-mesh in Figure 8.9(a) where processors P0 thru P6 are sending to P7.
which a QoS class is applied is the flow. A flow is described as a tuple (SourceIP, SourcePort, DestIP, DestPort). Packets are marked by the host or edge switch using either 1) port range, or 2) host (sender/client-side) marking. Since we are talking about end-to-end service levels, ideally the host which initiates the communication would request a specific level of service. This requires some client-side API for establishing the QoS requirements prior to sending a message. Alternatively, edge