Introduction
To enhance the performance of High Performance Computing(HPC), people have proposed many methods [1][2][3][4]. With HPC we can run application programs through parallel processors [5] efficiently and reliably. It can be applied in several advancing and promising areas, such as the cosmic cube[6] and Blue Gene/Q interconnection network[7]. The device technology evolution has reached its stagnation--no other hardware system outperforms CMOS—meaning that we cannot expect to improve large-scale computing capabilities on hardware levels. HPC has the potential to unclog computing bottleneck by taking full advantages of node interconnection among clustering computers to effectively process massive data [8] [9] [10]. The characteristics of HPC closely rely on inter-node communications. So, network topology and routing algorithms will greatly influence its computing performance. That’s why we focus on the progress of network topology.
Torus and fat tree are two prime interconnection topology infrastructures for HPC [11]. Fat tree can perform well in massive data processing, but its performance will be limited by scalability, reliability and power consumption as the scale of network increases. Compared with fat tree, torus network possesses desirable features—lower cost on hardware, more achievable on implementation and lower latency. [12]. In addition, routing algorithms, used to improve path diversity and failure detecting ability inside the interconnection network, can be better implemented on torus network.
Traditional Torus topology naturally has circular link paths in every dimension, and this architecture feature will inevitably result in circular waiting for node resources inside network when transaction comes from various nodes if we do not install packet routing. This will trigger deadlock incidents and correspondingly influence performance characteristics of networks [14] [15] [16] [17].
Experimental results and theoretical analysis show that each port of each node in torus network will require at least two virtual channels to avoid deadlock even if we introduce deterministic routing algorithm or partial adaptive routing algorithm inside network of this type. [18].
To solve problems this network may bring about as well as take advantage of its superiority, we propose a new torus-based topology called Mesh-of-Torus topology. In Mesh-of-Torus topology, we can achieve in deadlock free even without the deployment of virtual channels. This new structure will connect the network in three levels--a 4*4 torus network is considered as the basic cell to construct higher level network. And we connect cells of higher level network by mesh topology. In this way, four links are utilized to connect two adjacent torus networks. To be more specific, a second level cell consists of four basic cells and a third level cell consists of four second level cells in the same way. In addition, we implement a corresponding port assignment mechanism that can forestall deadlock in the network on the top level. We will discuss our new topology and related port assignment mechanism in this paper.
The remainder of this paper is organized as follows: In section 2, we place our work in the context of the deadlock nature of torus and deadlock avoidance mechanism without using virtual channels. And in section 3, we will offer detailed information of our new topology and we will compare it with mesh and torus topology, respectively, on their network parameters. A deadlock-free routing algorithm is illustrated in Section 4. And in Section 5 we will give various scaling rules of Mesh-of-Torus data center networks for plane and three-dimension forms. The simulations and results will be analyzed in Section 6. Finally, we end the paper in Section 7 by drawing conclusion and discussing prospect of the application of this new-type network topology.