Interconnecting Technologies for High Performance Computing

This article is to give and introduction to the technologies that are up against each other to provide high performance computing. We look at Ethernet, Fiber channel, Infiniband and the various consolidations that are proposed. We also take a peek at iSCSI.What is High Performance Computing (HPC)?

When there is a large problem or computation that involves processing a large amounts of data, generally that large computing is broken down in to smaller computing problems, processed separately (each called a cluster) and then the results are integrated. So, it may take several servers / processors to process each of such smaller, broken down computations. Here, since large no. of servers are interconnected to each other, the interconnecting technologies become very critical. Highest possible bandwidth with the lowest possible latency is very critical for clustering. So, we look at the different interconnecting technologies available for HPC.

The typical applications that require High Performance Computing include – Clustering, Fast multiple cores and virtualization.

Ethernet:

This is the most common technology for the edge and the core network of any enterprise. The advantages of ethernet include its flexibility, scalability, plug and play interface, maximum applications, familiarity, cost effectiveness and the presence of a huge installed base.

But for a High Performance computing and applications like clustering, the latency offered by Ethernet is unpredictable. More over there is limited scalability due to spanning tree protocol structure of the Ethernet. Packet drops in congested Ethernet networks and hence it is less reliable. Though enterprises have used Ethernet for HPC applications, it is quite restricted to smaller sizes, with 10Gb switches being most common implementations.

Fiber Channel:

This technology is actually popular in the storage networks (SAN). And most of the current SAN networks run Fiber Channel networks. The advantage of fiber channel is its reliability and decent performance speeds – but lesser than the Ethernet and infiniband. It is very critical that packets don’t drop (like Ethernet) as storage is a more sensitive application, and hence fiber channels are more popular for SAN’s. But when it comes to high performance computing, fiber channels may not exactly compete on the same level with the other two technologies. But their presence in the data centres cannot be ignored.

Infiniband:

This is a technology which gives low latency and high bandwidth. They are considered quite reliable for HPC and quite a number of clusters are already deployed with infiniband. Switches with a 40 Mb/sec kind of bandwidth per port is common for this technology and are already being used in big clusters. It is highly scalable (to even thousands of nodes). So, infiniband, with support from the major HPC vendor is clearly the main focus in this segment.

But the confusion comes with the requirement of managing three different technologies within a data centre. Fiber Channel is already an established technology for storage. Ethernet is irreplaceable (at least for now) for the edge and core networks. And Infiniband has emerged as the preferred choice for the high performance computing. Does this mean that every data centre has to deploy three different technologies and network adaptors/ switches for each of them?

Well, these three technologies are trying to interconnect with each other to try solve this problem. There is no clear strategy with Fiber Channel vendors but the other two – Ethernet and Infiniband are providing a route map to integrate the other technologies to run over them. Like FCOE – Fiber Channel Over Ethernet and Infiniband could support both FC and Ethernet over Infiniband.

What is iSCSI?

iSCSI is a block level storage protocol that allows for the creation of separate storage networks using Ethernet as transport for data between servers and storage area networks. iSCSI uses Ethernet, but is more suitable than Ethernet for storage applications. Though not directly connected to HPC, these things might influence the future of HPC. Which ever technology is used, administrators still have to consider the backward compatibilities and inter-connectivities.

excITingIP.com

You could stay up to date on the various computer networking technologies by subscribing to this blog with your email address in the sidebar box mentioned as “Get email updates when new articles are published”