Storage/Network Convergence with FCOE: Fiber Channel Over Ethernet – Need, Advantages & Limitations

What is the need for FCOE – Fiber Channel Over Ethernet?

High density storage for large organizations use SCSI and FC – Fiber Channel protocols. So, they reside in a separate network with their own adapters (HBA), FC Ports, FC Switches, Cables, etc. Fiber Channel has been reliable (lossless), and has good performance, speeds and hence they are very popular for large scale storage systems.

But, with our ever pervasive IP Networking protocol standardizing on the Ethernet medium, all companies have their IP Networks running over Ethernet. People saw the opportunity to converge both the Computer Network and FC SAN Network, and hence two major protocols were born to enable such convergence – iSCSI and FCOE.

iSCSI (SCSI over TCP/IP) replaced the FC stack with the TCP/IP stack so that the storage traffic could be transported over the low cost TCP/ IP Networks. This works well enough for small/ medium scale implementations but large networks still demanded the reliability of the Fiber Channel (FC) Network. Hence, FCOE – Fiber Channel Over Ethernet was conceived.

What is FCOE – Fiber Channel Over Ethernet?

FCOE or Fiber Channel Over Ethernet transports the SCSI storage data (used in Fiber Channel – FC networks) using Fiber Channel Protocol stack instead of TCP/IP stack; using the Ethernet infrastructure (NIC, Cables, Switches, etc). FCOE maps the FC commands and data directly into Ethernet frames (Fiber Channel frames are encapsulated in Ethernet frames) to create FCOE and the mapping is 1:1 meaning, there is no segmentation or compression of FC frames.

But, as we know, Ethernet is a lossy medium – it provides a single best effort pipe that drops packets during a network congestion. So, in FCOE, FC is simply encapsulated and run over a lossless Ethernet infrastructure.

So, how is that Lossless Ethernet Infrastructure created by FCOE?

FCOE has to create a lossless Ethernet environment in order to ensure the reliability of large scale storage data transportation. Two standards enable that – The Data Center Bridging (DCB) and Converged Enhanced Ethernet (CEE). A few of the enhancements to Ethernet (to make it lossless) are described below to get an idea about DCB and CEE.

IEEE 802.1Qbb: Priority Flow Control – IEEE had earlier defined the means to categorize traffic according to its priority – IEEE 802.1p (QoS Standard). So, the new standard IEEE 802.1bb takes advantage of this standard by first allowing to classify the traffic in to eight categories (lanes), each of which could be assigned a priority level. Priority Flow Control issues a ‘Pause’ command that allows to halt FCOE traffic during congestion so that the losses can be minimized and it uses the priority level to recognize FCOE from other types of traffic. So, the administrators can create lossless (virtual) lanes for FCOE traffic and lossy (virtual) lanes for normal IP based traffic.

IEEE 802.1Qau: Congestion Notification – Congestion is measured at the congestion point, where ever it is happening but the action (like rate limiting) is taken at the reaction point (originating point). For example, an aggregation switch can ask an edge switch to stop (or limit) its traffic from a particular port, if congestion is encountered, through this standard.

IEEE 802.1Qaz: Enhanced Transmission Selection – High Priority traffic (like FCOE) can be allocated with a minimum guaranteed bandwidth so that all the other traffic on the network does not congest the path due to their high volumes.  However, if FCOE traffic does not fully utilize the path (its reserved capacity), then the bandwidth is  allowed to be used by other types of traffic and this can be controlled dynamically by this protocol.

What are the components of FCOE – Fiber Channel Over Ethernet Infrastructure?

There are basically three components of FCOE infrastructure:

1. Converged Network Adapter (CNA) – The CNA is a single adapter in the server (that attaches to PCI Express Slot) which can provide the functionalities of both Ethernet NIC (Network Interface Cards) and FC HBA (Host Bus Adapters) virtually. That means, the server still sees two Interfaces and it sends the IP traffic to the NIC and FC traffic to the HBA. But the CNA collects both of them, and transports it over a single Ethernet cable after having wrapped all the Fiber Channel (FC) frames to Ethernet frames.

2. FCOE Links – The FCOE infrastructure utilizes the same Ethernet infrastructure used by the TCP/IP Network. So, it uses the UTP Copper Cables, Optical Fiber Cables and even the low cost Twinax cables that use the SFP+ Interface to carry 10 GE for short distances.

3. FCOE Switches/ Network Switches supporting FCOE protocol – The Fiber Channel SAN’s understand only the Fiber Channel Protocol and Fiber Channel Interfaces. So, there needs to be an intermediary that separates the FCOE traffic from the regular IP traffic and connects to the FC SAN’s directly. This intermediate functionality is given by FCOE Switches (or) Network switches (with FC Ports) supporting FCOE protocol. So, the HBA’s from the Servers connect to this FCOE Switch which in-turn connects to the SAN Network using FC ports and IP network using IP Ports.

What are the advantages of FCOE – Fiber Channel Over Ethernet?

• FCOE reduces the two network adapters (HBA for storage connectivity, NIC for computer network connectivity) and two individual cables from each server to just one, thereby simplifying the network.
• FCOE can carry traffic over the Ethernet medium and hence use the familiar and easily available copper UTP cables and Optical Fiber Cables.
• One Network Adapter instead of two gives some power savings for the server.
• There are certain I/O Virtualization solutions that support FCOE which reduces the total number of server adapters (CNA) for a group of servers by consolidating them on to a I/O Virtualization appliance and allowing the servers to share the common pool of adapters. This, reduces the total number of CNA’s required for a group of servers. The servers themselves connect to the I/O Virtualization appliance through interfaces like PCI Express / appropriate cables from there. But certain proprietary vendor based drivers may have to be installed, to complete this setup.
• The performance of FCOE network is is comparable to that of FC/ IP networks with FCOE supporting the speeds of the Ethernet network ( Up to 1/ 10 Gbps currently and expanding to 40 Gbps and 100 Gbps in the future).
• FCOE can be used in virtualized environment (Server Virtualization) and is quite advantageous to use in such circumstances.
• FCOE, unlike iSCSI, is a very reliable Storage transportation protocol, and can scale up to thousands of servers.
• Since FCOE just encapsulates the FC data onto Ethernet frames for transportation only, all the existing administration tools and workflows for FC (Fiber Channel) remain intact and hence the investments in existing Fiber Channel storage is preserved while the reliability of FC is also maintained.
• The support from Network Switch vendors for FCOE by offering converged switches with both Ethernet and Fiber Channel ports strengthens the case of FCOE.

Dis-advantages / Limitations of FCOE – Fiber Channel Over Ethernet:

• The only Ethernet component that is currently compatible with FCOE is the cables. Everything else will have to change for implementing FCOE. So, the actual component saving today would only be the amount and cost of cables!
• The cost of a Unified CNA (though it is coming down) might still be more than the cost of the HBA and NIC combined!
• FCOE is currently restricted to Access Networks only (Server to Switch Connections). The distribution and core storage networks are still in Fiber Channel and will continue to be in Fiber Channel till FCOE technology matures enough to create its own FCOE SAN networks.
• iSCSI proponents might still argue that changing one disparate network in to another does not much amount to convergence of storage and network infrastructures.
• Security on FCOE networks might have to be re-looked as the network is now running over Ethernet and hence is more easily accessible than their Fiber Channel counterparts.

excITingIP.com

In case you want to add any more points or have clarifications, you can use the comment form below to drop a comment on this article or contact us using the contact form. You could keep yourself updated on the various Computer Networking technologies by subscribing to this blog by submitting your email address in the box titled “Get Email Updates When New Articles are Published”.