Understanding the way Fibre Channel (FC) identifies domains, and a new mechanism for virtualizing your fabric, enables you to exploit these concepts to your advantage. Building a SAN isn’t difficult—you just plug things in—but to make it resilient in the face of changes, ah, there’s the rub. In this installment of Storage Networking 101, we […]
Understanding the way Fibre Channel (FC) identifies domains, and a new mechanism for virtualizing your fabric, enables you to exploit these concepts to your advantage. Building a SAN isn’t difficult—you just plug things in—but to make it resilient in the face of changes, ah, there’s the rub. In this installment of Storage Networking 101, we will learn about FC domains, address assignment and VSANs.
First, we must understand how a SAN fabric exists without loops. Everything you see here will look suspiciously familiar to Spanning Tree. A few terms are different, of course, but the same concept applies.
The Domain_ID is dynamically assigned to a FC switch when it comes online. The Principal Switch (PS) election begins, which is very similar to a root bridge election in Spanning Tree, followed by the Domain_ID Distribution process.
Before the switch can talk to other switches, it will first configure itself to know what’s attached. Skipping over link initialization, we simply need to know that the hardware works out what port mode is present, and determines the addresses of attached N_Ports. A switch assigns the FCID to each attached node, which is derived from the Domain_ID, Area_ID and WWN of the attached node.
Briefly, this is the election process for determining the PS:
After completion of the PS election, a switch must begin the Domain_ID Distribution process. Even if the Domain_ID is manually configured, the distribution process still occurs, because the PS needs to compile a list of Domain_IDs. The Domain_ID election process isn’t really important, because most people configure the domain manually. Just know that a change of a Domain_ID results in everyone sending an EFP frame with the updated information.
Configuring the Domain_ID is important, because merging fabrics can be disruptive if conflicting Domain_IDs are present. When you have a single switch, and want to extend the fabric by connecting the two together, everything goes fine unless they’re both Domain_ID 1, as some vendors set by default. Every new switch that’s brought online needs to be configured with a unique Domain_ID before connecting it to the fabric.
Conflicting Domain_IDs frequently happen when using VSANs. A VSAN is the same as a VLAN, but for FC networks. You can configure a VSAN-capable switch (usually a Cisco) to segment ports into separate fabrics. One node connected to switch port 1 may be in fabric 322, while the node right next to it lives in fabric 4; two completely separate fabrics. Each fabric may have a domain 31, for example. For the most part, excluding some fanciness implemented by a few vendors, there is no inter-fabric routing, so nodes in different fabrics won’t be able to talk to each other. This is wonderful, but often times it’s necessary to merge two fabrics together.
Merging two fabrics is normally accomplished by connecting multiple switches together. If a “core” switch already had a link to two switches, and suddenly decides to merge the fabrics by placing them in the same VSAN, those switches better have unique Domain_IDs. If not, traffic will suddenly be spotty, since the FCIDs include the Domain_ID. Furthermore, each PS in a domain runs its own name server containing information about N_Ports, and when receiving a frame, a switch will not know which way to send it if it has conflicting information.
Just like VLANs, a VSAN can be used to implement arbitrary boundaries, in ways that make administration much more tolerable, compared to manually moving wiring. The Cisco VSAN technology is gaining widespread adoption since ANSI blessed its implementation, calling it “Virtual Fabrics.” The neat thing about a VSAN is that it’s more capable than the Ethernet’s VLANs.
The Virtual Fabric model takes virtualization to the next level. It is possible to configure a zone server, so that all fabric-attached nodes know how to reach it. FC services run on a switch, unlike the IP world where services like DHCP and DNS normally run on a host. In a VSAN environment, the switch actually runs each service multiple times, once in each fabric.
Speaking of fabric services, there are a few well-known FC addresses associated with SAN services. The brief list is:
FC addresses (the FCID) aren’t actually necessary for SCSI over FC operation. Unicast FC frames are sent to and from the WWN of the node, so the FC address is really only needed in two cases: during link initialization, or when sending IP over FC. When sending IP over FC, and IP address needs to be turned into the FCID. Very similar to the Ethernet world, ARP is used in FC land. Either “ARP over FC” or FARP, which are two distinct protocols, is possible, depending on what the devices support. And you wondered why FC has so many interoperability issues?
Join us next time when we talk about Fabric Zoning and switch configuration.
Enterprise Networking Planet aims to educate and assist IT administrators in building strong network infrastructures for their enterprise companies. Enterprise Networking Planet contributors write about relevant and useful topics on the cutting edge of enterprise networking based on years of personal experience in the field.
Property of TechnologyAdvice. © 2025 TechnologyAdvice. All Rights Reserved
Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.
