Towards an Open Datacenter with an Interoperable Network
Part I – Why Standardize ?
Standards have played a pivotal role throughout history. Just ask my 9th grade daughter. This story is going to sound like a digression, but bear with me…like most stories, there’s an important morale at the end that will save money for your IT organization.
This past week, my daughter learned that the economic unification of China between 247 – 221 BC was due, in part, to the standardization of weights and measures, including the length of ox cart axles (which facilitated transport of goods on the road systems ). The history of technology contains many examples like this one, showing how standards are beneficial. They promote buying confidence by helping to future-proof purchases (no need to worry that your new ox cart won’t fit on the roads). They encourage competition and commoditization, which lowers capital expense (if all the ox carts are the same size, then I can buy the lowest priced cart that fits my needs). And they promote innovation, interoperability and avoid confusion in the marketplace (does it matter if my ox cart is red or blue, as long as it fits on the road? Probably not. But if I can build a cart with the same axel width that can hold twice as much produce, then I’ve created meaningful innovation and differentiated myself from the other ox carts).
In the same way, a standardized approach to more modern commodities, like data center switches, makes sense too. Much has been written about how we can standardize on parts of the solution that have long development times, like silicon ASICs, and differentiate through those aspects which have faster turnaround, like software.
But what about the benefits of buying all your ox carts from the same place? Doesn’t this mean you can get lower prices from buying in bulk and having a good relationship with a single ox cart provider? Won’t you have to train your mechanics to only fix one kind of ox cart, using a common set of tools, and thus save training expenses? Surprisingly, the answer is no to all of these questions, according to a study conducted by Gartner Group for major networking equipment vendors at a large number of customer deployments. For example, this study found that working with a single vendor actually costs a premium of up to 20% over multi-sourced environments, since that vendor isn’t constrained by competitive pricing. Since the tools to fix different types of ox carts (and network switches) are mostly common regardless of brand, there isn’t a need to increase staff or training.
In fact, according to this study, CIOs who don’t re-evaluate their single vendor networking choices aren’t living up to their fiduciary responsibilities. So check out this report for more details, and next time I’ll tell you how to distinguish between true industry standard networking implementations, and those who just want to take you for a ride in their ox cart.
Questions about how networking standards can save you money? Ask me through either my blog or Twitter feed.
Towards an Open Data Center with an Interoperable Network
Part II – What are we trying to fix?
Over the past several years, progressive data centers have undergone fundamental and profound architectural changes. Nowhere is this more apparent than in the data center network infrastructure. In this post, we’ll take a look at some of the problems with conventional networks, and next time we’ll introduce the fundamentals of an approach to deal with these issues.
Instead of under-utilized devices, multi-tier networks, and complex management environments, the modern data center is characterized by highly utilized servers running multiple VMs, flattened, lower latency networks, and automated, integrated management tools. Software defined network overlays are emerging which will greatly simplify the implementation of features such as dynamic workload provisioning, load balancing, redundant paths for high availability, and network reconfiguration. Cloud networks with multi-tenancy, resource pooling, and other features are becoming increasingly commonplace. Finally, to provide business continuity and backup/recovery of mission critical data, high bandwidth links between virtualized data center resources are extended across multiple data center locations.
Highly virtualized data centers offer greater resource utilization and lower costs. They can also simplify management if network issues such as latency, resilience, and multi-tenant support for public and private cloud environments are addressed. To realize the greatest benefits from virtualization, networks must be optimized to support high volumes of east-west traffic. This can be accomplished by flattening the network to a two-tier design, using Layer 2 domains to facilitate VM migration, and deploying network overlays to enable efficient virtual switches. While existing storage networks will likely continue in their present role for some time, the opportunity to converge networking and storage traffic is enabled by new lossless networking protocols that guarantee data frame delivery. Each of these exercises requires a non-trivial extension of the existing data network. Collectively, they present a daunting array of complex network infrastructure changes, with fundamental and far-reaching implications for the overall data center design.
The networking industry has responded to these changes with a bewildering array of standardized and proprietary solutions, making it difficult to determine the best course of action. IBM believes that the practical, cost-effective evolution of data networks must be based on open industry standards and end-to-end interoperability of multi-vendor solutions (for a few words on the importance of standards, see my last blog entry). That’s why IBM has recently published a series of technical briefs, endorsed by many industry leading companies, that lay out a path towards an open data center with an interoperable network (which we’ll call by its acronym ODIN….after the ruler of Asguard in ancient Norse mythology. Coincidentally, his symbol the valknut looks a bit like a 2 ties network topology).
Next time, we’ll give you an overview of the first series of ODIN documents and discuss why they’re important. Let me know the biggest problems in your network by responding to this post below, or for shorter problems on my Twitter feed.
I’m on the Edge….of storage…
(and I’m hangin’ on a modem with you)
If you haven’t guessed from the blatant pop culture reference in the title of my blog , I spent the first week of June at the IBM Edge storage conference (and I promise if you keep reading that I’ll refrain from making any puns on the Edge theme – despite the temptation to bring up a favorite Irish rock band hero ). Anyway, it would hardly be appropriate to mention another band when Foreigner did such an awesome job rockin’ the conference. Who knew when I was growing up that the 80’s would produce the greatest rock ballads of all time ?
Anyway, it’s been a great week at IBM Edge, hearing about all the latest advances in storage technology; in case you missed the talk on SVC Stretch Clusters as an example of the ODIN reference architecture, let me say a few words about it here. This will get a bit technical, but don’t worry…we’re not going to have a quiz at the end.
The problem we’re trying to solve is VM mobility over extended distance, and multi-site workload deployment across data centers. VM mobility not only improves availability of your applications, it’s a more efficient way to use limited storage resources. The most common reason for using this approach typically involves some form of business continuity or disaster avoidance/recovery solution, including such planned events as migrating one data center to another or eliminating downtime due to scheduled maintenance. But given an increasingly global work force, there are other good reasons to explore VM mobility. Many clients are realizing that this approach provides load balancing and enhanced user performance across multiple time zones (the so-called “follow the sun” approach). Others are realizing that by moving workloads over distance, it’s possible to optimize the cost of power to run the data center; since the lowest cost electricity is available at night, this strategy is known as “follow the moon”.
IBM has announced a software bundle featuring Storage Volume Controller (SVC), which includes Stretch Clustering over long distance. This provides read/write access to storage volumes located far apart from each other, enabling data replication across multiple data centers. SVC works in concert with Tivoli Productivity Center (TPC) to manage your storage, and integration with VMWare’s products like VMotion and vCenter enables transparent migration of virtual machines and their corresponding data or applications.
Let’s consider two data centers separated by up to 300 km (supported in SVC 6.3), and interconnected by a traditional IP network such as the internet or by dark optical fiber. We require many of the features for an Open Datacenter with an Interoperable Network (ODIN) for this solution, including lossless Ethernet fabrics, automated port profile migration, Layer 2 VLANS in each location, and an intersite Layer 2 VLAN supporting MPLS/VPLS (preferably with a 10G or 100G Ethernet line speed between sites, since the SAN infrastructure is likely running either 8G or 16G Fibre Channel). An SVC split cluster uses industry standard Fibre Channel links for both node-to-node communication and for host access to SVC nodes, so your production sites must be connected by Fibre Channel links or FC-IP.
Generally a business continuity solution will define one physical location as a failure domain, though this can vary depending on what you’re trying to protect against; a failure domain could also consist of a group of floors in a single building, or just different power domains in the same data center. In order for SVC to decide which storage nodes survive if we lose a failure domain, the solution uses a quorum disk (a management disk that contains a reserved area used exclusively for system management). At a minimum, you should have one active quorum disk on a separate power grid in one of your failure domains; up to three quorum disks can be configured with SVC, though only one is active at any given time. Metro mirroring is recommended for this type of solution; a maximum round trip delay of 80 ms is supported (note that routing is required, since the fabrics at each location are not merged).
Connectivity between sites may take several forms. First, if the regular Internet provides sufficient quality of service (QoS) and meets your business objectives for recovery time, recovery point, etc., the IBM SVC uses industry standard protocols (FC-IP) in conjunction with a Brocade switch infrastructure to transport storage over distance. This is typically a low cost option, though you might require multiple circuits with load balancing (a so-called virtual trunk). Second, it’s possible to run a Brocade inter-switch link (ISL) between SVC nodes (with SVC 6.3.0 or higher). Brocade switches provide ISL options including consolidation of up to four ISLs at 4 Gbit/s each (creating a 16G trunk), or up to eight ISLs at 16 Gbit/s each (creating a 128G trunk). Buffer credit support for up to 250 km (nearly the SVC limit) is available. SVC supports SAN routing (including FC-IP links) for intercluster storage connections. Finally, note that you can connect multiple locations with optical fiber and use a variety of protocol-agnostic wavelength division multiplexing (WDM) products in this solution. This may provide better QoS or dedicated bandwidth for large applications. A 10G passive WDM option is available on some Brocade switches (with options such as in-flight compression and encryption), or a stand-alone WDM product can be employed (IBM has qualified many such solutions, including those from ODIN participants Adva, Ciena, and Huawei). Your local service provider may also offer a variety of managed service backup options using a combination of these features. Attachment of each SVC node to both local and remote SAN switches (without ISLs) is typically done in this case. Both the ISL and non-ISL approaches are known as split I/O groups.
IBM SVC storage manager works in concert with vCenter through API plug-ins. This includes VADP (which provides data protection for snapshot backups at the VMware-level rather than the LUN level, allowing you to concentrate on the value of the VM rather than the physical location of the associated data). Performance improvements can be achieved by offloading some functions to the storage hypervisor, as well. The storage hypervisor includes a virtualization platform, controller, and management (TPC supports application aware snapshots of your data through Flash Copy Manager). At the management level, IBM also allows the storage hypervisor components to be managed as plug-ins for vCenter. VM location can be managed through vCenter with Global Server Load Balancer (GSLB), which works in concert with a Brocade API plug-in. Further, vCenter is integrated with Brocade Application Resource Broker (ARB), which can report VM status back to a Brocade ADX switch. vCenter and GSLB manage both VM and IP profiles, performing intelligent load balancing to redirect traffic to the VM’s new location.
With this combination of ODIN best practices, IBM SVC, and Brocade SAN/FC-IP connectivity, your data can rest easy, wherever it happens to be (and so can you).
Want to know more about IBM/Brocade partnerships , or SVC Stretch Cluster solutions ? Drop me a line, or ask a question on my Twitter feed.
SDN at the NSF ECC: or, the latest OpenFlow research
Guess I’ve been doing too much Twittering lately; for the acronym challenged, I’m talking about Software-Defined Networking research at the National Science Foundation’s conference on Enterprise Computing, hosted at Marist College (one of the prominent endorsers of the ODIN documents). This past week at the Enterprise Computing Conference has been really interesting; if you’ve never been to this event, I’d encourage you to consider it. Among other things, I learned about several recent case studies on proof points showing that IT education can lead to productive employment, and about a free IBM service that connects people with System Z skills to potential jobs. While there have been a lot of good talks, I’d like to spend some time today going over Marist’s contributions to software-defined networking and OpenFlow..
For those of you who don’t know what SDN and OpenFlow mean, beyond being some of the hottest buzzwords in the networking industry right now, you can check out the appropriate volume of the Open Datacenter Interoperable Network (ODIN) reference architecture for a detailed introduction to this topic and the problems it addresses. For those who just need a quick refresher, software-defined networking is an approach which allows the basic data flows through a switch to be manipulated through an external controller. It’s an industry standard approach being led by the Open Network Foundation (ONF), a consortium run by the world’s largest network users (Google, Facebook, Verizon, and more). OpenFlow is a relatively new industry standard which separates the data plane and control plane of a switch, creating flow table abstractions (in other words, you can match data flows based on content of the packets and perform actions associated with each flow match; if you don’t assign a flow, traffic can be blocked or filtered using this technique). Optimal paths through the network are defined by the OpenFlow controller, rather than some proprietary software within the switch.
One of the potential benefits of OpenFlow is that it allows you to innovate at Internet speeds, by just changing the software and not replacing or reconfiguring the switch hardware. There are still open questions about just how large an OpenFlow controller can scale, how many controllers we need, etc. Marist College has created an SDN lab which will contribute to the OpenFlow community, support research around SDN, and possibly support compliance testing in the near future. They are engaged with some large OpenFlow switch providers (including IBM) and some interested OpenFlow adopters (to be named later) to investigate use cases and performance limitations of the current OpenFlow protocol. Their current lab environment includes four IBM G8264 OpenFlow enabled 10/40G switches in a spine-leaf configuration, running under an open source FloodLight controller. These switches interconnect a server farm based on IBM x86, Power, and System Z enterprise servers. Many of the x86 servers run the VMWare hypervisor and the IBM 5000v virtual switch. The servers are connected via a separate Fibre Channel SAN to various enterprise storage devices.
One of Marist’s early contributions has been to create an open source FloodLight administrative control panel (FACP) that can be used for network administration. The FACP eliminates the need to write Python script to control the switches, thereby reducing management complexity. FACP provides an abstraction of the network, and a configuration application can be build against this abstraction. At the conference, Marist held a demo showing how this controller can provision quality of service and routing of Layer 2 & 3 VLANs in the network. Manipulation of firewall ACLs is also possible, and future extensions may include MPLS and other WAN related protocols. Ongoing work in this area is focused on creating a static flow pusher, which will allow a static programmable interface to write scripts which support flow tables across the network using the FloodLight rest API.
Further investigation will include such topics as demonstrating multi-vendor interoperability under a common FloodLight controller, and exploring the limits of scalability and security associated with OpenFlow networking. Keep up with their latest work and see their presentation from the NSF conference .
Want to suggest another TLA (three letter acronym) for my list ? Comment on this blog entry below, or drop me a line on my Twitter feed.
ODIN Sets the Standard for Open Networking
"If you want to go quickly, go alone. If you want to go far, go together." – African proverb
During InterOp 2012 in Las Vegas, IBM released a set of five technical briefs which lay out the path towards creating an Open Datacenter with an Interoperable Network (ODIN). This approach uses industry standards as the preferred means to address key issues in next generation data center networking. The response has been tremendous, and ODIN has been very well received across the industry. I've been posting a lot about this in my blog lately, but for your convenience here's the current list of everyone who's endorsed ODIN so far, in no particular order:
Juniper Networks noted in an endorsement from their Vice President of Global Alliances that there is an unprecedented array of technical challenges which ODIN will address, including cost effective scaling, highly virtualized data centers, and reliable delivery of data frames.
Brocade said that “using an approach like ODIN…facilitates the deployment of new technologies”
Huawei said that “ODIN addresses best practices and interpretations of networking standards that are vital to efficient data center operation.” Also, Huawei Fellow Peter Ashwood-Smith shows an ODIN view of the future data center network in his webinar for Interop, entitled “How to prepare your infrastructure for the cloud using open standards.”
Extreme Networks said in their endorsement that “Having open, interoperable, and standard-based technologies can enhance (these) cost savings by allowing choice of best-of-breed technologies.”
NEC noted that software-defined networking (SDN) is part of ODIN, and has emerged as the preferred approach to solving Big Data and network bottleneck issues.
BigSwitch said in their blog “The Importance of Being Open” that “ODIN is a great example of how we need to maintain openness and interoperability in next generation networks”
Adva Optical Networking in their blog on "the missing piece in the cloud computing puzzle" talked about the role of ODIN in the wide area network, including both dark fiber solutions, MPLS/GPLS, and emerging trends using SDN to manage cloud computing and the WAN. They also cited recent SDN work with the Ofelia project in Europe as an example of ongoing work towards open standards in the WAN.
Ciena pointed out in a post from their CTO and Senior Vice-President that “the use of open standards has been one of the fundamental “change agents” in the networking industry”. These standards are “associated with encouraging creativity by enabling a diverse and rapidly expanding user group” and “generally support the most cost-effective scaling”. They called ODIN “a nearly ideal approach” and said that ODIN “is on its way to becoming industry best-practice for transforming data-centers”.
Marist College provided a university’s perspective, as their CIO noted that their support of ODIN was part of their broader efforts to insure that the next generation of technology students are prepared for the challenges which await them. Marist also cited related work with their National Science Foundation funded lab for enterprise computing and their cloud computing computational resources.
Thanks to everyone for showing your support of open industry standards and the ODIN approach to data center networking. I’m honored and humbled by this strong show of support from so many industry leaders, and I’m very excited to be taking the first steps with all of you on this journey towards a more open, interoperable data center network. As we continue to develop more content for ODIN, both around new standards as well as deeper technical descriptions of reference architectures which implement the ODIN design principles, I’ll keep you posted on further activities with these and other companies.
Would you like to be next to endorse ODIN, and receive eternal fame and glory by being mentioned in my blog ? Let me know where I can point to your endorsement, or drop me a line on my Twitter feed
ODIN endorsed by Extreme Networks
Earlier today, IBM released a series of technical briefs describing the Open Datacenter Interoperable Network (ODIN) during InterOp. The ODIN approach to open networking has been endorsed by Extreme Networks, and you can read about it in their blog post. Both companies share a commitment to open industry standards within the data center network, an approach which should benefit clients with a lower total cost of ownership and superior performance.
NEC endorses ODIN
During the 2012 InterOp conference in Las Vegas, IBM introduced a set of technical briefs describing the path towards creating an Open Datacenter with an Interoperable Network (ODIN). The approach of using open industry standards in the data center network was recently endorsed by NEC Corporation on their corporate blog. In particular, NEC mentions IBM's work with the Open Network Foundation (ONF) and their efforts to create software-defined networking standards (including both OpenFlow and network overlays) for next generation data center networks. I'm very pleased by NEC's support for software-defined networking and other open standards in the data center network, stay tuned to this blog or my Twitter feed to hear more about this and related topics.
Marist College endorses ODIN
In addition to the many industry leading companies who have endorsed IBM's recently released technical briefs, describing an Open Datacenter with an Interoperable Network (ODIN), the first academic endorsement of ODIN has recently come from Marist College (Go Red Foxes !). In their endorsement, Marist notes that their support of ODIN was part of their broader efforts to insure that the next generation of technology students are prepared for the challenges which await them. Marist also cited their related work with the National Science Foundation funded lab for enterprise computing, their network interoperability lab, and their cloud computing computational resources. Also commenting on ODIN as part of their Twitter feed were IBM Vice President Ross Mauri (a member of the Marist Board of Directors) and Marist Vice President and Chief Information Officer Bill Thirsk. I'm sure there will be opportunities for IBM and other ODIN supporters to work with colleges such as Marist on research and interoperability that will benefit the open design principles set forth in the ODIN documents.
Juniper endorses ODIN Approach
I’m pleased to report that Juniper Networks has publicly endorsed the open data center interoperable network (ODIN) approach to designing data center networks. If you've been following this blog, then you know that on May 8, IBM released a set of technical briefs describing ODIN during the InterOp conference in Las Vegas. This approach to using industry standards as the preferred means to designing data center networks has been supported by Juniper, as discussed in this blog post from Liz King, Vice-President of Global Alliances. Many thanks to Juniper for their support of open networking standards; I’m sure we’ll have more to say about how these solutions should be designed in the near future.
Huawei mentions ODIN during InterOp webinar
During a webinar presented at InterOp 2012 describing how to prepare your infrastructure for the cloud using open standards, Huawei has indicated their support for the Open Datacenter Interoperable Network (ODIN) approach. Huawei joins a growing number of companies who recognize that the best path forward for next generation data centers lies in the use of open industry standards and protocols. You can read more about the importance of open standards and ODIN in my earlier blog posts or through my Twitter feed. Stay tuned for the latest news from InterOp and the world of data center networking !
Huawei says more about ODIN
As noted in a recent post on this blog, Huawei had included a mention of the Open Datacenter Interoperable Network (ODIN) in their InterOp Webinar on open standards for cloud networking. In addition, Huawei has now posted a more detailed endorsement of ODIN on their blog site. According to this site, " ODIN addresses best practices and interpretations of networking standards that are vital to efficient data center operation". For those of you who haven't reviewed the ODIN materials yet, they include a description of the transformation taking place in modern data center networks and how to best address these issues using open industry standards. Keep watching this space for more news on ODIN and other data center networking issues.
Ciena endorses ODIN
The list of companies endorsing IBM's recently announced Open Datacenter with an Interoperable Network (ODIN) continues to grow. Ciena is the most recent company to endorse ODIN, as noted in their blog post from their CTO and Senior Vice-President, Products and Technology, Steve Alexander. In this post, Ciena says that ODIN "looks to be a nearly ideal approach to allow the connect, compute, and store resources to be virtualized and operationally united for simplicity and scale". In fact, the use of industry standards to enable more tightly integrated solutions has been recently demonstrated in IBM's PureSystems offerings, which were announced on April 11; you can read more about PureSystems in my earlier blog posts. I'm very pleased that Ciena has endorsed the ODIN approach, and I'm sure we'll see more examples of this design approach in the coming months. Remember, let me know what you think about ODIN by commenting on this blog, or on my Twitter feed, and keep watching this site for the latest data center networking news.
BigSwitch adds their endorsement to ODIN
Following the recent release of the Open Datacenter Interoperable Network (ODIN) technical briefs at InterOp 2012, several companies have publicly pledged their support for the ODIN approach to open standards. Most recently, BigSwitch Networks has posted to their blog with a nice summary of recent open standards activities at InterOp, including their endorsement of the ODIN technical briefs. IBM deeply appreciates this show of support for open standards in the data center network, including the full breadth of software defined networking (SDN) approaches (both overlay networks and OpenFlow). IBM has demonstrated the industry's first 40G OpenFlow enabled switch, and continues to drive strong innovation in SDN and other aspects of the ODIN design approach. Keep watching this blog for more news on ODIN and InterOp 2012, or follow my Twitter feed.
Adva endorses ODIN
Following IBM's announcement this past week at InterOp, there has been a surge of interest in the recently proposed Open Datacenter Interoperable Network (ODIN) technical briefs. I'm pleased to report that Adva Optical Networking, a leading wavelength division multiplexing (WDM) company specializing in WAN transport, has endorsed the ODIN approach on their blog. There's never been a better opportunity for cloud data centers to get in on the ground floor by designing their next generation networks around the best practices and open industry standards referenced in ODIN. We appreciate Adva's support of this direction, and look forward to working with leading industry networking companies to implement the ODIN design recommendations in modern data center networks.