The Integration Chasm that is Killing Cloud

Cloud is eminently flexible. That’s the core technology proposition, underpinning architectural substrate, the central truth driving the development of this technology. But for all the flexibility, obstacles exist in the cloud computing model of service-based IT delivery. There’s no such thing as an easy-to-integrate multi-poly-cloud set of instances spread around a variety of private, public, and hybrid data center pipes.

Multi-poly-cloud Fundamentals

Before we look at the connection and integration chasm that exists (and hopefully also offer some routes to get around the gulf in front of us), let’s remind ourselves how diverse the cloud landscape is. As we know, there is private cloud and there is public cloud and there are hybrid deployments that span both resource types.

There is also multi-cloud, a deployment scenario where enterprise organizations use more than one cloud services provider (CSP) to host their workloads. The spread here is generally designed to optimize workloads according to different CSP  price points, the availability of specialized tools, the presence of cloud service optimizations (extra storage power, better transactional capacity, super-charged processing or analytics or other) and sometimes even as a result of brand loyalty.

Finally, within the scope of this discussion at least, there is also poly cloud. This is where individual component parts of an application or data service are ‘separated out’ across different clouds based on optimizations similar to those noted above.

Why does all of this matter? There is a weight of industry momentum driving multi (and also poly) cloud. Analyst house IDC thinks that multi-cloud will “achieve total ubiquity in enterprise IT by 2022” no less. At which point 90% of enterprises will depend on multiple private and public clouds.

As we now come through the wake of what the world prays might be the tail end of the COVID-19 pandemic, cloud deployments will obviously and clearly further expand; nobody needs to be reminded how cloud’s remote connectivity DNA has helped the world to still function throughout 2020 and onwards.

Also read: 5G Drives Collaboration Between Carriers, Cloud Infrastructure Providers

Cloud Flexibility isn’t Working

Despite all of this, multi-cloud isn’t delivering. IDG, in a separate report, revealed that 79% of organizations are struggling to realize synergies of employing multiple platforms. Flexera’s 2021 State of the Cloud report details what that looks like: workloads firmly siloed on different clouds in almost half of cases.

“Freedom of choice and the ability to shop around are great, but if the digital ecosystem of providers you use don’t work together then the teams who build, manage and secure your infrastructure will be saddled with disconnected tools, siloed technologies and manual processes,” says Mandi Walls, DevOps advocate at digital operations management company PagerDuty. “They will lack the information to see what’s happening in your infrastructure and an effective means to respond, delaying changes and taking longer to solve technology problems.”

Walls highlights the disconnects with some eloquence, but she also offers some suggestions as to how we can overcome the inherent lack of integration that is now hampering multi-poly-cloud integration, interoperability, and the subsequent orchestrate-ability that we all seek.

One method the PagerDuty team says is an option is to try to keep the technology stack simple. In practice that means working to standardize, say, on virtual machines, network fabric, load balancers and so on. But this loses the elastic benefits of being cloud native.

But other options exist. Wall points out that another option is to work only with common overlapping architectures, but this path is not simple: even with Kubernetes, there are at least 67 certified distributions, according to CNCF.

“Alternatively, you can build an engineering operation capable of taming different tools and services. This means becoming something akin to a systems integrator, with engineering teams building and maintaining multi-clouds around best of breed. It’s the kind of approach undertaken by U.S. retail giant Target, with 4,000 engineers working on its Google and Microsoft multi-cloud,” says Walls.

The core problem with an approach of this kind (or one that is redolent of it with similar constructs) is that it requires thousands of dollars of investment and engineering hours, all of which go towards ‘simply’ building and maintaining a control plane, rather than working to establish differentiation. 

Walls and team insist that a more accessible approach is to use a model where tools, technologies, services, and vendors are pre-integrated in a way that connects teams and information resources together in real time. This is a model that means infrastructure can be smoothly and reliably managed and that allows teams to communicate and collaborate effectively. It might even help enterprises realize the benefits of multi-cloud and crossing the chasm.

Also read: Public vs. Private Cloud: Cloud Deployment Models

Three Cornerstones of Integration

There are many routes, channels, types, platforms, tools, and processes that make up the technology sub-universe that is integration. With some IT vendors self-styling themselves as integration specialists (Tibco would be a good example, although the company has now moved on to wider cloud and data platform status), we can think about integration in the multi-poly-cloud arena from three perspectives.

Workflows

Having eaten its own dogfood and been through the processes to understand how workflows should be handled, PagerDuty’s position on this aspect could be valuable. 

According to Walls, workflows establish procedures that people, teams and systems follow in lifecycle management of applications and in response to IT emergencies. Integrated across the digital ecosystem, workflows provide the consistency DevOps teams need to work at scale; they help teams respond to events and work with IT to fix them quickly and efficiently according to a targeted and prescribed plan.

“Workflows can be codified for lists of approved tools, technologies and platforms; they can, for example, state how to run Python with containers. Workflows should start small and grow as your cloud changes and they evolve as your business and customer needs change,” says Walls.

Automation

Automation is the rails on which workflows run. Walls explains the process at work here and says that automation triggers workflows and processes without manual intervention that could very typically be inefficient and so risk delay in responding to events. 

Blasting, sweeping, silo-bursting

“Automation blasts through the process silos between teams and technologies; it’s a means to sweep up mundane activities such as provisioning a node or rolling out an update. It’s a way to ensure the right members of a team are alerted at the right time when there is an IT-related incident to solve,” clarifies Walls.

Visibility

We can reasonably suggest that integration is the cornerstone of visibility—a means of looking inside your technology operations. Done well, integration provides a means for the teams to gain the information needed to act. Combined with workflows and automation, visibility means teams can also work effectively with others as needed. 

With visibility shot throughout the ecosystem, teams have the added flexibility of working with the tools they want—supporting productivity—rather than having to use prescribed tools.

Walls arguably has exactly the right level of insight to surmise this whole analysis—she underlines the analysis by saying that multi-cloud may have prevailed, but its attributes can act as hurdles to its benefits. 

“Without integration across your digital ecosystem—between tools, technologies, and providers—siloes remain, workflows are truncated, and automation operates on an island. Integration provides the tunnel through which workflows can be combined, systems orchestrated, and processes automated for IT to scale and manage your infrastructure at a platform level,” she says.

If we can get all of these fundamentals under our belts, then we can perhaps feel better about the extended use of multi-poly-cloud implementations for the future.

Read next: What You Need to Know About Cloud Automation: Tools, Benefits, and Use Cases

Adrian Bridgwater
Adrian Bridgwater is a technology journalist with three decades of press experience. Primarily he works as a news analysis writer dedicated to a software application development ‘beat’; but, in a fluid media world, he is also an analyst, technology evangelist, and content consultant. As the previously narrow discipline of programming now extends across a wider transept of the enterprise IT landscape, his own editorial purview has also broadened. He has spent much of the last 10 years also focusing on open source, data analytics and intelligence, cloud computing, mobile devices, and data management.

Latest Articles

Follow Us On Social Media

Explore More