Why DevOps and the Cloud Are a Match Made in Heaven
DevOps and the cloud. They're an obvious match, but what elements of the cloud, and of DevOps, make the match so obvious?
A Look at DevOps
First, a very quick historical recap. DevOps grew out of the Agile movement, which was largely focused on applying streamlined (i.e. very agile) business-organisation practices to software development. DevOps extended Agile processes and philosophy to IT, combining development and operations into a single, unified set of practices.
DevOps eliminates silos (walled-off and largely non-interacting job/task categories), treats infrastructure itself as code, automates infrastructure processes, and replaces intermittent deployments with a continuous delivery chain. It relies heavily on virtualisation, both by deploying applications using virtual machines, and by breaking large applications down into microservices deployed in containers (stripped-down virtualised environments which rely on the host OS for many basic services).
Much of the current emphasis in DevOps is on moving to full containerization, and away from dependence on monolithic applications and virtual machines.
And a View of the Cloud
The cloud, in the meantime, grew more or less directly out of expanding hardware capabilities. Increases in processor speed, memory and storage capacity, and network speed and bandwidth all made it possible to pool a large set of individual servers into a virtualised, always-available super-server—something which major players in the IT and software industries (including Microsoft) had been trying to do with varying degrees of success for several years.
As the cloud became a practical reality and came into widespread use, it was natural (and probably inevitable) that it would give rise to cloud-based and cloud-oriented ways of doing software development and IT.
Not surprisingly, DevOps has turned out to be a near-perfect fit for the role of cloud-oriented development/IT methodology. Technology historians can debate about whether DevOps actually developed its core ideas in response to cloud computing or simply converged with it—what is clear is that DevOps and the cloud have been so closely allied in their development that it is becoming increasingly difficult to draw a boundary between them.
Virtual is Real
The cloud is by its nature a virtual environment. While a cloud service provider's IT staff may work at times with bare metal, cloud users at all levels (including IT teams deploying applications in the cloud) are insulated from the underlying hardware by several layers of abstraction.
If you develop or maintain software in the cloud, you are always working in a virtual environment; virtuality is reality. Virtual deployment on your part simply adds another layer of abstraction, generally with little or no performance penalty. Virtualisation-within-virtualisation becomes not only unavoidable, but a matter of everyday business.
DevOps is a natural framework for cloud deployment in this regard. In the context of the cloud, the DevOps emphasis on virtualisation no longer even seems like advocacy—It is merely a description of how things are done. The move to containerization is a way of making optimum use of the naturally-virtual cloud environment; it minimises any performance penalty that might exist, and in many ways, it is the most sensible and logical method of deploying services in a completely virtualised world.
Infrastructure is Nothing But Code
If you were to strip the technical side of DevOps down to a single slogan, it would probably be, "Infrastructure is code!" This isn't just a statement about the nature of infrastructure, of course. It's about what you should do with it.
If it's code, it's programmable (or scriptable, or accessible via API). And if it can be programmed or scripted, there is no reason why you should take the time and effort to manage it manually.
Manual control of infrastructure is expensive, it brings in added delays, and it encourages the proliferation of specially-configured snowflake systems, which can multiply cost, time, and effort. The ideal DevOps infrastructure is fully scripted and fully automated.
In a world of bare metal and individual servers, of course, achieving full infrastructure automation is likely to be an upstream swim against very strong currents. But once again, in the cloud, what appears to be pure blue-sky advocacy on the part of DevOps turns out to be a simple description of reality.
When you deploy in the cloud, the only kind of infrastructure that you can come in contact with is code. It is typically all scriptable, and access to the API is part of the package. Full automation in such an environment is more natural than manual configuration and hands-on control.
Continuous Really is Continuous
Continuous delivery is another key DevOps concept. In a world where software and services are often deployed via the Internet or on an enterprise-wide level, the kind of intermittent updates typical of traditional software development make no sense.
The sporadic nature of traditional deployment is for the most part a result of the logistical complexities of testing and distributing software for desktop systems, or installing it on mainframes. If all you need to do to deploy an update is run a script that uploads it to a server and handles the installation, the obvious next step is to streamline the development and testing processes so that the entire development/deployment chain becomes one continuous flow.
And the cloud is tailor-made for continuous deployment. It is fully scriptable, and updates can typically be uploaded to a single target system. In the cloud, there are no built-in delays.
A Revolution in Testing
Quite often in traditional deployment, the biggest delays come from testing. Non-cloud tests are generally sequential, and can add days or weeks to deployment time. The cloud has produced a rather quiet but profound revolution in testing: large-scale parallel tests run on cloud-based virtual machines have cut testing time down to a fraction of what it once was.
Tests that once took days can be completed in a few hours, tests that once required weeks may take a day or less, and tests that were previously impractical because of time constraints can now be included in the testing regime with little impact on testing time. The cloud makes continuous delivery possible with no reduction (and often an actual increase) in software quality.
DevOps and the cloud
In many ways, what appears to be advocacy of an ideal system in DevOps turns out to be nothing more than a description of everyday reality in the cloud. If DevOps hadn't grown out of the Agile movement then it, or something very DevOps-like, would almost certainly have developed from cloud deployment practices as a pragmatic response to the capabilities and challenges of the cloud.
DevOps and the cloud are a natural match for each other. But this isn’t simply a ‘nice-to-have’ - enterprises can save millions by using cloud platforms and DevOps ways of working in combination.
