App Testing From the Comfort of the Cloud
Organizations are starting to see that using a cloud approach to testing can serve to better accommodate demand dynamics. There's even been a shift in the actual approach to development. Instead of gathering up bushels of code for one big build, developers are taking a more agile tack, making more builds more often.
One area of cloud computing that has really taken off and generated a lot of interest is the development test and performance proofing of applications -- all from an elastic cloud services fabric. The build and test basis of development have traditionally proven complex, expensive, and inefficient. Periodic bursts of demand on runtime and build resources are the norm.
By using a cloud approach, the demand burst can be accommodated better through dynamic resources, pooling, and provisioning. We've seen this done internally for development projects, and now we're starting to see it applied increasingly to external cloud resource providers like Amazon Web Services.
Here to help explain the benefits of cloud models for development services and how to begin experimenting and leveraging external and internal clouds -- perhaps in combination -- for test resource demand and efficiency, are Martin Van Ryswyk, vice president of engineering at Electric Cloud. We're also joined by Mike Maciag, CEO at Electric Cloud.
Listen to the podcast (23:25 minutes).
Martin, let me take our first question to you. As I mentioned, we're seeing a lot of uptake in development aspects of the life cycle toward creation, test, and ultimately production for applications. Why is this such a big deal now? We've obviously had a lot of these technologies around for quite a while.
Martin Van Ryswyk: You're right. The technology and the need have been around for a long time. Folks have always wanted their builds to be fast and organized and to be done with as little hardware as possible.
In places I've worked, we've always struggled to get enough resources applied to the build process. One of the big changes is that folks like Amazon have come along and really made this accessible to a much wider set of build teams. Large companies have big data centers, but even those are filling up, and so Amazon has really made the difference.
The dev and test problem really lends itself to what's been provided by these new cloud players. The need for lots of different types of machines, the matrix problem of how you're going to test, the "burstiness" nature of it -- I might have only integration storms where I need machines -- all seems to lend itself to what's being provided.
Dana Gardner: I suppose there have been shifts in the actual approach to development. There was a time when you would get a lot of code together and you would get in one big build, but we seem to be now in a more agile-development mode, in many instances, where we need to do almost constant builds.
Van Ryswyk: Absolutely. There are more builds happening more often. The days of the nightly build are pretty much gone for most of the people we talk to.
The other part of that agile approach is bringing testing into the automated part of doing the nightly build. So, it's build, test, and deploy a lot of times -- deploying either to production or to other test environments.
There is no brick wall any more to throw code over to test teams. All of this is happening as one seamless operation, and it's happening more often. So you're doing more and doing it more often. It just puts a tax on the resources.
Gardner: Mike, what are the reasons the traditional approach to this has fallen short? There are economic reasons, but on the technical side, have we missed anything?
Mike Maciag: No. The traditional approaches of the concept of the overnight build or even to the point of what people refer to as "continuous integration" have fallen short, because they find problems too late. The best world is where engineers or developers find problems before they even check in their code and go to a preflight model, where they can run builds and tests on production class systems before checking in code in the source code control system.
Gardner: Martin, help us understand a little bit about what Electric Cloud brings to the table. With ElectricCommander, for example, what is the problem set that we're solving here?
Van Ryswyk: ElectricCommander solves the problem of automating and making it efficient to do the entire build process. There's a lot of creative juice that goes into making products -- developer, design, requirements. There are tools to help you store your code safely. All of that is part of the creative process.
But at a certain point, you just want it to happen like a factory. You want to be able to have builds run automatically. You want them to run at 3:00 in the morning. You want to run them in continuous integration style, based on a trigger from a software configuration management (SCM) system or before the SCM system even gets it, as Mike said. That's what ElectricCommander does. It orchestrates that whole process, tying in all the different tools, the SCM tools, defect tracking tools, reporting tools, and artifact management -- all of that -- to make it happen automatically.
When you do that, resources are a part of that, and that's really where the cloud part comes in, having to run all those in parallel a lot of times. At the same time, different architectures have different operating systems. Then, you're bringing it all back together for a cohesive end report, which says, "Yes, the build worked."
Gardner: So, we're in the early innings, but it seems to me that the same complexity of needing to test across a variety of platforms, environments, stacks, and configurations is now being bumped up into the cloud.
I have to assume that there is going to be more than one cloud, although we sometimes mistakenly refer to it as "the cloud." Is that correct -- that we're going to have, in essence, the same heterogeneity complexity that we had to manage internally but now at a higher abstraction?
Van Ryswyk: Absolutely. ElectricCommander, for example, was already allowing customers to manage the heterogeneity on physical machines and virtual machines (VMs). With some integrations we've added -- and I think people want to do this in general -- you can now extend that into the cloud. There will be times when you need a physical machine, there will be times when your virtual environment is right, and there will be times when the cloud environment is right.
Maciag: What we've seen is that the most sophisticated customers have been able to build private clouds to do exactly what Martin is talking about. The exciting part about the public cloud is that it's bringing those types of infrastructures into the affordability range and sophistication range of much smaller organizations.
Dana Gardner is president and principal analyst at Interarbor Solutions, which tracks trends, delivers forecasts and interprets the competitive landscape of enterprise applications and software infrastructure markets for clients. He also produces BriefingsDirect sponsored podcasts. Follow Dana Gardner on Twitter. Disclosure: Electric Cloud sponsored this podcast.