As the world continues to spin toward real-time processes, real-time apps, and real-time expectations, it’s easy to see why the practice of continuous testing is primed to hit mainstream in software development circles. Continuous testing, which came to the forefront with Agile, is the process of ensuring that a new feature is continuously tested at every stage from planning and design throughout the entire software delivery pipeline.

Continuous testing (CT) is grounded in the practice of continuous delivery (CD), the process of building software so that at any given point in time it can be released into production, as defined by Martin Fowler. In the traditional software development lifecycle, a release might take several weeks or months, or even longer. For many projects and organizations this makes sense and will continue to be the preferred approach for some time. Those organizations that want or need to follow the Agile path of CD, require the ability to deploy working software at any point by “pushing a button,” so to speak. The big umbrella here, of course, is DevOps, which encompasses CD, continuous integration (CI), and CT.

The job of CT is to make sure that at any point in the development lifecycle, the development team has deployable software. Because of this “deploy anywhere, anytime” mentality, traditional testing in longer-running phases won’t work.

The foundation of CD is rapid, iterative cycles and incremental development, which leads to incremental testing. In traditional development, testing happens in distinct, isolated phases, such as exploratory, regression, and acceptance testing. With CT, the testing focuses on a small unit of work that can be released to production as soon as it’s finished: think building blocks.

CT incorporates another popular Agile testing method called test-driven development or TDD, which relies on automated testing of features that run in parallel with development. TDD calls for test scenarios to be written before development begins. CT also incorporates the "automation first" approach, but extends it to include manual and exploratory testing. The reality of software development is that automated tests do not always provide full coverage. In CT, those test scenarios may also be used to define manual or exploratory tests. TDD is just one tool in the CT process.

By starting with a contract or test case that is well understood to incrementally test a feature requirement, you ensure that as a small iterative unit of work completes, it meets that contract in such a way that it is releasable, deployable software. Exploratory testing tools for new feature development come to play as do coverage tools that send data showing anomalies between releases back to the quality process. Coveralls.io is a great tool that’s easy to configure and has wonderful visualizations for the most popular languages, while Jenkins has a highly customizable dashboard.

Some of the tools that teams use for CT are also used for CD and CI, which focuses on automated builds and deployments. These applications include Jenkins, Team City, Puppet and many others. Such technologies help teams understand status of projects, collaborate and get instant feedback at any point in the software lifecycle, along with automating a lot of the work.

Containers and virtualization software have also played a critical role in the adoption of CD and CI on software teams, because this faster, highly iterative process requires ample compute power and storage. With newer infrastructure technologies that also happen to be a lot cheaper, such as the cloud, it’s much more viable to implement CD in companies.

Testers may also use open source tools such as the free-to-use CircleCI and Github Issues (for tracking) to integrate a continuous delivery pipeline.

Technology can’t solve all problems, however, so developers and testers will need to change some of their workflows to master CT. These concepts are closely linked to the Agile and DevOps practices you are probably already using, so adapting testing in this way should not be a huge shift.

• Developers should work on feature branches that can isolate the work and reduces risk to the master branch. This delivers quick, rapid feedback by running subsets of test cases. Add TDD to this process as you see fit to bring even more efficiencies.

• Pair developer with testers. This common Agile practice helps with the increased velocity of communications, which is required for CT, and also helps each role to educate and train the other.

• Adopt the AWS "two pizza teams" concept, which calls for small teams that focus on a single product from development through testing, to deployment.

• Teams should exercise the "Pull Request" model using automation and gating as much as possible during the process. A typical workflow might look like the following:
  • A developer creates a pull request based upon a feature branch he’s been working on.
  • Upon a successful build, the merge is completed into a testing branch.
  • A set of status checks run here, including unit tests with CI, potentially integration checks, and automated deployment to specific environments with tools like Docker Compose and Puppet.
  • If the automation and integration tests pass, the code is merged into a staging branch and is deployed to have regression/manual checks performed.
  • With tight integrations into your ALM, VCS, and CI systems, it’s possible to signal to each that this manual/regression testing has passed and failed.
  • Once these checks have passed, CT promotes CD (the ability to deploy now), and a team can move on to practicing CD into their production environment on the schedule they choose.

• Make metrics your friend, and use visualization. These are commonly called “information radiators.” Ideally, they indicate status of release, alerts for projects, code changes, and more, which leads to shared knowledge across the team to solve problems and keep the wheels churning.

• DevOps automation, fueled by many different powerful tools, including Chef, Amazon Web Services (AWS) Code Deploy, and Docker means that there should be no manual operators deploying software. This automation enables CT as a pipeline of deployment. If code passes automated status checks at stage 1, it automatically deploys to stage 2 without a different team having to manually intervene and track this process. Testers are notified by software where and when something is ready to test.

• Break testing tasks down into small, incremental steps at every point in the cycle. Teams run through the development/test/deploy cycles frequently so that testing does not wait until the end stage.

CT is the process of ensuring the quality of your software to the point that it is always releasable at a moment’s notice. If your company is delivering software to consumers, or users that need or expect continual enhancements to a product, CT is the means to get there. Teams practicing CD and CT can be confident that they can deploy to production, potentially several times in a week or even in a day, offering quality and functionality to their customers at a very fast pace.