DevOps is aimed at converting the distinct steps of application development and support into a continuous workflow, which in turn should make the enterprise itself more agile in an increasingly fast-paced digital economy.
One of the most fundamental aspects of this process is quality assurance, which, arguably, has long been the bane of development because it’s where rigorous testing provides the feedback on everything the developer needs to fix in order to launch the product into production environments. As such, it has long been a distinct, and rather time-consuming step, in what has traditionally been a slow, laborious route from initial concept to working app.
This is why QA and its associated test functions require a radical makeover in the world of DevOps. When new products and updates are expected weekly, if not daily and perhaps even hourly, testing can no longer remain a specialized area of expertise, with its own team and own ways of doing things, and it will have to embrace high levels of automation without sacrificing the personal touch that unique applications require in a complex operating environment.
According to Mike Sowers, executive vice president of agile software developer Coveros, proper integration of test procedures into the DevOps pipeline will help ensure the quality of not just the final product but the process used to create it.
“In DevOps, we are not only concerned with the quality of the application or system, but also the quality and security of the pipeline – the tools, code and infrastructure that we use to deliver our software value to the customer,” he said. “As we automate our pipeline, we have the ability to spin up test environments more quickly, ensure consistency in our dev, test, stage and pre-production environments and run acceptance, non-functional and operational tests earlier.”
This level of coordination requires a team effort, of course, which means QA is no longer solely responsible for ensuring proper functionality in the final product.
“In Agile, we now have the testers and developers working together and the quality responsibility embraced by the whole team,” he said. “This gives us the opportunity to apply systems thinking, to include the entire value stream and to keep driving quality left, checking at every step and evaluating the operational requirements early.”
The key to this transformation is the conversion from static infrastructure to “infrastructure as code” and the ability to automate much of the provisioning and configuration processes that currently slows IT management to a crawl. In this way, DevOps can provide repeatable and reliable software delivery, foster collective ownership of the entire application lifecycle and enable continuous improvement of both the application and its operating environment.
As expected, this will require some retooling of the testing platform.
Re-examining the Testing Platform
“The organization may have some existing tooling, however, in many cases, it may need additional tooling, particularly to support the automation of the pipeline,” Sowers said. “Both open source and commercial tools are available today with the necessary capabilities to support this goal.”
The more automated you are, the faster the feedback loop from test back to development, which means software bugs and other issues can be reported, and in some cases corrected, in a fraction of the time when done manually.
“Automating the conveyor belt that moves our software changes from development to code commit to unit, integration, acceptance and operation tests – and in some cases even into production – yields repeatability and consistency,” Sowers said. “Automating the creation and destruction of the dev, test, stage and pre-production environments which support this testing ensures that code does not suffer the dreaded ‘it worked fine in my environment’ syndrome and also allows us to streamline the creation, testing and delivery of emergency fixes or easily and quickly roll back code that is not performing as expected.”
The end game to all of this, of course, is speed. With processes running in an iterative fashion, organizations have the comfort of knowing that new services can be rolled out as needed, rather than in next year’s new version, and problems can be corrected before they start to seriously affect the user base.
It certainly is not as linear or as predictable as current test/dev environments, but it allows team members to gain a greater perspective on the needs of the full application pipeline, and it delivers a more usable product in the end.
The following are some leading DevOps test solutions:
AppVerify
AppVerify provides continuous testing for agile, DevOps and continuous delivery teams. It applies any test case to any application with no plug-ins and automatically updates scripts with new data in order to maintain consistency as apps change over time. As well, it provides for self-defined validation points that can be inserted anywhere in the DevOps pipeline to evaluate expected outcomes.
Eggplant DAIS
The Digital Automation Intelligence Suite (DAIS) provides functional, performance and usability testing on any platform. It auto-generates test cases to boost productivity while using both artificial intelligence and machine learning algorithms for detection and analysis of both applications and the test processes. It also features a REST API to coordinate test execution with third-party CI tools like Jenkins.
IBM Rational Function Tester
The Rational Function Tester supports a wide range of applications, including .Net, Java, SAP and terminal emulator-based products. It provides simplified visualization and editing using natural language and screenshots while enabling advanced test scripting by recording user actions and supporting intelligent script maintenance.
Katalon Studio
Katalon Studio enables automated testing without requiring substantial programming knowledge. It reduces the direction of regression lifecycle testing by half and comes with detailed documentation, tutorials and templates for easy start-up. The system is compatible with all frameworks, ALM integrations and CI plugins, as well as Windows, MacOS and Linux.
Selenium
Selenium automates test applications for web purposes, but is also extensible to other administration tasks. Selenium functions with multiple browsers and operating systems and can be run under many programming languages and testing frameworks. The latest server release also features built-in grid support.
SoapUI
SoapUI provides a complete test automation framework for Soap, Rest and other APIs. The system provides functional and performance testing, as well as security testing for public, third-party and even vulnerable internal APIs. It also supports data-driven testing and full reporting, metrics and analysis of test and measurement capabilities.
TestComplete
TestComplete allows complex, scalable automated UI functional tests to be created in seconds. It has a customizable object repository to ensure the accuracy of record tests and allows the reuse of data-driven tests using either an internal automation framework or by consuming Unit, Selenium or SoapUI tests.
Tosca
Tosca is a continuous testing platform that reduces regression testing from weeks to minutes while maximizing reuse and maintainability with model-based test automation. It provides deep insight into the risks of release candidates and provides cross-browser, mobile and even robotics testing.
Unified Functional Testing
UFT is designed to make manual testing more efficient and increase collaboration between test and development. It offers real device testing on iOS, Android and Windows environments as well as Chrome, Firefox, IE and other browsers. It also provides a visual test flow graphical canvas that manages actions, changes test orders, debugs tests and performs other functions.
Watir
Watir (Web Application Testing in Ruby) offers a number of open Ruby libraries for testing automated web browsers. Rather than simulating a browser, it drives it directly through the OLE protocol using interprocess communication and dynamic object creation and manipulation. It is also available as a RubyGems gem.
Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.
