Agile Testing Manifesto

Agile Testing is continuously discovering better ways of creating value for our customers through new testing approaches. Through this ongoing debate and collaboration we have come to value:

Testing Driven Development > Development Driven Testing

Thoughtful Testing > Scripted Testing

Business Driven Testing > Requirement Based Testing

Delivering Voice of Customer > Being a Critic

Highlighting Risk > Acting as Gatekeepers

Testing Understanding > Checking Functionality

Good Enough > Total Conformance

Preventing Defects > Detecting Defects

Delivering Value > Criticizing the Product

Testing with Others > Testing Alone

Improving Our Craft > Following Standard Practices

Timely Reporting of Progress > Weekly Reporting of Progress

Listening to People > Analyzing Quality Metrics

Testing Throughout > Testing at the End

Collaborative Elaboration > Speedy Automation

The Agile testing community invites QA Testers to collaboratively answer the following questions:

  • Are these the right values?
  • Which values should we eliminate?
  • Should we consolidate any?
  • Have we missed something?

Understanding the Agile Methodology
But in case you’re still wondering what, exactly, Agile is all about, its basic principles can probably best be explained in comparison to more traditional approaches to software development, such as “waterfall,” which Agile adherents believe to be an archaic holdover from an era when software first became a packaged commodity to be produced and sold to consumers like other physical goods.
According to software testing expert Scott Barber, between 1979 and 1985 virtually all software companies began to codify their production processes into the same waterfall method that had been successfully used by the manufacturing and engineering industries for decades.  In a waterfall approach, any given project’s workflow proceeds in a linear series of sequential steps, progressing down the chain of production from Requirements to Design to Implementation to Verification to Maintenance.
(Source http://scott-barber.blogspot.com/)
In a waterfall process the “design” and “implementation” stages exist prior to, and distinct from, the “verification” and “maintenance” stages.  This split between software developers and software testers, positioning them as separate entities at different points along a production cycle, is one of the fundamental problems that Agile seeks to resolve.
In the Agile approach, developers and testers are seen as two sides of the same production coin, two parallel lines that should always meet and compare notes daily.  From an Agile perspective, efficient production is severely hampered if your developers are striving to refine their code to a state of perfection before passing it on to a separate testing team, who then strive to break it in as many ways as they can before sending their damage report back to the dev team. This two-step process requires time, money, and frequently leads to internal division between a given company’s developers and testers. Instead, Agile suggests that these two essential functions be merged—not necessarily in terms of people, but in terms of time and process—thus bridging the illusory divide between code creators and code breakers, and even reducing the need for robust testing teams, while still respecting the necessity of both roles. One could even say that in Agile, developers are encouraged to think more like testers, continually checking their own code for potential errors, and testers are encouraged to think more like developers, tempering their natural destructive tendencies to engage more fully in the creative process itself. Barber refers to this as creating a “product delivery team with a unified vision of delivering production-worthy code the ‘first’ time by integrating developer thinking and tester thinking throughout the code writing process.” This integration implies both that developers enhance their tester thinking skills and embrace the notion of direct interaction, even pairing, with those who specialize in tester thinking while coding.
Continuous Integration and Quality
In short, Agile requires a depth of continuous collaboration between stakeholders—including managers, developers, testers, and customers—throughout the production process that isn’t found in a traditional waterfall workflow. Testing becomes an essential component of each and every phase of the developmental process, with quality being “baked in” to a product at every stage of its development through constant feedback from everyone holding a vision of the final product. As Martin Fowler, one of the original signatories of the Agile Manifesto, explains:
The key to this feedback is iterative development. This is not a new idea.  Iterative development has been around for a while under many names: incremental, evolutionary, staged, spiral… lots of names. The key to iterative development is to frequently produce working versions of the final system that have a subset of the required features. These working systems are short on functionality, but should otherwise be faithful to the demands of the final system. They should be fully integrated and as carefully tested as a final delivery.
When testing in an Agile way, there are few, if any, rigid adherences to requirements documents and checklists.  The goal, instead, is to simply do whatever’s necessary at any moment to satisfy a customer’s requests, replacing documentation with in-person meetings and replacing siloed functions with unified, self-organizing project teams.  In an Agile company culture, everyone is expected to work closely together, no matter his or her role, to achieve a single goal: a high-quality software product that fulfills all of the essential specifications a client or designer requires with each iteration.  Software developers, testers, and quality-assurance personnel wear each other’s hats from time to time, and while there may be a select group of people running most of the tests, the notion of a separate testing team disappears entirely for many product teams, and disappears within the core development cycle for those organizations who are required by external agencies (or even by law) to have formal and/or external release candidate testing. Even in these cases, this separate release candidate “testing” isn’t testing for the purpose of finding problems or improving the product, it’s an exercise in verification, regulatory compliance, and/or audit trail completion.
It is this dynamically collaborative dimension to the Agile approach that its proponents believe sets it apart from other methods, with continuous internal dialogue leading to better communication with clients and end users, better responsiveness to ever-changing project requirements, better working relationships between developers and testers, and a better and more efficiently delivered software product overall.
While seemingly more complex than traditional project management methodologies such as waterfall, Agile’s proponents contend that, once successfully implemented, it quickly leads to greater simplicity, freeing up the creative potential and efficiency of software production teams in ways that sequential, linear, and more strictly procedural, by-the-book approaches to software development rarely do.
Agile, as its name suggests, simply proposes to be a faster, more priority & risk focused, and more flexible, adaptable, and efficient way of conducting the complicated business of software production.  And, of course, it is not without its critics.
Criticisms of Agile
“For all its proponents,” writes Mike Brown of uTest, “Agile has its fair share of skeptics and detractors.  These are people who have a much different Agile experience—one characterized by chaotic processes, lower quality, miscommunication and numerous other problems.”
And those critics are not in the minority.  In 2012, Max Smolaks of TechWeek Europe,reporting on research conducted by Voke Media to determine what 200 different software companies thought of their attempt to embrace Agile, wrote:
Out of over 200 participants, 64 percent said that switching to Agile Development was harder than it initially seemed.  Forty percent of respondents did not identify an obvious benefit to the practice. Out of those who did, 14 percent thought it resulted in faster releases, and 13 percent—that it created more feedback. Seven percent of participants noted that Agile developers were happier due to reduced future planning and documentation.
These results strongly suggest that for organizations entrenched in years of non-Agile working practices, making the switch can be difficult, if not counterproductive.  As SmartBear’s Scott Barber explains:
I believe that the trend to “go Agile” is misguided.  If a company is developing good software, the people involved in developing that software are happy working there, the software development is sustainable, and the business is being adequately served by that software, there’s really no need for them to try to be more or less Agile.  Agile has challenges like any other culture, but the single biggest challenge I find is companies trying to solve development, process, management, and/or schedule problems by “going Agile.”  Teams who have grown up in a culture that is fundamentally different than Agile simply will not find it easy to “go Agile.”
Source (https://www.utest.com/articles/04)
In other words, companies expecting Agile to be a magic bullet to fix whatever ails their production efforts may be seriously disappointed, as it demands more of a systemic culture-shift than merely embracing a new set of tools and procedures.  Moreover, it isn’t a one-size-fits-all approach.  What works for a small San Francisco startup may be completely inappropriate for an enterprise with 5,000 employees distributed in cities around the world.  Agile might not also be the right practice for software companies and IT firms that are subject to regulatory compliance mandates, such as government agencies and their contractors, where extensive documentation and procedures rife with checks and balances are essential.
Naturally, proponents of Agile suggest that virtually all of the problems, complaints, and negative experiences that companies attempting to use Agile have reported are due not to a problem with Agile itself, but with a failure to understand the approach and its limitations. Others, such as technology strategist Lajos Moczar, claim to understand Agile fully and yet contend that its guiding principles are flawed.  In an article on CIO.com published in June 2013 and titled “Why Agile Isn’t Working,” Moczar ignited a firestorm, with pro-Agile and anti-Agile commenters on his post engaging in mild verbal warfare.  “The comments in this thread have taken a strangely negative religious tone,” noted one participant, “like watching one sectarian argue doctrine with a member of another sect.”  As with all ideological paradigms, some people embrace—or denounce—Agile with an almost spiritual fervor.
However, there are many signs suggesting that despite its potential drawbacks, the rapid adoption and popularization of Agile is largely based on the pragmatic demands placed on software production and information technology in the early 21st century.  When everything is changing so fast, some agility is simply required.
The Future of Software Development
As mentioned earlier, Agile has already won the battle for mainstream acceptance, and by all indications its general precepts are going to remain viable for some time. While much of this success is undoubtedly due to its effectiveness when used in appropriate circumstances, liberating software teams from old, tired ways of doing business, it’s also at least partially the result of the changing territory of computing itself.
A couple of decades ago, when consumers would head to their local Costco to purchase software magnetically encoded on floppy disks, taking those over-sized boxes up to the register along with their bulk packages of breakfast muffins and corndogs, it made sense for major enterprises and small companies alike to view software as just another mass-produced industrial product.  A waterfall-style production process seemed not only appropriate but also effective.  Testers were necessary in order to vet developers’ code up until the final commit, whereupon quality-assurance product teams would oversee its transference onto physical media—with floppy disks rolling down the assembly line like any other physical product.
But then the Internet happened, and broadband changed everything. With the possibility of downloading software directly from publishers, along with periodic bug fixes and updates, the whole game changed. Production processes had to become iterative, lightweight, context-dependent, and more responsive than ever to customer feedback. To many, the possibility of ever “finishing” a software project seemed increasingly tenuous.  The slow, step-by-step production-line mentality that had dominated software development in the 1980s was being challenged by the rapidly evolving infrastructure of computing itself.  Something more flexible and efficient was needed to streamline the process, something more…agile.
Today, we live in a world where digital downloads dominate sales and where continuous development is becoming the norm. At any given moment, your smartphone may be downloading background app updates to fix bugs you didn’t even know were there—pushed to your phone, from the cloud, by an Agile team that is used to refining new iterations of their software almost daily.  The speed of software sales is only increasing (and increasing exponentially at that), and judging by reviews in the app stores, customers’ patience for software glitches seems to be decreasing conversely.  For companies to remain competitive, running tests with the night tools as fast as developers can modify code, they have to realize that the future of software development is no longer linear—and perhaps never should have been.

Share on

linkedin sharing button twitter sharing button