Chris Pattinson, Director of Quality Assurance, spoke to us this week regarding his role at Embarcadero Technologies – a database tools and developer software company. After watching the video I hope you'll read an excerpt from a blog post Chris recently wrote (July 2009).
Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied, and may include ensuring conformance to one or more standards, such as ISO 9000 or CMMI.
Why Quality Assurance?
I’ve now had almost a decade in the Quality Assurance field of software development, and while all folks understand the importance of testing, the understanding and measuring of the real value of a quality assurance organization can be difficult to determine.
From my personal view, a well run quality organization achieves three majors goals:
- Visibility
- Control
- Reduces Development Debt incurred by Bugs
By visibility, the more mature the testing organization is, the faster and more effectively problems are found and the impact of problems are understood. When a bug is reported, a mature organization can analyze the frequency and impact of such an issue, then once it’s analyzed create an effective automated test to ensure that once the issue is fixed, it stays fixed.
Visibility is more then just detecting the bug as well - the requirement for a feature may be ambiguous or incomplete leading to inefficiencies in development. And then other processes such as code reviews, project planning, and release processes could all have areas that if improved, not only ensure a product is shipped of good quality but that the process to do so is lean, mean and very effective - less waste of one of the most precious resources : Time.
Control is achieved by clearly defined milestones and acceptance criteria to validate that milestone. It’s easy to say you have your first Beta build with no qualifications, however if you state the beta build needs to have certain functionality and bug thresholds, then you can ensure the team aims for those goals and can keep a project ‘health’ in good shape. You can also ensure that when you release a product to market, you clearly understand it’s state and can more confidently predict revenue. Also, public beta’s advertise how your organization cares about a product, and customers can feel part of the development process - as well as help other customers learn and use the product.
QA in itself does not fix a product - however QA ENABLES the R&D team to fix a product and better meet a customer’s needs. And with highly visible test systems that run frequently, this adds huge value to R&D - the sooner a bug is detected the easier it is to fix.
Now, the first two factors are hard to measure. Development Debt can be measured on the time it takes for developers to work on bug fixing during a product cycle. This work can be measured against customer satisfaction to understand the development cost of bugs to achieve certain satisfaction levels. A well run development process with mature quality organization can be truly agile and spend more time on features, and less on bug fixing since the cost of fixing a bug found quickly after introduced is hugely less then found later.
Take the simple scenario - an engineer modifies a feature and a manual tester looks at it, and certifies it’s ok. However a bug now occurs in another feature that depended on a property that was modified and this other feature wasn’t tested. Worst case, the customer finds the issue weeks or months later after many other code changes - the time to investigate, and the dependencies of OTHER code on this initial change would typically cause a lot of work and headaches for both R&D and QA, ultimately making it hard to fix the issue without causing OTHER side-effects.
Ultimately, R&D ‘feels’ less work was done, however if the issue was caught and fixed, the Development Debt is less for the next release, meaning more exciting features . It’s clear how this is a huge benefit of a well run QA organization.
There are other advantages too - once you have visibility you can find opportunities for new features and understand what the customer is really asking for, not what you are assuming they are asking for. You may find that you thought customers always use your product in one way, or one feature that is really important to them. You may also find areas in the development process that if you improve, dramatically improve overall team productivity.
No comments:
Post a Comment