SQA in Practice

A comprehensive definition of Software Quality Assurance SQA is presented here, in the form of a short history of the Software Quality Movement, an example of a theoretical CMMI SQA\SQC implementation and an example of what a typical Software Application QA engineer does day to day.


SQA, SQC in the Commercial Workplace

Having discussed a basic definition of Software Quality Assurance SQA and Software Quality Control SQC as well as a formal implementation of these defined terms, this paper documents a typical (or hybrid) framework which benefits from the main goals of SQA\SQC without the organizational structure of the pure implementation.

First a caveat, the danger in environments where SQA\SQC is not formally implemented is that the ‘Quality Group’ becomes a kind of ‘Engineering Services’ group and do what ever the main development team does not currently (or does not want to) do. In many environments the SQA\SQC team reports to an application development manager and the ‘Engineering Services’ approach is very attractive to this manager. For example, let’s say performance is an ‘after thought’ that is not specified as a non-functional requirement, not designed into the system but noticed when things go wrong. Under the “Engineering Services” scenario it is tempting for the manager to send the QA person on a LoadRunner (or other load testing tool course) install the tool and declare the QA person ‘our performance expert’. This type of approach will not work, the QA team member should be able to perform load testing (as part of SQC) but the performance requirements (that are tested for) should be part of the requirements and the Designer\Programmer should ‘own’ the performance objective.

So, accepting that SQA\SQC have to be separate from development activities and certain (mainly Requirement and Design) activities always need to take place, let’s look at a hybrid, or generalized, approach to SQA. SQA and SQC will be referred to as Software QA and the activities of this person(s) represent a good ROI for achieving the goals of SQA\SQC with a limited resource.

Note: This is only a sample of SDLC activities but role of Software QA against these activities should illustrate the kind of roles that a general Software QA person should undertake.



SQA and SQC roles against generic SDLC

Requirements
This is the starting point, everything flows from the requirements. Templates should be established, that can be referenced during reviews, which have sections for all Functional and Non-Functional requirements (see FURPS+). For example the performance requirements should be stated in terms of user population and transaction rates, in this way it will not be an after thought. A Traceability Matrix should also be started to assist Requirements Management. The Traceability matrix not only helps with test coverage but also encourages the analyst to reference individual requirements, so that they can be crossed reference.

Role of Software QA, this role verifies that the Requirements conform to the basic standard (Templates) and the requirements are free of any ambiguities. In terms of completeness, Software QA would also review the risks of not completing the Non-functional section. The Software QA would also review the Traceability Matrix in order to make sure all requirements were referenced.

Test Cases
At this stage, some test cases that refer to the requirements could be written by Software QA and cross referenced in the Traceability Matrix. In many cases this exercise will further verify the Requirements, as test cases (with data) are being built.

Interface Specification
If the system is component based then an Interface Specification should be produced. Again templates should exist for this. This document should be subjected to Verification by SQA.

Other Specifications
Lower level specifications (for commercial applications) should only be subjected to Software QA if they are critical (i.e. main loan rate calculations) modules. In general paying attention to Interfaces and the Requirements (Functional and Non-Functional) will provide a good ROI for limited Software QA resources.

Unit testing
This should be done by the developers themselves.

Component Testing
If a harness (Web Services or HTTP) is available, then Software QA should get involved with this ‘grey box’ testing. The test cases can be written ahead of time, by the Software QA team. For this reason the Interface Specification is an important deliverable.

Integration Testing
The actual staging and verification of the basic ‘hand shake’ should be done by the technical development team. If the systems ‘don’t talk’ then the Software QA team should not be involved. Once a component is integrated then Software QA can execute the Integration Testing.

SQA of Interface Specifications Both component and Integration testing will provide good measures on how effective the Interface Specification was. These measures will provide good feedback for continuous improvement of the Interface Specification process and documentation.

System Test against the Requirements
Following a complete system being built, Software QA should execute the main System test cases that validate the system against the Requirements.

Acceptance Testing This should be owned and executed by representatives of the “sign off” team. For small changes this could be Software QA.

Defect Tracking and Resolution
Software QA should track all defects, retest until testing is complete. During this process Software QA should determine the ‘origin’ of the defect. If the defect was introduced as a result a misunderstanding of the Requirements then this information should feed into the Requirements Development process improvement initiative.

Regression Testing
Software QA should have a documented set of test cases that can verify if any component or function, not already tested as part of a release, has been negatively impacted by the new change.

Performance Testing
Software QA should run some performance tests to verify performance parameters (response times etc.) are within the requirements. If performance issues are found then the development team needs to be involved in diagnosing and resolution

Move to Production
This should be verified by Software QA but the actual process should be carried out by someone else.

Support Software QA should analyze the origin of the defects and examine the SDLC to determine where the defect was introduced (Requirements\Design or Coding) and review these with the process owners for possible improvements.

Conclusion

Software QA, as defined, can mix the formal SQA\SQC activities but the overriding principle that they remain independent from the actual Software production activities should still hold. As soon as Software QA ‘owns’ the build process or does Usability Design or is the performance engineer or has some other non-QA role then you have moved to an “Engineering Services” model and real SQA\SQC is compromised.

No guarantee (or claim) is made regarding the accuracy of this information. Any questions or comments should be sent to:-