Software Metrics

A look into the extent to which software processes and components can be measured, as well as the value of such measurements for SQA, SQC and SPI (CMMi).

Article Purpose

The purpose of this article is to define the term software metrics and give examples of common software metrics (aka measurements). In addition this article also draws the relationships between software characteristics and software process improvement (SPI) initiatives, using CMMi for reference.

A definition of software quality metrics is:-

A measure of some property of a piece of software or its specifications.

Basically, as applied to the software product, a software metric measures (or quantifies) a characteristic of the software.
Some common software metrics (discussed later) are:-
  • Source lines of code.
  • Cyclomatic complexity, is used to measure code complexity.
  • Function point analysis (FPA), is used to measure the size (functions) of software.
  • Bugs per lines of code.
  • Code coverage, measures the code lines that are executed for a given set of software tests.
  • Cohesion, measures how well the source code in a given module work together to provide a single function.
  • Coupling, measures how well two software components are data related, i.e. how independent they are.
The above list is only a small set of software metrics, the important points to note are:-
  • They are all measurable, that is they can be quantified.
  • They are all related to one or more software quality characteristics.
The last point, related to software characteristics, is important for software process improvement. Metrics, for both process and software, tell us to what extent a desired characteristic is present in our processes or our software systems. Maintainability is a desired characteristic of a software component and is referenced in all the main software quality models (including the ISO 9126). One good measure of maintainability would be time required to fix a fault. This gives us a handle on maintainability but another measure that would relate more to the cause of poor maintainability would be code complexity. A method for measuring code complexity was developed by Thomas McCabe and with this method a quantitative assessment of any piece of code can be made. Code complexity can be specified and can be known by measurement, whereas time to repair can only be measured after the software is in support. Both time to repair and code complexity are software metrics and can both be applied to software process improvement.

From our previous definition of SQA and SQC, we now see the importance of measurement (metrics) for the SDLC and SPI. It is metrics that indicate the value of the standards, processes, and procedures that SQA assures are being implemented correctly within a software project. SQA also collects relevant software metrics to provide input into a SPI (such as a CMMi continuous improvement initiative). This exercise of constantly measuring the outcome, then looking for a causal relationship to standards, procedures and processes makes SQA and SPI pragmatic disciplines.

The whole process of setting up a SDLC, then selecting the correct metrics and then establishing causal relationships to parts of the SDLC is more of an art than a science. It is for this reason that there a few, if any, off the shelf answers to SQA, SQC and SPI. It is a question of where are the risks and challenges of a given environment. For example if you have one version of a system and this runs on a central server, then configuration issues are unlikely and you are less likely to be concerned with Portability issues than someone who targets multiple platform, multiple versions is concerned with portability (and configuration management).

That said the following section tries to pull the ideas of quality metrics, quality characteristics, SPI, SQC and SQA together with some examples by way of clarifying the definition of these terms.

The Software Assurance Technology Center (SATC), NASA, Software Quality Model includes Metrics

The table below cross references Goals, Attributes (software characteristics) and Metrics. This table is taken from the Software Assurance Technology Center (SATC) at NASA. Although the software quality model has different quality characteristics than those previously discussed on this website, namely ISO 9126, the relationship with Goals lends itself to giving examples of how this could be used in CMMi. If you look at the other quality models they have a focus on what comes under the Product (Code) Quality goal of the SATC model.

The SATC model of documenting metrics is useful for implementing the CMMi SPI for these reasons:-
  • The SATC model includes goals for processes, (i.e. Requirements, Implementation and Testing).
  • The SATC model can be used to reference all of the CMMi software engineering process areas, for example Requirements management (which includes traceability).
  • If desired the SATC model can be expanded to accommadate grater risk mitigation in the specified goal areas, or other goal areas can be created.
  • Demonstrating the relationship of metrics to quality characteristics and SPI (CMMI) is well served by the SATC quality model.
If you need to do this work in practice, you will need to select a single reference point for the Software Quality Model, then research the best metrics for evaluating the characteristics. The importance of being able to break down a model of characteristics into measurable components indicates why these models all have a hierarchal form.



The SATC Software Quality Model (which includes Goals and Metrics as well as the software attributes)

GOALSATTRIBUTESMETRICS
  Ambiguity Number of Weak Phrases. Number of Optional Phrases.
 Completeness Number of To Be Determined (TBDs) and To be Added (TBAs).
Requirements QualityUnderstandability Document Structure. Readability Index.
 Volatility Count of Changes / Count of Requirements. Life cycle stage when the change is made.
 Traceability Number of software requirements not traced to system requirements. Number of software requirements not traced to code and tests.

 Structure/Architecture Logic complexity. GOTO usage. Size.
 Maintainability Correlation of complexity/size.
Product (Code) QualityReusability Correlation of complexity/size.
 Internal Documentation Comment Percentage.
 External Documentation Readability Index.

Implementation EffectivityResource Usage Staff hours spent on life cycle activities.
 Completion Rates Task completions. Planned task completions.


Testing Effectivity
Correctness Errors and criticality. Time of finding of errors. Time of error fixes. Code Location of fault.

SATC's relationship with CMMi


The SATC Goals can be mapped to the following CMMi development processes:-

GOALSCMMi process.
Requirements Quality Requirements Development, Requirements Management.
Product (Code) Quality Technical Solution
Implementation Effectivity Project Management.
Testing Effectivity Verification and Validation.






Although all of the CMMi process areas are not covered by the SATC goals, the main stream development activities are, that is:-
  • Project Management.
  • Requirements.
  • Development (coding and implementing).
  • Testing.
One good addition to this model would be Supportability that includes number of faults and mean time to repair, such metrics would characterize the ongoing cost of ownership. That said the SATC model focuses on Project development activities and is primarily used for mitigating project development risks.

The measures can also be added to, the idea being to better measure the software attributes (characteristics). For example for Reusability data coupling and functional cohesion could be used instead of correlation of complexity/size. Data coupling measures how closely two components are tied together, with loose data coupling being desirable. Functional cohesion measures the extent to which a module performs one function, whenever the scope of a component is stretched to be multiple functional the code becomes less reusable. The main goal being to be continually experimenting with new metrics to make a better (i.e. more useful) determination of the extent to which the characteristic, that they are measuring, exists in the software.

The measures listed in the above table, i.e. for traceability, code complexity etc., would provide a good starting point for any software metrics program. What is important is to place software metrics in the context of contiunous improvement. In order to achieve this a well defined quality model is needed which lists the desired characteristics and how they are measured (i.e. metrics). The final essential relationship that needs to be established is the causal relationship between the metrics and the SDLC activities (i.e. processes, procedures and standards). When these models and relationships have been implemented then a continuous improvement life cycle can be implemented (such as CMMi), in short:-

We can only control what we can measure.

No guarantee (or claim) is made regarding the accuracy of this information. Any questions or comments should be sent to:-