U-QASAR Methodology WebBook

Section 4: Evolutionary Refinement of the Measurement Program ( Section 2.4 )

Checklist

Consider the evolution of the development domain
Reflect the quality management strategy
Refine the quality models, indicators, and metrics

Description

Technological systems with some importance tend to outlive the initial project that created them. Typically, the maturing of software systems is accompanied by revisions to the quality management strategy used to direct the focusing and attention to different quality objectives. Industrial software practice suggests that there are three generic types of quality management strategies:

Maximize customer satisfaction
Minimize engineering effort and schedule
Minimize defects

Which strategy is active at a particular point in time in a product's life cycle is a management decisions that can be influenced by market pressures, product strategies, portfolio considerations etc. With increasing speed of innovation, and shorter release cycles, product managers may need to postpone qualities such as maintainability and scalability for later. In consequence, a quality management methodology needs to accommodate a potentially evolving strategy for quality management.

Quality management is best helped by considering the current software life cycles stage. Depending on its status, from initial development through evolution, servicing, phaseout and closedown stages, the development needs different focus on a set of factors like staff expertise, software architecture and decay and economic factors.

Part 4.1 : Evolution of quality models ( Section 2.4.1 )

Flyers

The staged model for the software life cycle

Guidelines

Guideline	Description
Describe the quality model evolution	To understand the evolution of quality models there are two questions we must address: What elements of the quality model are likely to change? What elements of the quality model are likely to endure?
Mapping the product life cycle to appropriate measurements	There is a need to revise the practical (methodological) level with quality model evolution and explain what are the effects of such evolution on the measurements.

Software systems tend to pass through different stages (see flyer “The staged model for the software life cycle ”.)from its first version till it is finally replaced by an alternative system. These stages need to be mapped to the product under the development since they should steer the development of the quality models too. QA project is associated with a product, a development project and an overall quality model. The latter is a manifestation of the organization’s quality strategy. The overall quality model may include quality objectives that span beyond software projects, but will be influential in the creation of the particular QA project. A unique QA project is created upon the establishment of a new development project, even if the project concerns an existing product. This is the entity in which change is most likely – it is the main point of support for life cycle evolution of quality management.

Lower level concepts, the indicators and the metrics, may also be replaced, but probably at a slower pace. With time, the organization is likely to accumulate a relatively stable set of indicators and metrics. After all, the final objective of all the quality management strategies is customer satisfaction. Therefore, we should expect that the accumulated set of indicators and metrics cover the entities of interest. What is changing is the weighting of indicators and

The staged model for the software life cycle

Stage 1: Initial development

Software development starts with an initial development stage where software is built (often from scratch) to satisfy initial requirements. In this stage, the software teams build expertise about the domain and problem, necessities for the evolution of the product. In this phase, creating a good architecture that supports further development is essential. System components need to interact in smart ways, their properties such as functionality and efficiency hinders or helps later development.

Stage 2: Evolution

Evolutionary stage is defined by iterative changes, modifications and deletion of functionality. These changes occurs because of experienced changes in market needs, legislative actions or business and hardware changes such as expanding a system to changing platforms. Most software development evolves before being shipped, even if many companies choose to ship alfa and beta version to attract customer experiences already during this stage. While making these changes it is important to remember to maintain an open architecture and appropriate, and to keep a competent staff to be able to quickly meet market needs. As long as the software system is kept wellgroomed, by competent personnel, it is possible to stay in the evolutionary stage. Often, however, key personnel are shifted to other development projects and less experienced personnel take over. This unfortunately aids in increasing factors like software decay. Gradually changing and hacking at the system leads to an architecture that makes further changes more difficult, and concepts like refactoring and reengineering of large systems inevitably turn expensive. As a result of this, less and less changes are made possible without fear of breaking the existing system. Keeping software in the evolutionary stage is beneficial because it enables changes. It is also expensive because it needs constant vigilance and to be maintained by experienced developers.

Stage 3: Servicing

When making changes is too expensive and difficult to handle, software enters the servicing stage. It is considered decayed legacy code with an architecture that is too difficult and expensive to change, but it is still seen as important. In this stage only minor changes are made. Software can also be “wrapped” in a black box and left untouched. Some software, like safety-critical software, should never enter the servicing stage as the need for making changes and having control over the software is too important. However, this stage also provides some benefits. As the now legacy software is boxed in, in this allows the use of less experienced personnel to reduce costs. This of course comes at the cost of not being able to follow market trends and other key factors that make your software product competitive. Gradually even minor changes are seen as too difficult to handle.

Stage 4: Phaseout

As software servicing is discontinued, it enters the phaseout stage. The company may still make money from selling software that isn’t patched, but it will inevitably be too old and fall out of sync with market requirements. During this stage, maintenance and development costs are very low.

Stage 5: Closedown

In the end, users need to be directed to a new version of the product or alternative solutions. It may be important to provide migration solutions. During the closedown stage there may still be contractual obligations and other factors that mean the code cannot be fully phased out.

Based on the article “A Staged Model for the Software Life Cycle” by Rajlich and Bennet, 2000

Part 4.2 : Quality assurance in a continuous deployment environment ( Section 2.4.3 )

Guidelines

Guideline	Description
Fit QA activities into continuous deployment strategy	Customers are relying less and less on buying versions of software, they buy access to a product that is constantly changing and evolving to meet market demands. The product is no longer produced and sold as set versions with specific release dates – instead the product is constantly changing and evolving to cater for the needs of the users and technical needs. Tactics for testing new features or ideas on a subset of customers has been developed.
Enforce QA activities at the developer's side	In the world of continuous deployment, the goal is to reduce time between coding and deploying the code on the production server for actual use by customers. Continuous deployment means that software developers become responsible for deployments and the quality assurance team has not looked at the quality of the already deployed code. The developers are thus responsible for the quality of the code, and this increases pressure on the developers.
Enable roll-back	Lean startup focuses on using methods like A/B testing, and observing how the groups behave as they are using two versions of the same feature. By trying out new strategies and ideas on a continuous basis, small increments at the time, the idea is to spend as little effort as possible to check if the idea was beneficial or not. If it isn’t, it is simple to roll back to the previous version. By deploying small chunks of code there is also more control over deployment. You don’t deploy many features at the same, so if something goes wrong you stand a better chance to find out what is wrong.

Continuous deployment strategy aims to reduce time between coding and deploying the code on the production server for actual use by customers. Here we describe some of the main challenges of quality assurance when software development is seen as a continuous process like continuous deployment, and where bringing in knowledge on quality from other sources like operations can help pinpoint where to focus effort. Software systems now operate in very complex environments – battling for shared resources (i.e. web servers, network use), differing environments (operating system platforms, various web browsers and even browser versions) or changing environments (network access while on the move) – and this gives rise to a whole new set of problems as well.

Key to creating a successful application is to understand and make use of knowledge of operations. Knowledge from operations is pivotal to understanding and quantifying problems within the client/applications landscape. Operations should become an integrated part of development and provide statistics, pinpointing where to focus effort in a large list of change requests and bug reports. Focus lies on automating as much as possible of the release, QA and operations processes, but in order to do this one needs knowledge of all these factors. Hence they must cooperate. This forces developers to gain a clearer understanding of the operational constraints of the system, and at the same time it gives operations insight into what it being made and can make the necessary preparations.