Control over the IT process of Install and Accredit Solutions and Changes that satisfies the business requirement for IT of
- implementing new or changed systems that work without major problems after installation
- testing that applications and infrastructure solutions are fit for the intended purpose and free from errors, and planning releases to production
- Establishing test methodology
- Undertaking release planning
- Evaluating and approving test results by business management
- Performing post-implementation reviews
- Amount of application downtime or number of data fixes caused by inadequate testing
- Percent of systems that meet expected benefits as measured by the post-implementation process
- Percent of projects with a documented and approved testing plan
1 Non-existent
2 Initial/Ad Hoc
3 Repeatable but Intuitive
4 Defined
5 Managed and Measurable
6 Optimized
Benchmarks/Guidelines for Scoring
1 Non-existent when
There is a complete lack of formal installation or accreditation processes, and neither senior management nor IT staff members recognize the need to verify that solutions are fit for the intended purpose.
2 Initial/Ad Hoc when
There is an awareness of the need to verify and confirm that implemented solutions serve the intended purpose. Testing is performed for some projects, but the initiative for testing is left to the individual project teams, and the approaches taken vary. Formal accreditation and sign-off are rare or non-existent.
3 Repeatable but Intuitive when
There is some consistency amongst the testing and accreditation approaches, but typically they are not based on any methodology. The individual development teams normally decide the testing approach, and there is usually an absence of integration testing. There is an informal approval process.
4 Defined when
A formal methodology relating to installation, migration, conversion and acceptance is in place. IT installation and accreditation processes are integrated into the system life cycle and automated to some extent. Training, testing and transition to production status and accreditation are likely to vary from the defined process, based on individual decisions. The quality of systems entering production is inconsistent, with new systems often generating a significant level of post-implementation problems.
5 Managed and Measurable when
The procedures are formalized and developed to be well organized and practical with defined test environments and accreditation procedures. In practice, all major changes to systems follow this formalized approach. Evaluation of meeting user requirements is standardized and measurable, producing metrics that can be effectively reviewed and analyzed by management. The quality of systems entering production is satisfactory to management even with reasonable levels of post-implementation problems. Automation of the process is ad hoc and project-dependent. Management may be satisfied with the current level of efficiency despite the lack of post-implementation evaluation. The test system adequately reflects the live environment. Stress testing for new systems and regression testing for existing systems are applied for major projects.
6 Optimized when
The installation and accreditation processes have been refined to a level of good practice, based on the results of continuous improvement and refinement. IT installation and accreditation processes are fully integrated into the system life cycle and automated when appropriate, facilitating the most efficient training, testing and transition to production status of new systems. Well-developed test environments, problem registers and fault resolution processes ensure efficient and effective transition to the production environment. Accreditation usually takes place with no rework, and post-implementation problems are normally limited to minor corrections. Post-implementation reviews are standardized, with lessons learned channeled back into the process to ensure continuous quality improvement. Stress testing for new systems and regression testing for modified systems are consistently applied.