Release and Deployment Table of Contents
Release and Deployment Management is the United Van Lines of IT Service Management. It is their responsibility to ensure that changes get introduced into the live production environment efficiently and effectively. Release Management has no responsibility over the content of the 'release' - only in the manner in which it takes place. Process Requires Re-writing for conformance with ITIL Version 3. More...

Introduction to Release and Deployment Management

Release and Deployment Management is a process within the Transition module of the ITIL Service Lilfecycle.

Release Management is concerned with moving and not with the content of the move.

Release and Deployment Management

Objectives

• Driving the release strategy, which is the overarching design, plan, and approach for deployment of a change into production in collaboration with the change advisory board (CAB).
• Determining the readiness of each release based on release criteria (quality of release, release package and production environment readiness, training and support plans, rollout and backout plans, and risk management plan).
• Developing a release process and release policy which describes a framework for participation by all application areas while offering an immediate operational process based on ITIL best practices. This description includes recognition of the Change process as the authoritative gate for releases into the operating infrastructure. Release processes must be developed to integrate with this and other developed service management processes
• Increasing the quality of system production releases by implementing consistent release processes and enhanced release communication. This is required to address issues associated with inconsistent or non-existent processes.
• Establishing communication channels for release information notification, and determine the audiences, communication frequencies, and information to be communicated via the channels.
• Ensuring the secure movement of software from the build/purchase phase through implementation using the DSL to store master copies of all software.
• Coordinating comprehensive user acceptance testing and pilot staging that includes verification of rollout and backout procedures.

Critical Success Factors

• change management to approve changes and track them throughout the release process.
• configuration management information to build and verify valid test environments in the development phase of the new release, and, to assess the impact of changes to the IT environment and to provide a definitive store for the release package.
• the existence of pre-production facilities for testing releases in an environment which simulates the production environment as closely as possible.
• load testing procedures and applications to emulate the production environment under different stress conditions.
• established procedures governing User Acceptance testing

Process Coverage

Scope

In Scope

• application programs developed in-house
• externally developed software (including standard off-the-shelf software as well as customer-written software)
• utility software
• supplier-provided systems software
• hardware, and hardware roll-outs
• assembly instructions and documentation, including User manuals.
• User Acceptance Testing
• Configuration Item versioning methodologies
• Procedures to maintain copies of production software in a Definitive Software Library (DSL)

All deliverables need to be managed effectively, from development or purchasing, through customization and configuration, through testing and implementation, to operation in the live environment.

Usually Excluded

The Release process considers application builds from the moment of testing completion to "release" into the production environment. Project Management up to the point of pre-production testing is covered by Application Project Management. Of course, project administration and the end-user participation will appreciably affect the success of the release. Steps done early as part of the project plan will pay dividends during the release process.

Discretionary

• procedures associated with building and testing releases may or may not be included within the release process. If they are not, included in Release they must be completed as part of the applications overall system development lifecycle (SDLC).

Relationship to Other Processes

Relationship with Change and Configuration Management processes

Area Emphasized

Change Management

The CAB, as defined in the Change Management process, with advice from Release Management, is responsible for recommending the content and scheduling of Releases. Release Management is then responsible for implementing the agreed Releases. Release Management is normally represented on the CAB and is involved in establishing the organization's Release policy.

Although Release Management oversees the details of the roll out of a Change, it is under the control and authority of Change Management.

Configuration Management

Release Management may use various services of Configuration Management during the implementation of a Release (e.g. configuration audit to ensure that the target environment is as expected). It is possible to combine these two functions with or without Change Management to form one organizational unit.

Configuration Management Database (CMDB)- Definitive Software Library (DSL)

Service Desk and Incident Management

Problem Management

• any related Problems or enhancement requests should be closed
• Problem Management staff should be informed of the new Release so that they can support its use in the live environment. Such staff should receive training in any new or revised support procedures.

Problem Management is also often involved in identifying faults that will lead to Requests for Change (RFCs) and so eventually lead to new Releases.

Software from Developers and Suppliers

Availability Management

Service Continuity

Capacity Management

Security Administration

Project Management

Release Management should assist project management in planning and implementing a Release, but it does not take control.

Policies & Guidelines

• Ultimate approval for any change, including releases, into the production infrastructure resides with Change Management. All releases will ultimately be described in a Request for Change (RFC) and will undergo
• approval of the Change Concept in which a Release Plan and estimated release dates will be presented, and,
• requests for approval at each release point wherein implementation preparedness will be assessed. This stage will be accompanied by a "Release Readiness Report" which is attached to the RFC for consideration by the appropriate Change authority
• the Change Advisory Board (CAB) should make recommendations on a case by case basis, based upon all the relevant facts, on whether the Release unit stipulated in the Release policy is appropriate or whether a Delta Release is preferable
• all estimated release dates will be recorded in a common Change Calendar (Forward Schedule of Changes) which will constitute the authorized date for the change to be implemented. Changes to this date must be approved by the designated Change authority. The Release Manager should try to keep the Change Manager informed of changes to estimated dates as early as possible and, in any event, must secure authorization before implementing the release into the production environment
• there will be an Emergency Release process which will parallel the Urgent Change process in Change Management which will have expedited procedures to allow a release to be placed into the production infrastructure. In this case all testing including Build management will be completed post-implementation
• Build management becomes the responsibility of Release Management from the controlled test environment onwards. Every release to the production environment will be accompanied by a Build Book which describes the sequence of steps and resources involved in placing the release into production. The accuracy of the Build Book will have been tested in a pre-production environment which simulates to the greatest possible extent the environment in which the application will be placed. Following the Build Book, a complete build without stoppage due to an inaccuracy will be required before Change Management will approve the release.
• There will be a virtual Definitive Software Library (DSL) comprising all production software and including two previous versions of the released software. Each application area (including productivity and operating system software) will provide High Availability servers for retaining this software. Every discrete module, identified separately in the Build Book, will constitute a Configuration Item (CI) and will be identified by a Category, Type and Item (CTI) designation in the Configuration Management Database (CMDB). Prior to release all software must be tested from CIs checked out of the DSL. Following Release to the production environment the Release Coordinator is responsible for ensuring both the DSL and the CMDB are updated accordingly.
• A Defect List describing Known Errors with the release will be provided as part of the Request for Change and forwarded to the Service Desk prior to the release
• Attempts will be made to secure standard software and hardware when implementing a release based upon the principle that known, standard types with a longevity in the production environment are inherently less risky than new, untested equipment and software
• Change Management should ensure that there is a formal User acceptance and sign-off before Release Management can continue to roll out the Release .
• The production of back-out plans for each Change is the responsibility of Change Management, but Release Management has a role in ensuring that the back-out plans for each Change within a Release operate together to create a Release back-out plan.
• All releases should have an identifiable business contact who can formally sign-off that the release has been put into production successfully from the point of view of the business

How the Process Scales

Concepts

Release Unit

The general aim is to decide the most appropriate release unit level for each software item or type of software. Such a policy means that each time a Configuration Item (CI) forming part of a service is changed, a Full Release of the whole of that system is normally issued. The same organization may decide that a more appropriate Release unit for PC software should be at suite level, whilst the Release unit of a batch application should be at program level. The following factors should be taken into account when deciding the appropriate level for Release units:

• the amount of Change necessary at each possible level
• the amount of resources and time needed to build, test, distribute and implement Releases at each possible level
• ease of implementation
• the complexity of interfaces between the proposed unit and the rest of the IT infrastructure
• the disk space available in the build, test, distribution and live environments

Related to this matter is whether there needs to be a single definition of a release unit as an organizational guideline or whether multiple definitions can exist sensitive to the unique characteristics of applications and roll-out centers.

Release Type

Full Release

Any problems are therefore more likely to be detected and addressed before entry into the live environment. The disadvantage is that the amount of time, effort and computing resources needed to build, test, distribute and implement the Release will increase. Although in some circumstances the testing of a Delta Release may need to be as extensive as that for an equivalent Full Release, the amount of building effort required to test a delta Release is normally less than for a full Release.

Regression testing as part of the process of implementing a Full Release allows a large number of components to be re-tested to ensure that there is no degradation in system function or behaviour.

An example of a Full Release could consist of the complete Release of a new version of client desktop software, or client desktop hardware, or both.

Delta Release

There may be occasions when Release of a full unit cannot be justified. In such cases, a Delta Release may be more appropriate. A decision is required on whether delta Releases should be allowed, and under what circumstances. There is no single 'correct' choice. It is recommended that delta Releases be allowed, with the decision being taken case by case.

In each case the Change Advisory Board (CAB) should make a recommendation, based upon all the relevant facts, on whether the Release unit stipulated in the Release policy is appropriate or whether a delta Release is preferable. In making its recommendation, the CAB should take into account:

• the size of a delta Release in comparison with a full Release, and hence the resources and effort required
• the urgency of the need for the facilities to be provided by the Release to the Users
• the number of CIs (below the Release unit level) that have changed since the last full Release - a very large number may enforce a full Release
• the possible risk to the business if compatibility errors are found in the Release (e.g. would it be preferable to wait for a full Release than to risk interface problems arising with a delta Release?)
• the resources available for building, testing, distributing and implementing the delta Release (e.g. if implementation is to be via non-technical staff, is it easier to implement a complete new Release than a delta Release?)
• the completeness of impact analysis information to make an informed and objective decision.

Package Release

A package can, for example, contain an initial version of a new service, several new versions of batch programs, a number of new and initial versions of individual modules, together with the Release of a complete new desktop system (both hardware and software). Both full and delta Releases may be included.

The use of package Releases can reduce the likelihood of old or incompatible software being wrongly kept in use. It can encourage organizations to ensure that all Changes that should be made concurrently, in different suites and systems, are actually made concurrently. It can also encourage organizations to test the interworking of these suites and systems fully.

Care should be taken, however, not to exceed, in any particular package Release, the amount of Change that can be handled comfortably. When making a decision on what to include in the package, care should be taken to ensure that the full impact of all individual parts on each other part is understood and has been properly assessed.

Definitive Software Library (DSL)

Each application area has a file area where the current production version and the previous two revision versions of the complete application are maintained. Provision is made to include any build routines required to bridge patches/fixes etc which occurred between these versions.

The DSL will include:

• medium, physical location, hardware and software to be used
• naming conventions for filestore areas and physical media
• environments supported, e.g. test and live environments
• security arrangements for submitting Changes and issuing software, plus backup and recovery procedures
• the scope of the DSL: e.g. source code, object code from controlled builds and associated documentation
• retention period for old Releases of software
• capacity plans for the DSL and procedures for monitoring growth in size
• audit procedures
• procedures to ensure that the DSL is protected from erroneous or unauthorized Change (e.g. entry and exit criteria for items).

Release Risk Category

High Risk

• Any Production Support project that has a history of instability or negative business impact as a result of similar releases done in the past.
• Any project that is not frequently repeated and has any of the following characteristics:
  • Involves a change to any part of the system considered to be "business critical"
  • Impacts at least one other system
  • Impacts more than one department
  • Involves formal training (i.e., CBT, classroom, workshop, etc.)
  • Impacts customers external to the organization

Medium Risk

• Risk Type is assessed in order to determine a Proposed Release Date. A Medium Risk Type is a Project that is frequently repeated and does not include:
• A history of instability or negative business impact as a result of similar releases done in the past.
• Impacts to at least one other system
• Impacts to more than one department
• Formal training (i.e. CBT, classroom, workshop, etc.)
• Impacts to customers external to the organization

Low Risk

• Risk Type is assessed in order to determine a Proposed Release Date. A Low Risk Type is a project that is frequently repeated and does not include:
• A history of instability or negative business impact as a result of similar releases done in the past.
• Impacts to at least one other system
• Impacts to more than one department
• Formal training (i.e. CBT, classroom, workshop, etc.)
• Impacts to customers external to the enterprise

Release Communications

Development includes determining the information to be provided during the project, its audience, the communication channels, and the frequency of information delivery. Execution involves setting the plan into motion in the organization.

It is extremely important that the Communications Coordinator identify a reputable person within IT and/or user community leadership to issue all correspondence to users. Assigning a person with credibility to this role demonstrates management support and builds user confidence for the project. The goals of the communication are to:

• Increase understanding of the pending release.
• Develop user acceptance.
• Demonstrate management support for the release.
• Establish feedback mechanisms for employees to voice their concerns.
• Provide timely and accurate status reporting to appropriate project groups.
• Maintain employee morale and productivity by fostering an open atmosphere and relieving employee concerns about the pending change.

Pre-production Testing

Coordinating the establishment of the test environment with the configuration manager helps to determine the requirements for mirroring the live environment. The test coordinator must decide what tool to use for reporting the results of testing. A computer-based tool allows the coordinator to monitor test results, develop result reports that can be passed on to management and other stakeholders, and track testing status. The coordinator should also specify tools needed to monitor operating effects of the new release on the computing environment.

Release Build

At a minimum, four types of testing are performed prior to releasing an application:

• Conduct User Acceptance Testing

  UAT is designed to ensure that the system operates in the manner expected, and any supporting material such as procedures, forms etc. are accurate and suitable for the purpose intended. The testing checks for compliance with statutory regulations or organizational business processes and rules, and to observe how it will behave under operational conditions. The emphasis switches from finding defects to ensuring that the software is suitable for the tasks it is being implemented for. It is often the first time non-development staff will work with a new application release. UAT testing is often considered the last step in the developmental cycle before the "Movers" (i.e., Release Management) assume primary responsibility for the next phase of the System Development Lifecycle.

  Tests are performed in accordance with the schedule laid out in a test plan. This schedule shows the tests to be performed and the persons performing them. Every feature of the release should be tested. For each test script, include preparation procedures, test items, the expected results, and documentation requirements. Make support available to all testers to help them resolve problems. The test staff and/or the service desk can provide this support.

  When preparing testers for user acceptance testing, the test coordinator should ensure that they understand the importance of following test scripts. Users should also be given the leeway to develop their own tests based on the system requirements necessary to fulfill their business needs. Testers must document these tests fully and follow prescribed test scripts carefully so that the test coordinator can accurately assess results, reproduce the test, and compare results over time.

• Load and Stress Testing

  The emphasis here is to simulate the conditions of the real environment as closely as possible. However, because equipment and conditions are expensive and difficult to duplicate exactly, measures are frequently taken to accept some risk by utilizing less expensive duplicate equipment or by simulating (rather than duplicating) the conditions which might be expected to occur in the live environment

• Implementation Testing

  All major changes to the Production environment should be preceded by implementation tests to ensure that the change can be correctly placed into the Production environment. A completely successful build of the release should occur prior to any final approval to release new configuration items into the live business environment. Implementation Testing applies to all kinds of Major changes and is usually considered part of Change Management.

• Back-Out Testing

  In the event that an Implementation encounters problems within the time-frame established for the implementation there should be a plan to restore the environment to its' pre-implementation condition so that business can resume without interruption. The back-out test provides insurance that the 'known and trusted environment' can be re-created in the event of an implementation failure.

Testing Environments

• Production - Changes are made only under strict control. Everything in production needs to work all the time and if something doesn't work it's impact must be known and controllable or it must be removed to a version that was 'known and trusted'
• Production Mirror - An accurate reflection of the production environment as it was a short time ago (recognizing the necessity of a time lag in keeping the environment in synchronization with Production). Any updates to production, whether approved or otherwise, would be applied to the mirror after an agreed-upon period of time has passed.
• Quality Assurance (Production Test) - This is a test environment for applications that are believed to be production-ready. Quality assurance is the last stop for code before it is designated for release to the live business environment.
• Development - Environments for Work In Progress. Changes need not be monitored very closely or approved with the same degree of procedure as production.
• Laboratory - Often called a sandbox, this is where new hardware, new technologies and third-party applications get tested
• Business Contingency - usually located some distance from the main production site, this site provides a fail-safe site for the business in the event of a major catastrophe. Use of the site should be referenced within a Disaster Recovery strategy.

The need for financial economy may mean some consolidation of these different purposes within a physical setup. Using an environment for multiple purposes places additional importance on the need to maintain order and discipline in switching amongst the various roles. In doing this, attention is re-focused to the need to quickly create and re-create the environment to a baseline condition (as described by the CMDB).

Each major application may retain its' own set of associated testing environments. This ensures it will not have to 'compete' for testing time on generic devices, allows a more precise mirroring of the Production environment and means less preparation for the environment prior to each test. This is expedited by using tools and procedures, which ensure that the production environment is known exactly and can be re-created quickly. There are many tools for comparing two environments and for capturing and restoring environments to a 'baseline' image.

Test Results Analysis and Failure Correction

It is very unlikely during this stage of testing that a design problem will be identified as the cause of a failure. However, if a design problem is suspected, the potential impact to the live environment and the likeliness of problem occurrence must be considered. The test coordinator must report all findings to the release manager, who coordinates all problem corrections with release team members and development personnel.

Roles and Responsibilities

Release Process Owner

Release Manager / Change Initiator

For releases with large and complex scopes, the Release Manager forms a Migration Team to manage the many required release activities. The Release Manager selects the team members, obtains the appropriate management approval, and assigns team roles and responsibilities. The Release Manager facilitates team communication, thereby ensuring that releases are implemented according to the approved timeline with system integrity and availability maintained.

The Release Manager is responsible for ensuring active participation in Release Board meetings and providing monthly release metrics. The release Manager sponsors Release plans, schedules and implementations before the Corporate CAB. Responsibilities include:

• Ensure successful implementation of production releases and identify opportunities for quality improvement.
• Develop and maintain the Release Calendar
• Communicate and train on the release schedule and release process
• Develop and maintain the release process
• Obtain Security authorization for production migration process
• Provide back up to Migration Team
• Coordinate and facilitate Release Board meetings
• Coordinate and facilitate any required technical release meetings with Release Coordinators and Migration Team
• Maintain Detailed Release Listing
• Coordinate production migration approval/readiness with Migration Team
• Answer questions about release content and status

Migration Teams

• Migrate code to production and execute database updates
• Update production version of tool specific applications
• Answer calls for migration questions/issues (not application issues)
• Provide all controls, verifications, checks and balances for migrating code
• Provide metrics on migration errors and corrections
• Coordinate releases amongst all application and rollout areas
• Provide Release Support during business and non business hours during Releases

Release Board

• Review and approve a release schedule, including initial setup for the upcoming year and ongoing maintenance of the schedule throughout the year
• Review and approve planned release content - business request level, quarterly view
• Analyze cross-system and cross-organizational impacts in planned releases
• Review changes to release process
• Represent entire area of responsibility
• Communicate planned release content to area of responsibility
• Submit schedules to Change Advisory Board for approval

Change Advisory Board (CAB)

• Approves, in concept, all application and hardware releases to the infrastructure including associated schedules, annual release plans and in-year revisions by the Release Board.
• Approves implementation requests contained in release plans for the period under review by CAB ensuring that
• There is no risks associated with scheduling the release with other change events
• adequate testing has taken place
• back-out plans have been developed and tested

Release Coordinator

For each release that they have changes in the Release Coordinators are responsible for the following:

• Ensure quality of area production release.
• Ensure code is in appropriate staging library for move to production
• Attends all Release Board meetings
• Coordinates Releases across areas (with Project Manager)
• Complete Release Spreadsheet for entrance into release process (including change requests, area impacted, Risk Type, Release Type)
• Complete Production Release Entrance Criteria checklist
• Coordinate production checkout activities
• Perform Availability Management on release dates
• Ensure availability of backup to release coordinator if necessary
• Determine cause of any release problems and work with Release Manager on continuous improvement of Release Process
• Be available for representative on Corporate CAB for the implementation of parts of the release into the production environment

Communications Coordinator

• Develops and manages the communications plan
• Develops content based on input from all project members
• Finalizes, gains approval for, and distributes content
• Determines the channels of information dissemination
• Develops feedback mechanisms and a collection database for user comments
• Ensures issue resolution is communicated in a timely and effective manner
• Evaluates and updates the communications plan to maintain its effectiveness throughout the release process
• Communicates release goals and scope to users
• Communicates release status, progress, and issues to appropriate groups

Documentation Coordinator

• Assesses current user guides, administrator guides, and reference manuals
• Defines purpose and use of required documentation
• Identifies required documentation from developers and users
• Identifies delivery options and formats that meet the users' needs
• Designs, develops, and proofs documentation
• Tests and validates documentation with users
• Manages the modification of existing documentation to support the release
• Disseminates documentation to appropriate personnel

Rollout Coordinator

• Manages the planning of the pilot staging and organization-wide implementations
• Develops implementation plans and determine site locations for pilot rollouts
• Establishes implementation schedules
• Identifies and communicates problems and schedule changes based on feedback from release team coordinators
• Ensures that the rollout team is properly trained
• Validates rollout and back out plans during release testing

Testing Coordinator

• Designs, builds, and maintains the test environment
• Prepares test scripts and execute testing strategies
• Identifies testers and assigns testing responsibilities
• Develops and manages testing schedule
• Documents test procedures and problems
• Manages problem resolution and re-testing
• Gathers, analyzes, and documents test results
• Communicates results and provide recommendations to appropriate parties
• Validates readiness of release package to be integrated into the live environment

Training Coordinator

• Identifies user competency to use the release
• Performs gap analysis against target competencies and skill requirements
• Identifies individuals to be trained to meet goals and objectives of the release
• Selects training methodologies best suited for content and audience
• Designs, builds, and implements the training strategy
• Communicates training plan to audience
• Manages and coordinates daily training activities
• Collects and analyzes feedback from trainees for effectiveness of delivery methodology and content

Availability Management

• Undertakes availability modeling to assess the impact of future changes and to identify potential improvements
• Coordinates scheduling of release implementation in pilot staging and live environments

Capacity Management

Security Management

• Evaluates proposed releases for proper security
• Identifies security considerations associated with the planned release
• Develops and implements security changes in coordination with other release team members
• Manages recommendations for system security during all phases of the release process

Performance Measurement

Key Goal and Performance Indicators

By comparison, a Key Performance Indicator is a measure of "how well" the process is performing. They will often be a measure of a Critical Success Factor and, when monitored and acted upon, will identify opportunities for the improvement of the process. These improvements should positively influence the outcome and, as such, Key Performance Indicators have a cause-effect relationship with the Key Goal Indicators of the process.

A number of key performance indicators (KPIs) should be monitored to assess the effectiveness of the Release Management process. Consider choosing some measures that show a clear indication of at least some of the following:

• Releases built and implemented on schedule, and within budgeted resources (but care should be taken to isolate any problems that are outside the control or responsibility of Release Management, such as application development delays)
• very low incidence of Releases having to be backed out due to unacceptable errors
• low incidence of build failures
• secure and accurate management of the DSL
• no evidence of software in the DSL that has not passed quality checks and no evidence of reworks on any software that was extracted from the DSL
• DSL sizing matching the demand for space, and timely and accurate housekeeping of the DSL
• compliance with all legal restrictions relating to bought-in software
• accurate distribution of Releases to all remote sites
• implementation of Releases at all sites, including remote ones, on time
• no evidence of unauthorized reversion to previous versions at any site
• no evidence of use of unauthorized software at any site
• no evidence of payment of licence fees or wasted maintenance effort, for software that is not actually being used at any particular location
• no evidence of wasteful duplication in Release building (e.g. multiple builds of remote sites, when copies of a single build would suffice)
• accurate and timely recording of all build, distribution and implementation activities within the CMDB
• a post-mortem carried out on all Release activities, and all necessary corrective or follow-up action taken, together with any process improvements
• the planned composition of Releases matching the actual composition (which demonstrates good Release planning)
• IT and human resources required by Release Management being subject to good ongoing forward planning.

Metrics

• Number of Releases
• Number of Changes introduced as part of a Release,
• Average number of changes per release
• Number of CIs changed and affected (ie. associated with in CMDB) as part of a Release,
• Number of incidents reported attributable to CIs changed as part of a Release
• Average time to restore service caused by incidents attributable to CIs changed as part of a release
• Number of changes introduced as part of release without complete testing
• Number of Releases using CIs checked out of DSL
• Number of Urgent Changes as part of Release processes
• Number of releases undergoing rescheduling

Breakdowns

• By categories representing number of defects introduced into production as part of release
• By application / organization associated with the release
• By category of length of time for development project
• By category of cost of development project (eg., small, medium, large, very large)

Measurement Issues

• Definitional issues associated with "release" and "changes associated with release" may introduce behaviors specifically targeted at meeting the metrics by modifying the definition of the metric.
• Developing interval measures for number of defects introduced as part of Defect List into production as cross-comparison with ensuing incidents - intent to determine relationship and costs associated with ROI from greater efforts in testing to eliminate defects prior to release.

Processes

Release Management Summary

Inputs

• Project Lifecycle

• Project documentation

• Service-related deliverables

• Authorized RFC

• Release Policy

• Overview of business needs

• Constraints and dependencies

• Defect List

• Release Policy & Guidelines

• Change Guidelines

• Discovery tools

• Testing methodology and proceses

• Template

Outputs

• New SW in DSL

• Updated CMDB

• Known Errors

  Unresolved items remaining in the Defect List are reconstituted as Known Errors in the production environment which require being addressed through Problem Management

• Testing results

• User Training

Activities

Planning a Release involves:

• gaining consensus on the Release contents
• agreeing to the phasing over time and by geographical location, business unit and Customers
• producing an organization-level Release schedule
• conducting site surveys to assess existing hardware and software in use
• planning resource levels (including staff overtime)
• agreeing on roles and responsibilities
• obtaining detailed quotes and negotiating with suppliers for new hardware, software or installation services
• producing back-out plans
• developing a quality plan for the Release
• planning acceptance of support groups and the Customer.

Release planning inputs include:

• project life-cycle
• service-related deliverables
• authorized RFCs
• Release policy
• an overview of business needs
• constraints and dependencies
• CAB output
• templates

Release planning outputs include the plan for a particular Release, and high-level test plans and acceptance criteria for the Release. The outputs of Release planning are normally documented in the Change Management plan for a particular project.

RM1.1 Gain approval for Release Objectives

The Release Manager coordinates the articulation of the objectives of the release by:

• determining line of business (LOB) requirements for the release
• referencing these requirements in view of other release events maintained on the Change Calendar
• assessing the availability of pre-production test beds and detailing risks associated with the release
• estimating financial and resources required for release
• Getting the approval to proceed to develop the Release Plan

RM1.2 Gain internal approval on Release Plan

Achieve consensus on the Release contents and obtain agreement on the phasing over time and by geographical location, business unit and customers. Get approval by Release Board to proceed. The Release Board recommends the appropriate forum in which the Change is required to be heard. If the impacts are seen to be containable (by referencing OLA Guidelines between the Corporate and Local Change Boards) then the authorizing board will be the Local Change Board (LCB). If there are potential infrastructure-wide implications then the Change Board will be Corporate CAB.

Assess magnitude of changes involved and what Release/Change forums need to be approached for approval by one or more of:

• Executive CAB
• Corporate CAB
• Local CAB
• Corporate Change Mgr
• Change is pre-authorized (ensure adherence to pre-authorized procedures)

RM1.3 Develop High Level Release Schedule

If approval to proceed is granted by the Release Board then the Release Manager details based on best available estimates, the timing for the rollout. This is done in consultation with line of business representatives and using the Change Calendar as a reference to determine points of conflict and assessing the CMDB for potential points of infrastructure impact,. This schedule forms part of the Request for Proposal to be submitted to the Change authority.

CM1.2 RfC Filtering and Approval

Objective: Gaining approval from the authorized source which is best aware of all possible changes occurring and which best represents the total interests of the entire infrastructure.

The Release Manager develops the Request for Proposal to be submitted to Change Management to gain overall approval to proceed with the Release schedule.

The Change Management forum will be either the Corporate CAB or the Local CAB for the change depending upon the recommendation of the Release Board. Note: In most cases the RFC for approval in concept for the release will be presented before Corporate CAB because of the wide ranging and, as yet in the process, undefined overall impacts on the infrastructure. Where there is uncertainty it is better to err on the side of caution and put the release through the greater scrutiny associated with the Corporate CAB forum. Individual changes associated with the product rollout may, thereafter, be referred to a Local CAB after the initial testing and backout strategies have been presented.

RM2: Designing, Building and Configuring a Release

RM2.1 Develop Testing Strategy & Plan

Objective: Ensure that all tests to be performed have been identified, and formal processes and procedures have been developed to encapsulate all test scenarios.

Establish criteria to determine when and if the application is Fit for Release. The Defect List is a primary concern in this assessment as it establishes how 'fit' the application is for production. This assessment forms part of a Testing Strategy to be developed by the Release Manager.

The Test Coordinator is then responsible for developing a comprehensive test plan based on the Testing Strategy that includes: scope and objectives, schedule, risks, test scripts, documentation requirements, resource requirements, and problem resolution. ( See the Section on Criteria to Determine Amount of Testing Required at the end f this document).

The Testing Coordinator will have the Release Manager approve the Plan. The Plan will be part of the documentation to be placed in the DSL and recorded as a CI in the CMDB.

Testing strategy should include the following

• An outline of the aims and objectives, to assist with the clarification and purpose of the testing function, which have been agreed with the business and IT management.
• Detailed testing scenarios to be performed, and the various roles and responsibilities assigned to each.
• The methodology under which testing is to be performed, and the approach the organization is going to follow to ensure the testing phase is addressed accordingly.

Any requirements, either organization specific or regulatory which must be implemented during the testing phases.

RM2.2 Assemble Release Resources

Using the Build Book as a guide, obtain any additional IT staff and purchase or otherwise acquire hardware and software components. The receiving and storing of components (i.e., logistics) will need to be managed from a logistical standpoint. Then identify other resources needed both for reproducing the conditions set out in the Release Plan and for ongoing service delivery - the secured resources should include everything necessary to implement service support and control mechanisms.

Objective 2: Ensure smooth project execution by ensuring all necessary conditions are in place.

All software, parameters, test data, run-time software and any other software that is required for the Release should be under Configuration Management control. Quality control checks should be performed on this software before the application is built. A complete record of the build results should be logged in the CMDB. This ensures that a build can be repeated should it be necessary.

In securing testing resources two best practices are advanced:

• Testers should be independent of application developers in order to ensure independent, unbiased results
• Testing should include line of business participation to ensure the Customers viewpoint and concerns are represented and to ensure involvement by the Clint with Testing results

The high-level test plans for the Release need to be expanded to include specific tests to verify the successful rollout of the Release, by satisfying critical success factors and exit criteria.

RM2.3 Procure / Assemble Hardware

Move the components to designated target sites for final assembly, integration, and implementation.

RM2.4 Build / Assemble Hardware

Install any software not addressed by the initial component assembly and integration using available software control and distribution tools and methods. Any final configuration updates should be performed at this time as needed to provide the service support and control mechanisms detailed in the Release Plan. All localized data should be loaded.

RM3: Release Acceptance

Testing should cover the installation procedures and the functional integrity of the resultant system. There should be a sign-off for each stage of testing. The final acceptance and sign-off for the Release to go into the live environment should be an agreed stage of the Change Management process. All levels of support should be involved in the testing of major Releases.

Release acceptance should be performed in a controlled test environment that can be reinstated to known configurations of both software and hardware. These configurations should be described in the Release definitions and stored in the CMDB, along with any other related CIs.

Release acceptance inputs comprise:

• a controlled test environment(s) configured to replicate the current live versions of the application with documentation of the test configuration
• Release definition and plans
• Release test plans and acceptance criteria
• copies of the installation media and installation instructions
• test plans for the installation scripts
• documented back-out procedures.

The end result of the acceptance activity should be a sign-off on the completeness and accuracy of the whole Release.

The outputs should include:

• tested installation procedures
• tested Release components
• tested back-out procedures
• known defects to be carried forward into the live environment
• test results
• support documentation including the system overview; updated support procedures; diagnostic aids
• operating and administration instructions
• contingency and back-out plans
• a training schedule for Service Management, support staff and Customers
• acceptance test documentation signed by all relevant parties
• authorization to implement the Release (done through Change Management).

RM3.1 Migrate Configuration Items (CI) from DSL to UAT

The Testing Coordinator checks the CIs designated in the Release and Test Plans from the DSL into the Pre-production (testing) environment. Any additional check-outs are noted by ensuring modification to the Release and Test Plan documents prior to exit.

RM3.2 Conduct Acceptance Testing

Acceptance testing procedures should :

• Satisfy users and computer operations staff that the system as installed meets the contract conditions and/or acceptance test criteria such that the user is prepared to authorize a system acceptance document.
• Demonstrate that the software does not affect either the functions or service levels of other IT services.
• Enable customers to test their own procedures and to demonstrate that they are ready to use the new or amended IT service

These tests should include:

• Load / Stress testing - Creation of production environment (for data and Users), where every piece of system should be tested and monitored. Additionally, performance and response times of all hardware and software components should be analyzed. It is important that transaction volumes placed on the system should be of a similar extent to those that could be reasonably expected during system operation in the live environment.
• User acceptance testing - Should be performed by Users for final approval. Business Users should generally have been involved in the testing plan creation from an early stage. Their involvement in the development and evaluation of test plans and results is crucial to achieving a successful User Acceptance function.

UAT should be fully documented using standard methodology agreed during the strategic and design phases, and should indicate issues to be resolved prior to the system being implemented in the live environment.

UAT must have followed strict acceptability criteria as defined during the design phase, and should only be formally authorized subsequent to completion of testing by both the IT and Business Users for each key process.

The Testing Team and business management should provide written evidence of authorization and approval of each key testing process.

Testing should confirm acceptable system results and environment performance and The Testing Coordinator should confirm that test failure points have been resolved and re-tested and the entire process documented. Testing should confirm success of user acceptance shakedown and resolution of identified issues.

RM4: Rollout PLanning

Rollout planning involves:

• producing an exact, detailed timetable of events, as well as who will do what (i.e. a resource plan)
• verifying and firming up the CIs to install (tentatively itemized in Release Plan) as well as those to decommission, with details on the method of disposal for any redundant equipment and software
• documenting an action plan by site, noting any implications of different time zones on the overall plans (e.g. an international organization may well not have a single common Release window when none of its systems is being used throughout the world)
• producing Release notes and communications to end Users
• planning communication
• developing purchasing plans
• acquiring hardware and software where, because this often involves the acquisition and deployment of numerous The organization's-value assets, the rollout plan should include the procedures to be followed for their secure storage prior to rollout and the mechanisms to trace their deployment during the implementation (which could involve the use of asset tags or other electronically readable labels)
• scheduling meetings for managing staff and groups involved in the Release.

RM4.1 Develop Rollout Timetable

• produce a timetable of events and document an action plan by site, as well as who will do what (i.e. a resource plan)
• listing CIs to install and decommission, with details on the method of disposal for any redundant equipment and software

RM4.2 Develop Site Success Criteria

Establish Site Performance Goals and Metrics:

• Maximum Number of Users; Web Page Load Time; Percentage Compliance, etc.
• Simulated User Bandwidth (%) LIKE 28.8k, 56k, 128k, 256k, etc.
• Identify Cases for Testing like making a purchase and how the user logs in, selects a product, and enters credit card information

The next step is to design performance tests using the test plan's goals and metrics as a guideline to insure the most realistic simulated users. How users connect to the site (via modems, ADSL, LAN cable etc.) and how much time they take to do a transaction on the site can be simulated in order to track the performance and health of the system.

• Build the load test scripts in the test plan
• Data Preparation - Collect and input the test data into the server as the test scripts are written

RM5: Communications, Site Preparation & Training

It is important to publicize the Release mechanism. It is also important to publicize any constraints to end Users (for example that it may not be possible to effect an update to all PCs overnight in a large organization).

RM5.1 Develop Site and Send Out Communications

• produce Release notes and communications to end Users at specific sites
• plan communication templates and protocols to Sites
• establish contingency communications in the event of the need for back-out, incidents, problems, etc.
• schedule meetings for managing staff and groups involved in the Release.

RM5.2 Develop and Implement Site Training

• Determine onsite training needs to assist with the rollout
• Re-constitute Defect List as Known Errors
• make Service Desk aware of rollout schedule and any issues which might ensure

RM5.3 Prepare Site for Release

For certain releases, it may be appropriate to perform a survey of the production environment to ensure that the rollout plan contains all of the activities required to deploy the release into that environment. The survey might also uncover issues with the production environment that could significantly delay or even prevent the deployment of the release. The Release Manager will need to escalate these issues to change management. For example, A survey may reveal that the data center does not have sufficient (free) network connections to support the deployment of a file server, for example, and that a new network switch is required. The release manager will need to discuss the problem with change management and seek their approval for the purchase and installation of the required equipment. Assuming approval is given, the tasks and activities required to connect it into the production network must be factored into the rollout plan.

Objective 2: Ensure all elements required for the release are in place and operational in order to increase probability of successful release

Software distribution should be designed so that the integrity of software is maintained during handling, packaging and delivery. Automated software distribution to remote locations using SMS should be used whenever practical so as to save resources and reduce the distribution cycle time. After distributing software over a network, the Release manager should check that the Release is complete when it reaches its destination. It is quite common to distribute a new version of an application to a target installation, but to do so in a way that it remains dormant until activated. This should be accomplished by following the tested installation procedures. These may require running automated installation routines or other one-off utilities to effect the Change. To ensure a smooth rollout, an automated check of the target platform is required so as to ensure that it satisfies hardware and software prerequisites. This would call upon the audit services of Configuration Management, but Release Management drives the process.

• Check specific sites to ensure generalized site preparation steps remain valid and add any additional steps to ensure a smooth deployment.
• Tasks and activities are re-checked for accuracy. This includes communicating information about the release to users and other personnel, training service desk and technical support staff, and making backups of critical IT components.
• Once approval is given to prepare the production environment for a release, the Release Manager needs to check the rollout plan and confirm that they will be able to gain access to all of the resources. Resources in this context include equipment and materials as well as the individuals responsible for carrying out the tasks and activities listed on the rollout plan.
• Having confirmed resource availability, the Release Manager needs to ensure that users, support staff, and others are made aware of and are prepared for deployment. To achieve this, they should arrange for information about the release, the expected rollout dates, and details of training courses (if appropriate) to be sent to all interested parties
• A number of actions may be necessary to prepare the technology, people, and process components of the production environment. From a technology standpoint, for example, application source files may need to be copied to distribution points (file servers) within each business location before the application can be rolled out to client workstations.
• Information communicated to users, support staff, and others during the rollout preparation process often includes rollout plans and dates, details of where to find and sign up for training courses (if appropriate), and where to obtain more detailed information about the release. A detailed Q&A may also be provided to address the most common questions and concerns.
• It may also be appropriate to provide a list of actions the users need to carry out to prepare themselves and their workstations for the release. For example, if older workstations are being replaced because they do not meet the Windows XP minimum hardware specification, the information provided during rollout preparation could include the steps necessary to manually copy user files from the hard disk to a personal data area on the network file server.
• Once all the preparation tasks have been completed, the release is ready to be deployed into the production environment and the Release Manager should call a meeting to obtain agreement to proceed.

RM6 Distribution and installation

The processes for procurement, storage, dispatch, receipt and disposal of goods should ensure that equipment is delivered safely to its destination in its expected state. Storage areas should be secure. Checks on the receipt of goods against supporting delivery documentation are required for Configuration Management. Installation, environmental and electrical checks should be planned and completed before connection to the network.

Software distribution should be designed so that the integrity of software is maintained during handling, packaging and delivery. Automated software distribution to remote locations will save resources and reduce the distribution cycle time. After distributing software over a network, it is essential to check that the Release is complete when it reaches its destination.

Bringing application software Releases into active use in these environments is the final step in effecting the Change. It is quite common to distribute a new version of an application to a target installation, but to do so in a way that it remains dormant until activated. This should be accomplished by following the tested installation procedures. These may require running automated installation routines or other one-off utilities to effect the Change. To ensure a smooth rollout, an automated check of the target platform is required so as to ensure that it satisfies hardware and software prerequisites. This would call upon the audit services of Configuration Management, but Release Management drives the process.

The CMDB should be updated following installation or disposal of hardware or software, to ensure that it reflects the final position. It may be necessary to retrieve old versions of software that have been superseded, to prevent software licence rules from being violated.

RM6.1 Review and Revise Rollout Schedule

The Migration Team may often find that business requirements and priorities change during the course of release development and it might be necessary to change aspects of the release plan or the rollout order in response to these changes. Therefore, it is important to confirm the rollout order as the first step in rollout execution. If, for example, part of the organization that originally agreed to wait until the end of the rollout process to deploy the release now identifies a pressing and urgent need for the release, the release team may be forced to change the release plan to accommodate them. As well, prior to the proposed release date the Release Manager should ensure that all changes have been tested and signed off by the impacted business areas. Any changes not ready for release need to be pulled from the release and the remaining content verified as still working without the pulled changes. The Release Manager is notified of pulled changes as part of this process.

RM6.2 Site Survey

For certain releases, it may be appropriate to perform a survey of the production environment to ensure that the rollout plan contains all of the activities required to deploy the release into that environment. The survey might also uncover issues with the production environment that could significantly delay or even prevent the deployment of the release. The Release Manager will need to escalate these issues to change management.

CM6.2 Execute Release

Once the release is deployed, the Release Manager needs to confirm that it is working correctly before proceeding with any further deployments. For some releases, technical support staff can obtain confirmation by using a number of tools and technologies; for others, the Release Manager may need to ask the service desk to contact individual users for their feedback and comments.

If the release fails to meet expectations or serious problems are encountered during deployment, problem management may be needed to help identify and diagnose the root cause of the problem. If a suitable fix or workaround can be found, this should be documented and a request for change (RFC) created to deploy it into the production environment. If not, it may be appropriate to use the rollback procedures to remove the release from that environment.

Once the deployment phase is complete, the release tracking system should be used to record the results and information about any workarounds or request for change (RFC) raised in support of the release. If the release needs to be backed out, this should also be recorded, including any information that supports this decision.

CM4.1 Close RfC

Objective: Ensure integrity of Change and Release Management processes by ending this Change in the prescribed method.

If the change is successful the RfC is marked CLOSED. Important changes are then subject to a post-implementation review in process RM6.6

CM4.2 Perform Back-out

Plans should be developed beforehand to back-out changes in case significant problems occur during deployment. If the Change was not successfully implemented into the production environment the Change needs to be backed-out. If backout plans need to be invoked the Change Builder should be conscious of several come caveats regarding their use:

• Research other changes that may have occurred between the time of implementation and the completion of the back-out plan.
• Get assurance that the production environment has been restored to its' previous known state.
• Update the log with all information related the decision for backing out the change.

When back-out is completed, change the status of the RfC to "Completed" with a completion code of Backed Out (or use whatever other method has been agree to within Change management for denoting incomplete changes). Description to be developed.

RM6.6 Post-Release Review

A Change Review is conducted for all major changes as part of the Change Management process outlined in CM5 - Post-Implementation Review. In a similar fashion, a post-implementation release review should be conducted which assesses the the successes and problems encountered during the entire release process in order to codify the lessons for future release ventures.

Appendix

Release Plan Contents

Objectives

Scope of Release

Release Process Resource Requirements

Personnel

Test Infrastructure Preparation

Release Operational Requirements Following Implementation

Personnel

Training

User training is most effective when it is administered immediately prior to implementation and when it can be leveraged as a tool to improve operational performance. Effective training dispels employees' concerns about using the new system and also reduces downtime following implementation. If infrastructure changes are required to support the release, users must be trained on these modifications as well. Communicating the training plan to users keeps them involved in the release process and builds support for the new system.

Training must also be provided to the service desk staff and rollout team. Establishing training priorities for the service desk staff is important because this staff plays an important role in gaining user acceptance of the new system. An unprepared staff is less able to answer questions and resolve problems quickly, and this leads to loss of confidence by users of the new system. The service desk staff should be trained before user acceptance testing begins. Additional training should be conducted during the test phase to take advantage of the lessons learned through user assessment of the new system. The rollout team must be fully trained on the backout procedures before deployment to the live environment can begin. This training occurs in conjunction with user acceptance testing and pilot staging implementations in which the rollout team can get hands-on experience.

Documentation

Infrastructure

Communications Plan

Documenting the Release Process

User Acceptance Testing

Pilot Staging

• Guidelines for establishing the pilot environment.
• Requirements for user participation.
• Anticipated risks to the live environment.
• Backout procedures.
• The metrics used to evaluate the success of the pilot.
• The sites selected for pilot implementation.

The rollout coordinator makes the plan available to all affected group(s) within the organization. The plan includes pilot goals such as:

• Verifying that the business processes meet the business requirements.
• Testing the implementation procedures.
• Training the rollout team and the service desk staff.
• Verifying application capabilities and integration with existing systems.
• Building user acceptance.
• Full Release Implementation

Planning the release implementation in the live environment is the task of the rollout coordinator. This release plan section gives an overview of the release implementation activity, including the rollout type (parallel, phased, and so on) and the timeframe. It also outlines the backout procedures to be followed in the event of a system failure.

Plan Approval

Plan Revisions

Testing Plan

Scope and Objectives

Schedule

Risks

• Problems with establishing a comprehensive test environment.
• Inability to obtain the resources necessary to perform all tests adequately.
• Inadequate user participation.
• Changes not being made in a timely manner when errors are detected.

Test Scripts

Designing test scripts is a time-consuming task. Although it may be tempting to take shortcuts, the time spent designing test scripts pays off in the long run. Test scripts include information such as the following:

• Purpose of the test
• Special hardware requirements
• Special software requirements
• Specific setup or configuration requirements
• Description of how to perform the test
• Expected results or success criteria for the test

Test scripts must be designed to realistically represent the conditions and variations in the live environment if they are to accurately reflect how a release will execute. In many cases, time and environmental constraints do not permit complete testing of the system. Focus should be placed on areas having the greatest risk.

Documentation Requirements

• Tester's name and department.
• Date and time the test was performed.
• Full description of the results.
• Resolution of each test failure.
• Identification numbers for unresolved problems that have been entered into a problem-tracking database.

Resource Requirements

This section outlines the number of testers needed and the skill level(s) required to perform the test scripts. The test coordinator should include user and support personnel and avoid using application development personnel. Identify any training that the testers need to understand the technology they are testing and coordinate this training with the training manager. Specify all tools required for the testing activity, such as tools needed to track the test results, detect errors, and monitor the test environment.

Problem Resolution

Criteria to Determine Amount of Testing Required

Use this chart as a guide for determining what test scripting will be required for your particular User Acceptance Test Plan. If the application changes to be tested fall in more than one criterion then multiple test script types may be required.

Application Criteria-Test Criteria

New Application - (not replacing an existing application)

New Application - (replacing an existing application)

Database Change - (no other change to the application)

Application Enhancements - (the existing application is enhanced with new functionality)

Application Enhancement - (the existing application is being enhanced to change the existing functionality)