ICT SERVICES AGREEMENT SCHEDULES
SCHEDULE 6.2
TESTING PROCEDURES
Section A - Product Description
1. PRODUCT DESCRIPTION
ICT Services Agreement - Schedule 6.2 (Testing Procedures).
2. INTRODUCTION
-
The schedule details the requirements and procedures necessary for the conduct of the Testing Procedures in respect of receipt of the Services and Deliverables.
-
The objective of the Testing Procedures is to establish that the Services and Deliverables are ready and capable of being provided by the Contractor so as to meet or exceed the Service Levels.
-
The Testing Procedures are closely linked to the Milestones and implementation and set out the process for the parties to establish whether a given Milestone has been Achieved and, ultimately, for the Contractor to achieve Authority to Proceed.
3. COMPOSITION
-
Clause 4 (Testing) sets out the basic testing procedure, legal implications of testing and any retesting and commercial consequences. Whether a Deliverable requires Testing should be set out in schedule 6.1 (Implementation Plan).
-
Schedule 6.2 sets out details of the processes and procedures of Testing and defines when Testing will be deemed to be successful, namely:
-
the period during which Tests will be conducted;
-
a description of the Test Success Criteria;
-
detailed procedures for the Tests; and
-
determining the success of the Tests.
-
4. DERIVATION
-
Authority Requirements
-
Schedule 2.1 (Service Description)
-
Schedule 6.1 (Implementation Plan)
-
Guidance Note 2 (Payment, Affordability and Asset Ownership)
5. RELATED CLAUSES & SCHEDULES
Clauses: |
4 (Testing) and its associated guidance |
Schedules: |
2.1 (Service Description) |
6. ALLOCATION
Authority to provide outline schedule to bidders with the descriptive document. Test Success Criteria to be developed in conjunction with the Services Description and Contractor Solution.
A draft Test Strategy should be submitted by the Contractor as part of its proposal in response to the ITT. The final Test Strategy is then agreed post-contract award.
7. QUALITY / REVIEW
Authority Expertise: Technical, Project Management, Commercial, Legal.
Section B - Guidance
1. INTRODUCTION
1.1 Clause 4 of the Agreement and this schedule together govern the Testing of the Services and Deliverables. The Implementation Plan (Schedule 6.1) should specify which elements of the Services and Deliverables must be subject to Testing and to which Milestone they relate. The Testing will be carried out in accordance with the procedures set out in this schedule. Clause 4 of the Agreement stipulates that the parties should bear their own costs in respect of the Testing Procedures save that the Contractor should bear the costs of re-testing if a Milestone does not achieve the relevant Test Success Criteria.
1.2 Testing is a key part of an ICT services agreement and affects a number of critical events:
1.2.1 successful completion of the Tests for all Deliverables in respect of a Milestone will result in the issue by the Authority of a Milestone Achievement Certificate;
1.2.2 in the case of the Milestone denoting readiness to commence service delivery, issue of the Milestone Achievement Certificate will amount to Authority to Proceed;
1.2.3 depending on how the payment profile of the project is structured, the issue of a Milestone Achievement Certificate may also trigger a payment to the Contractor; and
1.2.4 failure by the Contractor to Achieve a Milestone by its associated Milestone Date due to Contractor Default will result in a number of remedies being available to the Authority pursuant to clause 6 of the Agreement (Delays due to Contractor Default).
2. SCHEDULE CONTENT
2.1 The schedule should include a detailed description of the following:
2.1.1 the nature of the Tests that the Contractor will carry out;
2.1.2 the Test Success Criteria that will be applied to each of the Deliverables (this may be indicated against each of the Deliverables in the Outline Implementation Plan);
2.1.3 the procedures to be followed by each of the parties in conducting Testing;
2.1.4 a Testing timetable (it may be that each Deliverable can be tested independently and will be tested at a different time);
2.1.5 the notice period that must be given by the Contractor when the Authority is required to carry out, participate in or assist with the conduct of Tests. It is important that the Authority is given adequate notice in order to ensure that suitable personnel are available to participate in the Testing.
If Testing is to be carried out at a site not owned or controlled by the Authority, then provisions dealing with access should be included;2.1.6 categories of Test Issue with applicable severity ratings;
2.1.7 the documentation that the Authority will require as evidence of the Test results (including Non-conformance Reports); and
2.1.8 procedures for re-testing and resolving Test Issues following a Test failure (also see clause 6 of the Agreement for detailed provisions concerning Test Issues and Test Reports).
2.2 The Authority should consider the value of developing, with the Contractor, a matrix of roles and responsibilities for testing, specifying roles such as "accountable", "responsible", "witness" etc. This will help add clarity and certainty to each party's role.
3. TEST SUCCESS AND ISSUE OF A MILESTONE ACHIEVEMENT CERTIFICATE
3.1 Confirmation by the Authority that a Deliverable has passed the Tests and satisfied the Test Success Criteria, or the issuing of a Milestone Achievement Certificate by the Authority, should not equate to an approval by the Authority of the means of delivering the Service. The Test Success Criteria should be drafted in a manner that enables an objective assessment of whether the Service is ready to commence or a particular Deliverable is ready to be put into service. The Authority should not take performance risk back from the Contractor. In agreeing that the Service is ready to commence, the Authority should not, in any way, prejudice its rights to reject the on-going performance of the Service as performed nor in any way prejudice the service performance regime i.e. performance risk should remain with the Contractor who must ensure that Service Levels are met throughout the Term.
3.2 The Test Success Criteria should be objective and reflect the outputs specified in the Service Description. The criteria derived for the purposes of Testing must be capable of establishing and/or verifying whether the Deliverable is capable of underpinning that element of the Services to which it relates.
3.3 Successful completion of the Tests constituting a Milestone should be evidenced by the issue of a Milestone Achievement Certificate. It is important that the Authority should issue a Milestone Achievement Certificate in a timely manner. To protect the Contractor in the event that, the actual date of issue of the Milestone Achievement Certificate is delayed by the Authority, its issue should take effect from the date on which it should have occurred, if the Authority had complied with the prescribed timescales. Please be aware that a provision deeming Authority to Proceed is not an appropriate remedy for failure by the Authority to comply with the timescales. The Authority must retain control over Authority to Proceed as it is critical to ensure that the Service is ready for delivery.
4. REGRESSION TESTING
4.1 In some projects service delivery will commence on a phased basis i.e. not all services will have the same commencement date and/or additional elements of a given service stream will be introduced at different stages. Typically this would arise in relation to a migration from a legacy system(s) where the former service is wound down as the new service comes on-stream, or where increased capacity or functionality is required incrementally over a period of time. Generally, each phase of the implementation will have a Milestone and set of Deliverables which will be subject to Testing. It is important that the Authority does not affirm through Testing that phase Deliverables are ready for Service, which are proven to work in isolation but which have not been proven to work in combination with Deliverables from another phase (as part of the Contractor System). The Deliverables for each phase should be tested in conjunction with the previous phases. At the final phase, the Contractor System must be tested in its entirety and the successful testing of all Deliverables together should be a pre-requisite of the final Milestone and Authority to Proceed to full Service provision. This form of testing is commonly known as "regression testing".
4.2 In the case of a phased implementation, clause 4 of the Agreement (Testing) should make it clear that testing of each phase/Milestone is an interim test. Final testing of Service readiness is not achieved until all related Deliverables have been comprehensively tested together as a whole. In the event that the Deliverables do not meet the Test Success Criteria and Authority to Proceed cannot be given by the Authority, it should reserve the right to reject all or some of the phases/Milestones regardless of their earlier interim testing. The Authority should also reserve the right to clawback any payments made to the Contractor in respect of Milestone Achievement (see paragraph 2.4 of Part A of Schedule 7.1 (Charges and Invoicing)). These provisions are fundamental for creating the appropriate risk profile for an Authority in an ICT service agreement, as they ensure that performance risk is not transferred back from the Contractor to the Authority. It is of paramount importance that these provisions are included and their effectiveness is not diluted by negotiation.
4.3 A phased testing has key advantages in ensuring successful project delivery for the following reasons:
4.3.1 the Deliverables can be linked to Delay Payments (liquidated damages) thus incentivising the Contractor to meet the delivery timetable and keep to the critical path; and
4.3.2 the parties will become aware of any fundamental problems much sooner in the development stages because issues may be identified before final delivery into Service.
5. TESTING PRIOR TO FULL IMPLEMENTATION
5.1 The service should be subject to 'service readiness testing' in order to determine at a generic level whether the service is capable of meeting the requirements. Successful completion of service readiness testing should result in the Contractor being granted Authority to Proceed.
5.2 If the Contractor fails to complete an implementation test, there may be circumstances (e.g. where Milestone Payments have been made) where the Authority should be able to recover any sums paid to date. Where Milestone Payments are potentially repayable, the Contractor will attach less value to these payments. The reason for this is that the Contractor may not be able to account for the milestone payments as revenue in its profit and loss account, but will have to book them as pre-payments in its balance sheet.
5.3 For complex implementations, the Agreement needs to set out a process that allows the Authority to sign off that everything appears to be in place to begin live service delivery to a user/group of users or sites. Again, this should be without prejudice to remedies in respect of subsequent poor performance.
6. TESTING AT FULL IMPLEMENTATION
6.1 To reach full implementation the key Milestones must have been completed. These key Milestones are likely to include some element of implementation testing to provide assurance to the parties that the services delivered work technically and are capable of meeting the Authority's requirements.
6.2 It is recommended that any Implementation Plan includes the following features:
6.2.1 full implementation must take place in an operational environment. Any implementation tests should be undertaken on the services in operation. This may be on a site or client group basis if appropriate, but not on non-operational pilot sites or 'model office' demonstrations;
6.2.2 it should involve a period of successful live running following, where appropriate, service readiness testing. Live operation of the services, for a pre-agreed time and to pre-agreed standards, should be required before full implementation;
6.2.3 it should take account of the transition from any existing services to the provision of new services. In addition to implementation tests, the implementation period will often involve the simultaneous delivery of transition services (such as interim/legacy services). Where this is the case it is essential that the parties set out in detail the plan for delivery of these services (the transition plan) in the agreement. The transition plan should also encompass the set up of the new services, any communication plans, and any staff and asset transfers.
7. INVOLVEMENT OF THE AUTHORITY IN IMPLEMENTATION TESTING
The Contractor should be responsible for carrying out Testing. The Authority should consider the extent of its involvement in the tests. As a general rule, where the Milestone payments for full implementation are relatively small, or for service readiness testing, the Authority should be represented but normally in the role of an observer. Where larger Milestone payments have been agreed, or where the service or component is business critical then the Authority should have a more active role. The Authority's role should be clearly defined to ensure that it is fully satisfied that the services have been delivered to specification before any payments are made to the Contractor.
Section C - Pro-forma/Example Schedule
[Guidance: subject to the agreement of this schedule, the following definition(s) will need to be added to schedule 1]
DEFINITIONS
"Expedited Dispute Timetable" |
[Guidance: see schedule 8.3 (Dispute Resolution Procedure)] |
"Material Test Issue" |
a Test Issue of severity levels [ ], the criteria for which are set out in paragraph 16.1 of schedule 6.2 (Testing Procedures); |
"Milestone Achievement Certificate" |
the certificate materially in the form of the document contained in Appendix B to schedule 6.2 (Testing Procedures) granted by the Authority when the Contractor has Achieved a Milestone; |
"Quality Assurance Review" |
an inspection of the Contractor's implementation of the Services to ensure the adherence to the Standards; |
"Quality Manager" |
the Contractor's representative responsible for ensuring that the Contractor adheres to the Quality Plan; |
"Test Certificate" |
the certificate materially in the form of the document contained in Appendix A to schedule 6.2 (Testing Procedures) issued by the Authority when a Deliverable has satisfied its relevant Test Success Criteria satisfactorily; |
"Test Issue" |
any variance or non-conformity of a Deliverable from its requirements as set out in the relevant Test Success Criteria; |
"Test Issue Management Log" |
a log for the recording of Test Issues as described further in paragraph 16.2 of schedule 6.2 (Testing Procedures); |
"Test Plan" |
a plan for the Testing of Deliverables and other agreed criteria related to the achievement of Milestones as described further in paragraph 12 of schedule 6.2 (Testing Procedures); |
"Test Report" |
a report setting out the results of any Tests which shall include without limitation the information set out in paragraph 8.5 of schedule 6.2 (Testing Procedure); |
"Test Specification" |
the specification that sets out how Tests will demonstrate that the Test Success Criteria have been satisfied, as described in more detail in paragraph 13 of schedule 6.2 (Testing Procedure); |
"Test Strategy" |
a strategy for the conduct of Testing as described further in paragraph 11 of schedule 6.2 (Testing Procedure); |
"Test Witness" |
any person appointed by the Authority pursuant to paragraph 17.1 of schedule 6.2 (Testing Procedure); |
8. INTRODUCTION
1.1 This schedule sets out the approach to Testing and the different Testing activities to be undertaken, including the preparation and agreement of the Test Success Criteria, Test Strategy, Test Plans and Test Specifications.
1.2 The Authority will issue a Test Certificate when a Deliverable satisfies the Test Success Criteria for the Tests related to that Deliverable. The Authority will grant a Milestone Achievement Certificate when all Tests relating to a Milestone have been completed and, subject to clause 6.2 (Delays Due to Contractor Default), the Authority has issued Test Certificates in respect of all of the Tests relating to that Milestone.
9. RISK
9.1 The issue of a Test Certificate shall not operate to transfer any risk that the Deliverable will meet and/or satisfy the Authority's requirements for that Deliverable. The grant of a Milestone Achievement Certificate shall not operate to transfer any risk that the Milestone is complete or that it will satisfy the Authority's requirements for that Milestone. The grant of a Milestone Achievement Certificate in respect of Authority to Proceed shall not operate to transfer any risk that the Contractor System or the Services will meet and/or satisfy the Authority's requirements for the Services.
9.2 Consequently, the grant of a Test Certificate and/or Milestone Achievement Certificate shall not affect the Authority's right subsequently to reject:
9.2.1 all or any element of the Deliverables to which a Test Certificate relates; or
9.2.2 any Milestone to which the Milestone Achievement Certificate relates.
10. TESTING OVERVIEW
10.1 All Tests conducted by the Contractor shall be conducted in accordance with the Test Strategy, the Test Plans and the Test Specifications.
10.2 The Contractor shall not submit any Deliverables for Testing:
10.2.1 unless it is reasonably confident that they will satisfy successfully the relevant Test Success Criteria and until the Authority has issued a Test Certificate in respect of any prior, dependant Deliverables;
10.2.2 until the parties have agreed the Test Plan and the Test Specification relating to those Deliverables; and
10.2.3 unless it has provided the Authority with at least [ ] Working Days notice in writing certifying that the relevant Deliverable is ready for Testing.
10.3 The Contractor shall use reasonable endeavours to submit a Deliverable for Testing or re-Testing by or before the date set out in the Implementation Plan for the commencement of Testing in respect of the relevant Milestone.
10.4 Prior to the issue of a Test Certificate, the Authority shall be entitled to review the relevant Test Reports and the Test Issue Management Log.
10.5 The provisions of clause 4 (Testing), clause 5 (Implementation Delays - General Provisions), clause 6 (Delays due to Contractor Default), clause 7 (Delays to Milestones due to Authority Cause) and clause 8 (Delays not due to one Party) shall apply to Testing.
10.6 Any disputes between the Authority and the Contractor regarding this Testing shall be referred to the Dispute Resolution Procedure using the Expedited Dispute Timetable.
10.7 The Authority shall issue Test Certificates and grant Milestone Achievement Certificates without unreasonable delay.
11. TEST STRATEGY
11.1 The Contractor shall develop the final Test Strategy as soon as practicable but in any case no later than [....... ] Working Days (or such other period as the parties may agree) after the Effective Date.
11.2 The final Test Strategy shall include:
11.2.1 an overview of how Testing will be conducted in relation to the Implementation Plan;
11.2.2 the process to be used to capture and record Test results and the categorisation of Test Issues;
11.2.3 the method for mapping the expected Test results to the Test Success Criteria;
11.2.4 the procedure to be followed should a Deliverable fail to satisfy the Test Success Criteria or to produce unexpected results, including a procedure for the resolution of Test Issues;
11.2.5 the procedure to be followed to sign off each Test;
11.2.6 the process for the production and maintenance of Test Reports and reporting, including templates for the Test Reports and the Test Issue Management Log, and a sample plan to resolve Test Issues;
11.2.7 the names and contact details of the Authority's and the Contractor's Test representatives;
11.2.8 a high level identification of the resources required for Testing, including facilities, infrastructure, personnel and Authority and/or third party involvement in the conduct of the Tests;
11.2.9 the technical environments required to support the Tests; and
11.2.10 the procedure for managing the configuration of the Test environments.
12. TEST PLANS
12.1 The Contractor shall develop Test Plans for the approval of the Authority as soon as practicable but in any case no later than [..... ] Working Days (or such other period as the parties may agree in the Test Strategy or otherwise) prior to the start date for the relevant Testing as specified in the Implementation Plan.
12.2 Each Test Plan shall include as a minimum:
12.2.1 the relevant Test definition and the purpose of the Test, the Milestone to which it relates, the requirements being Tested and, for each Test, the specific Test Success Criteria to be met;
12.2.2 a detailed procedure for the Tests to be carried out, including:
12.2.2.1 the timetable for the Tests including start and end dates;
12.2.2.2 the Testing mechanism;
12.2.2.3 dates and methods by which the Authority can inspect Test results or witness the Tests in order to establish that the Test Success Criteria have been met;
12.2.2.4 the mechanism for ensuring the quality, completeness and relevance of the Tests;
12.2.2.5 the format and an example of Test progress reports and the process with which the Authority accesses daily Test schedules;
12.2.2.6 the process with which the Authority will review Test Issues and progress on a timely basis;
12.2.2.7 the Test schedule;
12.2.2.8 the re-Test procedure, the timetable and the resources which would be required for re-Testing; and
12.2.2.9 the decision making process for escalation from a re-test situation to specific remedial action to resolve the problem / Test Issue.
12.3 The Authority shall not unreasonably withhold or delay its approval of the Test Plans and the Contractor shall implement any reasonable requirements of the Authority in the Test Plans.
13. TEST SUCCESS CRITERIA
The Test Success Criteria for the Milestone Achievement Certificates have been agreed by the parties prior to the Effective Date and are set out in Appendix C.
[Guidance: if all Test Success Criteria are not capable of determination at signature you should set out a timetable for them to be agreed, with provision for escalation in the event of failure to agree.]
14. TEST SPECIFICATIONS
14.1 Following approval of a Test Plan, the Contractor shall develop the Test Specification for the relevant Deliverables as soon as reasonably practicable and in any event at least [..... ] Working Days (or such other period as the parties may agree in the Test Strategy or otherwise) prior to the start of the relevant Testing as specified in the Implementation Plan.
14.2 Each Test Specification shall include as a minimum:
14.2.1 the specification of the Test data, including its source, scope, volume and management, a request (if applicable) for relevant Test data to be provided by the Authority and the extent to which it is equivalent to live operational data;
14.2.2 a plan to make the resources available for Testing;
14.2.3 Test scripts;
14.2.4 Test pre-requisites and the mechanism for measuring them; and
14.2.5 expected Test results, including:
14.2.5.1 a mechanism to be used to capture and record Test results; and
14.2.5.2 a method to process the Test results to establish their content.
15. TESTING
15.1 Before submitting any Deliverables for Testing the Contractor shall subject the relevant Deliverables to its own internal quality control measures.
15.2 The Contractor shall manage the progress of Testing in accordance with the relevant Test Plan and it shall carry out the Tests in accordance with the relevant Test Specification. Tests may be witnessed by the Test witnesses in accordance with paragraph 17 (Test Witnessing).
15.3 The Contractor shall notify the Authority at least [..... ] Working Days (or such other period as the parties may agree) in advance of the date, time and location of the relevant Tests and the Authority shall ensure that the Test Witnesses attend the Tests, except where the Authority has specified in writing that such attendance is not necessary.
15.4 The Authority may raise and close Test Issues during the Test witnessing process.
15.5 Not less than [ ] Working Days (or such other period as the parties may agree) prior to the date on which any Tests are planned to end the Contractor will provide the Authority with a draft Test Report and on completion of the Testing the Contractor will provide the Authority with the final Test Report. Each Test Report shall provide a full report on the Testing conducted in respect of the relevant Deliverables, including:
15.5.1 an overview of the Testing conducted;
15.5.2 identification of the relevant Test Success Criteria that have been satisfied;
15.5.3 identification of the relevant Test Criteria that have not been satisfied together with the Contractor's explanation of why those criteria have not been met;
15.5.4 the Tests that were not completed together with the Contractor's explanation of why those Tests were not completed;
15.5.5 the Test Success Criteria that passed, failed or which were not tested, and any other relevant categories, in each case grouped by severity level in accordance with paragraph 16.1 (Test Issues);
15.5.6 the specification for any hardware and software used throughout the Testing and any changes that were applied to that hardware and/or software during the Testing; and
15.6 In the event that a Deliverable does not meet the relevant Test Success Criteria the Test Report shall constitute a Non-Conformance Report for the purposes of clause 6.1 (Delays Due to Contractor Default).
16. TEST ISSUES
16.1 Where a Test Report identifies a Test Issue, the parties shall agree the classification of the Test Issue using the following criteria:
[Guidance: The Authority should specify a range of severity levels, from the most severe (where the service or critical elements of it cannot be performed) ranging to minor incidents whereby all elements of the service are operational but there is a minor impact on scope and/or performance of the services. The Authority will need to amend the definition of "Material Test Issue" to state which severity levels will be classed as material and will therefore allow for escalation and, ultimately, possible termination pursuant to clause 6.2.2 of the Agreement. The guidance to clause 6.2.2 gives further detail here.]
16.2 The Test Issue Management Log shall log Test Issues to reflect the severity classifications allocated to each Test Issue pursuant to paragraph 16.1.
16.3 The Contractor shall be responsible for maintaining the Test Issue Management Log and for ensuring that its contents accurately represent the current status of each Test Issue at all relevant times. The Contractor shall make the Test Issue Management Log available to the Authority upon request.
16.4 The Authority Representative shall confirm the classification of any Test Issue unresolved at the end of a Test in consultation with the Contractor. If the parties are unable to agree the classification of any unresolved Test Issue, the Dispute shall be dealt with in accordance with the Dispute Resolution Procedure using the Expedited Dispute Timetable.
17. TEST WITNESSING
17.1 The Authority may, in its sole discretion, require the attendance at any Test of one or more witnesses. Witnesses will be selected by the Authority, each of whom will have appropriate skills to fulfil the role of a Test witness.
17.2 The Contractor shall give the Test Witnesses access to any documentation and Testing environments reasonably necessary and requested by the Test Witnesses to perform their role as a Test Witness in respect of the relevant Tests.
17.3 The Test Witnesses will actively review the Test documentation and will attend and engage in the performance of the Tests on behalf of the Authority so as to enable the Authority to gain an informed view of whether a Test Issue may be closed or whether the relevant Component should be re-Tested. However, the Test Witnesses will not be involved in the execution of any Test.
17.4 The Test Witnesses will be required to verify that the Contractor conducted the Tests in accordance with the Test Success Criteria and the relevant Test Plan and Test Specification.
17.5 The Test Witnesses may produce and deliver their own, independent reports on Testing, which may be used by the Authority to assess whether the Tests have been Achieved.
17.6 The Test Witnesses may raise Test Issues on the Test Issue Management Log in respect of any Testing.
17.7 The Test Witnesses may require the Contractor to demonstrate the modifications made to any defective Deliverable before a Test Issue is closed.
18. TEST QUALITY AUDIT
18.1 Without prejudice to its rights pursuant to clause 24 (Audits), the Authority may perform on-going quality audits in respect of any part of the Testing (each a "Testing Quality Audit") subject to the provisions set out in the agreed Quality Plan.
[Guidance: a cross-reference to the Quality Plan should be inserted here if it is appended to the Agreement]
18.2 The focus of the Testing Quality Audits will be on:
18.2.1 adherence to an agreed methodology;
18.2.2 adherence to the agreed Testing process;
18.2.3 adherence to the Quality Plan;
18.2.4 review of status and key development issues; and
18.2.5 identification of key risk areas.
18.3 The Contractor shall allow sufficient time in the Test Plan to ensure that adequate responses to a Testing Quality Audit can be provided.
18.4 The Authority will give the Contractor at least [..... ] Working Days' written notice of the Authority’s intention to undertake a Testing Quality Audit and the Contractor may request, following receipt of that notice, that any Testing Quality Audit be delayed by a reasonable time period if in the Contractor’s reasonable opinion, the carrying out of a Testing Quality Audit at the time specified by the Authority will materially and adversely impact the Implementation Plan.
18.5 A Testing Quality Audit may involve document reviews, interviews with the Contractor personnel involved in or monitoring the activities being undertaken pursuant to this schedule, the Authority witnessing Tests and demonstrations of the Deliverables to the Authority. Any Testing Quality Audit shall be limited in duration to a maximum time to be agreed between the Contractor and the Authority on a case by case basis not to be unreasonably withheld or delayed. The Contractor shall provide all reasonable necessary assistance and access to all relevant documentation required by the Authority to enable it to carry out the Testing Quality Audit.
18.6 If the Testing Quality Audit gives the Authority concern in respect of the Testing Procedures or any Test, the Authority will discuss the outcome of the Testing Quality Audit with the Contractor, giving the Contractor the opportunity to provide feedback in relation to specific activities, and subsequently prepare a written report for the Contractor detailing the same to which the Contractor shall, within a reasonable timeframe, respond in writing.
18.7 In the event of an inadequate response to the written report from the Contractor, the Authority (acting reasonably) may withhold a Test Certificate (and consequently delay the grant of a Milestone Achievement Certificate) until the issues in the report have been addressed to the reasonable satisfaction of the Authority.
19. OUTCOME OF TESTING
19.1 Deliverables will complete Testing and a Test Certificate shall be issued when the Deliverables satisfy the Test Success Criteria in respect of that Test without any Test Issues.
19.2 If the Deliverables (or any relevant part) do not satisfy the Test Success Criteria then:
19.2.1 the Contractor shall rectify the cause of the failure and re-submit the Deliverables (or the relevant part) to Testing, provided that the parties agree that there is sufficient time for that action prior to the relevant Milestone Date; or
19.2.2 the parties shall treat the failure as a Contractor Default.
19.3 When the Authority has issued Test Certificates and/or conditional Test Certificates in respect of all the Deliverables related to a Milestone it will also grant a Milestone Achievement Certificate.
19.4 The grant of a Milestone Achievement Certificate will entitle the Contractor to the receipt of a payment in respect of that Milestone in accordance with the provisions of schedule 7.1 (Charges and Invoicing).
APPENDIX A - Test Certificate
To: [CONTRACTOR] |
FROM: [AUTHORITY] |
[Date] |
Dear Sirs, |
| TEST CERTIFICATE |
Deliverables: [insert description of Deliverables] |
Yours faithfully |
APPENDIX B - Milestone Achievement Certificate
To: [CONTRACTOR] |
FROM: [AUTHORITY] |
[Date] |
Dear Sirs, |
MILESTONE ACHIEVEMENT CERTIFICATE |
Milestone: [insert description of Milestone] |
Yours faithfully |
APPENDIX C - Test Success Criteria
[Guidance: You should set out the Test Success Criteria here in respect of all relevant service elements]
Service Element |
Pre-conditions |
Test Success Criteria |
|
|
|
|
|
|