Proposition P12 - Service Quality metrics (as per CMA Order)

Caveat

This document is currently the internal house view of the Open Banking Implementation Entity (OBIE) and does not yet represent the official view of the Open Banking programme.

However due to the requirement to publish a specification for this item in Feb 2018, OBIE is progressing designs for this specification based on the requirements currently considered in scope below. We will work on these requirements first. We may (at a later date) work on other requirements, if these subsequently become part of the agreed scope. But for now we will not spend time on designing for requirements we consider may be in scope or not in scope.

Version Control

Version
Date
Author(s)
Comments
0.1 

Initial draft 

0.2 Chris Michael (Unlicensed)Minor update to reflect communication to IESG

Introduction

The purpose of this paper is to define the overall proposition for item P12, so that participants (ASPSPs and TPPs) and stakeholders (FCA, HMT, CMA) are clear about what is and is not in scope for this item, and how this will support regulatory requirements and key use cases.

Roadmap Definition

The following table is taken 'as-is' from the published roadmap (https://www.openbanking.org.uk/wpcore/wp-content/uploads/2017/11/FAO-CMA_Proposed-Amendments-to-Agreed-Arrangements_v_final-1.pdf):

DescriptionRationaleAligns with

Scope Item description:

  • Provision of key service quality metrics by the CMA9 (including for example net promoter score) via OB API and without charge according to part 3 of the Order

Key activities:

  • A discovery phase by the OBIE to understand modifications required to the exiting standards

  • The development of standards by the OBIE for the products and functionality referred to above

  • The implementation of those standards by the CMA9 for Release 2

Regulatory alignment:

  • Fulfils the requirements of the Order (part 3)

Consumer / SME utility:

  • Consumers would be able to assess and benchmark the CMA9 according to objective metrics
CMA Order

Regulations

Regulatory references 

CMA Order

13. Release of service quality indicators

  • Providers must release and make continuously available without charge, in accordance with the Read-only Data Standard:
    • 13.1.1 service quality indicators that the Provider is required to publish and release pursuant to Article 15; and
    • 13.1.2 all underlying data anonymised so that it is no longer personal data for the purposes of the DPA generated from the responses to survey questions commissioned in accordance with Article 16 unless such information has already been made public via other means. Such information may include survey results relating to providers who are not subject to this Part 2.

15. Requirement to publish service quality indicators

  • 15.1 Providers shall publish, in relation to each of their Brands to which this Part 3 applies, service quality indicators showing the willingness of their customers holding a PCA who have used the account or a relevant service in a defined period prior to the survey taking place:
    • 15.1.1 to recommend the Brand to friends and family;
    • 15.1.2  to recommend the Brand’s online and mobile banking services to friends and family;
    • 15.1.3  to recommend the Brand’s branch services to friends and family; and
    • 15.1.4  to recommend the Brand’s overdraft services to friends and family.
  • 15.2 Providers shall publish, in relation to each of their Brands to which this Part 3 applies, service quality indicators showing the willingness of their customers holding a BCA who have used the account or relevant service in a defined period prior to the survey taking place:
    • 15.2.1  to recommend the Brand to other SMEs;
    • 15.2.2  to recommend the Brand’s relationship/account management to other SMEs;
    • 15.2.3  to recommend the Brand’s online and mobile banking services to other SMEs;
    • 15.2.4  to recommend the Brand’s branch and business centre services to other SMEs;
    • 15.2.5  to recommend the Brand’s credit (overdraft and loan) services to other SMEs.
  • 15.4 The first set of service quality indicators shall be published by all Providers on 15 August 2018 falling six weeks after all the data, incorporating results from October 2017 (at the latest) to June 2018, has been collected. The service quality indicators shall thereafter be updated on the first Working Day after 14 February and 14 August each year based on data collected on a rolling basis over the 12 months from, respectively:
    • 15.4.1 the beginning of January to the end of December of the previous calendar year; and
    • 15.4.2 the beginning of July to the end of June incorporating six months of results from the previous calendar year and six months from the prevailing calendar year.

Competition and Markets AuthorityBanking Remedies : Presentation of qualitative research findings (Service Quality Metrics)

Regulatory requirements

The OBIE interpretation of this is that OBIE will define an API standard to cater for each of the above indicators and then each of the CMA9 will be mandated to implement and support an API endpoint which conforms to this standard.

Use cases

These regulatory references and requirements are clearly specified. However, it is also important to consider a real world use case in order to define the scope, and corroborate the definition and rationale as stated in the roadmap item above:

  • As a personal or business customer, when evaluating a short list of current accounts recommended to me by a price comparison website, I want to access service quality metrics from multiple providers, so that I can make comparisons between brands on basis of their service quality.

Scope

In scope

  • As defined in the CMA Order, this covers a number of UK brands (CMA9 and non CMA9).
  • OBIE will define the API specification for this.
  • OBIE will also provide a centralised platform (which takes a feed from each of the 2 x research agencies).
  • The brands will each have regulatory responsibility for implementing and supporting the APIs, inlcuding the underlying integrity of data in conformance with the agreed specification.

Comments

N/A

Not in scope

  • Other brands and metrics not explicitly defined by the CMA Order.
  • Other metrics, including those currently being defined by the FCA (these will be covered by an additional roadmap item TBC).

Considerations

Constraints

  • There is a risk that API users may not understand and may mis-interpret the data due (e.g. due to potentially small sample sizes) and this may result in mis-representation and/or invalid comparisons.

Dependencies

  • The existing open data licence will need to be reviewed and adapted to cover this item.
  • For OBIE to host this, there will need to be some agreement between OBIE and in scope ASPSPs.
  • Exact architecure and support model for OBIE centralised platform to be determined.
  • The API specification is dependent on the data format being agreed by the CMA, in scope ASPSPs, and UK Finance Steering Committee.

Assumptions

  • Access to the API endpoints will be based on the same principles as for open data (i.e. no registration required, but users may be asked to agree to an appropriate licence).
  • The 2 x research agencies (one for each of PCA and BCA) will provide accurate, approved data on a timely manner in accordance with the agreed CMA methodology.

How we will measure adoption

The following metrics will be required to measure adoption:

  1. Number of API users accessing each endpoint.
  2. Volume of successful API calls to each endpoint.
  3. Volume of failed API calls to each endpoint. 

Deliverables, Estimated Schedule and Completion Criteria

The following high level milestones apply. A more detailed schedule of miletsones will be supplied seperately via PMG.

Deliverable Name
Planned Date
Governance Milestones for Completion Criteria 
Product Proposition Paper 
  
Acceptance by Policy Advisory Group (PAG) and the Implementation Entry Steering Group (IESG)
Technical Specification Version 2.0 rc1 
  
Endorsement by Information Security Working Group
Technical Specification Version 2.0 rc2 
  
Technical approval by the Technical Design Authority
Technical Specification Version 2.0 Final
  
Industry Acceptance via IESG