API and data governance in federated organisations: Weeknotes #2, Saturday 15 Jan 2022

Labs

8 min read

Written by Mark Boyd
Updated at Sat Jan 15 2022
API and data governance in federated organisations: Weeknotes #2, Saturday 15 Jan 2022

This week, as part of our client-facing work, I have been thinking a lot about how to support organisations to implement cohesive API strategies.

Business, governments and industry-wide systems often work via a federated model in which individual units are responsible for managing their own data or APIs. For example, across various industries, it is common for one enterprise to have lines of business that each have their own data and developer teams. So the data being collected and managed within that line of business will be independent from how other lines of business collect and manage their data. For API development, each team independently creates the APIs that they need when integrating systems, or when exposing data and functionalities internally or with partners.

The same happens in government, where multiple individual departments might each collect, maintain or buy their own datasets, or build their own APIs. 

In health data systems, this federated model even extends to how a country’s health system is structured. Under European health data policy arrangements, for example, Spain, Switzerland and some other European countries have devolved health systems to autonomous communities (in Spain) or cantons (in Switzerland). Each of those regional entities is responsible for investing and managing data and data sharing within their regional boundaries. When collecting population health statistics (for example, to monitor pandemic impacts or manage national health resources), this federated structure often creates challenges when individual regions use different data models or incompatible data systems. (There are also benefits of federated, decentralised systems: they can support local ownership of health data and can help ensure that health system resourcing is prioritised in a way that meets local community values and needs.)

With the support of Lorenzino Vaccari, Monica Posada, and Dietmar Gattwinkel, in 2019 and 2020, I had the honour of being lead author of the European Commission’s API Framework for Digital Governments. (This work sat alongside a broader research project we were also a part of and led by Lorenzino, with other co-authors and researchers including Mehdi Medjaoui, Shelby Switzer, Isabelle Reusa and Mattia Santoro, amongst others.) During research for that work, we found that, quite often, governments (like businesses before them) identified technical needs for APIs at a departmental level. But over time, there was a recognition that APIs could provide an opportunity for whole-of-government approaches to foster engagement with a wider ecosystem of stakeholders both across departments and external to government. The departmental-level approach to designing and delivering APIs, however, was holding them back, as the APIs were often only known to those in each department, each were built slightly differently without common designs, and were proliferating to create complexity. (This was pretty much the same trajectory that Lesley-Ann Vaughan and I identified amongst digital financial services that were moving to API approaches in May 2018.)

A key element of this API Framework (built on government workshops across Europe, industry and stakeholder consultations, and a literature review of 343 best practice documents sourced from government and industry) was to find a middle path that could support departments to maintain their APIs, while also improving them to align with common standards and approaches. What was needed was API governance practices. 

The European Commission's API Framework outlines 12 key proposals that create a cohesive whole-of-government API strategy
Source: European Commission's API Framework for Digital Governments


In the Framework, API governance is identified as a key element at the strategy, tactical and operational levels. At the strategy (whole-of-government) level, it is necessary to define the structures and decision-making processes around how all APIs will be managed. At this level, governments should bring together the necessary stakeholders in a platform approach to define use case needs, identify common APIs (and data systems) that would be needed across all departments to prevent duplication and speed up development, and establish the decision-making structures that could oversee common standards for the development of APIs. At the tactical, or departmental, level, APIs would be built that met subject or business operation needs, but drawing on whole-of-government templates and standards to ensure that APIs were consistent. From a data governance point of view, departments with specific business domain knowledge might be responsible for maintaining specific data registries which could then be shared with other departments so that only one dataset was being maintained and used by multiple departments. At the operational level (that is, how APIs were being used within a department or agency on a daily basis), a product ownership model was recommended in which a business/policy lead would work to ensure APIs were useful and generated value for users, while technical leads would ensure the APIs were performant and aligned with industry best practices, including ensuring robust security.

Growing an Effective API Governance Process

Recommendations from James Higginbotham

"In a federated governance model, a centralized API governance team establishes the core processes and standards by which the organization will operate. Representatives from a specific business unit or region are trained as delegates that can provide context-specific guidance and coaching. The centralized API governance team works with federated API coaches to gain insight into emerging needs, suggested improvements, and any clarifications required to maintain consistency across the organization.

By introducing a federated governance approach, organizations gain the speed and flexibility they need to scale governance across a large-scale organization, while supporting the varying needs of different business units and regions."

 

Here are several tools I use to support organisations to think through how to build an API governance approach that will work for federated organisations:

  • Our value models and datasets that describe case studies and examples from across industries that quantify the benefits that have been realised by all types of organisations implementing cohesive API strategies. Sharing these examples in presentations, and by linking organisations to the architects and builders who have created success, helps shift the API strategy discussion towards finding a balance between business and technical management, and draws on examples where both centralised and federated structures have worked in practice.
  • Matthew Reinbold’s work on polarity mapping overcomes the dichotomy of centralised versus federated organisational models, and instead focuses on how to find the right balance to ensure governance is able to be maintained rather than becoming a new set of bureaucratic processes (such as cumbersome, internal standards) that end up being sidestepped by teams when building new APIs. Gemma Sindall’s eight-step approach to API governance best practices follows a similar approach, finding the balance between centralised and federated practices.
  • The thinking from Divya Joshi , Podila Madhusudhana Rao and Sheetal Pratik at Thoughtworks on federated data governance proposes some foundational steps, particularly to support their data mesh model, which may be particularly useful to consider in health systems, like the European Health Data Space expected to be established this year.
  • Matthew Skelton and Manuel Pais’ Team Topologies model defines four types of teams across an organisation that can help support cohesive API strategy and implementation.
Team typologyTeam Function
Stream-aligned teamA line of business team focused on engineering or product management for a specific business or policy area
Enabling teamA team that helps stream-aligned teams overcome challenges and identifies needed capabilities, for example, a centre of excellence team or centre for enablement
Complicated subsystem teamA team requiring significant technical expertise such as a data science team
Platform teamA team that might be composed of members of other team types to create internal products that accelerate delivery by stream-aligned teams, for example, a team responsible for documenting internal standards (a style guide) or an internal API catalogue

Source: Team Topologies model of team structures that can support API governance across an organisation

Here’s what the Platformable team has been working on this week:

Open Ecosystems

Open Banking/Open Finance

  • We continued our work preparing for our next Open Banking/Open Finance Trends Report (to be launched next week)
  • We worked with a client on our API product monetization workshop and tool

Open Government

  • We continued work with clients on API strategy and API management technical architecture design

Open Health

  • Continued work on data governance projects for our community health organisation clients
  • Continued to add to our open health ecosystem database

Open Sustainability

  • Completed documenting our value model and impact taxonomy
  • Continued research for our trends report looking at how open banking and open finance APIs can support sustainability efforts (to be launched on 15 February)

Business Development

  • We attended the MigraCode job fair, where we hope to recruit a new trainee in the coming months. Our current trainee, the excellent Elmira Saifullina is available for a full-time, junior developer job in Barcelona. I highly recommend Elmira: she coded our new blog index page, took the initiative to do a site-wide review and proposed changes to streamline website components, and then implemented those changes to ensure the site’s responsiveness across a range of screen sizes.
  • We pushed some website updates, including new product pages for Open Sustainability, a contact page, and an online chat widget that directly connects to me during working hours, so that I can hear from you about your needs, and how we could work together, so give it a try!