Governance and Federation for Multi-Team API Development - The Virtual API Team

[ozanseymen]

Introduction

In this article, I would like to introduce the concept of a Virtual API Team as a way to solve governance and team federation concerns in large API programs. I will be going through different team structuring options and explain where responsibilities lie for each option.

As the number of teams in an API program increases, concerns around standardisation, common documentation, management of reusable artifacts, knowledge sharing, etc increases. APIs will be developed by multiple teams, perhaps across different business areas, all at the same time. This presents problems in the following broad categories:

1. API Standards: When it comes to exposing your APIs to the consumers (public, partners or internal), they all need to look and behave the same in order to present a unified view of the entire API catalog. This is important to increase developer adoption and lessen the learning-curve required by developers consuming your APIs. If they familiarise themselves with request/response structure and format, understand how to pass in tokens, do pagination, choose api version, etc, they should just follow the same patterns for all other APIs, without knowing which team developed the API or needing to consult documentation for each API in the catalog.
2. Development/Operations: API developers need to share knowledge, reuse common artifacts, devops practices, documentation, etc in order to cut down development time and decrease bugs.
3. Evangelism: Spreading the good word about APIs, increasing the adoption and awareness of the platform. Objective is to gain support and contributions from the rest of the organisation. Your API program will not be efficient and effective if it stays in the tech floor - whether this can even be called a “program” is open for debate. If APIs will be open to the public, once inside the organisation is conquered, public evangelism of the platform is very important in order to gain wider API developer adoption.
4. Knowledge Sharing: Includes meetups with all API teams and the creation/maintenance of a central document repository for all the good standards, principals, API docs, etc that are agreed throughout the process.

Virtual API Team concept, as an attempt to solve these problems, came up when I was doing a webinar with two Gamesys architects, Michael McCarthy and Zsolt Sztupak about their microservices journey. You can watch the full recording here.

There are lots of discussions on the Multiple API Teams problem in community but one notable one is this question in Apigee Community. In one of the answers in that question, Dom Couldwell talks about 3 different roles the API program management should consider in order to run multiple API teams successfully: Policeman, Teacher & Referee. It is a must-read to appreciate the severity of the problem.

The Platform Team

One of the options to solve the multiple API team problem is to create a special team called “The Platform Team”. This team will overarch the individual API teams to solve API and devops standardisation and tackle the categories of problems I talked about above. They will wear the Policeman, Teacher and Referee hats throughout the process like Dom described.

This is a great model for many organisations that we see a lot of in our day to day work in Apigee professional services. We generally see 2-4 resources assigned to this team who will be extremely busy for the first couple of months defining and arguing the best standardisation and reuse across the whole API program.

However, depending on the culture of your organisation, you might see the following fallbacks of this model:

1. You will most probably choose the brightest individuals who have proved themselves in competency and technical leadership into the platform team. As you cannot assign the entire IT department to this team, many people will be left behind - outside the new shiny team to make standards for a platform that people call “the future of this company”. This might create many disappointments and angry faces.
2. If your API teams have strong technical resources who are extremely knowledgeable (and opinionated) about API design and development, and if some of those resources do not end up in the platform team, they might challenge the adoption, impede the progress and sometimes declare their own (and the team’s) independence.
3. Depending on the culture of the organisation and nature of the developer community within the organisation, it may not be acceptable for one “elite” team to push standards and knowledge to other teams. This naturally forms an imbalance within the team structure and dynamics which doesn’t always end well.
4. Platform team, with its finite number of resources, can be a bottleneck especially in the initial months. Teams can start development but if Platform team is not catching up in unifying the standards, teams might progress in different directions. Platform team may try to stop API team’s progress but this will not go well with the PMO.

If you see the pattern here: items 1-3 is all to do with some resources feeling left out and some believing they could do the job better than some resources in the platform team. Item 4 is about classic resource allocation and work scheduling. If you think your organisation will suffer from these issues (based on your culture), read on...

The Virtual API Team

This model can be further improved by yet another team structure - but this one is “virtual”. Here is how it works: You have your platform team and API teams. You ask each team to allocate x resources (2 is a good number to start with) to form “The Virtual API Team”. For example, if you have 4 API development teams, you would end up with 10 resources (including 2 resources from the platform team) to form a virtual team. Each team decides who those 2 resources will be and they can rotate the resources amongst the team members to achieve equal contribution. All of the responsibilities we talked about above for the platform team, will be carried out by this virtual team now - the important difference here is that, in this model, decisions will be made by those collection of resources (consortium/quorum) representing all teams. Yes, there will be disagreements - especially when discussing topics such as API versioning, hypermedia, etc - but at least it will be done freely and openly with each team having a say in the matter.

This team is “virtual” in nature because it doesn’t really exist in the organisation as a legitimate team - it is just a group of people (may not always be the same people) getting together to collaborate on the whole decision making process.

Are you wondering what the purpose of the platform team now is in this new model?

1. To organise and administer virtual team meetings, scheduling meetings, calendar invites, etc.
2. Take actions on the decisions that are made by the virtual team, e.g. maintaining and documenting standards, principles and processes, as agreed by the virtual team. Maintaining the central document repository for everyone to use.
3. Evangelism - executed and managed by the platform team but with contributions from API teams, i.e. platform team is responsible for organised events, internal/external hackathons, discussions with the CTO, public documentation with marketing, etc.
4. Common code artifacts - developing reusable code/policies/flows, etc and maintaining the central code repository for those artifacts ready to be “cloned” by API teams. This can also include kickstarter templates for speedy API proxy development, i.e. “if you are starting a new API proxy, clone this template repo and start from there. It has all common flows, policies and code already configured”.
5. DevOps - Executing on the devops processes as agreed by the virtual API team. Implementing CI/CD processes. Owner of all centralised/common component, e.g. centralised Jenkins, docker hub/images, cloud resourcing, Apigee organisation, etc.
6. Training - Either train the trainer or train new API team resources directly. Prepare materials for training and distribute amongst API teams.

Conclusion

The concept of platform team is a very nice concept to tackle the governance, federation and standardisation of the API platform. However it may not play nicely in all organisations depending on their culture. When more open debate and collaboration is required, the virtual api team concept can be utilised to fill the gap.

We are always interested to know other ideas to solve the multi-team api development problem - so let us know your solutions in the comments section!