Migrating to Apigee Reference

[davissean]

When migrating from an existing platform to Apigee, there are a number of considerations. Here is a reference approach to aid you in this process.

Analyse

Existing Behaviour

In order to scope the migration, we need to understand the behaviour of our existing platform. This will be used to derive our process, required tests and migration scripts.

It can be useful to group this behaviour into the following categories:

• Northbound Security
• Southbound Security
• Traffic Management
• Caching
• Error Handling
• Mediation
• Orchestration
• Analytics
• Logging
• Monetization
• Custom Policies
• None of the above

At this stage, you can also understand if there is any functionality that cannot or should not be migrated. This can be the result of unused APIs, code rot, or an increase in API maturity over time.

Team Skill Set

In order to perform the migration, it is necessary for the team to have a solid understanding of the source and the target platforms. This can be addressed with self-service training, professional services or by leveraging partner organizations that have Apigee experience.

Define

In an ideal world, the definition of our requirements would be in the form of automated tests. This reduces human error, lowers the time for validation and allows us to demonstrate in black and white the current state of the migration.

Functional Requirements

For the API behaviour defined in the Analyse section, we can write functional tests. Typically this involves taking existing tests and enriching them to ensure the highest possible coverage. A common framework for functional API testing is Apickli.

Non-functional Requirements

Where possible, we would also like to automate the quality checks for our non-functional requirements. This may be through Performance Benchmarking, Penetration Testing or Developer Portal Usability testing.

Standards Compliance

Depending on the maturity of the API Platform, it is likely that a list of API Standards have been developed as part of a wider governance model. By automating the testing of these standards, it is possible to identify previously unaddressed gaps and improve the compliance of each API.

Industry specific regulations may also govern your API program. In many cases an automated test suite is provided, which can be included in your migration test suite. Examples of these can be found for Open Banking and Healthcare.

In addition, please review the Apigee Product Limits to ensure the best performance of the platform.

Configure

Following a test-driven approach, once our test suite is complete we can begin implementation.

Platform Provisioning

Initially we must ensure that we provision Apigee using the correct deployment type. This may be as a Software as a Service, Hybrid or On Premise model. Please seek guidance for this step, and particularly ensure that you have accounted for Traffic Isolation, High Availability and Disaster Recovery considerations.

Developer Portal

There are two aspects of Developer Portal migration that are key considerations - API Specifications and Self-service registration. API specifications typically leverage Open API Specifications, which we can simply update. This is discussed further below in the Reuse section.

For self-service registration, we must change the underlying datasource that the Portal connects to for creating Developers and Applications. In Apigee this uses Management APIs, so we must ensure, regardless of technology, that our portal connects to these. If the previous portal is also based on Drupal, this can be done using the Apigee Drupal module without writing custom code.

Environment Entities

Once the platform is configured, we can initially migrate the prerequisite environment configuration. The API-first nature of Apigee’s Management capability allows us to automate the creation and maintenance using the underlying Management APIs. These APIs can be called directly or driven by a community tool such as the CSV import tool or Maven Config plugin.

Key Value Maps are used for holding environment-level configuration values, much like you would find in a properties file. These can be encrypted to hold secrets. After extracting the keys and values from your existing platform, import these in.

Caches are used to improve performance by removing the need to call external services where possible. At this stage, we just need to identify the cache entities we need to create, not the actual conditions or implementation.

Virtual Hosts are the entry point of our APIs. We must ensure that we have defined a Virtual Host for each combination of Host name and TLS configuration. For example, we would create separate Virtual Hosts for api.brand1.com and api.brand2.com.

Target Servers contain the configuration for each backend. It is recommended to use them to abstract the TLS configuration and Target URL, rather than hardcoding them in each proxy.

Keystores and Truststores are used for managing our certificates. We should identify any certificates required for our Virtual Hosts, Target Servers or JSON Web Signatures.

User Management

Developers, Operations, Business Users and Administrators must be added to the Apigee platform. This generally involves a few steps.

Optionally, configure Single Sign-On integration into the Apigee platform using SAML. This will allow users to maintain their existing credentials to log into the platform.

Identify the roles that are present in the existing platform and map those to Apigee roles. It is possible to configure custom roles with a high level of granularity by using the Permissions Management API. Finally we can associate the Platform users to their role.

Reuse

API Specifications

API Specifications define the interface to our APIs. They are useful for a number of reasons, including generating documentation, mocks, API Proxy code skeletons and test code. There are a number of tools available to do this for Apigee.

Whilst the industry has converged on using OAS3 as the defacto standard for API specifications, there others that are using. In general, the steps for converting any specification format to OAS3 are as follows:

1. Merge all references into a single specification
2. Escape unsupported characters for YAML/JSON
3. Convert to OpenAPI
4. Replace Hostname and Basepath

Shared Flows

Apigee allows API policies to be grouped into flows that can be parameterised and reused across API proxies. By building reusable Shared Flows, we can cover the vast majority of the functional areas identified in the Analyse stage. A reference of these common shared flows can be found here.

Proxy Templates

Rather than building every API proxy from scratch, it is now sensible to script the conversion from the previous platform to Apigee API Proxies. By creating a parameterised Template Proxy that references our Shared Flows, we are able to generate each proxy just by providing the required variables. Any template language can be used, such as Jinja2 or EJS. A simple example using Linux tools can be found here.

Secure

In the Configure section, we have already created Keystores, Virtual Hosts and Target Servers for securing communication for Apigee at the transport layer. It is likely that in addition, we are also implementing API Key validation, Basic Authentication, OAuth 2 or Open ID Connect. The first step to move this behaviour into Apigee is to migrate Client IDs and Secrets.

Within Apigee, clients are called Developer Apps. If we want to validate client credentials inside Apigee, we can transfer the existing Key + Secret pairs into Apigee for each client. Documentation for this process is here. Alternatively, we can use a Service Callout policy to validate these credentials in an external system.

If we are using a token based approach such as OAuth or Open ID Connect, we must create an Apigee Identity API proxy to generate these tokens. In generate this is a one-off proxy rather than being generated from the template, as most of the functionality is identity-specific. This proxy can use Apigee or integration with a third-party identity provider to generate tokens.

Finally, we must consider existing valid tokens. If a client obtains a token on the previous platform, they will expect it to continue to work throughout the migration. As such, logic should be implemented to validate tokens that are unknown to Apigee against the previous token provider. This logic should be in place until there are no more valid tokens in the previous provider.

Verify

In order to verify the success of our implementation, we can deploy our APIs and run our entire test suite. It is recommended to use the following stages for verification:

• Mocks - ensure that the tests pass against mocks to isolate the API Management policies from their dependencies
• Integration - integrate into a non-production environment and ensure the tests pass against these systems
• Staging - ensure the tests pass with the APIs connected to a production-like environment, in case there are environmental issues that weren’t identified in non-production.

If there are discrepancies you will discover these as failing tests. You can then evaluate which tests must be resolved immediately, and which can be flagged as exceptions before continuing.

Promote

When confidence has been achieved it is time to deploy to the Apigee Production environment and migrate clients to the platform. There are two types of traffic that we must migrate - Runtime and Management traffic.

Runtime traffic can be migrated by switching the DNS name, such as api.organization.com from the existing provider to Apigee. This can be done using a big-bang approach or a canary technique in which a percentage of traffic is moved over to observe the effects, before the full switch is complete.

Management traffic relates to the Automation tools that may be calling the Management APIs. These are hosted by Apigee, so it is the clients responsibility to update their applications. This typically isn’t a problem as the clients are usually development teams managing deployments, and the deployment endpoint can be included in the proxy template. In exceptional cases, you may choose to put an API proxy in front of the Management APIs to mediate traffic for a short time while clients make their changes.

Iterate

Now that you have the existing functionality in production, you can shut down the old platform and implement new features available to you in Apigee.

[reddier]

Help me red,

#BinOpen @davissean