Agile Assurance: Advice for starting the Agile journey

[dcouldwell]

Overview

Many organisations starting in their Digital Transformation journey face challenges finding an approach that allows them to deliver with the required velocity but still deal with the reality of existing enterprise systems and multi speed IT.

Apigee recommend the adoption of an Agile development approach to meet these dual demands.

Adoption of a truly Agile approach can be a significant challenge and is most successful when it is accepted by the API team as opposed to being imposed on them.

There are many challenges when adapting to this new development model and I'm only going to cover some suggestions on the basics of how the core 'build' cycle can adapt over time assuming you're starting from a Waterfall standpoint. These examples assume Scrum is being adopted as the approach but other Agile methodologies can follow a similar approach.

Waterfall

Lets assume the organisation is starting from a traditional Waterfall approach. This is still a common approach and is still suitable in cases where an organisation is developing a large enterprise system of record with it's associated longer delivery timescales and heavy weight needs in terms of process and gatekeeping but does provide the agility and speed required for API programs.

Lets assume that in this model the development phases are broken down into the traditional steps of

1. Requirements Definition
2. Development
3. Test
4. Release

Where the timescale between steps 1 and 4 could be multiple months.

Scrumfall

A first step on the journey can be reducing the iteration cycle for steps 1 and 2

The following steps move into short iterations (Sprints) e.g. 2 weeks

1. Requirements Definition
2. Development

Feeding into a separate cycle of

3. Test

4. Release

Where steps 3 and 4 may still run on a cycle of many weeks / months

As a first step it allows development teams to get used to some of the Agile concepts such as Sprint Planning, Demos, Retrospectives and the general iterative feedback loop approach. Each Sprint should still have a clear definition of done (exit criteria) for any items completed e.g. items have passed basic developer generated unit tests and been deployed into the test environment. The definition of ready (entry criteria) at this point would just be that there was enough details in each user story for development to be completed but the full testability of the story may not be fully elaborated.

It's important to not get stuck at this stage as you aren't yet getting the full benefits of Agile. There are still significant inefficiencies in having to feed testing feedback as a separate loop and the release cycles are still not delivering the kind of velocity that provides real benefits.

Test Driven Development

As a next step the testing process can be brought into the Sprint cycle.

This introduces more of the concepts of Test Driven Development (TDD) where the the definition of ready for a given item is that the test approach is clearly defined before development starts and tests are being written before development starts. The Test phase may still add in additional tests as the testers test the solution. The definition of done should be providing you with production ready code.

At this stage each Sprint now comprises of the first three phases i.e.

1. Requirements Definition
2. Development
3. Test

With the release process on a different cycle which still be in the range of weeks.

Typically within the Sprint the Development and Test phases are working in series with testing feeding back to development within the Sprint cycle. At this stage testers may be adding in additional tests on top of those created by the developers. Tests should be automated i.e. repeatable and configurable across environments.

But don't get stuck here. We're still not getting the full benefits of a regular release cycle into production and we're still having to pass feedback from testing back into development which is inefficient.

Continuous Integration

In order to take the next step we need to improve a few areas.

1. Remove cycling between test and development
2. Fully automate testing
3. Fully automate deployment

In order to do this we need to ensure that all tests are written before development begins and are complete i.e. if the tests pass we need to be confident we're done and there is no need for a separate Test cycle or additional tests to be added after development. The lines are blurring between developers and testers and ideally they are one in the same and the previous Development and Test steps are combined into one 'Build' step.

Both testing and deployment should be fully automated so they can be run by your Continuous Integration (CI) tool of choice and deployments should be controlled through your CI tool and pushed through the different environments required before the production release.

The production release cycle itself can be tied closely to the Sprint cycle so the organisation is gaining the maximum benefit from each cycle.

Each Sprint should now consist of

1. Requirements Definition
2. Build
3. Release

This is commonly the end point for Agile maturity for many organisations and is providing the speed to market and agility required for a successful API program but there is a further step that you can take.

Continuous Delivery

Continuous Delivery is an extension of Continuous Integration where the build cycle can further be reduced so that the 3 steps outlined above can be iterated on even within a given Sprint. It requires an additional level of maturity for your testing, deployment and release process and also an organisational mindset change in terms of confidence in the build process. As soon as a development change is checked in and passed automated testing it will be pushed through the different testing environments before being automatically deployed into production.

By reducing the production release cycle to the minimum this provides the maximum benefit from the Agile concepts above.

Summary

This article presents a very simplified overview of the potential steps an organisation can take on the Agile journey.

There are many other aspects of the journey that I have not covered e.g. SCM approach and other areas that I have not covered in detail e.g. TDD and CI but hopefully this article gives you some ideas on how you might start the journey.

Although Continuous Delivery may not be a realistic goal for all organisations it's always worth keeping this in mind as a goal as even if this might not be our final destination it will keep you on the right path. The steps above will also need to be catered for your organisations level of maturity and hopefully you'll be able to jump right into the TTD stage and move to a CI centred approach in the short term.