How ING Increased Software Deployments to Twice a Day

During a recent webinar , ING’s IT Manager Andreas (A.J.) Prins, met online with Andrew Phillips, VP of DevOps Strategy at XebiaLabs to discuss the strategies ING used to build, improve, and maximize their continuous delivery pipeline.


ING was able to rebuild their continuous delivery pipeline, reducing their release times from 6 weeks to 34 minutes. Prins stressed, “Continuous delivery will never be the ‘goal,’ it is just part of delivering software”. Continuous delivery has one true purpose: delivering value, to either the business or the customer. Once your organization has decided it wants to make the most out of its continuous delivery pipeline, you need to ask yourselves, “how much time do we want to commit to improving our pipeline so that it meets organizational goals”.

“Continuous delivery will never be the ‘goal,’ it is just part of delivering software.” – A.J. Prins

 

Identify Bottlenecks and Release More Quickly

Map out your continuous delivery pipeline and measure how long each process takes. For example, your testing stage may require 15 minutes, while all of the other stages in your continuous delivery pipeline combined don’t even take half as long. Once you have identified your bottleneck, you should “decide to make that a focus point and explore how to cut it down,” says Prins.

 

Screen Shot 2016-02-09 at 2.12.09 PM

 

Factors for Success

  1. Take time to improve processes; you have to be willing to invest the time. “10-20% of your backlog for improvement,” says Prins.
  2. Take small steps and improve;“automate everything 100%, then move to the next phase. Do it step by step,” proclaimed Prins.
  3. Continuous delivery is a team effort; everybody involved needs to understand how continuous delivery adds value.
  4. Visibility of the pipeline; allows for analysis and continued improvement. Prins and his team at ING used XL Release to orchestrate their continuous delivery pipeline.

 

Strong Principles

Prins and his team lived by the following principles in order to build their pipeline and reduce their release cycle from 6 weeks to 34 minutes:

  1. We Focus on customer value; where some businesses may focus on adding business value instead.
  2. We Work AgileAppliance of principles described in the Agile Manifesto.
  3. Automate as much as possible
  4. We continuously improve; a necessary mindset in order to compete in IT. In today’s day and age, this is “the most important because it affects everything,” says Prins.
  5. Continuous Delivery is part of the product; CD will never be the goal, it is just part of delivering software. “Think of a car’s engine. You need the engine to run the car, but it is the driver who gets the car from point A to point B”.

 

General Tips

  1. Have a target release date. Setting a specific yet challenging release date will push your team and bring about improvements to your software delivery process.
  2. Be flexible. Don’t plan on a daily basis. Target and actual release dates will sometimes change. This will help your team to adapt to different situations with greater agility.
  3. You can improve your pipeline simultaneously. Following DevOps practices should be a giant learning curve, but there are plenty of experiences you can learn from along the way that will help your team to grow more efficient.

 

What the ING Pipeline Looks Like Today

  • Software (by team)
    • Multiple deliveries per weeklong sprint
    • Smaller increments
  • Testing Software
    • Automated
    • Minutes
    • High quality
  • Go Live
    • Only one incident in 8 months
    • Minutes to deploy
    • Predictable results
  • Security is Part of Development
    • Deliver business value through this quality

 

Reducing your release cycle from 6 weeks to 34 minutes is no easy feat. Combining hard work and commitment with a process that has proven its business value is how companies like ING are able to overcome such odds.

For more details, listen to the entire ING presentation here or read the case study here.

Editor’s note: This post was originally published in February 2016.


Alexander Moss-Bolaños

About the Author ()


  • Tim Coote

    The pipeline does not seem to include much by way of behavioural tests. And I cannot see the data-migration/rollback parts. Is this a pipeline of a representative enterprise application?

    • Hi Tim, thanks for the question. This is a representative pipeline of applications built and used by ING. Every organizations pipeline will be different because they have different governing models and different checks and balances along the pipeline. Some of those checks and balances may include testing at different stages along the pipeline. And that might include behavioral tests. Since this was just a presentation, it does not show the particular mechanics behind the different aspects of test data management, but our XL TestView product does just that.

      I’m not exactly sure what you were asking for about the data-migrations/rollback parts, but I can comfortably say our XL Deploy product manages rollbacks (https://xebialabs.com/products/xl-deploy/features/#automated-rollback)

      • Tim Coote

        I’d guess that there’s a scale issue. In my mind an enterprise application build would take person decades to complete. When replacing/extending existing systems a large chunk of the cost is typically for data migration. A CD approach means that this migration and its testing is the responsibility of the dev. team. For any significant deployment, I’d expect to see blue/green approaches used to avoid service outages. Further integration makes the challenge more fun.

        Getting these things right is more than a product issue. Clearly, life’s easier with good tool support. But you still have to do things ‘properly’ to get the engineering right.

        otoh, for a ‘simple’, standalone web type of application, there could be no data to transfer between releases and few behavioural concerns as there are no data-flows that could create behaviour variation in human noticeable timescales (e.g. the whole thing could run on one box).

        Larger scale applications following CD tend to release on a per sprint timescale – even though each commit potentially creates a releasable product. Smaller scale tend to follow more of a continuous deployment model. Decomposition into decoupled components has become popular as an approach to supporting higher rates of change (microservices), but that requires even more test automation.