How to stop worrying and start enjoying working with others — a CI/CD story (with a scoop of ice cream 🍨)
CI/CD stands for Continuous Integration, Delivery and Deployment. But if you still wonder what advantages CI/CD brings and why it is difficult to imagine modern software development without it, then this article is for you!
We’ll use an example of an imaginary web application and see how adding CI/CD pipeline helps at different stages of product evolution.
With that, let’s begin!
Let me introduce you to Kate. She is a product developer👩🏻💻 and she came up with an idea — an app to help people find the nearest ice cream shop🍦. After a week of work she created a simple website, a proof of concept, to find the nearest spots to get a refreshing ice cream. Just in time before the start of summer season!☀️
While working on her own, Kate kept a copy of the codebase on her machine and for simplicity deployed changes to the website manually.
She shared the link with her friends, who spread the word and soon Kate’s ice cream locator got popular and the number of visitors started rising🎉. More and more feature requests were coming. Happy Kate convinced her friend Bob 👨🏽💻 to join and to grow the ice cream locator app together.
There were now two of them in a team. Kate moved her project code to a shared repository, so that they could work collaboratively with Bob.
Bob started working on a feature to add search for ice cream🔍, while Kate was working on the visual improvements of the whole website.
In software development we rarely work individually, most of the time the success of the project depends on the quality of collaboration within the team⛹️♀️. And big chunk of this collaboration comes from working on the same code base. There are many challenges involved in changing code simultaneously: how to keep project in a consistent state? how not to break work done by other colleagues? how not to overwrite accidentally the changes done by someone else (who naturally will be vastly unhappy about this 😠)? how to merge your changes into constantly evolving project?
These problems often referred as ‘integration hell’ or ‘merge hell’. Let’s look closer what it means and how CI/CD helps preventing it.
Goal # 1: prevent merge hell🔥
Kate and Bob worked independently in their local branches, they got into a zone where they were modifying many lines of code, bringing new features and occasionally refactoring code in the files they were changing💃. One change was bringing another, and it felt ages till they were done with their commits.
At that point of time, their local versions became so different, it took them the whole day🤯 to deal with all merge conflicts and to find how their changes can coexist together without breaking the website! What a waste of precious time!😭
They decided that they can’t work effectively unless they follow continuous integration practices.💡
Continuous integration to the rescue!⛑️
Continuous integration helps minimising merge problems and keeping code coherent. How is it done? Let’s look at the changes in the development practices, which Kate and Bob embraced.
First, they could no longer work independently for a week and merge big amounts of changes at once. That’s why, they agreed to adopt a habit to frequently rebase their local branches and merge the changes in smaller chunks on a daily basis.
But this alone is not sufficient, another condition necessary for continuous integration is that every commit is kept atomic and not breaking other functionality in order to keep the project in a consistent and stable state. So, when Kate pulls the latest changes from Bob, the project is in complete working state.
How can they be certain that every new commit does not break the project? For this a set of checks is run every time someone pushes a change into the repository. This can be achieved through multiple tools, such as GitLab, CircleCI, Jenkins, AWS CodePipeline or a set of scripts to run the build for each new commit before code could be merged.
Now they had a very simple code pipeline with continuous integration in place.
With these changes to their practices, they continued working on the project and making smaller commits several times throughout a day. This helped Kate and Bob work faster, bring new cool features to the website, such as searching ice cream by flavour or finding options for allergic eaters.
The ice cream locator had big success🎀 and at the end of the summer Kate and Bob started thinking of areas to expand the product. They decided to cover other delicious street food options🍢🌮🍤! This would mean a lot of work. They needed help!
Three more people joined their team: Ada and Jin joined to help with development work and Carl took a role of quality engineer, since with more and more features, ice cream locator became buggier and buggier🐛🐛🐛.
Coordinating changes in the project is a tough task, and the bigger the team is, more challenges pop up. With new people on the team, who are less familiar with existing features, it is easy to overlook logic and make a regression in existing functionality. That’s why it is important to have a contract and mechanisms to support testability and increase feedback loop to fail, improve and learn faster. That’s why our next goal is focused on the quality of releases.
Goal # 2: boost the quality
With multiple changes to the project it is easy to lose track of global picture and with new functionality break old features. To prevent this from happening our team decided to introduce two new types of tests: smoke tests and user acceptance tests.
Smoke tests are a subset of all tests focused on main functionality of the application. Since the number of tests is limited, they can be run quickly and more frequently. 🏎️
Acceptance tests or functional tests are used to verify that the functionality works as expected from the end user point of view. An acceptance test is a scenario with clearly defined steps describing user action and following system behaviour. In agile practices these tests are defined as part of user stories and written in human language.
Both smoke and acceptance tests should be run in an environment close to production one. This is usually a staging environment, but there can be a dedicated environment for a quality engineer.
To introduce the staging environment our team needed to expand their code pipeline one more time, now with continuous delivery.
Continuous delivery to boost quality
The goal of continuous delivery is to prepare the code changes to be released to production. This can serve different purposes — deploying to separate environments, running integration tests, performance tests, smoke tests and acceptance tests.
Continuous delivery is a powerful tool to look at a product increment from different standpoints. It also gives a possibility to share latest changes with stakeholders to get their feedback before the changes are released into production.
Goal # 3: shorten time to market
Our imaginary team has set up continuous integration and continuous delivery, but someone still has to press the button to deploy the changes to production.
But imagine, that every commit an engineer does can be available to the end users within hours. An old version will be retired and end users can enjoy new features and fixed bugs sooner. And as a result the team reaches quicker time to market and has a faster feedback loop. All of this affect the happiness of the customers and removes the burden from engineers by taking away boring manual error-prone tasks.
This is where continuous deployment is used:
Many more goals: learn customer preferences, do gradual roll-outs and many other things
When teams and projects grow they uncover new needs and there are many scenarios where CI/CD can help. For example, you can add instruments to apply AB testing to learn more about the customers, roll out features gradually, notify other teams/members to make specific reviews, run visual tests to check for regressions and many other things.
It’s also common to have way more than one pipeline in order to separate concerns and simplify projects.
Conclusions and what’s next
CI/CD can be seen as a big toolbox, where at the core we have continuous integration, continuous delivery and deployment. Once the fundamentals are in place, many smaller tools can be applied. Every tool has its own scenario, where it performs best. This can depend on project nature, team size, technology stack. At the end of the day, the goal is not to use as many instruments as possible, but to use the right tools to reach the goals.
Why do I love CI/CD? It gives me, an engineer, predictability and transparency. CI/CD represents a contract between team members, establishing practices and norms for collaboration and work with code repositories. What’s more important, CI/CD provides a toolbox to translate contract into manageable mechanisms. Not to mention that I’m terrified by merge hell😨, so continuous integration practices help me keep a clear conscience.
If you read till here, you might wonder, how can we work with CI/CD having features which are not possible to finish within a day, or even a week. This is where feature flags come to help. But this is a story for a different time!
Thank you for reading. Let me know, do you use CI/CD in your projects? What do you like about it and what challenges do you have?
