It is always crucial to test new functionality as soon as possible to make sure it does not have a negative impact on the existing features. Our experience has shown that continuous integration is a viable approach when developing complex embedded systems. It usually means that you must invest effort in developing an integrated Automated Build System (ABS) and an Automated Testing System (ATS), but it also results in less effort required for testing and bug fixing. This approach enabled us to create high quality and reliable solutions.
Let’s take a closer look at some of the points we needed to address when deciding on the approach to a specific project.
REASONS AND BENEFITS FOR BUILDING ATS AND ABS ON EMBEDDED PROJECTS
1.) Deployments on embedded systems are complicated
When you develop software for custom hardware, it is usually complicated and time-consuming to upload it to the hardware. To save time, software releases are usually less frequent and comprised of multiple code base changes. The problem with this approach is that some failures are much harder to resolve because of the inability to pinpoint the cause to a specific code change.
2.) It is harder to debug after more commits
The traditional way of testing complex embedded software consists of a test engineer manually deploying and testing the new software release (multiple software commits from different developers) on the hardware.
What is potentially extremely time-consuming and unpredictable is the time spent on debugging and bug fixing in case tests fail. Finding the reason for the failure can be significantly faster if the code changes between consecutive software releases are small and done by a single developer.
With increasing complexity, the probability of complex system failures with non-trivial connections between triggers and resulting fails also increases. Believing that no complex bugs will arise in your system is just not feasible/affordable.
3.) With ATS and ABS, you avoid a black hole and save unpredicted time
As you may know, continuous integration (CI) is by definition “the practice of merging all developer working copies to a shared mainline several times a day”.
With an Automated Build System (ABS) and Automated Testing System (ATS) you can test new functionality (changes in the code base) as soon as possible. Being able to continuously run tests for small changes to the code base allows you to find and fix bugs more efficiently.
Developing ABS and ATS takes effort, but this effort can be planned and is predictable. To reduce the scheduling risk in your project, using continuous integration with ATS and ATB is the best choice.
Picture 1: Automated Testing System
AT SQD FIND OUT HOW CONTINUOUS INTEGRATION REALLY WORKS IN PRACTICE!
Visit my session at Software Quality Days 2018 on 18 January 2018 in Vienna, Austria, where I will talk about how and why we developed an ATS and ABS for a specific embedded system for one of our customers and why you need to think about developing your own when working on embedded projects.
See you at SQD2018!