Embedded Test Automation – From Unit Tests to HiL

I’ve never worked with an embedded team that considered testing unimportant. On the contrary, everyone on the development team understands how crucial testing is for a stable overall system.
In practice, however, testing still sometimes stops at “works on my machine”. If it runs on the setup on someone’s desk, it’s considered fine.

Dedicated test systems for automated testing or well-defined manual test routines not only help uncover bugs through reproducible environments, they also strengthen developers’ confidence in their own code.

Gut feeling instead of real confidence – uncertainty in development

In my view, the biggest downside of missing tests is the uncertainty.
Every developer knows that moment right after a commit:

“I hope my fix didn’t break something else.”

This feeling comes up when:

• there are no defined test cases
• tests are carried out based on gut feeling
• results are not reproducible
• there is no test report that makes failures visible

To eliminate this uncertainty, you need a clear testing routine with concrete, detailed test steps and a clean test report.

A simple testing strategy

Even a simple testing strategy can have a big impact.
Once basic tests are run continuously in a defined test environment, confidence across the whole team increases.

Continuous test runs should, where possible and reasonable, be integrated into the CI/CD pipeline. For unit tests this is usually straightforward: they can be executed directly on the host if the software is decoupled from the hardware. Unit tests should be portable, easy to run, and ideally fast, because as a developer you don’t want to wait for hours after a commit to see whether it introduced any errors.

Integration tests, on the other hand, are usually executed in a simulated test environment (Software-in-the-Loop, SiL) or on real hardware (Hardware-in-the-Loop, HiL). Which setup makes sense depends on the operating system, vendor, programming language, use case, and overall complexity.

Different test environments can be combined, but they should be chosen deliberately, and only if the extra benefit justifies the additional effort. The more environments and test cases you run in parallel, the higher the maintenance overhead in the QA process. I’ll go into the differences between host-, SiL-, and HiL-based environments in a separate article.

From manual to automated integration tests

When defining integration tests, a simple rule of thumb applies:
Start with manual tests, then automate.

A good sequence is:

1. Document what you are currently testing manually
2. Describe the test steps in enough detail that even a non-developer could follow them
3. Run these steps against different software versions
4. Note down boundary conditions and typical failure patterns
5. Identify test cases that
   • take the most time
   • provide the most value
   • are easiest to automate

Only then does the actual automation start.
Manual tests are candidates for automation, but you should always begin with manual testing before you automate except in the case of trivial unit tests.

These more complex tests are usually executed in SiL or HiL environments. Depending on the use case, one of the two setups or even a mix of both may be the better choice.

HiL test environments

Pure Software-in-the-Loop (SiL) tests are often not enough when you need to test closer to the real hardware, or when building a simulation environment would be too much effort.

In these cases, Hardware-in-the-Loop (HiL) test environments are used. In practice, a combination of HiL and SiL components is very common.

Benefits of a HiL/SiL mix

This hybrid approach is often chosen to ensure safety while increasing test efficiency. A good example is controlling a battery storage system:

1. Safety: Directly integrating a real battery into the HiL setup without sufficient safety measures involves significant risk and can endanger lives.
2. Efficiency: Tests with a physical battery storage system are extremely time-consuming, because state changes (e.g. jumping from 3% to 95% state of charge) cannot be forced artificially. You would have to wait for realistic charging cycles instead.

ChatGPT said:

That’s why the battery storage system is usually simulated, while the controller under test is physically present.
For specific validations that require a real battery, a separate test bench with adequate safety measures can be set up in parallel.

Components of a HiL environment

To run HiL tests, the following components are essential:

• DuT (Device under Test): The hardware you want to test (e.g. an evaluation board or a custom PCB).
• Flashing mechanism: An automated way to flash the DuT with the software under test.
• Power supply: To power the DuT and perform resets.
• Connection to the test system: An interface between the DuT and the test system to execute tests and validate results. For embedded Linux, for example, an SSH connection works well.

Optional components that can extend the setup:

• Signalgeneratorensignal generators
• oscilloscopes
• spectrum analyzers
• logic analyzers
• communication interfaces (UART, CAN, SPI, I²C, etc.)
• additional power supplies and loads
• etc.

These tests can also be integrated into the CI/CD pipeline (Continuous Integration/Continuous Delivery). They are often executed after every commit to specific branches, overnight, or on merge requests into main. The exact strategy depends on the scope and runtime of the tests. For complex systems with multiple subsystems, it is often useful to split integration tests into several stages to enable step-by-step and more efficient validation.

It can also make sense to set up multiple HiL environments with different levels of functionality to maximize utilization and run tests more frequently. For example, long-running tests and fast integration tests should be executed on separate environments or at least at different times (e.g. on every commit vs. nightly runs).

Hardware-in-the-Loop environments are usually highly tailored to the specific device or system under test. However, parts of the HiL architecture can be kept generic and reused across projects. I’ll go into more detail on that in a separate article.

Conclusion

Uncertainty and lack of confidence don’t come from a lack of developer skills, but from a lack of structure in the testing process. With a clearly defined testing routine, reproducible test environments, and simple tests that run continuously, you can already eliminate a large part of that uncertainty.

Unit tests are the fastest way to get started. When you complement them with manual test definitions, SiL tests, and HiL setups, you gradually build a stable and scalable QA process.

Test automation doesn’t start with tools; it starts with clear, understandable manual tests. Reliable automation is built on top of that.