How Can Robot Firmware Run Before the Hardware Exists?

• Logic tested on the computer
• Emulated microcontroller
• Real board with simulated input

What you will learn

• Name the three ways to run firmware without the finished robot and what each one can show.
• Structure firmware so its decisions can be tested with fake sensors, a fake clock and an injected link loss.
• Script a scenario such as a dropped heartbeat, write its assertions and run it on every change.

Before you start

Read the article on the layers of a mop-robot simulation and the one on how an ESP32 board runs a program. The behaviour under test is described in how a robot returns home when its link drops. Other terms are explained where they first appear.

Which levels of testing run before the robot exists?

There are three levels: tests on the host computer, emulation of the microcontroller, and a real board with simulated inputs. The first is the cheapest and the last is the closest to the robot.

A unit test checks one part of the code alone. The PlatformIO documentation says its unit testing lets you separate each part of the firmware and check that each part works correctly. Host tests need no robot at all: running on the host needs no specific hardware except the host machine.

An emulator runs the compiled firmware. Espressif's QEMU fork emulates the CPU, memory and several peripherals of ESP32, which is a way to test applications without flashing them to real hardware. Hardware-in-the-loop (HIL) means the real board runs the real firmware while a test rig supplies simulated sensor values and link events.

• Host test
• Emulated microcontroller
• Hardware-in-the-loop
• Left column, top to bottom: decision logic, timing, peripherals, radio
• Filled circle: can be shown
• Hollow circle: cannot be shown
• Half-filled circle: only in part

How do you make firmware testable?

Keep the decisions in code that does not touch the chip, and reach the hardware through a thin layer that tests can replace. ESP-IDF supports the same idea: its Linux-host tests are unit tests in which part of the hardware can be abstracted via mocks. A mock is a stand-in object that returns scripted values.

For the mop robot the logic layer holds the state machine: normal, suspect link, safe, heading home. The hardware layer holds the pump pin, the wheel drivers, the clock and the radio. Four fakes are enough to start:

• Fake sensors return values the test chooses, such as a stall or a low battery.
• Fake clock moves time forward on command, so a long timeout takes no real time.
• Fake link delivers heartbeats until the test stops them.
• Fake outputs record every command, so the test can ask whether the pump was switched off.

An illustrative sketch of the decision, written as pseudocode and not tied to any library:

on_tick(now):
  if heartbeat_expired(now) and state == MOPPING:
      pump_off()
      wheels_stop()
      state = SAFE
  if state == SAFE and local_return_is_valid():
      state = HEADING_HOME

The ESP-IDF documentation adds two cautions. Running applications on the host is still an experimental feature, and Linux-host tests are still under development, with only a small fraction of its components supporting them. Keeping your own logic free of chip calls avoids depending on that support.

• Decision logic (state machine)
• Thin interface layer
• Fakes: sensors, clock, link
• Real hardware: chip, wheel, pump

Which scenarios and assertions are worth scripting?

Script the situations that are risky or rare on a real robot: a dropped heartbeat, a jam, a low battery and a busy dock. An assertion is a statement the test checks, for example "the pump and wheels receive stop commands within the deadline, before any return motion is permitted". Function-call ordering must never delay a wheel stop.

Walk the first row through both levels. In a host test, the test sets the fake clock, delivers heartbeats, stops them, advances the clock past the timeout and reads the recorded outputs. It passes only when both stop commands are issued within the deadline; lost-localization input must keep the robot safe. Recorded output commands do not prove the real pump or wheels stopped. The run takes a fraction of a second.

In an emulated run the same real firmware image starts inside the emulator. Espressif's idf.py qemu monitor command builds the application, starts QEMU and connects the monitor to the emulated UART port, so the console log shows the state change. Wokwi offers a simulated Wi-Fi network with full internet access and a virtual access point for the link, and MQTT servers can be reached to send sensor data. Dropping that connection in the scenario plays the part of the heartbeat loss.

• Regular heartbeats
• Silence until the timeout
• Pump off
• Wheels stop
• Heading home only with valid local conditions
• Passed assertions

How do the tests run on every change?

Run the host tests on every change, the emulated runs on every merge and the hardware-in-the-loop runs before a release. Renode is meant to run in the background on a CI server, and it is integrated with the Robot Framework test suite. Continuous integration (CI) means a server builds the code and runs the tests automatically after each change.

The NASA handbook places such work under analysis: analysis includes the use of modeling and simulation as analytical tools. A passing simulation predicts; it does not replace the real robot.

Some things stay invisible at every level above the real board. The ESP-IDF documentation puts it plainly: simulating the environment and mocking dependencies does not fully represent the target device, so testing on the device stays necessary. By design reasoning, three limits matter for this robot: real-time timing of the control loops, the behaviour of real peripherals such as motor drivers and sensors, and the real radio in a real building. An emulated Wi-Fi link has no walls and no interference.

• Code change
• Server builds and runs the tests
• Result of each test
• Report to the author

Real-world example

This is an illustrative scenario, not a measured result. A builder has a controller board but no robot yet. They write the heartbeat rule as a pure function of the clock and the last heartbeat, and add the four fakes. The heartbeat-loss test fails because the pump misses its stop deadline. They fix the stop branch to issue both commands and keep the robot safe without localization; the test passes, and the CI server runs it on every change from then on.

Next they start the same firmware in an emulator and cut the virtual link; the log shows the state change. Only when the first real board arrives do they find that the timeout is too short for a Wi-Fi roam. The earlier scenario omitted roaming latency; they add that scenario and tune the timeout against measurements.

Common misunderstandings

A passing host test means the robot will behave. Host tests prove the logic, not the timing, the peripherals or the radio.

An emulator is the same as the real chip. The QEMU fork emulates several peripherals, so check that the ones your robot uses are among them.

Testing needs the robot to be finished. Most of the safety logic, such as heading home after a lost link, can be tested before any wheel turns.

Frequently asked questions

Can I test ESP32 firmware without the board?

Yes, in part. You can test the logic on your computer and run the firmware in an emulator such as Espressif's QEMU fork or Wokwi. A real board is still needed for timing, real peripherals and the radio.

What is the difference between a mock and an emulator?

A mock replaces one part of the code with a scripted stand-in for a test. An emulator models the supported chip and peripherals so an appropriate firmware image can run; it does not guarantee every board or peripheral works unchanged.

Do I need a CI server to start?

No. Run the host tests by hand first. Move them to a CI server once they are stable, so every change is checked without anyone remembering to run them.

Summary

• Firmware can be tested on the host, in an emulated microcontroller and on a real board with simulated inputs; the cheaper the level, the less it can say about the real robot.
• Separate decisions from hardware, and use fake sensors, a fake clock, a fake link and recorded outputs, so a scenario such as a dropped heartbeat becomes an automatic test.
• Run host tests on every change; timing, real peripherals and the real radio still need the real board.